<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Nov 10, 2015, at 5:47 PM, Richard Smith <<a href="mailto:richard@metafoo.co.uk" class="">richard@metafoo.co.uk</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote">On Tue, Nov 10, 2015 at 4:50 PM, Ben Langmuir via cfe-dev <span dir="ltr" class=""><<a href="mailto:cfe-dev@lists.llvm.org" target="_blank" class="">cfe-dev@lists.llvm.org</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><div class=""><div class="h5"><br class=""><div class=""><blockquote type="cite" class=""><div class="">On Nov 10, 2015, at 4:39 PM, Richard Smith <<a href="mailto:richard@metafoo.co.uk" target="_blank" class="">richard@metafoo.co.uk</a>> wrote:</div><br class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote">On Tue, Nov 10, 2015 at 3:39 PM, Ben Langmuir via cfe-dev <span dir="ltr" class=""><<a href="mailto:cfe-dev@lists.llvm.org" target="_blank" class="">cfe-dev@lists.llvm.org</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><div class=""><span class="">Hey all,</span></div><div class=""><b class=""><br class=""></b></div><div class=""><b class="">Background</b></div><div class=""><br class=""></div><div class="">Suppose you’re developing a system library FancyLib that installs a module map file into a default search path, and you’re using implicit module maps. When you’re developing locally, you will likely end up with multiple versions of your module that are reachable in your search paths:</div><div class="">* The system version /usr/include/FancyLib/module.modulemap</div><div class="">* The local development version ~/…/Build/Products/Debug/FancyLib/module.modulemap</div><div class=""><br class=""></div><div class="">You generally want to use the local version, and ignore the one from the system. Unfortunately, this currently causes module-redefinition errors if both module map files are ever parsed. For framework modules, we have been getting away with this because the compiler almost never looks into a framework directory if it has previously looked into a framework directory of the same name. Sadly the same is not true of non-frameworks.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><b class="">Proposed change for users</b></div><div class=""><br class=""></div><div class="">Users developing non-framework system modules would add:</div><div class=""> -fmodule-map-file=path/to/local/copy/of/module.modulemap</div><div class=""><br class=""></div><div class="">Typically, the “local copy” of the module map file would be to somewhere inside their build products.</div><div class=""><br class=""></div><div class=""><b class=""><br class=""></b></div><div class=""><b class="">Proposed clang changes</b></div><div class=""><br class=""></div><div class="">The flag -fmodule-map-file already exists in clang, and it causes clang to explicitly parse the specified module map, making any modules defined in that file available for @import (or via auto-import from a #import).</div><div class=""><br class=""></div><div class="">I propose we extend -fmodule-map-file so that modules found by this mechanism are allowed to shadow modules that we found implicitly in header search paths. When searching for a module by name, we will not consider the shadowed module at all. It will be an error for header search to find a header that is considered part of a shadowed module.</div><div class=""><br class=""></div><div class="">If two modules with the same name are *both* found in the contents of -fmodule-map-file arguments, then it is a redefinition error. Since these flags are ordered, we could theoretically allow shadowing between them, but I cannot think of a good use-case for this and suspect it would usually be unexpected to users.</div><div class=""><br class=""></div><div class="">If there are two modules with the same name and *neither* of them comes from -fmodule-map-file, then it is a redefinition error just like it is today.</div><div class=""><br class=""></div><div class=""><b class="">Why not use header search path order to do shadowing?</b></div><div class=""><br class=""></div><div class="">I originally thought we should just allow modules that come from earlier search paths to shadow modules from later ones. This would avoid having to use an extra flag -fmodule-map-file, and let us infer the module configuration “for free”, since users are already accustomed to setting up their header search paths. However, there are a number of problems with this approach:</div><div class=""><br class=""></div><div class="">* Module map files from “later” search paths are often parsed ahead of module map files from “earlier” paths. We would either need to eagerly parse module map files from all search paths in order (slow), or we would need to be able to retroactively make a module shadowed (brittle).</div><div class="">* Module map lookup does not know about search paths, so when searching for a module map file by umbrella directory we don't know when we cross search path boundaries</div><div class="">* We would need to lock down the HeaderSearch interface to prevent modifying search path order in the middle of a build (although that might be a generally good thing…)</div></div></blockquote><div class=""><br class=""></div><div class="">The direction here makes complete sense to me.</div><div class=""> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><div class="">My patch implementing this is attached for reference.</div></div></blockquote><div class=""><br class=""></div><div class="">This seems to be somewhat more general than we need for the above proposal -- it supports a fully-general set of scopes, whereas it would seem sufficient to track a flag to indicate whether the module came from a -fmodule-map-file= file or from an implicit module map file. Do you have some future extension in mind that would need more than 2 different scopes?</div></div></div></div>
</div></blockquote><br class=""></div></div></div><div class="">Nothing concrete, although it feels like if we might someday be able to get at least one other scope for the default system search paths. The reason I chose to not just use a flag was - perhaps embarrassingly - that I haven’t been able to come up with a nice way to express what that “flag” is to the ModuleMap class without defining it in terms of our command-line interface, which feels like a conceptual layering violation. I’m open to suggestions thought :-)</div></div></blockquote><div class=""><br class=""></div><div class="">The flag seems to be whether the module map is explicit or implicit, but that's more at the level of HeaderSearch's world view than ModuleMap’s.</div></div></div></div></div></blockquote><div><br class=""></div><div>In that case it should apply to -fmodule-file as well, right?</div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class=""><br class=""></div><div class="">I also think it's probably unnecessary to build shadow modules as your patch does (unless you expect those files to be reachable via #include_next?). Instead, you could do the same thing we do when the definition of a module was provided by a loaded module file: skip to the close brace of the module definition and return. </div></div></div></div></div></blockquote><div><br class=""></div><div>I built the unavailable module because I want to diagnose #include of headers from the shadowed module. Including headers from the shadowed module likely indicates a bug in how the module or header search paths are set up.</div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class="">We particularly don't want to stat all the files named in the shadow module, since the extra stat syscalls can be moderately expensive. It seems like a cleaner semantic model to say that the shadowed module is completely ignored than that it exists and has the same name but is a somehow different module.</div></div></div></div>
</div></blockquote><br class=""></div><div><br class=""></div><br class=""></body></html>