[cfe-dev] RFC: Supporting private module maps for non-framework headers

Vassil Vassilev vvasilev at cern.ch
Thu Nov 13 12:36:43 PST 2014

On 12/11/14 23:37, Richard Smith wrote:
> On Wed, Nov 12, 2014 at 12:48 PM, Vassil Vassilev <vvasilev at cern.ch 
> <mailto:vvasilev at cern.ch>> wrote:
>     On 11/11/14 04:48, Richard Smith wrote:
>>     On Mon, Nov 10, 2014 at 4:00 PM, Argyrios Kyrtzidis
>>     <kyrtzidis at apple.com <mailto:kyrtzidis at apple.com>> wrote:
>>         Hi all,
>>         For frameworks Clang currently supports adding a separate
>>         module map file for the private headers of the framework. It
>>         looks specifically for the presence of
>>         ‘module.private.modulemap’ inside the .framework and parses
>>         both the public and the private module maps when it processes
>>         its module. We would like to extend support for private
>>         module maps for non-framework headers as well.
>>         In the Darwin platform, the public SDK headers are located in
>>         '/usr/include', while the associated private SDK headers are
>>         located in '/usr/local/include’. '/usr/local/include’ comes
>>         before '/usr/include’ in the header search paths.
>>     I worry that this will be fragile. If for any reason we look in
>>     /usr/include but not in /usr/local/include, we'll not load the
>>     private extension map and things will probably go quite badly
>>     from that point onwards. If the presence of the
>>     /usr/local/include headers is a fundamental part of a
>>     /usr/include module, then it seems better to me to specify that
>>     within the /usr/include module map.
>>     So here's one possibility: allow 'extern module' declarations to
>>     be nested within other modules, then write your /usr/include
>>     module map as:
>>     module MyModule {
>>       <...>
>>       extern module SomethingPrivate
>>     "/usr/local/include/module.private.map"
>>     }
>     Maybe off topic (sorry if I misunderstood): would that 'somehow'
>     allow placing a modulemap outside the /usr folder? (For cases like
>     /gcc's libstdc++/).
> There are a few related problems with this. One is that we need to be 
> able to map from a #included file's name to the module map file, if 
> we're loading that module map lazily. Another is that files named in a 
> module map file are found relative to that flie.
> We can solve the first problem with -fmodule-map-file=<libstdc++ 
> module map>. For the second half, I've been discussing with a few 
> people the idea of allowing a module map file to specify a "module 
> root" directory relative to which its files are found, which need not 
> be the directory in which the map is placed. (This also helps with 
> another problem: diagnostics when building or using a module point to 
> files relative to the module map file, which can result in some rather 
> contorted and unnatural paths.)
Thanks for the pointers. This makes sense. Would I be able to to specify 
in the a framework's directory modulemaps for external dependency. In my 
particular case, I'd like to be able to express that this is the 
modulemaps for the external dependency.

I was thinking what if we could accommodate more than one modulemap per 
file. Say:
cat module.modulemap:
modulemap Map1 {
   module M1{}
modulemap Map2 {
   modulemap_root /usr/include // Will use the virtual file system 
pretending the modulemap was found at the modulemap_root
   module N1{}

IMO this would allow the 'external dependencies' to be organized in 
different configurations. For example, a module per header of bunch of 
headers for module, whichever decides the framework fits best. For our 
use-cases that would be great. Maybe this could simplify also the cross 
referencing modules and visibility also...
>     Vassil
>>     (in addition to the other changes you suggest here). Then only
>>     allow a module to be extended if the extension is listed via an
>>     'extern module' in the definition of the module.
>>         We propose to make the following changes to Clang’s module
>>         mechanism:
>>         - When looking up a module through the search paths, in
>>         addition to ‘module.modulemap’ also lookup for a standalone
>>         ‘module.private.modulemap’ file. I will refer to this as the
>>         "private extension" module map.
>>         - When parsing a private extension map allow extending a
>>         module that was not defined before, without providing the
>>         full definition. To clarify, I refer to a module definition
>>         as this:
>>         module MyModule {
>>          <…>
>>         }
>>         while an extension is this:
>>         module MyModule.SomethingPrivate {
>>          <…>
>>         }
>>         An extension is a nested module with any depth.
>>         We can reuse the “extern module” syntax to indicate that we
>>         are extending a module whose definition is in a different
>>         module map:
>>         extern module MyModule
>>         module MyModule.SomethingPrivate {
>>          <…>
>>         }
>>         - After parsing the private extension map, we are still
>>         missing the module definition so module lookup will continue
>>         looking in the following header search paths. If the module
>>         we are looking for is not found then Clang will a emit a
>>         “module not found” error.
>>         - It may seem backwards that module search will find and
>>         parse the private extension ahead of the public one, but it
>>         is actually advantageous because this allows us to continue
>>         searching only until we find the module definition, at which
>>         point we will stop looking. If module search worked the other
>>         way then, after we had the module definition, we would need
>>         to always keep looking through the rest of the search paths
>>         in case there is a private extension map that we need to take
>>         into account, or treat certain paths specially and only look
>>         for private extensions in those.
>>         By finding the extension map early on, we keep the current
>>         semantics of doing the minimal search necessary to find and
>>         complete the module definition, without treating any
>>         particular search path specially.
>>         - After Clang finds and parses the public module map for
>>         ‘MyModule’, the module definition will be complete. Clang
>>         will keep track that there is a private extension map
>>         associated with the module and it will pass the paths of both
>>         the public module map and the private extension one to the
>>         module building invocation. This will result in one module
>>         file containing both the public and private APIs, similar to
>>         what we do with frameworks.
>>         - A module definition inside a private extension will be
>>         disallowed. The rationale is that otherwise it will be a very
>>         common mistake for users to write
>>         *module.modulemap:*
>>         module Foo {
>>           <public headers>
>>         }
>>         *module.private.modulemap:*
>>         module Foo {
>>           <private headers>
>>         }
>>         and then be left scratching their heads wondering why things
>>         are broken (things missing, headers included textually,
>>         etc.). Being more strict in private extension maps will be
>>         beneficial.
>>         Let me know what you think!
>>     _______________________________________________
>>     cfe-dev mailing list
>>     cfe-dev at cs.uiuc.edu  <mailto:cfe-dev at cs.uiuc.edu>
>>     http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20141113/1a6ec039/attachment.html>

More information about the cfe-dev mailing list