[cfe-dev] RFC: Supporting private module maps for non-framework headers

[Richard Smith]

On Tue, Nov 11, 2014 at 10:00 AM, Argyrios Kyrtzidis <kyrtzidis at apple.com>
wrote:

>
> On Nov 10, 2014, at 7:48 PM, Richard Smith <richard at metafoo.co.uk> wrote:
>
> On Mon, Nov 10, 2014 at 4:00 PM, Argyrios Kyrtzidis <kyrtzidis at apple.com>
> wrote:
>
>> Hi all,
>>
>> For frameworks Clang currently supports adding a separate module map file
>> for the private headers of the framework. It looks specifically for the
>> presence of ‘module.private.modulemap’ inside the .framework and parses
>> both the public and the private module maps when it processes its module.
>> We would like to extend support for private module maps for non-framework
>> headers as well.
>>
>> In the Darwin platform, the public SDK headers are located in
>> '/usr/include', while the associated private SDK headers are located in
>> '/usr/local/include’. '/usr/local/include’ comes before '/usr/include’ in
>> the header search paths.
>>
>
> I worry that this will be fragile. If for any reason we look in
> /usr/include but not in /usr/local/include, we'll not load the private
> extension map and things will probably go quite badly from that point
> onwards. If the presence of the /usr/local/include headers is a fundamental
> part of a /usr/include module, then it seems better to me to specify that
> within the /usr/include module map.
>
> So here's one possibility: allow 'extern module' declarations to be nested
> within other modules, then write your /usr/include module map as:
>
> module MyModule {
>   <...>
>   extern module SomethingPrivate "/usr/local/include/module.private.map"
> }
>
>
> This has drawbacks:
>
> - Details of the private SDK, “leak out” to the public one. It should work
> similar to frameworks, in that the public SDK remains the same irrespective
> if there is or not a private API, and the private API is a straight
> addition on top of the public one without needing to modify something in
> the public SDK.
> - It is a bit weak as guarantee anyway because the public module map must
> necessarily function even when the extension map is missing, which means
> pointing at the wrong path or missing the private map when you really need
> it will not be detected.
> - Flexibility to extend a module from any path may be valuable for testing.
>

OK, I'm not sure I understand what problem you're solving. If the
/usr/local/include stuff works as a layer on top of /usr/include, why do
you need them to be built as part of the same module? (Do your
/usr/local/include headers override / #include_next some of the
/usr/include headers, perhaps? If so, do you need the #includes in
/usr/include to find the /usr/local/include headers rather than the
/usr/include headers?)

> (in addition to the other changes you suggest here). Then only allow a
> module to be extended if the extension is listed via an 'extern module' in
> the definition of the module.
>
> We propose to make the following changes to Clang’s module mechanism:
>>
>> - When looking up a module through the search paths, in addition to
>> ‘module.modulemap’ also lookup for a standalone ‘module.private.modulemap’
>> file. I will refer to this as the "private extension" module map.
>> - When parsing a private extension map allow extending a module that was
>> not defined before, without providing the full definition. To clarify, I
>> refer to a module definition as this:
>>
>> module MyModule {
>>  <…>
>> }
>>
>> while an extension is this:
>>
>> module MyModule.SomethingPrivate {
>>  <…>
>> }
>>
>> An extension is a nested module with any depth.
>> We can reuse the “extern module” syntax to indicate that we are extending
>> a module whose definition is in a different module map:
>>
>> extern module MyModule
>> module MyModule.SomethingPrivate {
>>  <…>
>> }
>>
>> - After parsing the private extension map, we are still missing the
>> module definition so module lookup will continue looking in the following
>> header search paths. If the module we are looking for is not found then
>> Clang will a emit a “module not found” error.
>>
>> - It may seem backwards that module search will find and parse the
>> private extension ahead of the public one, but it is actually advantageous
>> because this allows us to continue searching only until we find the module
>> definition, at which point we will stop looking. If module search worked
>> the other way then, after we had the module definition, we would need to
>> always keep looking through the rest of the search paths in case there is a
>> private extension map that we need to take into account, or treat certain
>> paths specially and only look for private extensions in those.
>> By finding the extension map early on, we keep the current semantics of
>> doing the minimal search necessary to find and complete the module
>> definition, without treating any particular search path specially.
>>
>> - After Clang finds and parses the public module map for ‘MyModule’, the
>> module definition will be complete. Clang will keep track that there is a
>> private extension map associated with the module and it will pass the paths
>> of both the public module map and the private extension one to the module
>> building invocation. This will result in one module file containing both
>> the public and private APIs, similar to what we do with frameworks.
>>
>> - A module definition inside a private extension will be disallowed. The
>> rationale is that otherwise it will be a very common mistake for users
>> to write
>>
>> *module.modulemap:*
>> module Foo {
>>   <public headers>
>> }
>>
>> *module.private.modulemap:*
>> module Foo {
>>   <private headers>
>> }
>>
>> and then be left scratching their heads wondering why things are broken
>> (things missing, headers included textually, etc.). Being more strict in
>> private extension maps will be beneficial.
>>
>>
>> Let me know what you think!
>>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20141111/da4e8f1e/attachment.html>