<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Nov 12, 2014, at 2:34 PM, Richard Smith <<a href="mailto:richard@metafoo.co.uk" class="">richard@metafoo.co.uk</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div class="gmail_extra"><div class="gmail_quote">On Tue, Nov 11, 2014 at 8:04 PM, Argyrios Kyrtzidis<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:kyrtzidis@apple.com" target="_blank" class="">kyrtzidis@apple.com</a>></span><span class="Apple-converted-space"> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div style="word-wrap: break-word;" class=""><br class=""><div class=""><div class=""><div class="h5"><blockquote type="cite" class=""><div class="">On Nov 11, 2014, at 7:38 PM, Richard Smith <<a href="mailto:richard@metafoo.co.uk" target="_blank" class="">richard@metafoo.co.uk</a>> wrote:</div><br class=""><div class=""><div dir="ltr" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class=""><div class="gmail_extra"><div class="gmail_quote">On Tue, Nov 11, 2014 at 6:49 PM, Argyrios Kyrtzidis<span class=""> </span><span dir="ltr" class=""><<a href="mailto:kyrtzidis@apple.com" target="_blank" class="">kyrtzidis@apple.com</a>></span><span class=""> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div style="word-wrap: break-word;" class=""><div class=""><div class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">On Nov 11, 2014, at 6:37 PM, Richard Smith <<a href="mailto:richard@metafoo.co.uk" target="_blank" class="">richard@metafoo.co.uk</a>> wrote:</div><br class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote">On Tue, Nov 11, 2014 at 3:45 PM, Argyrios Kyrtzidis<span class=""> </span><span dir="ltr" class=""><<a href="mailto:kyrtzidis@apple.com" target="_blank" class="">kyrtzidis@apple.com</a>></span><span class=""> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div style="word-wrap: break-word;" class=""><span class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">On Nov 11, 2014, at 12:34 PM, Richard Smith <<a href="mailto:richard@metafoo.co.uk" target="_blank" class="">richard@metafoo.co.uk</a>> wrote:</div><br class=""><div class=""><div dir="ltr" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class=""><div class="gmail_extra"><div class="gmail_quote">On Tue, Nov 11, 2014 at 10:00 AM, Argyrios Kyrtzidis<span class=""> </span><span dir="ltr" class=""><<a href="mailto:kyrtzidis@apple.com" target="_blank" class="">kyrtzidis@apple.com</a>></span><span class=""> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div style="word-wrap: break-word;" class=""><br class=""><div class=""><span class=""><blockquote type="cite" class=""><div class="">On Nov 10, 2014, at 7:48 PM, Richard Smith <<a href="mailto:richard@metafoo.co.uk" target="_blank" class="">richard@metafoo.co.uk</a>> wrote:</div><br class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote">On Mon, Nov 10, 2014 at 4:00 PM, Argyrios Kyrtzidis<span class=""> </span><span dir="ltr" class=""><<a href="mailto:kyrtzidis@apple.com" target="_blank" class="">kyrtzidis@apple.com</a>></span><span class=""> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div style="word-wrap: break-word;" class=""><div class="">Hi all,</div><div class=""><br class=""></div><div class="">For frameworks Clang currently supports adding a separate module map file for the private headers of the framework. It looks specifically for the presence of ‘module.private.modulemap’ inside the .framework and parses both the public and the private module maps when it processes its module. We would like to extend support for private module maps for non-framework headers as well. </div><br class="">In the Darwin platform, the public SDK headers are located in '/usr/include', while the associated private SDK headers are located in '/usr/local/include’. '/usr/local/include’ comes before '/usr/include’ in the header search paths.<br class=""></div></blockquote><div class=""><br class=""></div><div class="">I worry that this will be fragile. If for any reason we look in /usr/include but not in /usr/local/include, we'll not load the private extension map and things will probably go quite badly from that point onwards. If the presence of the /usr/local/include headers is a fundamental part of a /usr/include module, then it seems better to me to specify that within the /usr/include module map.</div><div class=""><br class=""></div><div class="">So here's one possibility: allow 'extern module' declarations to be nested within other modules, then write your /usr/include module map as:</div><div class=""><br class=""></div><div class="">module MyModule {</div><div class=""> <span class=""> </span><...></div><div class=""> <span class=""> </span>extern module SomethingPrivate "/usr/local/include/module.private.map"</div><div class="">}</div></div></div></div></div></blockquote><div class=""><br class=""></div></span><div class="">This has drawbacks:</div><div class=""><br class=""></div><div class="">- Details of the private SDK, “leak out” to the public one. It should work similar to frameworks, in that the public SDK remains the same irrespective if there is or not a private API, and the private API is a straight addition on top of the public one without needing to modify something in the public SDK.</div><div class="">- It is a bit weak as guarantee anyway because the public module map must necessarily function even when the extension map is missing, which means pointing at the wrong path or missing the private map when you really need it will not be detected.</div><div class="">- Flexibility to extend a module from any path may be valuable for testing.</div></div></div></blockquote><div class=""><br class=""></div><div class="">OK, I'm not sure I understand what problem you're solving. If the /usr/local/include stuff works as a layer on top of /usr/include, why do you need them to be built as part of the same module? (Do your /usr/local/include headers override / #include_next some of the /usr/include headers, perhaps? If so, do you need the #includes in /usr/include to find the /usr/local/include headers rather than the /usr/include headers?)</div></div></div></div></div></blockquote><div class=""><br class=""></div></div></span>There are some cases of cycles between public/private headers which would be accommodated by a single module build but the primary motivation is that we would like the module public/private interface to be under the same namespace, so you’d do<div class=""><br class=""></div><span style="white-space: pre-wrap;" class=""> </span>@import Dispatch;<br class=""><span style="white-space: pre-wrap;" class=""> </span>@import Dispatch.Private;<br class=""><br class=""><span style="white-space: pre-wrap;" class=""> </span>@import Darwin.POSIX.Foo.Bar;<br class=""><div class=""><span style="white-space: pre-wrap;" class=""> </span>@import Darwin.POSIX.Foo.Bar.Private;</div><div class=""><br class=""></div><div class="">and generally any kind of private extension:</div><div class=""><br class=""></div><div class=""><span style="white-space: pre-wrap;" class=""> </span>@import Dispatch.SuperCoolButPrivate;</div></div></blockquote><div class=""><br class=""></div><div class="">Do you want / need them to be built as a single module file, or not?</div></div></div></div></div></blockquote></div><div class=""><br class=""></div></div></div>As I said, cycles may make things difficult for separate module files, but how are we going to get new submodules under the same module name with separate module files ?<br class=""></div></blockquote></div><br class=""></div><div class="gmail_extra">Well, the restriction that module files correspond to top-level module names is arbitrary and artificial. (It's also a bad idea for a few reasons. It makes incremental refactoring very hard, for instance, because you're required to have no cycles at any point between things in different top-level modules.)</div><div class="gmail_extra"><br class=""></div><div class="gmail_extra">Splitting up the description of how to build a module file across various module maps seems like a very error-prone strategy, especially if you're intending to be able to stop looking before you've read all of the relevant module maps.</div></div></div></blockquote><div class=""><br class=""></div></div></div><div class="">I think that the high level parts of my proposal are not dependent on whether we build one .pcm file or multiple ones, this is an implementation detail.</div><div class="">To be more specific, if we have </div><div class=""><br class=""></div><div class=""><div style="margin: 0px;" class=""><b class="">module.private.modulemap<span class="Apple-converted-space"> </span></b><span class="">(extension):</span></div><div style="margin: 0px;" class=""><span class="">extern module Dispatch</span></div><div style="margin: 0px;" class="">module Dispatch.Private {</div><div style="margin: 0px;" class=""> <span class="Apple-converted-space"> </span><headers></div><div style="margin: 0px;" class="">}</div><div class=""><br class=""></div></div><div class=""><div style="margin: 0px;" class=""><b class="">module.modulemap:</b></div><div style="margin: 0px;" class="">module Dispatch {</div><div style="margin: 0px;" class=""> <span class="Apple-converted-space"> </span><headers></div><div style="margin: 0px;" class="">}</div><div class=""><br class=""></div></div><div class="">It is an implementation detail whether we buiild one Dispatch.pcm file or a Dispatch.Private.pcm file that depends on another Dispatch.pcm; it should make no difference on user code.</div><div class="">Is this incorrect ?</div></div></div></blockquote><div class=""><br class=""></div><div class="">Whether we build one .pcm or multiple is observable in some circumstances. 1) We concatenate together all the header files built as part of one .pcm file, and parse them all at once, and that is not always semantically equivalent to building them in two separate passes. 2) If you have one big Dispatch.pcm file which also contains the private bits, and by any sequence of events you end up also pulling in another Dispatch.pcm that contains the public headers but not the private ones, you may get ambiguity errors. 3) We do not allow circular references across .pcm files but do allow them within a .pcm file.</div></div></div></div></div></blockquote><div><br class=""></div><div>(2) is a bug.</div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div class="gmail_extra"><div class="gmail_quote"><div class=""><br class=""></div><div class="">If your Dispatch.Private is simply a layer on top of Dispatch, then building them as two separate .pcm files seems like the right choice; it keeps your Dispatch module (for want of a better word) modular. If on the other hand, you need includes/imports in Dispatch to pull in headers / submodules from Dispatch.Private, then one big Dispatch .pcm is probably the right answer, and we'd need something like your proposal so we could say "here is a Dispatch module that's like the one in /usr/include but different in the following ways”.</div></div></div></div></div></blockquote><blockquote type="cite" class=""><div class=""><div dir="ltr" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div class="gmail_extra"><div class="gmail_quote"><div class=""><br class=""></div><div class="">(Whichever of these options we pick, we can make the "@import Dispatch.Private" syntax do the right thing.)</div></div></div></div></div></blockquote></div><div class=""><br class=""></div><div class=""><div>In both cases we would need to extend module map parsing to allow submodule extensions. I think the original proposal accommodates that with the possible need that once clang supports separating a top module to different pcm files that we may need to control whether there is a combined module file or multiple ones.</div><div>Given that separating a top module in multiple pcm files is a rather intrusive change and the combined pcm file is sufficient for our needs, I’d like to proceed for now with the extending module map functionality which will result in one top level .pcm file. Is this reasonable ?</div></div><div class=""><br class=""></div></body></html>