<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 14/11/14 18:57, Ben Langmuir wrote:<br>
</div>
<blockquote
cite="mid:46492423-93D9-4638-A80D-E3BA67D3E34E@apple.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<br class="">
<div>
<blockquote type="cite" class="">
<div class="">On Nov 13, 2014, at 12:36 PM, Vassil Vassilev
<<a moz-do-not-send="true" href="mailto:vvasilev@cern.ch"
class="">vvasilev@cern.ch</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div bgcolor="#FFFFFF" text="#000000" class="">
<div class="moz-cite-prefix">On 12/11/14 23:37, Richard
Smith wrote:<br class="">
</div>
<blockquote
cite="mid:CAOfiQqnzUostvGO8=MTDJa+vDR5aSe4SODepBF5F4CzTuhXbig@mail.gmail.com"
type="cite" class="">
<div dir="ltr" class="">
<div class="gmail_extra">
<div class="gmail_quote">On Wed, Nov 12, 2014 at
12:48 PM, Vassil Vassilev <span dir="ltr"
class=""><<a moz-do-not-send="true"
href="mailto:vvasilev@cern.ch" target="_blank"
class="">vvasilev@cern.ch</a>></span>
wrote:<br class="">
<blockquote class="gmail_quote" style="margin:0 0
0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000" class=""><span
class="">
<div class="">On 11/11/14 04:48, Richard
Smith wrote:<br class="">
</div>
<blockquote type="cite" class="">
<div dir="ltr" class="">
<div class="gmail_extra">
<div class="gmail_quote">On Mon, Nov
10, 2014 at 4:00 PM, Argyrios
Kyrtzidis <span dir="ltr" class=""><<a
moz-do-not-send="true"
href="mailto:kyrtzidis@apple.com"
target="_blank" class="">kyrtzidis@apple.com</a>></span>
wrote:<br class="">
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div style="word-wrap:break-word"
class="">
<div class="">Hi all,</div>
<div class=""><br class="">
</div>
<div class="">For frameworks
Clang currently supports
adding a separate module map
file for the private headers
of the framework. It looks
specifically for the presence
of ‘module.private.modulemap’
inside the .framework and
parses both the public and the
private module maps when it
processes its module. We would
like to extend support for
private module maps for
non-framework headers as
well. </div>
<br class="">
In the Darwin platform, the
public SDK headers are located
in '/usr/include', while the
associated private SDK headers
are located in
'/usr/local/include’.
'/usr/local/include’ comes
before '/usr/include’ in the
header search paths.<br class="">
</div>
</blockquote>
<div class=""><br class="">
</div>
<div class="">I worry that this will
be fragile. If for any reason we
look in /usr/include but not in
/usr/local/include, we'll not load
the private extension map and
things will probably go quite
badly from that point onwards. If
the presence of the
/usr/local/include headers is a
fundamental part of a /usr/include
module, then it seems better to me
to specify that within the
/usr/include module map.</div>
<div class=""><br class="">
</div>
<div class="">So here's one
possibility: allow 'extern module'
declarations to be nested within
other modules, then write your
/usr/include module map as:</div>
<div class=""><br class="">
</div>
<div class="">module MyModule {</div>
<div class=""> <...></div>
<div class=""> extern module
SomethingPrivate
"/usr/local/include/module.private.map"</div>
<div class="">}</div>
</div>
</div>
</div>
</blockquote>
</span> Maybe off topic (sorry if I
misunderstood): would that 'somehow' allow
placing a modulemap outside the /usr folder?
(For cases like <span class=""> <em class="">gcc's
libstdc++</em>).</span></div>
</blockquote>
<div class=""><br class="">
</div>
<div class="">There are a few related problems
with this. One is that we need to be able to map
from a #included file's name to the module map
file, if we're loading that module map lazily.
Another is that files named in a module map file
are found relative to that flie.</div>
<div class=""><br class="">
</div>
<div class="">We can solve the first problem with
-fmodule-map-file=<libstdc++ module map>.
For the second half, I've been discussing with a
few people the idea of allowing a module map
file to specify a "module root" directory
relative to which its files are found, which
need not be the directory in which the map is
placed. (This also helps with another problem:
diagnostics when building or using a module
point to files relative to the module map file,
which can result in some rather contorted and
unnatural paths.)</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
<div><br class="">
</div>
</div>
</blockquote>
Sorry for the delay... :(<br>
<blockquote
cite="mid:46492423-93D9-4638-A80D-E3BA67D3E34E@apple.com"
type="cite">
<div>
<div>I guess the minimum viable solution would be something like
-fmodule-map-file-with-root=<module map
file>,/path/to/module/root. This has the advantage that
your module map file doesn’t tie itself to a particular
directory.</div>
</div>
</blockquote>
Yes that would work for me. This would be better than my original
proposal because the build system can expand the path to the
module's root, giving some extra flexibility.<br>
<blockquote
cite="mid:46492423-93D9-4638-A80D-E3BA67D3E34E@apple.com"
type="cite">
<div><br class="">
<blockquote type="cite" class="">
<div class="">
<div bgcolor="#FFFFFF" text="#000000" class="">
<blockquote
cite="mid:CAOfiQqnzUostvGO8=MTDJa+vDR5aSe4SODepBF5F4CzTuhXbig@mail.gmail.com"
type="cite" class="">
<div dir="ltr" class="">
<div class="gmail_extra">
<div class="gmail_quote"> </div>
</div>
</div>
</blockquote>
Thanks for the pointers. This makes sense. Would I be able
to to specify in the a framework's directory modulemaps
for external dependency. In my particular case, I'd like
to be able to express that this is the modulemaps for the
external dependency. <br class="">
<br class="">
I was thinking what if we could accommodate more than one
modulemap per file. Say:<br class="">
cat module.modulemap:<br class="">
modulemap Map1 {<br class="">
module M1{}<br class="">
...<br class="">
}<br class="">
modulemap Map2 {<br class="">
modulemap_root /usr/include // Will use the virtual file
system pretending the modulemap was found at the
modulemap_root<br class="">
module N1{}<br class="">
...<br class="">
}<br class="">
</div>
</div>
</blockquote>
<div><br class="">
</div>
<div>In this scheme, I would make your hypothetical root
declaration part of the module, not the module map file. i.e</div>
<div><br class="">
</div>
<div>module M1 { }</div>
<div>module N1 {</div>
<div> module_root “/usr/include”</div>
<div> …</div>
<div>}</div>
<div><br class="">
</div>
</div>
</blockquote>
<blockquote
cite="mid:46492423-93D9-4638-A80D-E3BA67D3E34E@apple.com"
type="cite">
<div>
<div>That might result in some duplication if you wanted to
describe a lot of modules in a single directory from an
external location, but it seems cleaner to me than a modulemap
{ } syntax.</div>
</div>
</blockquote>
Yes, this duplication is what I was trying to avoid. With the
modulemap {} syntax I wanted to argue for a way of specifying more
than one modulemap per file. The idea is to provide more than one
'view' of the headers describing the libraries and exposing its
interfaces. ( See below )<br>
<blockquote
cite="mid:46492423-93D9-4638-A80D-E3BA67D3E34E@apple.com"
type="cite">
<div><br class="">
<blockquote type="cite" class="">
<div class="">
<div bgcolor="#FFFFFF" text="#000000" class=""> <br
class="">
IMO this would allow the 'external dependencies' to be
organized in different configurations. For example, a
module per header of bunch of headers for module,
whichever decides the framework fits best. For our
use-cases that would be great. Maybe this could simplify
also the cross referencing modules and visibility also...
<br class="">
</div>
</div>
</blockquote>
<div><br class="">
</div>
<div>Not sure what you mean here. Would you mind expanding on
this a bit?</div>
</div>
</blockquote>
For our case, where we interpret C++, we have the mapping between
header files -> .so files. Whenever the user dlopen-s a shared
library (at runtime), we load a set of header files corresponding to
this library, such that the user can call library functions, for
example. We want to replace the headers and our own concept of
modulemaps with clang's ones. I.e instead of header files we will
use C++ modules and the mapping will happen by using modulemaps. I
call that 'runtime' modules.<br>
<br>
The library writer can decide to combine libA.so and libB.so into
libC.so, providing a module for it. Even worse (but rare) he can
decide to hide some stuff from libB.so (i.e things that he considers
private from libC standpoint) and libC should not provide a
module/header for them. With the current implementation of the
modulemaps would that be possible?<br>
<br>
In a way I though it is similar to private module maps... I hope it
is a bit clearer. I do realize that current implementation is a bit
rough and it will mature with time and I thought it is a good time
to throw our use-case in when talking about private module maps.<br>
<br>
Vassil<br>
<blockquote
cite="mid:46492423-93D9-4638-A80D-E3BA67D3E34E@apple.com"
type="cite">
<div><br class="">
<blockquote type="cite" class="">
<div class="">
<div bgcolor="#FFFFFF" text="#000000" class="">
<blockquote
cite="mid:CAOfiQqnzUostvGO8=MTDJa+vDR5aSe4SODepBF5F4CzTuhXbig@mail.gmail.com"
type="cite" class="">
<div dir="ltr" class="">
<div class="gmail_extra">
<div class="gmail_quote">
<div class=""><br class="">
</div>
<blockquote class="gmail_quote" style="margin:0 0
0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000" class=""><span
class=""><span class="HOEnZb"><font class=""
color="#888888"> Vassil<br class="">
</font></span></span>
<blockquote type="cite" class=""><span
class="">
<div dir="ltr" class="">
<div class="gmail_extra">
<div class="gmail_quote">
<div class=""><br class="">
</div>
<div class="">(in addition to the
other changes you suggest here).
Then only allow a module to be
extended if the extension is
listed via an 'extern module' in
the definition of the module.</div>
<div class=""><br class="">
</div>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div style="word-wrap:break-word"
class="">We propose to make the
following changes to Clang’s
module mechanism:<br class="">
<br class="">
- When looking up a module
through the search paths, in
addition to ‘module.modulemap’
also lookup for a standalone
‘module.private.modulemap’ file.
I will refer to this as the
"private extension" module map.<br
class="">
- When parsing a private
extension map allow extending a
module that was not defined
before, without providing the
full definition. To clarify, I
refer to a module definition as
this:<br class="">
<br class="">
module MyModule {<br class="">
<…><br class="">
}<br class="">
<br class="">
while an extension is this:<br
class="">
<br class="">
module MyModule.SomethingPrivate
{<br class="">
<…><br class="">
}<br class="">
<br class="">
An extension is a nested module
with any depth.<br class="">
We can reuse the “extern module”
syntax to indicate that we are
extending a module whose
definition is in a different
module map:
<div class=""><br class="">
</div>
<div class="">extern module
MyModule</div>
<div class="">module
MyModule.SomethingPrivate {<br
class="">
<…><br class="">
}<br class="">
</div>
<div class=""><br class="">
- After parsing the private
extension map, we are still
missing the module definition
so module lookup will continue
looking in the following
header search paths. If the
module we are looking for is
not found then Clang will a
emit a “module not found”
error.<br class="">
<br class="">
- It may seem backwards that
module search will find and
parse the private extension
ahead of the public one, but
it is actually advantageous
because this allows us to
continue searching only until
we find the module definition,
at which point we will stop
looking. If module search
worked the other way then,
after we had the module
definition, we would need to
always keep looking through
the rest of the search paths
in case there is a private
extension map that we need to
take into account, or treat
certain paths specially and
only look for private
extensions in those.<br
class="">
By finding the extension map
early on, we keep the current
semantics of doing the minimal
search necessary to find and
complete the module
definition, without treating
any particular search path
specially.<br class="">
<br class="">
- After Clang finds and parses
the public module map for
‘MyModule’, the module
definition will be complete.
Clang will keep track that
there is a private extension
map associated with the module
and it will pass the paths of
both the public module map and
the private extension one to
the module building
invocation. This will result
in one module file containing
both the public and private
APIs, similar to what we do
with frameworks.</div>
<div class=""><br class="">
</div>
<div class="">- A module
definition inside a private
extension will be disallowed.
The rationale is that
otherwise <span class="">it
will be a very common
mistake for users to write</span></div>
<div
style="margin:0px;min-height:14px"
class=""><br class="">
</div>
<div style="margin:0px" class=""><b
class="">module.modulemap:</b></div>
<div style="margin:0px" class="">module
Foo {</div>
<div style="margin:0px" class="">
<public headers></div>
<div style="margin:0px" class="">}</div>
<div
style="margin:0px;min-height:14px"
class=""><br class="">
</div>
<div style="margin:0px" class=""><b
class="">module.private.modulemap:</b></div>
<div style="margin:0px" class="">module
Foo {</div>
<div style="margin:0px" class="">
<private headers></div>
<div style="margin:0px" class="">}</div>
<div
style="margin:0px;min-height:14px"
class=""><br class="">
</div>
<div style="margin:0px" class="">and
then be left scratching their
heads wondering why things are
broken (things missing,
headers included textually,
etc.). Being more strict in
private extension maps will be
beneficial.</div>
<div style="margin:0px" class=""><br
class="">
</div>
<div style="margin:0px" class=""><br
class="">
</div>
<div style="margin:0px" class="">Let
me know what you think!</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</div>
<br class="">
<fieldset class=""></fieldset>
<br class="">
</span><span class="">
<pre class="">_______________________________________________
cfe-dev mailing list
<a moz-do-not-send="true" href="mailto:cfe-dev@cs.uiuc.edu" target="_blank" class="">cfe-dev@cs.uiuc.edu</a>
<a moz-do-not-send="true" href="http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev" target="_blank" class="">http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev</a>
</pre>
</span></blockquote>
<br class="">
</div>
</blockquote>
</div>
<br class="">
</div>
</div>
</blockquote>
<br class="">
<br class="">
</div>
_______________________________________________<br class="">
cfe-dev mailing list<br class="">
<a moz-do-not-send="true" href="mailto:cfe-dev@cs.uiuc.edu"
class="">cfe-dev@cs.uiuc.edu</a><br class="">
<a class="moz-txt-link-freetext" href="http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev">http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev</a><br
class="">
</div>
</blockquote>
</div>
<br class="">
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
--------------------------------------------
Q: Why is this email five sentences or less?
A: <a class="moz-txt-link-freetext" href="http://five.sentenc.es">http://five.sentenc.es</a>
</pre>
</body>
</html>