[cfe-dev] Modules TS Work

Tue Nov 6 13:16:38 PST 2018

On Mon, Oct 15, 2018 at 10:12 AM Whisperity <whisperity at gmail.com> wrote:

> In my previous post I have talked about the Modules >>TS<< system.
> I've also mentioned "-fprebuilt-modules-path" and also earlier on I
> think I've also mentioned "TS". I think the whole modulemap thing will
> always be a legacy or a compiler-specific extension, simply because
> every effort towards improving Modules TS and eventually getting it
> into the real standard does not consider this whole "module map"
> thing.
>

True, though the standard doesn't talk about a lot of things (like header
search paths) that implementations take for granted these days.

Specifically I think module maps will be part of things for standard legacy
modules. Currently my understanding is that legacy modules would be
discoverable only by their use as the target of a legacy import - so in
theory a compiler/build system could discover and classify headers as such
that way, but it's probably not super practical that way (in part because
then there'd be lots of duplication for all the parsing of any header that
wasn't a target of a legacy module import - but only used indirectly by
legacy modules).

So I think there might still be some room/motivation for modulemap style
support.

> With this in mind, maybe the best course of action is to improve upon
> the "Modules TS"(-ish) implementation and get more results out of it
> and hit more roadblock to get data on which the eventual "real deal"
> is improved with.
>
> Unfortunately, my CMake patch was rejected (or put on hold?) today
> because the whole Modules TS thing still being rather new, never
> standardised, compiler developers arguing about the internals (just
> see this current thread ;) )
>

Yeah, perhaps the best that can be done is to come up with prototypes to
help inform both the standards effort and the cmake direction when it comes
to that.

That said, maybe the fact that modulemaps are already a validated
implementation in clang might be a gateway for adding cmake support for
that, a shipping/in-use feature. Certainly Clang/LLVM could benefit from
being built with that sort of mode (& many other C++ code bases could opt
into this).

- Dave

>
> David Blaikie <dblaikie at gmail.com> ezt írta (időpont: 2018. okt. 15., H,
> 17:30):
> >
> >
> >
> > On Sat, Oct 13, 2018 at 4:41 AM Whisperity <whisperity at gmail.com> wrote:
> >>
> >> As I have now delved a bit into the Modules TS implementation from a
> >> user standpoint (due to having implemented support for it in CMake,
> >> see the patch here:
> >> https://gitlab.kitware.com/cmake/cmake/merge_requests/2482 )
> >> (Some of these concerns are already voiced by Boris in
> >> https://build2.org/article/cxx-modules-misconceptions.xhtml#build )
> >>
> >>  1. The first big issue is that while the "C++98 Modules" is somewhat
> >> well-documented, the "Modules TS" has absolutely no workable
> >> documentation available at all... I had to resort to looking up
> >> Clang's option td files to figure out what this thing is supposed to
> >> do.
> >
> >
> > Fair cop - work in progress, etc. Though it's far enough along it could
> probably use some documentation even if it said clearly "this is in flux,
> may be out of date, etc".
> >
> >>
> >>  2. Currently, Clang is wonky on how modules can be found. You need to
> >> explicitly specify the modules' location which (the prebuilt, if you
> >> want to allow caching of PCMs and no re-read of CPPM files all the
> >> time...) has to be a single folder.
> >
> >
> > What command line are you referring to here? -fmodule-cache-path?
> >
> > That's more used around the "implicit modules" support, which isn't
> likely to be supported/useful for ModulesTS style modules (becaues you'll
> want, maybe need, to build object files from the module files - so your
> build system would need to be aware of building module files).
> >
> >>
> >> The PCM must be available prior to
> >> compilation so the main file is semantically correct, but *it also has
> >> to be linked* against the resulting binary, otherwise implementations
> >> will provide linker errors. (If I link .o instead of .pcm, it will
> >> still work, and .o files are smaller than PCMs, but compilation will
> >> have to semantically analyse the CPPM file providing no speedup on
> >> module contents.) There is no direct sanity check that I'm linking the
> >> same PCM that I used for compiling, and it looks like the PCM must be
> >> linked *explicitly* (so the '-fprebuilt-modules-path' doesn't work
> >> here.)
> >
> >
> > yep, the build model currently is entirely explicit, something like this
> >
> > COMMON_FLAGS="-cc1 -xc++ -emit-module -fmodules -w"
> > CLANG=clang
> > $CLANG $COMMON_FLAGS -fmodule-name=foo foo.cppmap -o foo.pcm
> -fmodules-codegen
> > $CLANG $COMMON_FLAGS -fmodule-name=bar bar.cppmap -o bar.pcm
> -fmodules-codegen -fmodule-file=foo.pcm
> > $CLANG -c foo.pcm
> > $CLANG -c bar.pcm
> > $CLANG -cc1 -emit-obj -fmodules -fmodule-file=foo.pcm
> -fmodule-file=bar.pcm use.cpp
> > $CLANG foo.o bar.o use.o
> >
> > Basically - build the .pcms, use those pcms to build object files for
> the pcms, and use the pcms to help build the object file for the user, then
> link all the objects together as usual.
> >
> >> 3. This might be very CMake specific or I only found this issue
> >> because working on CMake recently, but it is a real pain the behind to
> >> figure out on which modules a translation unit depends. Unlike '-M'
> >> which nicely spits out imports, the only option I saw so far is to
> >> grep for the import statements and then try to map these to the module
> >> files under the prebuild folder. (This is one of the issues I just
> >> didn't succeed (perhaps yet?) in CMake because CMake is also
> >> *somewhat* wonky, at dependency check time it does not know about
> >> other targets and configuration, so I just can't implictly and
> >> automatically say "Okay now please depend on
> >> ./whatever/path/mymodule.pcm or it's corresponding cppm input...)
> >> I think this eventually needs to be considered because -M respects
> >> compiler options when resolving the headers, so I think it's logical
> >> that -M (or "-fmodules-list-dependencies" or whatever...) could also
> >> respect where the prebuild dir is and spit out the PCM paths (or even
> >> better the original CPPMs' -- I think PCM contains a backwards link to
> >> where it was built from?) the TU depends on.
> >
> >
> > Yep, build integration with modules is still very much
> uncertain/unknown. Experiments/ideas/prototypes are all being played with -
> but more experiments/design suggestions are welcome.
> >
> > GCC is prototyping a "callback" system of sorts - when the compiler
> needs a BMI (binary module interface - Clang's are the .pcm files) it would
> callback into the build system to ask for the file path of the BMI (so the
> build system could go off and build it, then inform the compiler where it
> is).
> >
> > Clang requires it all explicit at the moment, as I showed above - the
> build system could either have its own import/inclusion discovery or maybe
> -M could be expanded to support imports, etc.
> >
> > - Dave
> >
> >>
> >>
> >> Regards,
> >> Whisperity.
> >> David Blaikie via cfe-dev <cfe-dev at lists.llvm.org> ezt írta (időpont:
> >> 2018. okt. 4., Cs, 20:02):
> >> >
> >> > Yeah, Richard seems to be working on modules TS features when he has
> time amongst other priorities.
> >> >
> >> > Any particular areas you've found gaps you've been interested in
> filling?
> >> >
> >> > On Sat, Sep 29, 2018 at 8:15 AM Matt Asplund (mwasplund) via cfe-dev <
> cfe-dev at lists.llvm.org> wrote:
> >> >>
> >> >>
> >> >> Hello,
> >> >>
> >> >> I am curious if there is any active work being done on the Modules
> TS implementation. I would like to help fill in the gaps that I am finding
> while playing around with it and wanted to make sure I wasn’t duplicating
> work.
> >> >>
> >> >> -Matt
> >> >> _______________________________________________
> >> >> cfe-dev mailing list
> >> >> cfe-dev at lists.llvm.org
> >> >> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
> >> >
> >> > _______________________________________________
> >> > cfe-dev mailing list
> >> > cfe-dev at lists.llvm.org
> >> > http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20181106/78ea2a2c/attachment.html>