[llvm-dev] [cfe-dev] FYI: ENABLE_MODULES would make building faster
David Blaikie via llvm-dev
llvm-dev at lists.llvm.org
Tue Jul 11 14:21:07 PDT 2017
On Mon, Jul 10, 2017 at 2:44 AM NAKAMURA Takumi via cfe-dev <
cfe-dev at lists.llvm.org> wrote:
> I was testing efficiency with LLVM_ENABLE_MODULES to build clang/llvm tree.
Awesome - thanks for trying it out & gathering all this data!
> * Summary
> ** Efficiency of Modules increases as the degree of parallelism decreases.
> For example with -j8, Modules is 67% of elapsed time than no-modules.
> ** With higher parallelism, Modules is inefficient.
> For example with -j72, Modules is just 23 seconds faster than no-modules.
> Then, processor usage of Modules is about 55%.
> (Assuming (user+sys)/72 is ideal)
As Vasil mentioned, probably implicit modules.
I have some hope/aspirations of implementing explicit modules* support in
cmake & so it'll be interesting to compare how much more parallelism can be
achieved by that. If anyone else is interesting in doing/helping with this
work, I'd love any help - I've never touched cmake... so it'll be an
adventure. (I assume it'll need changes to cmake itself, but I could be
* Explicit modules are used at Google & implemented in Clang (though only
accessible via cc1 at the moment) - where an explicit clang invocation must
be made by the build system to build a .pcm file, and then explicit
arguments given to a clang invocation of a file using those modules, etc.
> ** If each module(s) is not rebuilt, rebuilding is sufficiently efficient.
> For example with -j72 to remove just *.o, processor usage is 84%.
> * Random notes for improvements
> - Get rid of -DCLANG_ENABLE_(ARCMT|REWRITER|SATATIC_ANALYZER), =>
> - Propagate definitions in unittests to whole the tree.
> Modules is sensitive of -D in command line.
> - Recognize CMake and Ninja to rebuild module cache.
> IIRC, there was the discussion about Fortran modules.
> - Parse and issue "module rebuilder" from modules.cache in advance of
> building the tree.
> Anyways, Ninja doesn't do anything while each compilation unit is
> waiting for module lock.
> I expect developers and users would be happier with Modules.
> Below, building clang with "/usr/bin/time ninja -jN clang"
> Host compiler is clang with libc++ and lld, -Asserts
> The host is Xeon 36 cores, 72 logical processors.
> Columns are;
> N, Number of jobs -jN
> user, user time (sec)
> system, system time (sec)
> elapsed, elapsed time (sec)
> Ideal:(u+s)/N, Ideal elapsed time w/o idle
> (Ideal/elapsed): Efficiency -- elapsed processor usage
> *ENABLE_MODULES_ON building to remove just *.o
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the llvm-dev