[llvm-dev] [cfe-dev] FYI: ENABLE_MODULES would make building faster
Sean Silva via llvm-dev
llvm-dev at lists.llvm.org
Tue Jul 11 19:10:07 PDT 2017
On Tue, Jul 11, 2017 at 2:21 PM, David Blaikie via cfe-dev <
cfe-dev at lists.llvm.org> wrote:
> On Mon, Jul 10, 2017 at 2:44 AM NAKAMURA Takumi via cfe-dev <
> cfe-dev at lists.llvm.org> wrote:
>> I was testing efficiency with LLVM_ENABLE_MODULES to build clang/llvm
> Awesome - thanks for trying it out & gathering all this data!
>> * Summary
>> ** Efficiency of Modules increases as the degree of parallelism decreases.
>> For example with -j8, Modules is 67% of elapsed time than no-modules.
>> ** With higher parallelism, Modules is inefficient.
>> For example with -j72, Modules is just 23 seconds faster than no-modules.
>> Then, processor usage of Modules is about 55%.
>> (Assuming (user+sys)/72 is ideal)
> As Vasil mentioned, probably implicit modules.
> I have some hope/aspirations of implementing explicit modules* support in
> cmake & so it'll be interesting to compare how much more parallelism can be
> achieved by that. If anyone else is interesting in doing/helping with this
> work, I'd love any help - I've never touched cmake... so it'll be an
> adventure. (I assume it'll need changes to cmake itself, but I could be
I vaugely looked at this at one point in the past, though I got side
tracked before I could try it out.
Basically, what it seemed like could be done was something like:
1. use a PRE_BUILD add_custom_command for a given add_library command to
build the pcm (
2. use an INTERFACE target_compile_definitions to add the command line flag
that dependent code needs to add to the clang invocation (
3. an IMPORTED target library could be used to model external system
dependencies, and you would have one such IMPORTED target for each
different libc or C++ standard library
Of course, to have it properly integrated into CMake, ideally add_library
would take a list of headers and do 1. and 2. by itself. CMake would need
to have built-in knowledge of 3. also.
-- Sean Silva
> * Explicit modules are used at Google & implemented in Clang (though only
> accessible via cc1 at the moment) - where an explicit clang invocation must
> be made by the build system to build a .pcm file, and then explicit
> arguments given to a clang invocation of a file using those modules, etc.
>> ** If each module(s) is not rebuilt, rebuilding is sufficiently efficient.
>> For example with -j72 to remove just *.o, processor usage is 84%.
>> * Random notes for improvements
>> - Get rid of -DCLANG_ENABLE_(ARCMT|REWRITER|SATATIC_ANALYZER), =>
>> - Propagate definitions in unittests to whole the tree.
>> Modules is sensitive of -D in command line.
>> - Recognize CMake and Ninja to rebuild module cache.
>> IIRC, there was the discussion about Fortran modules.
>> - Parse and issue "module rebuilder" from modules.cache in advance of
>> building the tree.
>> Anyways, Ninja doesn't do anything while each compilation unit is
>> waiting for module lock.
>> I expect developers and users would be happier with Modules.
>> Below, building clang with "/usr/bin/time ninja -jN clang"
>> Host compiler is clang with libc++ and lld, -Asserts
>> The host is Xeon 36 cores, 72 logical processors.
>> Columns are;
>> N, Number of jobs -jN
>> user, user time (sec)
>> system, system time (sec)
>> elapsed, elapsed time (sec)
>> Ideal:(u+s)/N, Ideal elapsed time w/o idle
>> (Ideal/elapsed): Efficiency -- elapsed processor usage
>> *ENABLE_MODULES_ON building to remove just *.o
>> cfe-dev mailing list
>> cfe-dev at lists.llvm.org
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the llvm-dev