<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 10/07/17 11:44, NAKAMURA Takumi via
cfe-dev wrote:<br>
</div>
<blockquote
cite="mid:CADiQthM4sgg5kYhAgnuC+AUzXTL+BFAJCdz-3ut6ehhTr1d+aQ@mail.gmail.com"
type="cite">
<div dir="ltr">I was testing efficiency with LLVM_ENABLE_MODULES
to build clang/llvm tree.<br>
</div>
</blockquote>
Thanks for sharing this summary.<br>
<blockquote
cite="mid:CADiQthM4sgg5kYhAgnuC+AUzXTL+BFAJCdz-3ut6ehhTr1d+aQ@mail.gmail.com"
type="cite">
<div dir="ltr"><br>
* Summary<br>
<br>
** Efficiency of Modules increases as the degree of parallelism
decreases.<br>
For example with -j8, Modules is 67% of elapsed time than
no-modules.<br>
</div>
</blockquote>
Do you have some numbers about the performance if build with
libstdc++?<br>
<blockquote
cite="mid:CADiQthM4sgg5kYhAgnuC+AUzXTL+BFAJCdz-3ut6ehhTr1d+aQ@mail.gmail.com"
type="cite">
<div dir="ltr"><br>
** With higher parallelism, Modules is inefficient.<br>
For example with -j72, Modules is just 23 seconds faster than
no-modules.<br>
Then, processor usage of Modules is about 55%.<br>
(Assuming (user+sys)/72 is ideal)<br>
</div>
</blockquote>
I assume this is the penalty of using implicit modules. Building
modules locks which might lead to quadratic compile times (we had an
issue describing the problem somewhere in bugzilla).<br>
<br>
I've seen in the past using make that we build modules but we fail
to pick them up. I tried to fix the issue but didn't test
thoroughly. If you compile with -H you should probably see which
files are still textually included.<br>
<blockquote
cite="mid:CADiQthM4sgg5kYhAgnuC+AUzXTL+BFAJCdz-3ut6ehhTr1d+aQ@mail.gmail.com"
type="cite">
<div dir="ltr"><br>
** If each module(s) is not rebuilt, rebuilding is sufficiently
efficient.<br>
For example with -j72 to remove just *.o, processor usage is
84%.<br>
</div>
</blockquote>
Do you mean we are 84% faster?<br>
<blockquote
cite="mid:CADiQthM4sgg5kYhAgnuC+AUzXTL+BFAJCdz-3ut6ehhTr1d+aQ@mail.gmail.com"
type="cite">
<div dir="ltr"><br>
* Random notes for improvements<br>
- Get rid of -DCLANG_ENABLE_(ARCMT|REWRITER|SATATIC_ANALYZER),
=> clang-config.h<br>
</div>
</blockquote>
+1<br>
<br>
Thanks a lot for working on this!<br>
<br>
--Vassil<br>
<blockquote
cite="mid:CADiQthM4sgg5kYhAgnuC+AUzXTL+BFAJCdz-3ut6ehhTr1d+aQ@mail.gmail.com"
type="cite">
<div dir="ltr">- Propagate definitions in unittests to whole the
tree.<br>
Modules is sensitive of -D in command line.<br>
- Recognize CMake and Ninja to rebuild module cache.<br>
IIRC, there was the discussion about Fortran modules.<br>
- Parse and issue "module rebuilder" from modules.cache in
advance of building the tree.<br>
Anyways, Ninja doesn't do anything while each compilation unit
is waiting for module lock.<br>
<br>
I expect developers and users would be happier with Modules.<br>
Thanks,<br>
Takumi<br>
<br>
<br>
Below, building clang with "/usr/bin/time ninja -jN clang"<br>
Host compiler is clang with libc++ and lld, -Asserts<br>
The host is Xeon 36 cores, 72 logical processors.<br>
<br>
Columns are;<br>
N,user,system,elapsed,Ideal:(u+s)/N,(Ideal/elapsed)<br>
<br>
N, Number of jobs -jN<br>
user, user time (sec)<br>
system, system time (sec)<br>
elapsed, elapsed time (sec)<br>
Ideal:(u+s)/N, Ideal elapsed time w/o idle<br>
(Ideal/elapsed): Efficiency -- elapsed processor usage<br>
<br>
*ENABLE_MODULES=OFF<br>
<div>96,11959.10,413.57,184.52,128.882,69.8%</div>
<div>80,12000.47,411.62,184.67,155.151,84.0%</div>
<div>72,11952.46,407.66,184.98,171.668,92.8%</div>
<div>64,10970.09,375.14,189.08,177.269,93.8%</div>
<div>48,8716.43,310.69,198.75,188.065,94.6%</div>
<div>41,7651.71,274.48,202.32,193.322,95.6%</div>
<div>40,7496.75,270.23,205.38,194.175,94.5%</div>
<div>39,7377.94,266.18,206.45,196.003,94.9%</div>
<div>38,7227.33,259.33,206.22,197.017,95.5%</div>
<div>37,7068.51,254.84,207.64,197.928,95.3%</div>
<div>36,6914.62,250.31,208.13,199.026,95.6%</div>
<div>35,6815.70,247.86,210.31,201.816,96.0%</div>
<div>34,6728.49,244.93,214.57,205.101,95.6%</div>
<div>33,6608.13,239.37,216.54,207.500,95.8%</div>
<div>32,6585.52,235.59,221.93,213.160,96.0%</div>
<div>28,6502.79,231.50,248.85,240.510,96.6%</div>
<div>24,6451.13,230.06,289.14,278.383,96.3%</div>
<div>20,6386.95,225.27,342.18,330.611,96.6%</div>
<div>16,6183.61,222.80,411.88,400.401,97.2%</div>
<div>8,5558.17,205.07,728.88,720.405,98.8%</div>
<div><br>
</div>
*ENABLE_MODULES=ON<br>
<div>96,6396.47,330.73,169.28,70.075,41.4%</div>
<div>88,6249.93,329.12,160.22,74.762,46.7%</div>
<div>80,6259.91,322.27,163.59,82.277,50.3%</div>
<div>72,6092.58,315.70,161.55,89.004,55.1%</div>
<div>64,5727.81,297.64,168.78,94.148,55.8%</div>
<div>56,5421.81,283.95,168.71,101.889,60.4%</div>
<div>48,4896.81,260.07,171.05,107.435,62.8%</div>
<div>40,4375.71,235.90,177.60,115.290,64.9%</div>
<div>32,3959.32,214.67,188.10,130.437,69.3%</div>
<div>24,3892.54,206.40,230.70,170.789,74.0%</div>
<div>16,3690.52,201.41,294.12,243.246,82.7%</div>
<div>8,3298.95,185.68,488.59,435.579,89.2%</div>
<div><br>
</div>
*ENABLE_MODULES_ON building to remove just *.o<br>
<div>96,6898.51,347.36,120.62,75.478,62.6%</div>
<div>88,6908.61,345.52,121.14,82.433,68.0%</div>
<div>80,6823.66,338.48,118.72,89.527,75.4%</div>
<div>72,6819.25,339.82,118.30,99.432,84.1%</div>
<div>64,6311.53,310.03,120.06,103.462,86.2%</div>
<div>56,5729.12,287.76,123.73,107.444,86.8%</div>
<div>48,5108.16,264.21,127.25,111.924,88.0%</div>
<div>40,4449.20,231.17,131.42,117.009,89.0%</div>
<div>32,3933.69,205.94,142.74,129.363,90.6%</div>
<div>24,3844.17,201.83,181.55,168.583,92.9%</div>
<div>16,3669.73,193.59,251.15,241.458,96.1%</div>
<div>8,3225.63,178.68,434.85,425.539,97.9%</div>
<div><br>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
cfe-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:cfe-dev@lists.llvm.org">cfe-dev@lists.llvm.org</a>
<a class="moz-txt-link-freetext" href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev">http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</a>
</pre>
</blockquote>
<p><br>
</p>
</body>
</html>