[cfe-dev] Slowness of clang build

[Edward Diener]

On 12/29/2012 12:08 PM, David Chisnall wrote:
> On 29 Dec 2012, at 16:47, Edward Diener wrote:
>
>> I am periodically retrieving the latest sources from the llvm/clang repository, as specified in the Getting Started page.
>>
>> When I do a build of clang, through the 'make' command from the build directory, the build runs very slow.
>
> Are you doing a parallel build?

I am just executing 'make' as in the clang Getting Started instructions.

>  On an i7 laptop, I can do a clean build in 5-10 minutes, with -j4, on a faster desktop it's about 3 minutes.  I usually use ninja (just add -G Ninja to your cmake command), rather than make, although there isn't much difference in performance between the two when doing a clean build.  For incremental builds, ninja is so much faster that make isn't even in the same league.

That is all nice, but a full build using 'make' after updating to the 
latest llvm and clang literally takes hours to complete. Im fact I can 
start it at noon, go away for most of the afternoon, and hope i 
completes by the time I get back.

So obviously something is ver different between your 5-10 minutes and my 
hours and hours. Which is why I originally posted.

>
>> Evidently the default is a Debug+Assert build. Compiling a single source file averages 5-10 seconds and overall the build takes hours considering how many modules must be built. It is all successful in the end and when I am finished I have the latest clang, which is great.
>
> The Debug+Asserts build is usually faster to compile, because it turns off optimisations, but slower to link, because the debug info is very large.  While it can take 5-10 seconds to compile each file, that only matters if you're doing one file at a time.

It is compiling one file at a time. Now it appears my originally 
estimate was wrong and it really takes on average 10-20 seconds to 
compile a single file.

>
>> I have plenty of memory, plenty of disk space, a fairly fast CPU, and am doing the build on Fedora 17 with hardly any other applications running.
>
> ninja -j8, if you've got enough RAM.  If you're not on an SSD, then more processes than you have cores is generally a good idea, because it lets some run while the other are waiting for the disk, but be careful not to have to many more than you have GBs of RAM.  A few of the files take about 1GB to compile, and for a debug build with ld-bdf you'll go over 2GB for the final linking step (enabling shared libraries speeds this up, but results in about a 5% slower executable at the end on x86-64).

What is 'ninja -j8' ? Is that a command line option to 'make' ?

>
>> Why is compiling clang so slow ? Is there any way to speed up the build ? If I do a release build ( how ? )is it significantly quicker ?
>
> I've not found the compilation time to be much different for release and debug builds (I've not measured it, but I do both fairly regularly and neither feels much slower than the other).  The big difference is the time spent linking.  If the linker runs out of physical memory and starts to swap then linking time can easily hit 5+ minutes.  I've seen it go over 2GB with a statically-linked debug+asserts build of clang.
>
> If you're doing a bootstrap build, make sure that you do a release build first, because the builds with asserts run a lot slower (factor of 10 or so), and trying to build LLVM/Clang with a build with asserts enabled is quite painful.

I am not sure what you mean by 'bootstrap build'. I did an original 
"configure" when I first got llvm and clang, then a 'make' Afterward, 
every time I get the latest I simpy do 'make'.

I am following the instructions on the clang 'Getting Started' page at 
http://clang.llvm.org/get_started.html.