[cfe-dev] Slowness of clang build

[David Blaikie]

On Sat, Dec 29, 2012 at 11:27 AM, Edward Diener
<eldlistmailingz at tropicsoft.com> wrote:
> On 12/29/2012 01:59 PM, David Blaikie wrote:
>>
>> On Sat, Dec 29, 2012 at 10:50 AM, Edward Diener
>> <eldlistmailingz at tropicsoft.com> wrote:
>>>
>>> On 12/29/2012 12:02 PM, David Blaikie wrote:
>>>>
>>>>
>>>> On Sat, Dec 29, 2012 at 8:47 AM, Edward Diener
>>>> <eldlistmailingz at tropicsoft.com> wrote:
>>>>>
>>>>>
>>>>> I am periodically retrieving the latest sources from the llvm/clang
>>>>> repository, as specified in the Getting Started page.
>>>>>
>>>>> When I do a build of clang, through the 'make' command from the build
>>>>> directory, the build runs very slow. Evidently the default is a
>>>>> Debug+Assert
>>>>> build. Compiling a single source file averages 5-10 seconds and overall
>>>>> the
>>>>> build takes hours considering how many modules must be built. It is all
>>>>> successful in the end and when I am finished I have the latest clang,
>>>>> which
>>>>> is great.
>>>>>
>>>>> I have plenty of memory, plenty of disk space, a fairly fast CPU, and
>>>>> am
>>>>> doing the build on Fedora 17 with hardly any other applications
>>>>> running.
>>>>
>>>>
>>>>
>>>> Could you be more specific about the CPU, cores, and available memory?
>>>
>>>
>>>
>>> I have 8 GB of memory and 4 CPUs.
>>
>>
>> I'd expect a full/clean build to take maybe 10 minutes across 4 cores
>> (but if it's 4 cpus each with multiple cores, then less time/more
>> parallelism).
>>
>>>
>>>
>>>> (have you watched the utilization while building Clang? Is the build
>>>> topping out on RAM/using swap? using all your CPU cores?)
>>>>
>>>> The simplest thing to do is probably to pass "-j N" where N is the
>>>> number of cores you have available (by default the make build isn't
>>>> parallelized).
>>>
>>>
>>>
>>> Use "make -j 4" ?
>>
>>
>> Correct.
>
>
> I have tried the above with some improvement. Only a few seconds for each
> compilation now. Thanks !
>
>
>>
>>>
>>>
>>>>
>>>> Other ideas:
>>>> * what compiler are you using to perform the build?
>>>
>>>
>>>
>>> I just execute 'make' from my buid directory. I have no idea what
>>> compiler
>>> clang uses. Is there a way to find out ? I would expect it uses 'clang'
>>> itself.
>>
>>
>> That'd only be possible if you have an existing clang installation?
>> (otherwise there's the bootstrap problem: how could it build clang
>> with clang if it hasn't build clang yet?) Assuming you didn't do
>> anything special, it should be using your system compiler (which means
>> that's probably not the performance problem - since you'd only get
>> really heinous performance from a debug/asserts build of Clang being
>> used to build Clang, and you wouldn't have a debug+asserts built
>> compiler as your system compiler)
>
>
> I did install a 3.0-14 earlier binary version of clang just so I can run
> certain tests against both the earler version and the latest I am building.
> Whether it is using the earlier clang or gcc 4.7.2-2, which I also have
> installed, to build the latest version of clang I do not know.
>
>
>>
>>>
>>>
>>>> If it's a previous
>>>> Debug+Asserts build of Clang, then that's really going to slow things
>>>> down - don't use that, use Release+Asserts or Release-Asserts.
>>>
>>>
>>>
>>> How do I specify Release+Asserts in the 'make' command line ?
>>>
>>>
>>>> * you could try the cmake+ninja build (ninja will run maximally
>>>> parallel by default & exploits more parallelism than the make build)
>>>
>>>
>>>
>>> Again I am just following clang's own instructions in their getting
>>> started
>>> page. I have not tried cmake.
>>
>>
>> The getting started docs do briefly mention cmake.
>
>
> Yes, I see it there. But aside from mentioning CMake it does not tell how to
> actually use it AFAICS.

Yeah, doesn't go into much detail.

If you have ninja & cmake with ninja support, you'd run:

cmake -G Ninja ../src

and then simply run "ninja"

>
>
>>
>>>
>>>
>>>> * to do a release build of clang (which might speed things up a little
>>>> since you won't be dealing with debug info, etc) I think it's
>>>> "--disable-optimized" when configuring the make based build
>>>
>>>
>>>
>>> Is this a configure option ? Do I go back and rerun ./configure passing
>>> it
>>> '--disable-optimized' ?
>>
>>
>> That's the rough idea, yes. I'm not sure I've remembered the exact
>> spelling of the option off-hand, though.
>
>
> OK, thanks. I will check that out also to try to speed up my builds. I don't
> really need the debug version since I am just using clang rather than being
> a clang developer.

& of course I had this quite backwards, --disable-optimized is how you
get a Debug build not how you avoid it (derr - I blame the weekend).

As David pointed out, the release V debug tradeoff isn't strictly
obvious in terms of compile-time, but disabling assertions might get
you a little bit of a compile-time improvement. (--disable-asserts or
some similar spelling)