[llvm-dev] [cfe-dev] llvm.org pre-built clang significantly slower than apple/xcode clang

Thu Nov 29 14:57:10 PST 2018

(Sorry for the late response, I was out of town for the holiday)

There is a little bit of internal magic for how AppleClang is built, but much of the essential configuration is in the public tree and documented here:
https://llvm.org/docs/AdvancedBuilds.html#apple-clang-builds-a-more-complex-bootstrap <https://llvm.org/docs/AdvancedBuilds.html#apple-clang-builds-a-more-complex-bootstrap>

Apple also uses PGO and order files for the AppleClang builds, and there is PGO infrastructure publicly documented here:
https://llvm.org/docs/AdvancedBuilds.html#multi-stage-pgo <https://llvm.org/docs/AdvancedBuilds.html#multi-stage-pgo>

That PGO infrastructure can also generate linker order files on Darwin. Unfortunately Apple's PGO training data is internal, so you'll need to come up with your own test cases for that. I would love to see more comprehensive training data in the open source tree, right now the only test we have is under clang/utils/perf-training, and it is a simple c++ hello world.

-Chris

> On Nov 21, 2018, at 7:43 AM, Jean-Daniel via cfe-dev <cfe-dev at lists.llvm.org> wrote:
> 
> Thank you for the correction. So they actually did change that at some point.
> 
>> Le 20 nov. 2018 à 23:54, Jack Howarth <howarth.mailing.lists at gmail.com <mailto:howarth.mailing.lists at gmail.com>> a écrit :
>> 
>> Jean-Daniel,
>>      The latest available release posted on their open source web site argues that that they are disabling assertions in clang.
>> 
>> https://opensource.apple.com/source/clang/clang-800.0.42.1/Makefile.auto.html <https://opensource.apple.com/source/clang/clang-800.0.42.1/Makefile.auto.html>
>> 
>>      Jack
>> 
>> On Tue, Nov 20, 2018 at 5:21 PM Jean-Daniel <mailing at xenonium.com <mailto:mailing at xenonium.com>> wrote:
>> I don’t think Apple disable assertion on the release build. I remember having clang and llvm crash because of assertion failure regularly at some point in the past.
>> Nowadays, it is far more unusual to get a clang crash, so I can’t tell, but I doubt they change the configuration.
>> 
>>> Le 20 nov. 2018 à 16:32, Jack Howarth via cfe-dev <cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>> a écrit :
>>> 
>>> The obvious question is whether the llvm.org <http://llvm.org/> builds are using -DLLVM_ENABLE_ASSERTIONS:OFF -DCMAKE_BUILD_TYPE:STRING=Release -DLLVM_LINK_LLVM_DYLIB:BOOL=ON which would improve the load time of the compiler by combining all of the llvm libs into a single dylib and would eliminate the speed decrease from using the default use of assertions in the built compiler.
>>>        Jack
>>> 
>>> On Tue, Nov 20, 2018 at 6:56 AM Tobias Hieta via cfe-dev <cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>> wrote:
>>> Hello LLVM/Clang developers,
>>> 
>>> We recently switched to use the same clang version on all our platforms. This included switching from apple-clang from xcode to a pre-built binary we downloaded from llvm.org <http://llvm.org/>. We noticed that this actually came with a pretty big performance regression in compile times.
>>> 
>>> If we do the simplest test program like this:
>>> 
>>> #include <string>
>>> #include <iostream>
>>> int main()
>>> {
>>>     std::cout << "Hello world" << std::endl;
>>> }
>>> 
>>> and compile that with Xcode Clang (Xcode 10.1 apple-clang clang-1000.11.45.5):
>>> clang++ test.cpp -o test  0.31s user 0.06s system 97% cpu 0.380 total
>>> 
>>> with clang 7 binaries found on llvm.org <http://llvm.org/> 7.0.0:
>>> ~/Downloads/clang+llvm-7.0.0-x86_64-apple-darwin/bin/clang++ -o test test.cpp  0.53s user 0.11s system 62% cpu 1.032 total
>>> 
>>> If we now run that on our whole project:
>>> with xcode clang:
>>> 368.17s user 32.00s system 663% cpu 1:00.30 total
>>> 
>>> with clang 7:
>>> 423.31s user 31.65s system 662% cpu 1:08.69 total
>>> 
>>> That's a pretty hefty difference. Any ideas what can account for this discrepancy? Does apple-clang contain any special patches or build flags that differ a lot from the binaries on llvm.org <http://llvm.org/>?
>>> 
>>> I know about PGO - and I guess the best we could do is to get profile data out of compiling my whole tree and use that when building clang - but this process seems not very well documented and unsure if this would even help.
>>> 
>>> Thankful for any ideas or feedback.
>>> Tobias
>>> _______________________________________________
>>> cfe-dev mailing list
>>> cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev>
>>> _______________________________________________
>>> cfe-dev mailing list
>>> cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev>
>> 
> 
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20181129/7f626e18/attachment-0001.html>