[cfe-dev] speed up clang build after modifying checker code of static analyzer
Lou Wynn via cfe-dev
cfe-dev at lists.llvm.org
Fri Sep 21 19:01:16 PDT 2018
After doing lots of experiments, ranging from installing clang-6 from
its official build (http://apt.llvm.org/xenial) to checking Cmake
failure logs and tweaking its settings, I've finally happened to find
out what was wrong.
I used
-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang
when I ran CMake because I saw clang and clang++ are symlinks to
bin/clang-x. This caused the Cmake failure, and it was gone when I
happened to use:
-DCMAKE_CXX_COMPILER=clang++-x
My guess is that the clang code must check which original command was
used to invoke it. When it's invoked through the clang symlink, it acts
like the C compiler; when through the clang++ symlink, like the C++
compiler.
Thank all of you who helped. Now with clang++-6.0 and lld-6.0, I can
build clang in 9s after changing a checker. Yeh! The entire cmake
command line looks like this:
cmake -G "Unix Makefiles" -DLLVM_OPTIMIZED_TABLEGEN=true
-DCMAKE_BUILD_TYPE=Debug -DBUILD_SHARED_LIBS=ON ../llvm/
-DLLVM_USE_LINKER=lld-6.0 -DCMAKE_C_COMPILER=clang-6.0
-DCMAKE_CXX_COMPILER=clang++-6.0
Thanks,
Lou
On 09/19/2018 02:58 PM, Kristóf Umann wrote:
> Uhh. I never encountered that error, so I can't offer anything that
> google can not :(
>
> All the other tips however others and I gave should speed up
> compilation and linking, so gcc is fine if you are having a lot of
> trouble building with clang, especially if linking is the most ime
> consuming part. Though it would be beneficial to get to the bottom of
> this if you have the time.
>
> One more thing, building shared libraries will make linking a whole
> lot faster (the resulting binary will be slower, but that won't hurt
> much in a debug build anyway) -DBUILD_SHARED_LIBS=ON.
>
> On 19 Sep 2018 23:44, "Lou Wynn" <lewisurn at gmail.com
> <mailto:lewisurn at gmail.com>> wrote:
>
> ~/work/git-repos/llvm/release-git$ which clang
> /usr/bin/clang
>
> ~/work/git-repos/llvm/release-git$ clang -###
> clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final)
> Target: x86_64-pc-linux-gnu
> Thread model: posix
> InstalledDir: /usr/bin
>
>
> I also tried with a self-built clang, and got the same error.
>
>
> Thanks,
> Lou
>
>
> On 09/19/2018 02:41 PM, Kristóf Umann wrote:
>> Are you using a prebuild binary? Are you sure you're using the
>> clang you'd like to compile with? (`which clang`?)
>>
>> On 19 Sep 2018 23:38, "Lou Wynn" <lewisurn at gmail.com
>> <mailto:lewisurn at gmail.com>> wrote:
>>
>> When I use clang to compile (gcc is fine), I got cmake error:
>>
>> -- Performing Test HAVE_CXX_ATOMICS_WITHOUT_LIB
>> -- Performing Test HAVE_CXX_ATOMICS_WITHOUT_LIB - Failed
>> -- Looking for __atomic_fetch_add_4 in atomic
>> -- Looking for __atomic_fetch_add_4 in atomic - found
>> -- Performing Test HAVE_CXX_ATOMICS_WITH_LIB
>> -- Performing Test HAVE_CXX_ATOMICS_WITH_LIB - Failed
>> CMake Error at cmake/modules/CheckAtomic.cmake:50 (message):
>> Host compiler must support std::atomic!
>> Call Stack (most recent call first):
>> cmake/config-ix.cmake:323 (include)
>> CMakeLists.txt:577 (include)
>>
>>
>> How can I fix it? I checkout source code from the git mirror.
>> I'm running ubuntu 16.04.1 LTS.
>>
>>
>> Thanks,
>> Lou
>>
>>
>> On 09/19/2018 12:02 AM, Kristóf Umann wrote:
>>> Hi!
>>>
>>> A couple tips I can give:
>>> * Compile with clang. Make sure you use a release build
>>> for this purpose, optimized with LTO. I usually just
>>> download a prebuild binary and go with that. Clang tends to
>>> be about 2-3 times faster than gcc.
>>> * For linking, use LLD, a linker part of the LLVM family.
>>> Especially with -j8, it will make a significant difference.
>>> Your next best option is gnu gold.
>>>
>>> Great to hear about a new checker! I'd encourage you to
>>> upload your work in progress early on to phabricator, even
>>> if it only contains an empty callback, because an entire
>>> working solution (if lengthy enough) can drastically
>>> increase the time it takes to review it.
>>>
>>> I know because I made that mistake :).
>>>
>>> Cheers,
>>> Kristóf Umann
>>>
>>>
>>> On 19 Sep 2018 04:06, "Lou Wynn via cfe-dev"
>>> <cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>> wrote:
>>>
>>> I do need debug information because tracing code is a
>>> great way for me to get familiarized with clang and the
>>> static analyzer.
>>>
>>> I've managed to run "make -j8 clang" at the top of the
>>> build directory. However, the final linking process
>>> takes too long now. Is there a way to build a thin
>>> version that only runs the static analyzer? I think that
>>> I can at least cut the code gen part out of the build
>>> process but have no idea how. The only command I run is
>>> "clang -cc1 -analyze -analyzer-checker=xxx" It'd be
>>> great if I could reduce the final object file to few
>>> hundred megabytes.
>>>
>>> [ 96%] Built target clangStaticAnalyzerCore
>>>
>>>
>>> Scanning dependencies of target clangStaticAnalyzerCheckers
>>> [ 96%] Building CXX object
>>> tools/clang/lib/StaticAnalyzer/Checkers/CMakeFiles/clangStaticAnalyzerCheckers.dir/IntegerOverflowChecker.cpp.o
>>> [ 96%] Linking CXX static library
>>> ../../../../../lib/libclangStaticAnalyzerCheckers.a
>>> [100%] Built target clangStaticAnalyzerCheckers
>>> [100%] Linking CXX executable ../../../../bin/clang <--
>>> this take most time now
>>> [100%] Built target clang
>>>
>>> real 2m22.707s
>>> user 2m10.812s
>>> sys 0m14.936s
>>>
>>> Thanks,
>>> Lou
>>>
>>> On 09/18/2018 06:25 PM, Reid Kleckner wrote:
>>>> If the clang binary is 2GB, I would recommend building
>>>> in Release mode with assertions enabled. Linking
>>>> binaries without debug information is dramatically
>>>> faster. This means passing -DCMAKE_BUILD_TYPE=Release
>>>> -DLLVM_ENABLE_ASSERTIONS=ON to cmake.
>>>>
>>>> If you use debug information, I recommend explicitly
>>>> building one target, the target you are testing
>>>> (usually clang), until you feel the need to run the
>>>> complete clang test suite.
>>>>
>>>> On Tue, Sep 18, 2018 at 5:42 PM Lou Wynn via cfe-dev
>>>> <cfe-dev at lists.llvm.org
>>>> <mailto:cfe-dev at lists.llvm.org>> wrote:
>>>>
>>>> Hi,
>>>>
>>>> I'm writing my first clang static analyzer checker.
>>>> I wrote the scaffold
>>>> code to add the checker into clang. But every time
>>>> when I change the
>>>> checker implementation code, the build process
>>>> after generating the
>>>> checker object file is very slow. It's over 4
>>>> minutes in my laptop,
>>>> which has the i7-6820HQ four core CPU and 32G
>>>> memory. The generated
>>>> clang binary is over 2G. The affected artifacts are
>>>> as follows:
>>>>
>>>> ---------------------
>>>>
>>>> Scanning dependencies of target
>>>> clangStaticAnalyzerCheckers
>>>> [ 95%] Building CXX object
>>>> tools/clang/lib/StaticAnalyzer/Checkers/CMakeFiles/clangStaticAnalyzerCheckers.dir/IntegerOverflowChecker.cpp.o
>>>> [ 95%] Linking CXX static library
>>>> ../../../../../lib/libclangStaticAnalyzerCheckers.a
>>>> [ 98%] Built target clangStaticAnalyzerCheckers
>>>> [ 98%] Linking CXX executable
>>>> ../../../../bin/arcmt-test
>>>> [ 98%] Linking CXX executable
>>>> ../../../../bin/clang-check
>>>> [ 98%] Linking CXX executable ../../../../bin/clang
>>>> [ 98%] Linking CXX shared library
>>>> ../../../../lib/libclang.so
>>>> [ 98%] Built target arcmt-test <--- this seems to
>>>> take the most time
>>>> [ 98%] Built target libclang
>>>> [ 98%] Linking CXX executable
>>>> ../../../../bin/c-arcmt-test
>>>> [ 98%] Linking CXX executable
>>>> ../../../../bin/c-index-test
>>>> [ 98%] Built target c-arcmt-test
>>>> [ 98%] Built target clang-check
>>>> [ 98%] Built target clang
>>>> [100%] Built target c-index-test
>>>>
>>>> real 4m23.014s
>>>> user 8m4.032s
>>>> sys 0m51.992s
>>>>
>>>> ---------------------
>>>>
>>>> How can I speed up the build process so that I
>>>> don't have to wait for
>>>> almost 4.5 minute for a single change in the
>>>> implementation code? I only
>>>> work on the checker of the static analyzer.
>>>>
>>>> --
>>>> Thanks,
>>>> Lou
>>>>
>>>> _______________________________________________
>>>> cfe-dev mailing list
>>>> cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>> <http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev>
>>>>
>>>
>>>
>>> _______________________________________________
>>> cfe-dev mailing list
>>> cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>> <http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev>
>>>
>>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20180921/5603dca3/attachment.html>
More information about the cfe-dev
mailing list