[llvm-dev] Building with LLVM_PARALLEL_XXX_JOBS

Tue Mar 1 12:17:21 PST 2016

> On Mar 1, 2016, at 10:01 AM, Mehdi Amini <mehdi.amini at apple.com> wrote:
> 
> 
>> On Mar 1, 2016, at 9:57 AM, Chris Bieneman <cbieneman at apple.com> wrote:
>> 
>> There are a few notes I'd like to add to this thread.
>> 
>> (1) we have a number of places throughout out CMake build where we use features from newer CMakes gated by version checks. Most of these features are performance or usability related. None of them are correctness. Using the latest CMake release will often result in faster builds, so I encourage it.
>> 
>> (2) CMake's "install" target will pretty much always be slower from clean than the old autoconf/make "install" target. This is because in CMake "install" depends on "all", and our CMake builds more stuff in "all" than autoconf did. To help with this or CMake system has lots of convenient "install-${name}" targets that support component-based installation. Not every component has one of these rules, but if one you need is missing let me know. I also recently (r261681) added a new option (LLVM_DISTRIBUTION_COMPONENTS) that allows you to specify a list of components that have custom install targets. It then creates a new "install-distribution" target that wraps just the components you want. For Apple this is almost a 40% speed up over "ninja install".
> 
> That sounds great, I want to use it!
> It would even be more awesome with an description/example in docs/CMake.rst :)

Once I get the last of the kinks worked out for our internal adoption I'm going to open source our config files that use it.

I've also made a note to remind myself to document it in docs/CMake.rst. I need to do a pass updating that with a bunch of the cool new things we're doing with CMake. Thanks for the reminder.

-Chris

> 
> -- 
> Mehdi
>> 
>> On Feb 25, 2016, at 11:08 AM, Sedat Dilek via llvm-dev <llvm-dev at lists.llvm.org> wrote:
>> 
>>>> Which combination of cmake/ninja versions are you using (latest are
>>>> v3.4.3 and v1.6.0)?
>>> 
>>> With this combination I could reduce build-time down from approx. 3h
>>> down to 01h20m.
>>> 
>>> $ egrep -i 'jobs|ninja' llvm-build/CMakeCache.txt
>>> //Program used to build from build.ninja files.
>>> CMAKE_MAKE_PROGRAM:FILEPATH=/opt/cmake/bin/ninja
>>> //Define the maximum number of concurrent compilation jobs.
>>> LLVM_PARALLEL_COMPILE_JOBS:STRING=3
>>> //Define the maximum number of concurrent link jobs.
>>> LLVM_PARALLEL_LINK_JOBS:STRING=1
>>> CMAKE_GENERATOR:INTERNAL=Ninja
>>> 
>>> $ LC_ALL=C ls -alt logs/3.8.0rc3_clang-3-8-0-rc3_cmake-3-4-3_ninja-1-6-0/
>>> total 360
>>> drwxr-xr-x 2 wearefam wearefam   4096 Feb 25 19:58 .
>>> drwxr-xr-x 6 wearefam wearefam   4096 Feb 25 19:58 ..
>>> -rw-r--r-- 1 wearefam wearefam 130196 Feb 25 19:54
>>> install-log_llvm-toolchain-3.8.0rc3.txt
>>> -rw-r--r-- 1 wearefam wearefam 205762 Feb 25 19:51
>>> build-log_llvm-toolchain-3.8.0rc3.txt
>>> -rw-r--r-- 1 wearefam wearefam  14331 Feb 25 18:30
>>> configure-log_llvm-toolchain-3.8.0rc3.txt
>>> 
>>> $ LC_ALL=C du -s -m llvm* /opt/llvm-toolchain-3.8.0rc3
>>> 315     llvm
>>> 941     llvm-build
>>> 609     /opt/llvm-toolchain-3.8.0rc3
>>> 
>>> - Sedat -
>>> 
>>> [1] https://cmake.org/files/v3.5/cmake-3.5.0-rc3-Linux-x86_64.tar.gz
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 