[llvm-dev] (Thin)LTO llvm build

[Carsten Mattner via llvm-dev]

On Sat, Sep 17, 2016 at 3:17 AM, Mehdi Amini <mehdi.amini at apple.com> wrote:
>
>> On Sep 16, 2016, at 6:13 PM, Carsten Mattner <carstenmattner at gmail.com> wrote:
>>
>> On Sat, Sep 17, 2016 at 2:07 AM, Teresa Johnson via llvm-dev
>> <llvm-dev at lists.llvm.org> wrote:
>>
>>> Yes and to add on - the ThinLTO backend by default will
>>> kick off  std::thread::hardware_concurrency # of threads, which I'm finding is
>>
>> Is it just me or does that sound not very intuitive or at least a
>> little unexpected?
>> It's good that it uses the resources eagerly, but in terms of build systems this
>> is certainly surprising if there's no control of that parameter via
>> make/ninja/xcode.
>
> You can control the parallelism used by the linker, but the option is linker dependent
> (On MacOS:   -Wl,-mllvm,-threads=1)

That's what I meant. Maybe lld can gain support for that and allow us to use
the same ld pass-through via the compile driver so that it works on Linux and
BSD too.

>>> too much for machines with hyperthreading. If that ends up being an issue I can
>>> give you a workaround (I've been struggling to find a way that works on various
>>> OS and arches to compute the max number of physical cores to fix this in the source).
>>
>> I've been using ninja -jN so far. I suppose when building with ThinLTO I should
>> run ninja -j1. Would that
>>
>> What's the workaround?
>
> Seems like you missed my previous email: : cmake -DLLVM_PARALLEL_LINK_JOBS=1

I didn't miss that and it will hopefully help with limiting LTO link phase
resource use, but I do wonder if it means it's either linking one binary
or compiling objects and not both at parallel.

> Also, ninja is parallel by default, so no need to pass -j.
>
> This way you get nice parallelism during the compile phase,
> and ninja will issue only one link job at a time.

I know, but I use ninja for C++ project because those are the most
frequent CMake user, and compiling C++ often requires limited it
to less than NUM_CORES.