[llvm-dev] LLD issue on a massively parallel build machine

[Itaru Kitayama via llvm-dev]

Then I’d suggest this should be renamed.
IMO it’s pretty much confusing.

it’s so easy to set ninja parallelism with the direct -j option as all we
know.

On Sun, Apr 5, 2020 at 2:30 Mehdi AMINI <joker.eph at gmail.com> wrote:

>
> On Thu, Apr 2, 2020 at 11:35 AM Itaru Kitayama via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Setting LLVM_PARALLEL_LINK_JOBS
>> did not help a week or two weeks ago’s lld.
>>
>> But recent commits to lld might reflect the variable correctly.
>>
>
> FYI: the variable has nothing to do with lld itself (not commits to lld
> would change the behavior of this flag), as far as I know this is purely
> instructing ninja to limit the number of parallel invocation to the linker.
> (It does not help limiting the parallelism when running the lld test-suite
> though)
>
> --
> Mehdi
>
>
>
>>
>> On Thu, Apr 2, 2020 at 22:52 Robinson, Paul <paul.robinson at sony.com>
>> wrote:
>>
>>>
>>>
>>> > -----Original Message-----
>>> > From: llvm-dev <llvm-dev-bounces at lists.llvm.org> On Behalf Of Tom
>>> Stellard
>>> > via llvm-dev
>>> > Sent: Wednesday, April 1, 2020 7:49 PM
>>> > To: Itaru Kitayama <itaru.kitayama at gmail.com>
>>> > Cc: Nemanja Ivanovic via llvm-dev <llvm-dev at lists.llvm.org>
>>> > Subject: Re: [llvm-dev] LLD issue on a massively parallel build machine
>>> >
>>> > On 04/01/2020 04:12 PM, Itaru Kitayama wrote:
>>> > > On another login node which is 256 (GB)/48 (nodes) JURECA at JSC, I
>>> > never had an LLD issue without setting -j when executing ninja
>>> > > in the past few weeks.
>>> > >
>>> > > On Thu, Apr 2, 2020 at 7:17 AM Itaru Kitayama <
>>> itaru.kitayama at gmail.com
>>> > <mailto:itaru.kitayama at gmail.com>> wrote:
>>> > >
>>> > >     Tom,
>>> > >     Then what ratio do you think it’s minimal?
>>> > >
>>> >
>>> > It really depends on your configuration, but I usually try to have at
>>> > least 2 GB
>>> > of memory per core.  However, I usually do Release builds, so Debug
>>> builds
>>> > might
>>> > need more.  If you aren't using LLVM_PARALLEL_LINK_JOBS it's pretty
>>> easy
>>> > to
>>> > run out of memory once ninja starts linking the tools and unittests.
>>> >
>>> > -Tom
>>>
>>> For Debug (or RelWithDebInfo) I usually figure on around 5GB per thread
>>> to avoid swapping.  Compiling is never an issue, it's the linking phase
>>> that uses memory.  LLVM_PARALLEL_LINK_JOBS works well for ninja builds,
>>> it has been a real help.
>>> --paulr
>>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200405/684ae393/attachment.html>