[llvm-dev] LLD issue on a massively parallel build machine

[Itaru Kitayama via llvm-dev]

Setting LLVM_PARALLEL_LINK_JOBS
did not help a week or two weeks ago’s lld.

But recent commits to lld might reflect the variable correctly.

On Thu, Apr 2, 2020 at 22:52 Robinson, Paul <paul.robinson at sony.com> wrote:

>
>
> > -----Original Message-----
> > From: llvm-dev <llvm-dev-bounces at lists.llvm.org> On Behalf Of Tom
> Stellard
> > via llvm-dev
> > Sent: Wednesday, April 1, 2020 7:49 PM
> > To: Itaru Kitayama <itaru.kitayama at gmail.com>
> > Cc: Nemanja Ivanovic via llvm-dev <llvm-dev at lists.llvm.org>
> > Subject: Re: [llvm-dev] LLD issue on a massively parallel build machine
> >
> > On 04/01/2020 04:12 PM, Itaru Kitayama wrote:
> > > On another login node which is 256 (GB)/48 (nodes) JURECA at JSC, I
> > never had an LLD issue without setting -j when executing ninja
> > > in the past few weeks.
> > >
> > > On Thu, Apr 2, 2020 at 7:17 AM Itaru Kitayama <
> itaru.kitayama at gmail.com
> > <mailto:itaru.kitayama at gmail.com>> wrote:
> > >
> > >     Tom,
> > >     Then what ratio do you think it’s minimal?
> > >
> >
> > It really depends on your configuration, but I usually try to have at
> > least 2 GB
> > of memory per core.  However, I usually do Release builds, so Debug
> builds
> > might
> > need more.  If you aren't using LLVM_PARALLEL_LINK_JOBS it's pretty easy
> > to
> > run out of memory once ninja starts linking the tools and unittests.
> >
> > -Tom
>
> For Debug (or RelWithDebInfo) I usually figure on around 5GB per thread
> to avoid swapping.  Compiling is never an issue, it's the linking phase
> that uses memory.  LLVM_PARALLEL_LINK_JOBS works well for ninja builds,
> it has been a real help.
> --paulr
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200403/76f7a971/attachment.html>