[llvm-dev] LLD issue on a massively parallel build machine

[Robinson, Paul via llvm-dev]

> -----Original Message-----
> From: llvm-dev <llvm-dev-bounces at lists.llvm.org> On Behalf Of Tom Stellard
> via llvm-dev
> Sent: Wednesday, April 1, 2020 7:49 PM
> To: Itaru Kitayama <itaru.kitayama at gmail.com>
> Cc: Nemanja Ivanovic via llvm-dev <llvm-dev at lists.llvm.org>
> Subject: Re: [llvm-dev] LLD issue on a massively parallel build machine
> 
> On 04/01/2020 04:12 PM, Itaru Kitayama wrote:
> > On another login node which is 256 (GB)/48 (nodes) JURECA at JSC, I
> never had an LLD issue without setting -j when executing ninja
> > in the past few weeks.
> >
> > On Thu, Apr 2, 2020 at 7:17 AM Itaru Kitayama <itaru.kitayama at gmail.com
> <mailto:itaru.kitayama at gmail.com>> wrote:
> >
> >     Tom,
> >     Then what ratio do you think it’s minimal?
> >
> 
> It really depends on your configuration, but I usually try to have at
> least 2 GB
> of memory per core.  However, I usually do Release builds, so Debug builds
> might
> need more.  If you aren't using LLVM_PARALLEL_LINK_JOBS it's pretty easy
> to
> run out of memory once ninja starts linking the tools and unittests.
> 
> -Tom

For Debug (or RelWithDebInfo) I usually figure on around 5GB per thread
to avoid swapping.  Compiling is never an issue, it's the linking phase
that uses memory.  LLVM_PARALLEL_LINK_JOBS works well for ninja builds,
it has been a real help.
--paulr