[llvm-dev] Tip: How to use less RAM linking LLVM

[via llvm-dev]

Hi all,

I've seen it pop up a few times now: linking LLVM uses too much memory. 
For me, using a VM with 10GB RAM was not enough, it still started swapping 
and the job took way too long (I had to kill it actually).

Using the following CMake options, I reduced peak memory usage to < 4 GB 
for a RelWithDebInfo build (LLVM only, including tools but not cfe/clang).
Do NOT set LLVM_USE_LINKER or set it to gnu.ld
LLVM_PARALLEL_LINK_JOBS=1
CMAKE_EXE_LINKER_FLAGS="-Wl,--reduce-memory-overheads 
-Wl,--hash-size=1021"

The `--reduce-memory-overheads` and the `--hash-size` flags are only 
supported by the GNU linker as far as I know.

The first option actually implies the 1021 value set in the second one 
(and thus `-Wl,--hash-size=1021` could be omitted), but I suspect that you 
can play around with the value. It is supposed to set the size of some 
internal hash-table sizes of the linker and not effect the output. (Look 
up your ld manpage for more info: https://linux.die.net/man/1/ld) 
Increasing the number might increase your memory usage and speed up 
compilation a bit, until you start swapping. If I had the time to 
recompile my LLVM repo a few times, I would try to tune this number to use 
up most, but not all of my RAM for optimal speed.

I hope this helps someone. If you have any additional tips, comments or 
gotchas with this approach, please add them.

All the best,
Boldizsár Palotás
This message is intended only for the recipient(s) named above. It may contain proprietary information and/or
protected content. Any unauthorised disclosure, use, retention or dissemination is prohibited. If you have received
this e-mail in error, please notify the sender immediately. ESA applies appropriate organisational measures to protect
personal data, in case of data privacy queries, please contact the ESA Data Protection Officer (dpo at esa.int).

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180920/de8feff7/attachment.html>