[PATCH] D72402: [build][NFC] limit parallel link to reduce memory usage

[Kókai Péter via Phabricator via llvm-commits]

Kokan created this revision.
Kokan added reviewers: LLVM, chandlerc.
Herald added subscribers: llvm-commits, mgorny.
Herald added a project: LLVM.

      

Measurerament I took with the following configuration:

- 32 GB memory
- 16 thread
- compiler clang/clang++ (version: 9.0.1)

  With ld as a linker (version: 2.33.1) ====

`-DLLVM_PARALLEL_LINK_JOBS=any` => out of memory
`-DLLVM_PARALLEL_LINK_JOBS=5` => no more than 30 GB memory
`-DLLVM_PARALLEL_LINK_JOBS=2` => no more than 14 GB memory
`-DLLVM_PARALLEL_LINK_JOBS=1` => no more than 10 GB memory

With lld as a linker (version: 2.33.1) => `-DLLVM_USE_LINKER=lld`
 ====

`-DLLVM_PARALLEL_LINK_JOBS=any` => no more then 9 GB memory
`-DLLVM_PARALLEL_LINK_JOBS=2` => no more than 6 GB memory

The current defaults for LLVM_PARALLEL_LINK_JOBS is empty, meaning any number
(only limited by ninja/make parallel option).

The LLVM_PARALLEL_LINK_JOBS=2 is a better default option,
 if the linker is not lld.

Signed-off-by: Kokai Peter <kokaipeter at gmail.com>


https://reviews.llvm.org/D72402

Files:
  llvm/cmake/modules/HandleLLVMOptions.cmake


Index: llvm/cmake/modules/HandleLLVMOptions.cmake
===================================================================
--- llvm/cmake/modules/HandleLLVMOptions.cmake
+++ llvm/cmake/modules/HandleLLVMOptions.cmake
@@ -41,6 +41,9 @@
     message(STATUS "ThinLTO provides its own parallel linking - limiting parallel link jobs to 2.")
     set(LLVM_PARALLEL_LINK_JOBS "2")
   endif()
+  if (NOT LLVM_PARALLEL_LINK_JOBS AND NOT LINKER_IS_LLD_LINK)
+    set(LLVM_PARALLEL_LINK_JOBS "2")
+  endif()
   if(LLVM_PARALLEL_LINK_JOBS)
     set_property(GLOBAL APPEND PROPERTY JOB_POOLS link_job_pool=${LLVM_PARALLEL_LINK_JOBS})
     set(CMAKE_JOB_POOL_LINK link_job_pool)


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D72402.236841.patch
Type: text/x-patch
Size: 660 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20200108/9d2c845d/attachment.bin>