[PATCH] D75153: [ThinLTO] Allow usage of all SMT threads in the system

[Alexandre Ganea via Phabricator via cfe-commits]

aganea created this revision.
aganea added reviewers: tejohnson, thakis, rnk, RobRich999.
Herald added subscribers: cfe-commits, dang, dexonsmith, mikhail.ramalho, steven_wu, MaskRay, aheejin, hiraditya, arichardson, inglorion, sbc100, emaste.
Herald added a reviewer: espindola.
Herald added projects: clang, LLVM.

Before this patch, it wasn't possible to extend the ThinLTO threads to all SMT/CMT/hyper-threads in the system. Only one thread per core was allowed, instructed by usage of `llvm::heavyweight_hardware_concurrency()` in places in the ThinLTO objects initialization.

Any number passed to the LLD flag `/opt:lldltojobs=...`, or any other ThinLTO-specific flag, would be interpreted in the context of `llvm::heavyweight_hardware_concurrency()`, which means SMT disabled.

After this patch, one can say in LLD: `/opt:lldltojobs=all` -- which means to ignore the `llvm::heavyweight_hardware_concurrency()` requirement, and instead use all SMT/CMT/hyper-threads in the system, which is equivalent to using `llvm::hardware_concurrency()`.

All cmd-line flags and code paths that lead to initialize ThinLTO have been modified by this patch: `-flto-jobs=...`, `-threads=`, `--thinlto-jobs=...`, `-thinlto-threads=...`, `--plugin-opt=jobs=...`.

Summary
-------

To sum up:
`/opt:lldltojobs=0` -- use all threads available, but constrained by usage of `heavyweight_hardware_concurrency()` or `hardware_concurrency()`.
`/opt:lldltojobs=N` -- limit usage to N threads, but constrained by usage of `heavyweight_hardware_concurrency()` or `hardware_concurrency()`.
`/opt:lldltojobs=all` -- use all threads available, regardless of usage of `heavyweight_hardware_concurrency()` or `hardware_concurrency()`.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D75153

Files:
  clang/lib/Driver/ToolChains/CommonArgs.cpp
  clang/lib/Driver/ToolChains/CommonArgs.h
  clang/lib/Driver/ToolChains/Darwin.cpp
  lld/COFF/Config.h
  lld/COFF/Driver.cpp
  lld/COFF/LTO.cpp
  lld/ELF/Config.h
  lld/ELF/Driver.cpp
  lld/ELF/LTO.cpp
  lld/test/COFF/thinlto.ll
  lld/test/ELF/basic.s
  lld/test/ELF/lto/thinlto.ll
  lld/test/wasm/lto/thinlto.ll
  lld/wasm/Config.h
  lld/wasm/Driver.cpp
  lld/wasm/LTO.cpp
  llvm/include/llvm/LTO/LTO.h
  llvm/lib/LTO/LTO.cpp
  llvm/test/Transforms/PGOProfile/thinlto_samplepgo_icp3.ll
  llvm/tools/gold/gold-plugin.cpp
  llvm/tools/llvm-lto2/llvm-lto2.cpp

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D75153.246621.patch
Type: text/x-patch
Size: 22071 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20200226/4897b4e0/attachment-0001.bin>