[PATCH] D147493: [ELF] Cap parallel::strategy to 8 cores when --threads= is unspecified

[Fangrui Song via Phabricator via llvm-commits]

MaskRay created this revision.
MaskRay added reviewers: peter.smith, mjguzik.
Herald added subscribers: pengfei, arichardson, emaste.
Herald added a project: All.
MaskRay requested review of this revision.
Herald added a project: LLVM.
Herald added a subscriber: llvm-commits.

On x86-64 hosts, for all tested applications of mine, parallelism
overhead outweighs optimization when parallel::strategy uses more than 8
cores. Cap parallel::strategy when --threads= is unspecified.

--thinlto-jobs= is unchanged since ThinLTO backend compiles are
embarrassingly parallel.

https://discourse.llvm.org/t/avoidable-overhead-from-threading-by-default/69160


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D147493

Files:
  lld/ELF/Driver.cpp


Index: lld/ELF/Driver.cpp
===================================================================
--- lld/ELF/Driver.cpp
+++ lld/ELF/Driver.cpp
@@ -1401,8 +1401,12 @@
     config->mllvmOpts.emplace_back(arg->getValue());
   }
 
+  config->threadCount = parallel::strategy.compute_thread_count();
+
   // --threads= takes a positive integer and provides the default value for
-  // --thinlto-jobs=.
+  // --thinlto-jobs=. If unspecified, cap the number of threads since
+  // overhead outweighs optimization for used parallel algorithms for the
+  // non-LTO parts.
   if (auto *arg = args.getLastArg(OPT_threads)) {
     StringRef v(arg->getValue());
     unsigned threads = 0;
@@ -1411,10 +1415,11 @@
             arg->getValue() + "'");
     parallel::strategy = hardware_concurrency(threads);
     config->thinLTOJobs = v;
+  } else if (config->threadCount > 8) {
+    parallel::strategy = hardware_concurrency(8);
   }
   if (auto *arg = args.getLastArg(OPT_thinlto_jobs_eq))
     config->thinLTOJobs = arg->getValue();
-  config->threadCount = parallel::strategy.compute_thread_count();
 
   if (config->ltoPartitions == 0)
     error("--lto-partitions: number of threads must be > 0");


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D147493.510679.patch
Type: text/x-patch
Size: 1188 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20230404/64ce5bfd/attachment.bin>