[clang-tools-extra] 8404aeb - [Support] On Windows, ensure hardware_concurrency() extends to all CPU sockets and all NUMA groups
Alexandre Ganea via cfe-commits
cfe-commits at lists.llvm.org
Fri Feb 14 07:26:00 PST 2020
Author: Alexandre Ganea
Date: 2020-02-14T10:24:22-05:00
New Revision: 8404aeb56a73ab24f9b295111de3b37a37f0b841
URL: https://github.com/llvm/llvm-project/commit/8404aeb56a73ab24f9b295111de3b37a37f0b841
DIFF: https://github.com/llvm/llvm-project/commit/8404aeb56a73ab24f9b295111de3b37a37f0b841.diff
LOG: [Support] On Windows, ensure hardware_concurrency() extends to all CPU sockets and all NUMA groups
The goal of this patch is to maximize CPU utilization on multi-socket or high core count systems, so that parallel computations such as LLD/ThinLTO can use all hardware threads in the system. Before this patch, on Windows, a maximum of 64 hardware threads could be used at most, in some cases dispatched only on one CPU socket.
== Background ==
Windows doesn't have a flat cpu_set_t like Linux. Instead, it projects hardware CPUs (or NUMA nodes) to applications through a concept of "processor groups". A "processor" is the smallest unit of execution on a CPU, that is, an hyper-thread if SMT is active; a core otherwise. There's a limit of 32-bit processors on older 32-bit versions of Windows, which later was raised to 64-processors with 64-bit versions of Windows. This limit comes from the affinity mask, which historically is represented by the sizeof(void*). Consequently, the concept of "processor groups" was introduced for dealing with systems with more than 64 hyper-threads.
By default, the Windows OS assigns only one "processor group" to each starting application, in a round-robin manner. If the application wants to use more processors, it needs to programmatically enable it, by assigning threads to other "processor groups". This also means that affinity cannot cross "processor group" boundaries; one can only specify a "preferred" group on start-up, but the application is free to allocate more groups if it wants to.
This creates a peculiar situation, where newer CPUs like the AMD EPYC 7702P (64-cores, 128-hyperthreads) are projected by the OS as two (2) "processor groups". This means that by default, an application can only use half of the cores. This situation could only get worse in the years to come, as dies with more cores will appear on the market.
== The problem ==
The heavyweight_hardware_concurrency() API was introduced so that only *one hardware thread per core* was used. Once that API returns, that original intention is lost, only the number of threads is retained. Consider a situation, on Windows, where the system has 2 CPU sockets, 18 cores each, each core having 2 hyper-threads, for a total of 72 hyper-threads. Both heavyweight_hardware_concurrency() and hardware_concurrency() currently return 36, because on Windows they are simply wrappers over std::thread::hardware_concurrency() -- which can only return processors from the current "processor group".
== The changes in this patch ==
To solve this situation, we capture (and retain) the initial intention until the point of usage, through a new ThreadPoolStrategy class. The number of threads to use is deferred as late as possible, until the moment where the std::threads are created (ThreadPool in the case of ThinLTO).
When using hardware_concurrency(), setting ThreadCount to 0 now means to use all the possible hardware CPU (SMT) threads. Providing a ThreadCount above to the maximum number of threads will have no effect, the maximum will be used instead.
The heavyweight_hardware_concurrency() is similar to hardware_concurrency(), except that only one thread per hardware *core* will be used.
When LLVM_ENABLE_THREADS is OFF, the threading APIs will always return 1, to ensure any caller loops will be exercised at least once.
Differential Revision: https://reviews.llvm.org/D71775
Added:
Modified:
clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
clang-tools-extra/clangd/TUScheduler.cpp
clang-tools-extra/clangd/index/Background.cpp
clang-tools-extra/clangd/index/Background.h
clang-tools-extra/clangd/index/BackgroundRebuild.h
clang/lib/Tooling/AllTUsExecution.cpp
clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp
clang/tools/clang-scan-deps/ClangScanDeps.cpp
lld/ELF/SyntheticSections.cpp
llvm/include/llvm/LTO/LTO.h
llvm/include/llvm/Support/ThreadPool.h
llvm/include/llvm/Support/Threading.h
llvm/lib/CodeGen/ParallelCG.cpp
llvm/lib/DWARFLinker/DWARFLinker.cpp
llvm/lib/DebugInfo/GSYM/DwarfTransformer.cpp
llvm/lib/ExecutionEngine/Orc/LLJIT.cpp
llvm/lib/LTO/LTO.cpp
llvm/lib/LTO/LTOBackend.cpp
llvm/lib/LTO/ThinLTOCodeGenerator.cpp
llvm/lib/Support/Host.cpp
llvm/lib/Support/Parallel.cpp
llvm/lib/Support/ThreadPool.cpp
llvm/lib/Support/Threading.cpp
llvm/lib/Support/Unix/Threading.inc
llvm/lib/Support/Windows/Threading.inc
llvm/tools/dsymutil/dsymutil.cpp
llvm/tools/gold/gold-plugin.cpp
llvm/tools/llvm-cov/CodeCoverage.cpp
llvm/tools/llvm-cov/CoverageExporterJson.cpp
llvm/tools/llvm-cov/CoverageReport.cpp
llvm/tools/llvm-lto2/llvm-lto2.cpp
llvm/tools/llvm-profdata/llvm-profdata.cpp
llvm/unittests/Support/Host.cpp
llvm/unittests/Support/TaskQueueTest.cpp
llvm/unittests/Support/ThreadPool.cpp
llvm/unittests/Support/Threading.cpp
mlir/lib/Pass/Pass.cpp
Removed:
################################################################################
diff --git a/clang-tools-extra/clang-doc/tool/ClangDocMain.cpp b/clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
index eae0d84721c4..cfc8ff653f16 100644
--- a/clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
+++ b/clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
@@ -268,8 +268,7 @@ int main(int argc, const char **argv) {
Error = false;
llvm::sys::Mutex IndexMutex;
// ExecutorConcurrency is a flag exposed by AllTUsExecution.h
- llvm::ThreadPool Pool(ExecutorConcurrency == 0 ? llvm::hardware_concurrency()
- : ExecutorConcurrency);
+ llvm::ThreadPool Pool(llvm::hardware_concurrency(ExecutorConcurrency));
for (auto &Group : USRToBitcode) {
Pool.async([&]() {
std::vector<std::unique_ptr<doc::Info>> Infos;
diff --git a/clang-tools-extra/clangd/TUScheduler.cpp b/clang-tools-extra/clangd/TUScheduler.cpp
index 5a1caa964520..f59c19e8031e 100644
--- a/clang-tools-extra/clangd/TUScheduler.cpp
+++ b/clang-tools-extra/clangd/TUScheduler.cpp
@@ -842,13 +842,7 @@ std::string renderTUAction(const TUAction &Action) {
} // namespace
unsigned getDefaultAsyncThreadsCount() {
- unsigned HardwareConcurrency = llvm::heavyweight_hardware_concurrency();
- // heavyweight_hardware_concurrency may fall back to hardware_concurrency.
- // C++ standard says that hardware_concurrency() may return 0; fallback to 1
- // worker thread in that case.
- if (HardwareConcurrency == 0)
- return 1;
- return HardwareConcurrency;
+ return llvm::heavyweight_hardware_concurrency().compute_thread_count();
}
FileStatus TUStatus::render(PathRef File) const {
diff --git a/clang-tools-extra/clangd/index/Background.cpp b/clang-tools-extra/clangd/index/Background.cpp
index ab80113a4a95..c2541237c3c9 100644
--- a/clang-tools-extra/clangd/index/Background.cpp
+++ b/clang-tools-extra/clangd/index/Background.cpp
@@ -148,9 +148,10 @@ BackgroundIndex::BackgroundIndex(
CDB.watch([&](const std::vector<std::string> &ChangedFiles) {
enqueue(ChangedFiles);
})) {
- assert(ThreadPoolSize > 0 && "Thread pool size can't be zero.");
+ assert(Rebuilder.TUsBeforeFirstBuild > 0 &&
+ "Thread pool size can't be zero.");
assert(this->IndexStorageFactory && "Storage factory can not be null!");
- for (unsigned I = 0; I < ThreadPoolSize; ++I) {
+ for (unsigned I = 0; I < Rebuilder.TUsBeforeFirstBuild; ++I) {
ThreadPool.runAsync("background-worker-" + llvm::Twine(I + 1), [this] {
WithContext Ctx(this->BackgroundContext.clone());
Queue.work([&] { Rebuilder.idle(); });
diff --git a/clang-tools-extra/clangd/index/Background.h b/clang-tools-extra/clangd/index/Background.h
index b11008de15d0..2ae11c72d5d4 100644
--- a/clang-tools-extra/clangd/index/Background.h
+++ b/clang-tools-extra/clangd/index/Background.h
@@ -135,7 +135,7 @@ class BackgroundIndex : public SwapIndex {
Context BackgroundContext, const FileSystemProvider &,
const GlobalCompilationDatabase &CDB,
BackgroundIndexStorage::Factory IndexStorageFactory,
- size_t ThreadPoolSize = llvm::heavyweight_hardware_concurrency(),
+ size_t ThreadPoolSize = 0, // 0 = use all hardware threads
std::function<void(BackgroundQueue::Stats)> OnProgress = nullptr);
~BackgroundIndex(); // Blocks while the current task finishes.
diff --git a/clang-tools-extra/clangd/index/BackgroundRebuild.h b/clang-tools-extra/clangd/index/BackgroundRebuild.h
index d74c28be5cfb..295f705c98e8 100644
--- a/clang-tools-extra/clangd/index/BackgroundRebuild.h
+++ b/clang-tools-extra/clangd/index/BackgroundRebuild.h
@@ -49,7 +49,9 @@ class BackgroundIndexRebuilder {
public:
BackgroundIndexRebuilder(SwapIndex *Target, FileSymbols *Source,
unsigned Threads)
- : TUsBeforeFirstBuild(Threads), Target(Target), Source(Source) {}
+ : TUsBeforeFirstBuild(llvm::heavyweight_hardware_concurrency(Threads)
+ .compute_thread_count()),
+ Target(Target), Source(Source) {}
// Called to indicate a TU has been indexed.
// May rebuild, if enough TUs have been indexed.
diff --git a/clang/lib/Tooling/AllTUsExecution.cpp b/clang/lib/Tooling/AllTUsExecution.cpp
index d85075f59607..777857a49e81 100644
--- a/clang/lib/Tooling/AllTUsExecution.cpp
+++ b/clang/lib/Tooling/AllTUsExecution.cpp
@@ -114,8 +114,7 @@ llvm::Error AllTUsToolExecutor::execute(
auto &Action = Actions.front();
{
- llvm::ThreadPool Pool(ThreadCount == 0 ? llvm::hardware_concurrency()
- : ThreadCount);
+ llvm::ThreadPool Pool(llvm::hardware_concurrency(ThreadCount));
for (std::string File : Files) {
Pool.async(
[&](std::string Path) {
diff --git a/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp b/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp
index b4d5a29ca695..b1b87e7fa573 100644
--- a/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp
+++ b/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp
@@ -106,7 +106,8 @@ DependencyScanningFilesystemSharedCache::
// sharding gives a performance edge by reducing the lock contention.
// FIXME: A better heuristic might also consider the OS to account for
// the
diff erent cost of lock contention on
diff erent OSes.
- NumShards = std::max(2u, llvm::hardware_concurrency() / 4);
+ NumShards =
+ std::max(2u, llvm::hardware_concurrency().compute_thread_count() / 4);
CacheShards = std::make_unique<CacheShard[]>(NumShards);
}
diff --git a/clang/tools/clang-scan-deps/ClangScanDeps.cpp b/clang/tools/clang-scan-deps/ClangScanDeps.cpp
index c499048fdb52..9105c616786f 100644
--- a/clang/tools/clang-scan-deps/ClangScanDeps.cpp
+++ b/clang/tools/clang-scan-deps/ClangScanDeps.cpp
@@ -485,15 +485,9 @@ int main(int argc, const char **argv) {
DependencyScanningService Service(ScanMode, Format, ReuseFileManager,
SkipExcludedPPRanges);
-#if LLVM_ENABLE_THREADS
- unsigned NumWorkers =
- NumThreads == 0 ? llvm::hardware_concurrency() : NumThreads;
-#else
- unsigned NumWorkers = 1;
-#endif
- llvm::ThreadPool Pool(NumWorkers);
+ llvm::ThreadPool Pool(llvm::hardware_concurrency(NumThreads));
std::vector<std::unique_ptr<DependencyScanningTool>> WorkerTools;
- for (unsigned I = 0; I < NumWorkers; ++I)
+ for (unsigned I = 0; I < Pool.getThreadCount(); ++I)
WorkerTools.push_back(std::make_unique<DependencyScanningTool>(Service));
std::vector<SingleCommandCompilationDatabase> Inputs;
@@ -508,9 +502,9 @@ int main(int argc, const char **argv) {
if (Verbose) {
llvm::outs() << "Running clang-scan-deps on " << Inputs.size()
- << " files using " << NumWorkers << " workers\n";
+ << " files using " << Pool.getThreadCount() << " workers\n";
}
- for (unsigned I = 0; I < NumWorkers; ++I) {
+ for (unsigned I = 0; I < Pool.getThreadCount(); ++I) {
Pool.async([I, &Lock, &Index, &Inputs, &HadErrors, &FD, &WorkerTools,
&DependencyOS, &Errs]() {
llvm::StringSet<> AlreadySeenModules;
diff --git a/lld/ELF/SyntheticSections.cpp b/lld/ELF/SyntheticSections.cpp
index bd01bc02617c..3a0251c90692 100644
--- a/lld/ELF/SyntheticSections.cpp
+++ b/lld/ELF/SyntheticSections.cpp
@@ -2747,8 +2747,8 @@ createSymbols(ArrayRef<std::vector<GdbIndexSection::NameAttrEntry>> nameAttrs,
size_t numShards = 32;
size_t concurrency = 1;
if (threadsEnabled)
- concurrency =
- std::min<size_t>(PowerOf2Floor(hardware_concurrency()), numShards);
+ concurrency = std::min<size_t>(
+ hardware_concurrency().compute_thread_count(), numShards);
// A sharded map to uniquify symbols by name.
std::vector<DenseMap<CachedHashStringRef, size_t>> map(numShards);
@@ -3191,8 +3191,8 @@ void MergeNoTailSection::finalizeContents() {
// operations in the following tight loop.
size_t concurrency = 1;
if (threadsEnabled)
- concurrency =
- std::min<size_t>(PowerOf2Floor(hardware_concurrency()), numShards);
+ concurrency = std::min<size_t>(
+ hardware_concurrency().compute_thread_count(), numShards);
// Add section pieces to the builders.
parallelForEachN(0, concurrency, [&](size_t threadId) {
diff --git a/llvm/include/llvm/LTO/LTO.h b/llvm/include/llvm/LTO/LTO.h
index e864e05df340..df2853345682 100644
--- a/llvm/include/llvm/LTO/LTO.h
+++ b/llvm/include/llvm/LTO/LTO.h
@@ -227,7 +227,8 @@ using ThinBackend = std::function<std::unique_ptr<ThinBackendProc>(
AddStreamFn AddStream, NativeObjectCache Cache)>;
/// This ThinBackend runs the individual backend jobs in-process.
-ThinBackend createInProcessThinBackend(unsigned ParallelismLevel);
+/// The default value means to use one job per hardware core (not hyper-thread).
+ThinBackend createInProcessThinBackend(unsigned ParallelismLevel = 0);
/// This ThinBackend writes individual module indexes to files, instead of
/// running the individual backend jobs. This backend is for distributed builds
diff --git a/llvm/include/llvm/Support/ThreadPool.h b/llvm/include/llvm/Support/ThreadPool.h
index 4bcbaa3142fd..2036f46c6d56 100644
--- a/llvm/include/llvm/Support/ThreadPool.h
+++ b/llvm/include/llvm/Support/ThreadPool.h
@@ -13,7 +13,9 @@
#ifndef LLVM_SUPPORT_THREAD_POOL_H
#define LLVM_SUPPORT_THREAD_POOL_H
+#include "llvm/ADT/BitVector.h"
#include "llvm/Config/llvm-config.h"
+#include "llvm/Support/Threading.h"
#include "llvm/Support/thread.h"
#include <future>
@@ -38,12 +40,11 @@ class ThreadPool {
using TaskTy = std::function<void()>;
using PackagedTaskTy = std::packaged_task<void()>;
- /// Construct a pool with the number of threads found by
- /// hardware_concurrency().
- ThreadPool();
-
- /// Construct a pool of \p ThreadCount threads
- ThreadPool(unsigned ThreadCount);
+ /// Construct a pool using the hardware strategy \p S for mapping hardware
+ /// execution resources (threads, cores, CPUs)
+ /// Defaults to using the maximum execution resources in the system, but
+ /// excluding any resources contained in the affinity mask.
+ ThreadPool(ThreadPoolStrategy S = hardware_concurrency());
/// Blocking destructor: the pool will wait for all the threads to complete.
~ThreadPool();
@@ -68,6 +69,8 @@ class ThreadPool {
/// It is an error to try to add new tasks while blocking on this call.
void wait();
+ unsigned getThreadCount() const { return ThreadCount; }
+
private:
/// Asynchronous submission of a task to the pool. The returned future can be
/// used to wait for the task to finish and is *non-blocking* on destruction.
@@ -94,6 +97,8 @@ class ThreadPool {
/// Signal for the destruction of the pool, asking thread to exit.
bool EnableFlag;
#endif
+
+ unsigned ThreadCount;
};
}
diff --git a/llvm/include/llvm/Support/Threading.h b/llvm/include/llvm/Support/Threading.h
index bacab8fa23b6..d3d4a37e69c6 100644
--- a/llvm/include/llvm/Support/Threading.h
+++ b/llvm/include/llvm/Support/Threading.h
@@ -14,6 +14,7 @@
#ifndef LLVM_SUPPORT_THREADING_H
#define LLVM_SUPPORT_THREADING_H
+#include "llvm/ADT/BitVector.h"
#include "llvm/ADT/FunctionExtras.h"
#include "llvm/ADT/SmallVector.h"
#include "llvm/Config/llvm-config.h" // for LLVM_ON_UNIX
@@ -143,20 +144,52 @@ void llvm_execute_on_thread_async(
#endif
}
- /// Get the amount of currency to use for tasks requiring significant
- /// memory or other resources. Currently based on physical cores, if
- /// available for the host system, otherwise falls back to
- /// thread::hardware_concurrency().
- /// Returns 1 when LLVM is configured with LLVM_ENABLE_THREADS=OFF
- unsigned heavyweight_hardware_concurrency();
-
- /// Get the number of threads that the current program can execute
- /// concurrently. On some systems std::thread::hardware_concurrency() returns
- /// the total number of cores, without taking affinity into consideration.
- /// Returns 1 when LLVM is configured with LLVM_ENABLE_THREADS=OFF.
- /// Fallback to std::thread::hardware_concurrency() if sched_getaffinity is
- /// not available.
- unsigned hardware_concurrency();
+ /// This tells how a thread pool will be used
+ class ThreadPoolStrategy {
+ public:
+ // The default value (0) means all available threads should be used,
+ // excluding affinity mask. If set, this value only represents a suggested
+ // high bound, the runtime might choose a lower value (not higher).
+ unsigned ThreadsRequested = 0;
+
+ // If SMT is active, use hyper threads. If false, there will be only one
+ // std::thread per core.
+ bool UseHyperThreads = true;
+
+ /// Retrieves the max available threads for the current strategy. This
+ /// accounts for affinity masks and takes advantage of all CPU sockets.
+ unsigned compute_thread_count() const;
+
+ /// Assign the current thread to an ideal hardware CPU or NUMA node. In a
+ /// multi-socket system, this ensures threads are assigned to all CPU
+ /// sockets. \p ThreadPoolNum represents a number bounded by [0,
+ /// compute_thread_count()).
+ void apply_thread_strategy(unsigned ThreadPoolNum) const;
+ };
+
+ /// Returns a thread strategy for tasks requiring significant memory or other
+ /// resources. To be used for workloads where hardware_concurrency() proves to
+ /// be less efficient. Avoid this strategy if doing lots of I/O. Currently
+ /// based on physical cores, if available for the host system, otherwise falls
+ /// back to hardware_concurrency(). Returns 1 when LLVM is configured with
+ /// LLVM_ENABLE_THREADS = OFF
+ inline ThreadPoolStrategy
+ heavyweight_hardware_concurrency(unsigned ThreadCount = 0) {
+ ThreadPoolStrategy S;
+ S.UseHyperThreads = false;
+ S.ThreadsRequested = ThreadCount;
+ return S;
+ }
+
+ /// Returns a default thread strategy where all available hardware ressources
+ /// are to be used, except for those initially excluded by an affinity mask.
+ /// This function takes affinity into consideration. Returns 1 when LLVM is
+ /// configured with LLVM_ENABLE_THREADS=OFF.
+ inline ThreadPoolStrategy hardware_concurrency(unsigned ThreadCount = 0) {
+ ThreadPoolStrategy S;
+ S.ThreadsRequested = ThreadCount;
+ return S;
+ }
/// Return the current thread id, as used in various OS system calls.
/// Note that not all platforms guarantee that the value returned will be
@@ -184,6 +217,14 @@ void llvm_execute_on_thread_async(
/// the operation succeeded or failed is returned.
void get_thread_name(SmallVectorImpl<char> &Name);
+ /// Returns a mask that represents on which hardware thread, core, CPU, NUMA
+ /// group, the calling thread can be executed. On Windows, threads cannot
+ /// cross CPU boundaries.
+ llvm::BitVector get_thread_affinity_mask();
+
+ /// Returns how many physical CPUs or NUMA groups the system has.
+ unsigned get_cpus();
+
enum class ThreadPriority {
Background = 0,
Default = 1,
diff --git a/llvm/lib/CodeGen/ParallelCG.cpp b/llvm/lib/CodeGen/ParallelCG.cpp
index 7dbd830666fb..c19ed1f8f71d 100644
--- a/llvm/lib/CodeGen/ParallelCG.cpp
+++ b/llvm/lib/CodeGen/ParallelCG.cpp
@@ -51,7 +51,7 @@ std::unique_ptr<Module> llvm::splitCodeGen(
// Create ThreadPool in nested scope so that threads will be joined
// on destruction.
{
- ThreadPool CodegenThreadPool(OSs.size());
+ ThreadPool CodegenThreadPool(hardware_concurrency(OSs.size()));
int ThreadCount = 0;
SplitModule(
diff --git a/llvm/lib/DWARFLinker/DWARFLinker.cpp b/llvm/lib/DWARFLinker/DWARFLinker.cpp
index 03919c805130..715ad24b5521 100644
--- a/llvm/lib/DWARFLinker/DWARFLinker.cpp
+++ b/llvm/lib/DWARFLinker/DWARFLinker.cpp
@@ -2446,7 +2446,7 @@ bool DWARFLinker::link() {
}
EmitLambda();
} else {
- ThreadPool Pool(2);
+ ThreadPool Pool(hardware_concurrency(2));
Pool.async(AnalyzeAll);
Pool.async(CloneAll);
Pool.wait();
diff --git a/llvm/lib/DebugInfo/GSYM/DwarfTransformer.cpp b/llvm/lib/DebugInfo/GSYM/DwarfTransformer.cpp
index e01b6b6ebc0c..c3bf71f21cda 100644
--- a/llvm/lib/DebugInfo/GSYM/DwarfTransformer.cpp
+++ b/llvm/lib/DebugInfo/GSYM/DwarfTransformer.cpp
@@ -445,7 +445,7 @@ Error DwarfTransformer::convert(uint32_t NumThreads) {
// Now parse all DIEs in case we have cross compile unit references in a
// thread pool.
- ThreadPool pool(NumThreads);
+ ThreadPool pool(hardware_concurrency(NumThreads));
for (const auto &CU : DICtx.compile_units())
pool.async([&CU]() { CU->getUnitDIE(false /*CUDieOnly*/); });
pool.wait();
diff --git a/llvm/lib/ExecutionEngine/Orc/LLJIT.cpp b/llvm/lib/ExecutionEngine/Orc/LLJIT.cpp
index f81e584b3b2d..4218ca4e481f 100644
--- a/llvm/lib/ExecutionEngine/Orc/LLJIT.cpp
+++ b/llvm/lib/ExecutionEngine/Orc/LLJIT.cpp
@@ -157,7 +157,8 @@ LLJIT::LLJIT(LLJITBuilderState &S, Error &Err)
if (S.NumCompileThreads > 0) {
TransformLayer->setCloneToNewContextOnEmit(true);
- CompileThreads = std::make_unique<ThreadPool>(S.NumCompileThreads);
+ CompileThreads =
+ std::make_unique<ThreadPool>(hardware_concurrency(S.NumCompileThreads));
ES->setDispatchMaterialization(
[this](JITDylib &JD, std::unique_ptr<MaterializationUnit> MU) {
// FIXME: Switch to move capture once we have c++14.
diff --git a/llvm/lib/LTO/LTO.cpp b/llvm/lib/LTO/LTO.cpp
index 1d23c6bab36d..f8affcb20cef 100644
--- a/llvm/lib/LTO/LTO.cpp
+++ b/llvm/lib/LTO/LTO.cpp
@@ -477,8 +477,7 @@ LTO::RegularLTOState::RegularLTOState(unsigned ParallelCodeGenParallelismLevel,
LTO::ThinLTOState::ThinLTOState(ThinBackend Backend)
: Backend(Backend), CombinedIndex(/*HaveGVs*/ false) {
if (!Backend)
- this->Backend =
- createInProcessThinBackend(llvm::heavyweight_hardware_concurrency());
+ this->Backend = createInProcessThinBackend();
}
LTO::LTO(Config Conf, ThinBackend Backend,
@@ -1095,7 +1094,8 @@ class InProcessThinBackend : public ThinBackendProc {
const StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
AddStreamFn AddStream, NativeObjectCache Cache)
: ThinBackendProc(Conf, CombinedIndex, ModuleToDefinedGVSummaries),
- BackendThreadPool(ThinLTOParallelismLevel),
+ BackendThreadPool(
+ heavyweight_hardware_concurrency(ThinLTOParallelismLevel)),
AddStream(std::move(AddStream)), Cache(std::move(Cache)) {
for (auto &Name : CombinedIndex.cfiFunctionDefs())
CfiFunctionDefs.insert(
diff --git a/llvm/lib/LTO/LTOBackend.cpp b/llvm/lib/LTO/LTOBackend.cpp
index b85471555b09..ec57744cf480 100644
--- a/llvm/lib/LTO/LTOBackend.cpp
+++ b/llvm/lib/LTO/LTOBackend.cpp
@@ -375,7 +375,8 @@ void codegen(const Config &Conf, TargetMachine *TM, AddStreamFn AddStream,
void splitCodeGen(const Config &C, TargetMachine *TM, AddStreamFn AddStream,
unsigned ParallelCodeGenParallelismLevel,
std::unique_ptr<Module> Mod) {
- ThreadPool CodegenThreadPool(ParallelCodeGenParallelismLevel);
+ ThreadPool CodegenThreadPool(
+ heavyweight_hardware_concurrency(ParallelCodeGenParallelismLevel));
unsigned ThreadCount = 0;
const Target *T = &TM->getTarget();
diff --git a/llvm/lib/LTO/ThinLTOCodeGenerator.cpp b/llvm/lib/LTO/ThinLTOCodeGenerator.cpp
index a4f270240005..152f0afcf12e 100644
--- a/llvm/lib/LTO/ThinLTOCodeGenerator.cpp
+++ b/llvm/lib/LTO/ThinLTOCodeGenerator.cpp
@@ -80,8 +80,8 @@ extern cl::opt<std::string> RemarksFormat;
namespace {
-static cl::opt<int>
- ThreadCount("threads", cl::init(llvm::heavyweight_hardware_concurrency()));
+// Default to using one job per hardware core in the system
+static cl::opt<int> ThreadCount("threads", cl::init(0));
// Simple helper to save temporary files for debug.
static void saveTempBitcode(const Module &TheModule, StringRef TempDir,
@@ -1042,7 +1042,7 @@ void ThinLTOCodeGenerator::run() {
// Parallel optimizer + codegen
{
- ThreadPool Pool(ThreadCount);
+ ThreadPool Pool(heavyweight_hardware_concurrency(ThreadCount));
for (auto IndexCount : ModulesOrdering) {
auto &Mod = Modules[IndexCount];
Pool.async([&](int count) {
diff --git a/llvm/lib/Support/Host.cpp b/llvm/lib/Support/Host.cpp
index ef38c1c09413..7e772b2b1378 100644
--- a/llvm/lib/Support/Host.cpp
+++ b/llvm/lib/Support/Host.cpp
@@ -1266,7 +1266,7 @@ StringRef sys::getHostCPUName() { return "generic"; }
// On Linux, the number of physical cores can be computed from /proc/cpuinfo,
// using the number of unique physical/core id pairs. The following
// implementation reads the /proc/cpuinfo format on an x86_64 system.
-static int computeHostNumPhysicalCores() {
+int computeHostNumPhysicalCores() {
// Read /proc/cpuinfo as a stream (until EOF reached). It cannot be
// mmapped because it appears to have 0 size.
llvm::ErrorOr<std::unique_ptr<llvm::MemoryBuffer>> Text =
@@ -1312,7 +1312,7 @@ static int computeHostNumPhysicalCores() {
#include <sys/sysctl.h>
// Gets the number of *physical cores* on the machine.
-static int computeHostNumPhysicalCores() {
+int computeHostNumPhysicalCores() {
uint32_t count;
size_t len = sizeof(count);
sysctlbyname("hw.physicalcpu", &count, &len, NULL, 0);
@@ -1326,6 +1326,9 @@ static int computeHostNumPhysicalCores() {
}
return count;
}
+#elif defined(_WIN32)
+// Defined in llvm/lib/Support/Windows/Threading.inc
+int computeHostNumPhysicalCores();
#else
// On other systems, return -1 to indicate unknown.
static int computeHostNumPhysicalCores() { return -1; }
diff --git a/llvm/lib/Support/Parallel.cpp b/llvm/lib/Support/Parallel.cpp
index 523665d14b02..0272a53beb39 100644
--- a/llvm/lib/Support/Parallel.cpp
+++ b/llvm/lib/Support/Parallel.cpp
@@ -39,20 +39,21 @@ class Executor {
/// in filo order.
class ThreadPoolExecutor : public Executor {
public:
- explicit ThreadPoolExecutor(unsigned ThreadCount = hardware_concurrency()) {
+ explicit ThreadPoolExecutor(ThreadPoolStrategy S = hardware_concurrency()) {
+ unsigned ThreadCount = S.compute_thread_count();
// Spawn all but one of the threads in another thread as spawning threads
// can take a while.
Threads.reserve(ThreadCount);
Threads.resize(1);
std::lock_guard<std::mutex> Lock(Mutex);
- Threads[0] = std::thread([&, ThreadCount] {
- for (unsigned i = 1; i < ThreadCount; ++i) {
- Threads.emplace_back([=] { work(); });
+ Threads[0] = std::thread([this, ThreadCount, S] {
+ for (unsigned I = 1; I < ThreadCount; ++I) {
+ Threads.emplace_back([=] { work(S, I); });
if (Stop)
break;
}
ThreadsCreated.set_value();
- work();
+ work(S, 0);
});
}
@@ -90,7 +91,8 @@ class ThreadPoolExecutor : public Executor {
}
private:
- void work() {
+ void work(ThreadPoolStrategy S, unsigned ThreadID) {
+ S.apply_thread_strategy(ThreadID);
while (true) {
std::unique_lock<std::mutex> Lock(Mutex);
Cond.wait(Lock, [&] { return Stop || !WorkStack.empty(); });
diff --git a/llvm/lib/Support/ThreadPool.cpp b/llvm/lib/Support/ThreadPool.cpp
index 40982d777914..5aa5815d7272 100644
--- a/llvm/lib/Support/ThreadPool.cpp
+++ b/llvm/lib/Support/ThreadPool.cpp
@@ -20,16 +20,15 @@ using namespace llvm;
#if LLVM_ENABLE_THREADS
-// Default to hardware_concurrency
-ThreadPool::ThreadPool() : ThreadPool(hardware_concurrency()) {}
-
-ThreadPool::ThreadPool(unsigned ThreadCount)
- : ActiveThreads(0), EnableFlag(true) {
+ThreadPool::ThreadPool(ThreadPoolStrategy S)
+ : ActiveThreads(0), EnableFlag(true),
+ ThreadCount(S.compute_thread_count()) {
// Create ThreadCount threads that will loop forever, wait on QueueCondition
// for tasks to be queued or the Pool to be destroyed.
Threads.reserve(ThreadCount);
for (unsigned ThreadID = 0; ThreadID < ThreadCount; ++ThreadID) {
- Threads.emplace_back([&] {
+ Threads.emplace_back([S, ThreadID, this] {
+ S.apply_thread_strategy(ThreadID);
while (true) {
PackagedTaskTy Task;
{
@@ -108,12 +107,10 @@ ThreadPool::~ThreadPool() {
#else // LLVM_ENABLE_THREADS Disabled
-ThreadPool::ThreadPool() : ThreadPool(0) {}
-
// No threads are launched, issue a warning if ThreadCount is not 0
-ThreadPool::ThreadPool(unsigned ThreadCount)
- : ActiveThreads(0) {
- if (ThreadCount) {
+ThreadPool::ThreadPool(ThreadPoolStrategy S)
+ : ActiveThreads(0), ThreadCount(S.compute_thread_count()) {
+ if (ThreadCount != 1) {
errs() << "Warning: request a ThreadPool with " << ThreadCount
<< " threads, but LLVM_ENABLE_THREADS has been turned off\n";
}
@@ -138,8 +135,6 @@ std::shared_future<void> ThreadPool::asyncImpl(TaskTy Task) {
return Future;
}
-ThreadPool::~ThreadPool() {
- wait();
-}
+ThreadPool::~ThreadPool() { wait(); }
#endif
diff --git a/llvm/lib/Support/Threading.cpp b/llvm/lib/Support/Threading.cpp
index 48750cef5ec2..de5adaddd9d3 100644
--- a/llvm/lib/Support/Threading.cpp
+++ b/llvm/lib/Support/Threading.cpp
@@ -45,10 +45,6 @@ void llvm::llvm_execute_on_thread(void (*Fn)(void *), void *UserData,
Fn(UserData);
}
-unsigned llvm::heavyweight_hardware_concurrency() { return 1; }
-
-unsigned llvm::hardware_concurrency() { return 1; }
-
uint64_t llvm::get_threadid() { return 0; }
uint32_t llvm::get_max_thread_name_length() { return 0; }
@@ -57,6 +53,13 @@ void llvm::set_thread_name(const Twine &Name) {}
void llvm::get_thread_name(SmallVectorImpl<char> &Name) { Name.clear(); }
+llvm::BitVector llvm::get_thread_affinity_mask() { return {}; }
+
+unsigned llvm::ThreadPoolStrategy::compute_thread_count() const {
+ // When threads are disabled, ensure clients will loop at least once.
+ return 1;
+}
+
#if LLVM_ENABLE_THREADS == 0
void llvm::llvm_execute_on_thread_async(
llvm::unique_function<void()> Func,
@@ -78,30 +81,19 @@ void llvm::llvm_execute_on_thread_async(
#else
-#include <thread>
-unsigned llvm::heavyweight_hardware_concurrency() {
- // Since we can't get here unless LLVM_ENABLE_THREADS == 1, it is safe to use
- // `std::thread` directly instead of `llvm::thread` (and indeed, doing so
- // allows us to not define `thread` in the llvm namespace, which conflicts
- // with some platforms such as FreeBSD whose headers also define a struct
- // called `thread` in the global namespace which can cause ambiguity due to
- // ADL.
- int NumPhysical = sys::getHostNumPhysicalCores();
- if (NumPhysical == -1)
- return std::thread::hardware_concurrency();
- return NumPhysical;
-}
+int computeHostNumHardwareThreads();
-unsigned llvm::hardware_concurrency() {
-#if defined(HAVE_SCHED_GETAFFINITY) && defined(HAVE_CPU_COUNT)
- cpu_set_t Set;
- if (sched_getaffinity(0, sizeof(Set), &Set))
- return CPU_COUNT(&Set);
-#endif
- // Guard against std::thread::hardware_concurrency() returning 0.
- if (unsigned Val = std::thread::hardware_concurrency())
- return Val;
- return 1;
+unsigned llvm::ThreadPoolStrategy::compute_thread_count() const {
+ int MaxThreadCount = UseHyperThreads ? computeHostNumHardwareThreads()
+ : sys::getHostNumPhysicalCores();
+ if (MaxThreadCount <= 0)
+ MaxThreadCount = 1;
+
+ // No need to create more threads than there are hardware threads, it would
+ // uselessly induce more context-switching and cache eviction.
+ if (!ThreadsRequested || ThreadsRequested > (unsigned)MaxThreadCount)
+ return MaxThreadCount;
+ return ThreadsRequested;
}
namespace {
diff --git a/llvm/lib/Support/Unix/Threading.inc b/llvm/lib/Support/Unix/Threading.inc
index afb887fc1096..8cacaa83e961 100644
--- a/llvm/lib/Support/Unix/Threading.inc
+++ b/llvm/lib/Support/Unix/Threading.inc
@@ -267,3 +267,27 @@ SetThreadPriorityResult llvm::set_thread_priority(ThreadPriority Priority) {
#endif
return SetThreadPriorityResult::FAILURE;
}
+
+#include <thread>
+
+int computeHostNumHardwareThreads() {
+#if defined(HAVE_SCHED_GETAFFINITY) && defined(HAVE_CPU_COUNT)
+ cpu_set_t Set;
+ if (sched_getaffinity(0, sizeof(Set), &Set))
+ return CPU_COUNT(&Set);
+#endif
+ // Guard against std::thread::hardware_concurrency() returning 0.
+ if (unsigned Val = std::thread::hardware_concurrency())
+ return Val;
+ return 1;
+}
+
+void llvm::ThreadPoolStrategy::apply_thread_strategy(
+ unsigned ThreadPoolNum) const {}
+
+llvm::BitVector llvm::get_thread_affinity_mask() {
+ // FIXME: Implement
+ llvm_unreachable("Not implemented!");
+}
+
+unsigned llvm::get_cpus() { return 1; }
diff --git a/llvm/lib/Support/Windows/Threading.inc b/llvm/lib/Support/Windows/Threading.inc
index 9456efa686ff..eb9229621226 100644
--- a/llvm/lib/Support/Windows/Threading.inc
+++ b/llvm/lib/Support/Windows/Threading.inc
@@ -16,6 +16,8 @@
#include "WindowsSupport.h"
#include <process.h>
+#include <bitset>
+
// Windows will at times define MemoryFence.
#ifdef MemoryFence
#undef MemoryFence
@@ -122,3 +124,163 @@ SetThreadPriorityResult llvm::set_thread_priority(ThreadPriority Priority) {
? SetThreadPriorityResult::SUCCESS
: SetThreadPriorityResult::FAILURE;
}
+
+struct ProcessorGroup {
+ unsigned ID;
+ unsigned AllThreads;
+ unsigned UsableThreads;
+ unsigned ThreadsPerCore;
+ uint64_t Affinity;
+};
+
+template <typename F>
+static bool IterateProcInfo(LOGICAL_PROCESSOR_RELATIONSHIP Relationship, F Fn) {
+ DWORD Len = 0;
+ BOOL R = ::GetLogicalProcessorInformationEx(Relationship, NULL, &Len);
+ if (R || GetLastError() != ERROR_INSUFFICIENT_BUFFER) {
+ return false;
+ }
+ auto *Info = (SYSTEM_LOGICAL_PROCESSOR_INFORMATION_EX *)calloc(1, Len);
+ R = ::GetLogicalProcessorInformationEx(Relationship, Info, &Len);
+ if (R) {
+ auto *End =
+ (SYSTEM_LOGICAL_PROCESSOR_INFORMATION_EX *)((uint8_t *)Info + Len);
+ for (auto *Curr = Info; Curr < End;
+ Curr = (SYSTEM_LOGICAL_PROCESSOR_INFORMATION_EX *)((uint8_t *)Curr +
+ Curr->Size)) {
+ if (Curr->Relationship != Relationship)
+ continue;
+ Fn(Curr);
+ }
+ }
+ free(Info);
+ return true;
+}
+
+static ArrayRef<ProcessorGroup> getProcessorGroups() {
+ auto computeGroups = []() {
+ SmallVector<ProcessorGroup, 4> Groups;
+
+ auto HandleGroup = [&](SYSTEM_LOGICAL_PROCESSOR_INFORMATION_EX *ProcInfo) {
+ GROUP_RELATIONSHIP &El = ProcInfo->Group;
+ for (unsigned J = 0; J < El.ActiveGroupCount; ++J) {
+ ProcessorGroup G;
+ G.ID = Groups.size();
+ G.AllThreads = El.GroupInfo[J].MaximumProcessorCount;
+ G.UsableThreads = El.GroupInfo[J].ActiveProcessorCount;
+ assert(G.UsableThreads <= 64);
+ G.Affinity = El.GroupInfo[J].ActiveProcessorMask;
+ Groups.push_back(G);
+ }
+ };
+
+ if (!IterateProcInfo(RelationGroup, HandleGroup))
+ return std::vector<ProcessorGroup>();
+
+ auto HandleProc = [&](SYSTEM_LOGICAL_PROCESSOR_INFORMATION_EX *ProcInfo) {
+ PROCESSOR_RELATIONSHIP &El = ProcInfo->Processor;
+ assert(El.GroupCount == 1);
+ unsigned NumHyperThreads = 1;
+ // If the flag is set, each core supports more than one hyper-thread.
+ if (El.Flags & LTP_PC_SMT)
+ NumHyperThreads = std::bitset<64>(El.GroupMask[0].Mask).count();
+ unsigned I = El.GroupMask[0].Group;
+ Groups[I].ThreadsPerCore = NumHyperThreads;
+ };
+
+ if (!IterateProcInfo(RelationProcessorCore, HandleProc))
+ return std::vector<ProcessorGroup>();
+
+ // If there's an affinity mask set on one of the CPUs, then assume the user
+ // wants to constrain the current process to only a single CPU.
+ for (auto &G : Groups) {
+ if (G.UsableThreads != G.AllThreads) {
+ ProcessorGroup NewG{G};
+ Groups.clear();
+ Groups.push_back(NewG);
+ break;
+ }
+ }
+
+ return std::vector<ProcessorGroup>(Groups.begin(), Groups.end());
+ };
+ static auto Groups = computeGroups();
+ return ArrayRef<ProcessorGroup>(Groups);
+}
+
+template <typename R, typename UnaryPredicate>
+static unsigned aggregate(R &&Range, UnaryPredicate P) {
+ unsigned I{};
+ for (const auto &It : Range)
+ I += P(It);
+ return I;
+}
+
+// for sys::getHostNumPhysicalCores
+int computeHostNumPhysicalCores() {
+ static unsigned Cores =
+ aggregate(getProcessorGroups(), [](const ProcessorGroup &G) {
+ return G.UsableThreads / G.ThreadsPerCore;
+ });
+ return Cores;
+}
+
+int computeHostNumHardwareThreads() {
+ static unsigned Threads =
+ aggregate(getProcessorGroups(),
+ [](const ProcessorGroup &G) { return G.UsableThreads; });
+ return Threads;
+}
+
+// Assign the current thread to a more appropriate CPU socket or CPU group
+void llvm::ThreadPoolStrategy::apply_thread_strategy(
+ unsigned ThreadPoolNum) const {
+ ArrayRef<ProcessorGroup> Groups = getProcessorGroups();
+
+ assert(ThreadPoolNum < compute_thread_count() &&
+ "The thread index is not within thread strategy's range!");
+
+ // In this mode, the ThreadNumber represents the core number, not the
+ // hyper-thread number. Assumes all NUMA groups have the same amount of
+ // hyper-threads.
+ if (!UseHyperThreads)
+ ThreadPoolNum *= Groups[0].ThreadsPerCore;
+
+ unsigned ThreadRangeStart = 0;
+ for (unsigned I = 0; I < Groups.size(); ++I) {
+ const ProcessorGroup &G = Groups[I];
+ if (ThreadPoolNum >= ThreadRangeStart &&
+ ThreadPoolNum < ThreadRangeStart + G.UsableThreads) {
+
+ GROUP_AFFINITY Affinity{};
+ Affinity.Group = G.ID;
+ Affinity.Mask = G.Affinity;
+ SetThreadGroupAffinity(GetCurrentThread(), &Affinity, nullptr);
+ }
+ ThreadRangeStart += G.UsableThreads;
+ }
+}
+
+llvm::BitVector llvm::get_thread_affinity_mask() {
+ GROUP_AFFINITY Affinity{};
+ GetThreadGroupAffinity(GetCurrentThread(), &Affinity);
+
+ static unsigned All =
+ aggregate(getProcessorGroups(),
+ [](const ProcessorGroup &G) { return G.AllThreads; });
+
+ unsigned StartOffset =
+ aggregate(getProcessorGroups(), [&](const ProcessorGroup &G) {
+ return G.ID < Affinity.Group ? G.AllThreads : 0;
+ });
+
+ llvm::BitVector V;
+ V.resize(All);
+ for (unsigned I = 0; I < sizeof(KAFFINITY) * 8; ++I) {
+ if ((Affinity.Mask >> I) & 1)
+ V.set(StartOffset + I);
+ }
+ return V;
+}
+
+unsigned llvm::get_cpus() { return getProcessorGroups().size(); }
diff --git a/llvm/tools/dsymutil/dsymutil.cpp b/llvm/tools/dsymutil/dsymutil.cpp
index 6b2608160aea..54adeaa11c1a 100644
--- a/llvm/tools/dsymutil/dsymutil.cpp
+++ b/llvm/tools/dsymutil/dsymutil.cpp
@@ -258,7 +258,7 @@ static Expected<DsymutilOptions> getOptions(opt::InputArgList &Args) {
if (opt::Arg *NumThreads = Args.getLastArg(OPT_threads))
Options.LinkOpts.Threads = atoi(NumThreads->getValue());
else
- Options.LinkOpts.Threads = thread::hardware_concurrency();
+ Options.LinkOpts.Threads = 0; // Use all available hardware threads
if (Options.DumpDebugMap || Options.LinkOpts.Verbose)
Options.LinkOpts.Threads = 1;
@@ -541,9 +541,10 @@ int main(int argc, char **argv) {
// Shared a single binary holder for all the link steps.
BinaryHolder BinHolder;
- unsigned ThreadCount =
- std::min<unsigned>(Options.LinkOpts.Threads, DebugMapPtrsOrErr->size());
- ThreadPool Threads(ThreadCount);
+ unsigned ThreadCount = Options.LinkOpts.Threads;
+ if (!ThreadCount)
+ ThreadCount = DebugMapPtrsOrErr->size();
+ ThreadPool Threads(hardware_concurrency(ThreadCount));
// If there is more than one link to execute, we need to generate
// temporary files.
diff --git a/llvm/tools/gold/gold-plugin.cpp b/llvm/tools/gold/gold-plugin.cpp
index f68f7183e034..2a9398fed9f3 100644
--- a/llvm/tools/gold/gold-plugin.cpp
+++ b/llvm/tools/gold/gold-plugin.cpp
@@ -134,8 +134,8 @@ namespace options {
static unsigned OptLevel = 2;
// Default parallelism of 0 used to indicate that user did not specify.
// Actual parallelism default value depends on implementation.
- // Currently only affects ThinLTO, where the default is
- // llvm::heavyweight_hardware_concurrency.
+ // Currently only affects ThinLTO, where the default is the max cores in the
+ // system.
static unsigned Parallelism = 0;
// Default regular LTO codegen parallelism (number of partitions).
static unsigned ParallelCodeGenParallelismLevel = 1;
diff --git a/llvm/tools/llvm-cov/CodeCoverage.cpp b/llvm/tools/llvm-cov/CodeCoverage.cpp
index 52e9958e92da..625e2342e4bb 100644
--- a/llvm/tools/llvm-cov/CodeCoverage.cpp
+++ b/llvm/tools/llvm-cov/CodeCoverage.cpp
@@ -947,9 +947,7 @@ int CodeCoverageTool::doShow(int argc, const char **argv,
// If NumThreads is not specified, auto-detect a good default.
if (NumThreads == 0)
- NumThreads =
- std::max(1U, std::min(llvm::heavyweight_hardware_concurrency(),
- unsigned(SourceFiles.size())));
+ NumThreads = SourceFiles.size();
if (!ViewOpts.hasOutputDirectory() || NumThreads == 1) {
for (const std::string &SourceFile : SourceFiles)
@@ -957,7 +955,7 @@ int CodeCoverageTool::doShow(int argc, const char **argv,
ShowFilenames);
} else {
// In -output-dir mode, it's safe to use multiple threads to print files.
- ThreadPool Pool(NumThreads);
+ ThreadPool Pool(heavyweight_hardware_concurrency(NumThreads));
for (const std::string &SourceFile : SourceFiles)
Pool.async(&CodeCoverageTool::writeSourceFileView, this, SourceFile,
Coverage.get(), Printer.get(), ShowFilenames);
diff --git a/llvm/tools/llvm-cov/CoverageExporterJson.cpp b/llvm/tools/llvm-cov/CoverageExporterJson.cpp
index 216b5e3fd226..ba8ff5c8fe52 100644
--- a/llvm/tools/llvm-cov/CoverageExporterJson.cpp
+++ b/llvm/tools/llvm-cov/CoverageExporterJson.cpp
@@ -163,11 +163,9 @@ json::Array renderFiles(const coverage::CoverageMapping &Coverage,
ArrayRef<FileCoverageSummary> FileReports,
const CoverageViewOptions &Options) {
auto NumThreads = Options.NumThreads;
- if (NumThreads == 0) {
- NumThreads = std::max(1U, std::min(llvm::heavyweight_hardware_concurrency(),
- unsigned(SourceFiles.size())));
- }
- ThreadPool Pool(NumThreads);
+ if (NumThreads == 0)
+ NumThreads = SourceFiles.size();
+ ThreadPool Pool(heavyweight_hardware_concurrency(NumThreads));
json::Array FileArray;
std::mutex FileArrayMutex;
diff --git a/llvm/tools/llvm-cov/CoverageReport.cpp b/llvm/tools/llvm-cov/CoverageReport.cpp
index 82259542c597..187e2dc4f553 100644
--- a/llvm/tools/llvm-cov/CoverageReport.cpp
+++ b/llvm/tools/llvm-cov/CoverageReport.cpp
@@ -356,11 +356,8 @@ std::vector<FileCoverageSummary> CoverageReport::prepareFileReports(
// If NumThreads is not specified, auto-detect a good default.
if (NumThreads == 0)
- NumThreads =
- std::max(1U, std::min(llvm::heavyweight_hardware_concurrency(),
- unsigned(Files.size())));
-
- ThreadPool Pool(NumThreads);
+ NumThreads = Files.size();
+ ThreadPool Pool(heavyweight_hardware_concurrency(NumThreads));
std::vector<FileCoverageSummary> FileReports;
FileReports.reserve(Files.size());
diff --git a/llvm/tools/llvm-lto2/llvm-lto2.cpp b/llvm/tools/llvm-lto2/llvm-lto2.cpp
index 8deedd49e050..fc86fd969efb 100644
--- a/llvm/tools/llvm-lto2/llvm-lto2.cpp
+++ b/llvm/tools/llvm-lto2/llvm-lto2.cpp
@@ -65,8 +65,8 @@ static cl::opt<bool>
"import files for the "
"distributed backend case"));
-static cl::opt<int> Threads("thinlto-threads",
- cl::init(llvm::heavyweight_hardware_concurrency()));
+// Default to using all hardware cores in the system.
+static cl::opt<int> Threads("thinlto-threads", cl::init(0));
static cl::list<std::string> SymbolResolutions(
"r",
diff --git a/llvm/tools/llvm-profdata/llvm-profdata.cpp b/llvm/tools/llvm-profdata/llvm-profdata.cpp
index 424edf446d03..f05c7e637cd5 100644
--- a/llvm/tools/llvm-profdata/llvm-profdata.cpp
+++ b/llvm/tools/llvm-profdata/llvm-profdata.cpp
@@ -307,8 +307,11 @@ static void mergeInstrProfile(const WeightedFileVector &Inputs,
// If NumThreads is not specified, auto-detect a good default.
if (NumThreads == 0)
- NumThreads =
- std::min(hardware_concurrency(), unsigned((Inputs.size() + 1) / 2));
+ NumThreads = std::min(hardware_concurrency().compute_thread_count(),
+ unsigned((Inputs.size() + 1) / 2));
+ // FIXME: There's a bug here, where setting NumThreads = Inputs.size() fails
+ // the merge_empty_profile.test because the InstrProfWriter.ProfileKind isn't
+ // merged, thus the emitted file ends up with a PF_Unknown kind.
// Initialize the writer contexts.
SmallVector<std::unique_ptr<WriterContext>, 4> Contexts;
@@ -320,7 +323,7 @@ static void mergeInstrProfile(const WeightedFileVector &Inputs,
for (const auto &Input : Inputs)
loadInput(Input, Remapper, Contexts[0].get());
} else {
- ThreadPool Pool(NumThreads);
+ ThreadPool Pool(hardware_concurrency(NumThreads));
// Load the inputs in parallel (N/NumThreads serial steps).
unsigned Ctx = 0;
diff --git a/llvm/unittests/Support/Host.cpp b/llvm/unittests/Support/Host.cpp
index 2c17a5094101..62252347d62a 100644
--- a/llvm/unittests/Support/Host.cpp
+++ b/llvm/unittests/Support/Host.cpp
@@ -37,7 +37,8 @@ class HostTest : public testing::Test {
// Initially this is only testing detection of the number of
// physical cores, which is currently only supported/tested for
// x86_64 Linux and Darwin.
- return (Host.getArch() == Triple::x86_64 &&
+ return Host.isOSWindows() ||
+ (Host.getArch() == Triple::x86_64 &&
(Host.isOSDarwin() || Host.getOS() == Triple::Linux));
}
diff --git a/llvm/unittests/Support/TaskQueueTest.cpp b/llvm/unittests/Support/TaskQueueTest.cpp
index 0a8aeca4e2d6..4d8c3e4064b4 100644
--- a/llvm/unittests/Support/TaskQueueTest.cpp
+++ b/llvm/unittests/Support/TaskQueueTest.cpp
@@ -22,7 +22,7 @@ class TaskQueueTest : public testing::Test {
};
TEST_F(TaskQueueTest, OrderedFutures) {
- ThreadPool TP(1);
+ ThreadPool TP(hardware_concurrency(1));
TaskQueue TQ(TP);
std::atomic<int> X{ 0 };
std::atomic<int> Y{ 0 };
@@ -66,7 +66,7 @@ TEST_F(TaskQueueTest, OrderedFutures) {
}
TEST_F(TaskQueueTest, UnOrderedFutures) {
- ThreadPool TP(1);
+ ThreadPool TP(hardware_concurrency(1));
TaskQueue TQ(TP);
std::atomic<int> X{ 0 };
std::atomic<int> Y{ 0 };
@@ -96,7 +96,7 @@ TEST_F(TaskQueueTest, UnOrderedFutures) {
}
TEST_F(TaskQueueTest, FutureWithReturnValue) {
- ThreadPool TP(1);
+ ThreadPool TP(hardware_concurrency(1));
TaskQueue TQ(TP);
std::future<std::string> F1 = TQ.async([&] { return std::string("Hello"); });
std::future<int> F2 = TQ.async([&] { return 42; });
diff --git a/llvm/unittests/Support/ThreadPool.cpp b/llvm/unittests/Support/ThreadPool.cpp
index a16adbbb78a7..237be875909b 100644
--- a/llvm/unittests/Support/ThreadPool.cpp
+++ b/llvm/unittests/Support/ThreadPool.cpp
@@ -8,11 +8,13 @@
#include "llvm/Support/ThreadPool.h"
+#include "llvm/ADT/DenseSet.h"
#include "llvm/ADT/STLExtras.h"
#include "llvm/ADT/SmallVector.h"
#include "llvm/ADT/Triple.h"
#include "llvm/Support/Host.h"
#include "llvm/Support/TargetSelect.h"
+#include "llvm/Support/Threading.h"
#include "gtest/gtest.h"
@@ -69,6 +71,8 @@ class ThreadPoolTest : public testing::Test {
void SetUp() override { MainThreadReady = false; }
+ void TestAllThreads(ThreadPoolStrategy S);
+
std::condition_variable WaitMainThread;
std::mutex WaitMainThreadMutex;
bool MainThreadReady = false;
@@ -131,7 +135,7 @@ TEST_F(ThreadPoolTest, Async) {
TEST_F(ThreadPoolTest, GetFuture) {
CHECK_UNSUPPORTED();
- ThreadPool Pool{2};
+ ThreadPool Pool(hardware_concurrency(2));
std::atomic_int i{0};
Pool.async([this, &i] {
waitForMainThread();
@@ -162,3 +166,45 @@ TEST_F(ThreadPoolTest, PoolDestruction) {
}
ASSERT_EQ(5, checked_in);
}
+
+#if LLVM_ENABLE_THREADS == 1
+
+void ThreadPoolTest::TestAllThreads(ThreadPoolStrategy S) {
+ // FIXME: Skip these tests on non-Windows because multi-socket system were not
+ // tested on Unix yet, and llvm::get_thread_affinity_mask() isn't implemented
+ // for Unix.
+ Triple Host(Triple::normalize(sys::getProcessTriple()));
+ if (!Host.isOSWindows())
+ return;
+
+ llvm::DenseSet<llvm::BitVector> ThreadsUsed;
+ std::mutex Lock;
+ unsigned Threads = 0;
+ {
+ ThreadPool Pool(S);
+ Threads = Pool.getThreadCount();
+ for (size_t I = 0; I < 10000; ++I) {
+ Pool.async([&] {
+ waitForMainThread();
+ std::lock_guard<std::mutex> Guard(Lock);
+ auto Mask = llvm::get_thread_affinity_mask();
+ ThreadsUsed.insert(Mask);
+ });
+ }
+ ASSERT_EQ(true, ThreadsUsed.empty());
+ setMainThreadReady();
+ }
+ ASSERT_EQ(llvm::get_cpus(), ThreadsUsed.size());
+}
+
+TEST_F(ThreadPoolTest, AllThreads_UseAllRessources) {
+ CHECK_UNSUPPORTED();
+ TestAllThreads({});
+}
+
+TEST_F(ThreadPoolTest, AllThreads_OneThreadPerCore) {
+ CHECK_UNSUPPORTED();
+ TestAllThreads(llvm::heavyweight_hardware_concurrency());
+}
+
+#endif
diff --git a/llvm/unittests/Support/Threading.cpp b/llvm/unittests/Support/Threading.cpp
index 183c9aa7d71c..c76e6e4a5bd1 100644
--- a/llvm/unittests/Support/Threading.cpp
+++ b/llvm/unittests/Support/Threading.cpp
@@ -21,7 +21,8 @@ TEST(Threading, PhysicalConcurrency) {
auto Num = heavyweight_hardware_concurrency();
// Since Num is unsigned this will also catch us trying to
// return -1.
- ASSERT_LE(Num, thread::hardware_concurrency());
+ ASSERT_LE(Num.compute_thread_count(),
+ hardware_concurrency().compute_thread_count());
}
#if LLVM_ENABLE_THREADS
diff --git a/mlir/lib/Pass/Pass.cpp b/mlir/lib/Pass/Pass.cpp
index fc40ff18ce06..41adb623e74d 100644
--- a/mlir/lib/Pass/Pass.cpp
+++ b/mlir/lib/Pass/Pass.cpp
@@ -411,7 +411,8 @@ void OpToOpPassAdaptorParallel::runOnOperation() {
// Create the async executors if they haven't been created, or if the main
// pipeline has changed.
if (asyncExecutors.empty() || hasSizeMismatch(asyncExecutors.front(), mgrs))
- asyncExecutors.assign(llvm::hardware_concurrency(), mgrs);
+ asyncExecutors.assign(llvm::hardware_concurrency().compute_thread_count(),
+ mgrs);
// Run a prepass over the module to collect the operations to execute over.
// This ensures that an analysis manager exists for each operation, as well as
More information about the cfe-commits
mailing list