[llvm] [Support] Join threads when stopping the ThreadPoolExecutor (PR #166054)

[via llvm-commits]

llvmbot wrote:


<!--LLVM PR SUMMARY COMMENT-->

@llvm/pr-subscribers-llvm-support

Author: Maurice Heumann (momo5502)

<details>
<summary>Changes</summary>

When building LLVM as DLL, the `ThreadPoolExecutor` can cause deadlocks on Windows.

The threads wre previously only joined when the destructor of the `ThreadPoolExecutor` is called. This happens when unloading the DLL, when the global destructors are called.

On Windows, `std::thread`s use `FreeLibraryAndExitThread` to cleanup a thread. This requires access to the loader lock, when a thread terminates.
However, when destroying the pool, the loader lock is also held, as the DLL is being unloaded.
If the threads did not end fast enough, the destructor would wait for them to join. At the same time, the threads would wait for the destructor to release the loader lock.

Joining the threads when stopping the pool fixes that, as it ensures the threads are stopped when calling `llvm_shutdown`, outside the loader lock.


---
Full diff: https://github.com/llvm/llvm-project/pull/166054.diff


1 Files Affected:

- (modified) llvm/lib/Support/Parallel.cpp (+7-5) 


``````````diff
diff --git a/llvm/lib/Support/Parallel.cpp b/llvm/lib/Support/Parallel.cpp
index 8e0c724accb36..31b51536aaa9e 100644
--- a/llvm/lib/Support/Parallel.cpp
+++ b/llvm/lib/Support/Parallel.cpp
@@ -82,26 +82,28 @@ class ThreadPoolExecutor : public Executor {
   ThreadPoolExecutor() = delete;
 
   void stop() {
+    std::vector<std::thread> ThreadsToJoin;
+
     {
       std::lock_guard<std::mutex> Lock(Mutex);
-      if (Stop)
+      if (Stop && Threads.empty())
         return;
       Stop = true;
+      ThreadsToJoin.swap(Threads);
     }
     Cond.notify_all();
     ThreadsCreated.get_future().wait();
-  }
 
-  ~ThreadPoolExecutor() override {
-    stop();
     std::thread::id CurrentThreadId = std::this_thread::get_id();
-    for (std::thread &T : Threads)
+    for (std::thread &T : ThreadsToJoin)
       if (T.get_id() == CurrentThreadId)
         T.detach();
       else
         T.join();
   }
 
+  ~ThreadPoolExecutor() override { stop(); }
+
   struct Creator {
     static void *call() { return new ThreadPoolExecutor(strategy); }
   };

``````````

</details>


https://github.com/llvm/llvm-project/pull/166054