[PATCH] D105506: [LLD] [COFF] Avoid thread exhaustion on 32-bit Windows host

[Jeremy Drake via Phabricator via llvm-commits]

jeremyd2019 created this revision.
jeremyd2019 added reviewers: mstorsjo, rnk.
jeremyd2019 added a project: lld.
jeremyd2019 requested review of this revision.
Herald added a project: LLVM.

LLD on 32-bit Windows would frequently fail on large projects with an exception "thread constructor failed: Exec format error".  The stack trace pointed to this usage of std::async, and looking at the implementation in libc++ it seems using std::async with std::launch::async results in the immediate creation of a new thread for every call.  This could result in a potentially unbounded number of threads, depending on the number of input files.  This seems to be hitting some limit in 32-bit Windows host.

I took the easy route, and only use threads on 64-bit Windows, not all Windows as before.  I was thinking a more proper solution might involve using a thread pool rather than blindly spawning any number of new threads, but that may have other unforeseen consequences.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D105506

Files:
  lld/COFF/Driver.cpp


Index: lld/COFF/Driver.cpp
===================================================================
--- lld/COFF/Driver.cpp
+++ lld/COFF/Driver.cpp
@@ -149,9 +149,10 @@
 // Create a std::future that opens and maps a file using the best strategy for
 // the host platform.
 static std::future<MBErrPair> createFutureForFile(std::string path) {
-#if _WIN32
+#if _WIN64
   // On Windows, file I/O is relatively slow so it is best to do this
-  // asynchronously.
+  // asynchronously.  But 32-bit has issues with potentially launching tons
+  // of threads
   auto strategy = std::launch::async;
 #else
   auto strategy = std::launch::deferred;


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D105506.356799.patch
Type: text/x-patch
Size: 637 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20210706/705320a8/attachment.bin>