[llvm] [lit] Migrate lit to ProcessPoolExecutor (PR #202681)

[via llvm-commits]

================
@@ -71,6 +82,38 @@ def execute(self):
                 if test.result is None:
                     test.setResult(skipped)
 
+    def _abort_executors(self, executors):
+        """SIGKILL all workers on abort (ctrl-C, --max-failures, --max-time,
+        worker crash). Pre-3.14 ProcessPoolExecutor has no force-stop."""
+        try:
+            # Killing worker processes can corrupt the executor's queues, which makes it
+            # unsafe for its atexit hooks to join their threads. Disable those hooks
+            # before terminating workers (a second ctrl-C should not bypass this cleanup).
+            # This applies to call-queue feeder threads and management threads.
+            # Otherwise, a thread blocked on a partially written pipe may require multiple
+            # ctrl-C to unblock.
+            # See: https://github.com/python/cpython/issues/125886
+            # These threads are daemonic on Python 3.8, so disabling them is harmless.
+            for ex in executors:
+                if hasattr(ex, "_call_queue") and ex._call_queue is not None:
+                    ex._call_queue.cancel_join_thread()
+            if hasattr(concurrent.futures.process, "_threads_wakeups"):
+                concurrent.futures.process._threads_wakeups.clear()
+            tree_kill_ok, _ = lit.util.killProcessAndChildrenIsSupported()
+            for ex in executors:
+                for pid, proc in list((ex.processes or {}).items()):
+                    if tree_kill_ok:
+                        lit.util.killProcessAndChildren(pid)
+                    else:
+                        proc.kill()
+            # Avoid cancel() and shutdown() as both can hang after queue corruption.
----------------
prasoon054 wrote:

Done!
Does `lit` run ruff though (I couldn't find `[tool.ruff]`)?
Left `kill_workers()` as a TODO. The tree-kill has to stay to reap the llc / FileCheck grandchildren, so it can't fully replace this block.

https://github.com/llvm/llvm-project/pull/202681