[llvm] [lit] Migrate lit to ProcessPoolExecutor (PR #202681)

[Douglas Yung via llvm-commits]

dyung wrote:

Have there been any issues reported with this change? I'm not sure that this change is the culprit, but I've been noticing many time outs on one of my MacOS bots where the testing just seems to be hanging and doesn't seem to do anything until it gets auto killed.

Doing a `ps -a` shows the following processes:
```
86387 ttys000    0:07.86 ninja check-all
95075 ttys000    0:00.00 /bin/sh -c cd /Users/buildbot/buildbot-root/aarch64-darwin/build && /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/bin/python3.9 /Users/buildbot/buildbot-root/aarch64-darwin/build/./bin/llvm-lit --verbose --timeout=900 /Users/buildbot/buildbot-root/aarch64-darwin/build/utils/mlgo-utils /Users/buildbot/buildbot-root/aarch64-darwin/build/projects/cross-project-tests /Users/buildbot/buildbot-root/aarch64-darwin/build/tools/lld/test /Users/buildbot/buildbot-root/aarch64-darwin/build/tools/clang/tools/extra/include-cleaner/test /Users/buildbot/buildbot-root/aarch64-darwin/build/tools/clang/tools/extra/clangd/test/../unittests /Users/buildbot/buildbot-root/aarch64-darwin/build/tools/clang/tools/extra/clangd/test /Users/buildbot/buildbot-root/aarch64-darwin/build/tools/clang/tools/extra/test /Users/buildbot/buildbot-root/aarch64-darwin/build/tools/clang/test /Users/buildbot/buildbot-root/aarch64-darwin/build/utils/lit /Users/buildbot/buildbot-root/aarch64-darwin/build/test
95076 ttys000    0:03.18 /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python /Users/buildbot/buildbot-root/aarch64-darwin/build/./bin/llvm-lit --verbose --timeout=900 /Users/buildbot/buildbot-root/aarch64-darwin/build/utils/mlgo-utils /Users/buildbot/buildbot-root/aarch64-darwin/build/projects/cross-project-tests /Users/buildbot/buildbot-root/aarch64-darwin/build/tools/lld/test /Users/buildbot/buildbot-root/aarch64-darwin/build/tools/clang/tools/extra/include-cleaner/test /Users/buildbot/buildbot-root/aarch64-darwin/build/tools/clang/tools/extra/clangd/test/../unittests /Users/buildbot/buildbot-root/aarch64-darwin/build/tools/clang/tools/extra/clangd/test /Users/buildbot/buildbot-root/aarch64-darwin/build/tools/clang/tools/extra/test /Users/buildbot/buildbot-root/aarch64-darwin/build/tools/clang/test /Users/buildbot/buildbot-root/aarch64-darwin/build/utils/lit /Users/buildbot/buildbot-root/aarch64-darwin/build/test
95204 ttys000    0:00.03 /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python -c from multiprocessing.resource_tracker import main;main(5)
95205 ttys000    0:00.07 /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=6, pipe_handle=19) --multiprocessing-fork
95206 ttys000    0:00.09 /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=6, pipe_handle=21) --multiprocessing-fork
95207 ttys000    0:00.17 /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=6, pipe_handle=23) --multiprocessing-fork
95208 ttys000    0:00.10 /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=6, pipe_handle=25) --multiprocessing-fork
95209 ttys000    0:01.18 /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=6, pipe_handle=27) --multiprocessing-fork
95210 ttys000    0:01.20 /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=6, pipe_handle=29) --multiprocessing-fork
95211 ttys000    0:01.49 /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=6, pipe_handle=31) --multiprocessing-fork
95212 ttys000    0:01.19 /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=6, pipe_handle=33) --multiprocessing-fork
95213 ttys000    0:01.22 /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=6, pipe_handle=35) --multiprocessing-fork
95214 ttys000    0:01.22 /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=6, pipe_handle=37) --multiprocessing-fork
95215 ttys000    0:00.10 /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=6, pipe_handle=39) --multiprocessing-fork
95216 ttys000    0:00.18 /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=6, pipe_handle=4
1) --multiprocessing-fork
```
But looking in top, the processes aren't doing anything and seem to just be sitting idle. Any idea what might be happening?

https://github.com/llvm/llvm-project/pull/202681