[llvm] Reland "[lit] Migrate lit to ProcessPoolExecutor (#202681)" (PR #209076)

[Paul Kirth via llvm-commits]

ilovepi wrote:

> I noticed check-llvm has seemed slower for me recently and found this PR increases testing time by ~56% (~46s to ~72s). Did you measure performance?
> 
> FWIW this is on AArch64 (64 core Graviton 2), build config is here: https://github.com/c-rhodes/llvm-tools/blob/d78ef649b5445d853764b2d772c3c3d63388a5c8/configure-llvm-build.sh#L72-L85
> 
> ```
> ➜  llvm-project git:(main) ninja -C build check-llvm
> ninja: Entering directory `build'
> [291/292] Running the LLVM regression tests
> 
> Testing Time: 72.98s
> 
> Total Discovered Tests: 76323
>   Skipped          :   241 (0.32%)
>   Unsupported      : 41091 (53.84%)
>   Passed           : 34941 (45.78%)
>   Expectedly Failed:    50 (0.07%)
> 
> ➜  llvm-project git:(main) git revert 14bfc424ae77976e05582ff38896caec399f2837
> [main c5b6a7a31772] Revert "Reland "[lit] Migrate lit to ProcessPoolExecutor (#202681)" (#209076)"
>  2 files changed, 48 insertions(+), 172 deletions(-)
> 
> ➜  llvm-project git:(main) ninja -C build check-llvm
> ninja: Entering directory `build'
> [290/291] Running the LLVM regression tests
> 
> Testing Time: 46.70s
> 
> Total Discovered Tests: 76323
>   Skipped          :   241 (0.32%)
>   Unsupported      : 41091 (53.84%)
>   Passed           : 34941 (45.78%)
>   Expectedly Failed:    50 (0.07%)
> ```

Yes. it was tested on Linux, Mac, and Windows on some combination of x86_64 and arm64 cpus. 

@prasoon054 do you still have the numbers.? IIRC this was basically neutral in your evaluation. I thought we had numbers on the original PR about it, but I'm not seeing it.

https://github.com/llvm/llvm-project/pull/209076