[llvm] [CMake][LIT] Add option to run lit testsuites in parallel (PR #82899)

[Mehdi Amini via llvm-commits]

joker-eph wrote:

> My use-case is CI builds, where we are running the whole build + all the tests, with the goal of being as fast as possible.

This is weird, your motivation stated earlier was "I want to run a very small set of the tests", I'm having a hard time following now what is the motivating example really!


> It artificially splits the build into first a compile phase, then a test phase. This means that tests which could be running against early compilation targets get delayed until the end of the build

This is true in theory, however in practice in LLVM the check targets depends on all the binaries needed for **all** the tests (unless you have many lit test suites in the first place, but I'd like to see an example). Another problem is that running the tests in parallel concurrently with the build is again oversubscribing the machine.

> It doesn't allow flexibility for different sizes of tests. i.e. N small unit tests would have poor utilization of available cpu/mem/etc. while N larger end-to-end integration tests could run into the oversubscribing issue you've described

I don't understand your point about small/large? As long as each test is single threader, it does not matter if they are small or large.

> The wins I am seeing come from moving parallelism logic out of lit and into Ninja. So we end up with a much smoother build graph because Ninja knows more about the build than lit does.

If you want to do this in a principle way, then we should 
1) Discover the lit tests targets using lit
2) Feed these individual tests to ninja for scheduling and let ninja do the scheduling. 

If you care about solving this properly, I'd be happy to see something in this direction, however this isn't the granularity you operate at right now.

I rather see this reverted and a more principled approach discussed first.



https://github.com/llvm/llvm-project/pull/82899