[PATCH] D97046: [lit] Add "late_tests" test suite config option

[James Henderson via Phabricator via llvm-commits]

jhenderson added a comment.

In D97046#2578480 <https://reviews.llvm.org/D97046#2578480>, @thopre wrote:

> In D97046#2578450 <https://reviews.llvm.org/D97046#2578450>, @jhenderson wrote:
>
>> Could you clarify more what the benefit of running the tests later actually is? Is this intended to be useful in conjunction with something else? On the face of it, I'd expect people to run the full testsuite to completion, and then observe the full results. Ultimately, it doesn't matter whether a test runs first or last in this order for that case.
>
> You want long running tests to run first and shorter one last as they will use to balance the imbalance from long running test. It's just to maximize the parallelism. If you have some medium to big tests running at the end, the risk is you are waiting for one core to finish while all the others are sitting idle. If you keep lots of small tests at the end, the cores that are idle will pick from it and hopefully reduce the imbalance.

I get the idea of running long tests early, but from observation, the vast majority of lit tests in the LLVM test suite are very similar in length, lasting less than a second. As such, it doesn't make any significant difference whether they are run late, or in the middle (as long as the long ones are run early). I would assume the same applies as much for these Swift tests as any other equivalent ones. I could see a case if we're suggesting that this feature is for a downstream test suite that is mostly comprised of long and slow tests (i.e. ones that take several seconds each), so that it's easier to mark the ones that are quick, but I don't think that's the case here?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97046/new/

https://reviews.llvm.org/D97046