[PATCH] D97046: [lit] Generalize `early_tests`

[David Zarzycki via Phabricator via llvm-commits]

davezarzycki abandoned this revision.
davezarzycki added a comment.

In D97046#2609772 <https://reviews.llvm.org/D97046#2609772>, @jdenny wrote:

> In D97046#2609763 <https://reviews.llvm.org/D97046#2609763>, @davezarzycki wrote:
>
>> I have a patch that implements the record-and-reorder feature. It turned out to be simpler than I feared. Now I just need to figure out how to write a test for it that makes sense and is reliable. For `ninja check-llvm`, the record-and-reorder mode completes the test suite 51% faster on my 48-core machine. I think it's time to abandon this change proposal.
>
> Nice!  Thanks for all your work.
>
> If I understand how the record-and-reorder feature works, I assume you can test the record and reorder phases separately.  The reorder phase should be easy to test with -j1.  The record phase is probably harder to test because it depends on test times.  You could at least check that it produces a sanely formatted record.  I suppose it might be possible to also have one test sleep for a long time and then check that it is ordered before one that terminates immediately, but I'm not sure how brittle that will be on the bots, and it will slow down lit's test suite.

Correct and correct. As with all tests that rely on timing, a sufficiently busy/slow machine will break whatever timing assumptions are made.

I'll finish the work and create a new Phab review tomorrow.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97046/new/

https://reviews.llvm.org/D97046