[libcxx-commits] [PATCH] D98179: [lit] Sort test start times based on prior test timing data

Nuno Lopes via Phabricator via libcxx-commits libcxx-commits at lists.llvm.org
Sat Mar 20 04:25:31 PDT 2021


nlopes added a comment.

I'm talking about sorting just the summary of failed tests, not the whole output. We need the whole -vv output, but that can be out of order.

Why are timeouts important? Our use case is running Alive2 with the test suite. Alive2 is heavy machinery and runs into timeouts. Running the tests in roughly the same order every time is important to avoid timeout tests flipping to failed or vice-versa. Plus slow tests = tests that consume a lot of memory (in our scenario), so we can't bundle slow tests together.
Adding a `--ignore-timing-data` would be great, yes! But I still feel that sorting the list of failed tests is important for user experience. I diff these all the time.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D98179/new/

https://reviews.llvm.org/D98179



More information about the libcxx-commits mailing list