[llvm] [llvm][lit] Add option to run only the failed tests (PR #158043)

[Michael Buch via llvm-commits]

Michael137 wrote:

> Hi, I found the cause of failure.
> 
> 
> 
> On Windows (with python <= 3.12 only, 3.13 works fine), the contents of `filter-failed.py.tmp/.lit_test_times.txt` after test ran is below:
> 
> ```
> 
> -2.408028e-02 fail.txt
> 
> 1.904011e-02 pass.txt
> 
> -0.000000e+00 unresolved.txt
> 
> 2.004886e-02 xfail.txt
> 
> -2.453256e-02 xpass.txt
> 
> ```
> 
> 
> 
> It says that unresolved.txt finished in 0 seconds. However, since [`previous_failure` is set by `time < 0`](https://github.com/llvm/llvm-project/blob/fbb587c65653395dc97a360e0cd0d1cf9ed0e875/llvm/utils/lit/lit/Test.py#L282), a test that finishes in 0 seconds is treated as a successful test.
> 
> 
> 
> All of failed tests should have a strictly negative elapsed time (or, modifying to `time <= 0` also might be an option.)
> 
> This patch works fine for me:
> 
> ```diff
> 
> diff --git a/llvm/utils/lit/lit/TestTimes.py b/llvm/utils/lit/lit/TestTimes.py
> 
> index a2c0e0527b84..0c18e3e01582 100644
> 
> --- a/llvm/utils/lit/lit/TestTimes.py
> 
> +++ b/llvm/utils/lit/lit/TestTimes.py
> 
> @@ -22,7 +22,7 @@ def record_test_times(tests, lit_config):
> 
>              continue
> 
>          if not t.suite.exec_root in times_by_suite:
> 
>              times_by_suite[t.suite.exec_root] = read_test_times(t.suite)
> 
> -        time = -t.result.elapsed if t.isFailure() else t.result.elapsed
> 
> +        time = min(-t.result.elapsed, -1.0e-6) if t.isFailure() else t.result.elapsed
> 
>          # The "path" here is only used as a key into a dictionary. It is never
> 
>          # used as an actual path to a filesystem API, therefore we use '/' as
> 
>          # the canonical separator so that Unix and Windows machines can share
> 
> ```

Thanks so much for investigating! I didnt have a windows machine handy so I put this PR on the backburner. Let me apply the patch and check

https://github.com/llvm/llvm-project/pull/158043