[llvm] 7ce39d8 - [Lit] Sort previously failed tests early

[David Green via llvm-commits]

Author: David Green
Date: 2022-07-27T17:55:11+01:00
New Revision: 7ce39d80a86a4afbcf1be932b6c98b348b5b6549

URL: https://github.com/llvm/llvm-project/commit/7ce39d80a86a4afbcf1be932b6c98b348b5b6549
DIFF: https://github.com/llvm/llvm-project/commit/7ce39d80a86a4afbcf1be932b6c98b348b5b6549.diff

LOG: [Lit] Sort previously failed tests early

D98179 added a mechanism to sort tests by test time to run slow tests
early, increasing potential parallelism. It also added a feature where
negative tests would be marked as negative, allowing subsequent test
runs to run them earlier. Unfortunately it never actually stored the
negative time, even if all the other code seemed to be inplace to sort
them early. Luckily the fix seems simple.

Differential Revision: https://reviews.llvm.org/D130570

Added: 
    llvm/utils/lit/tests/Inputs/reorder/fff.txt

Modified: 
    llvm/utils/lit/lit/TestTimes.py
    llvm/utils/lit/tests/Inputs/reorder/.lit_test_times.txt
    llvm/utils/lit/tests/reorder.py

Removed: 
    


################################################################################
diff  --git a/llvm/utils/lit/lit/TestTimes.py b/llvm/utils/lit/lit/TestTimes.py
index 621343f29d262..8379cf5dfe182 100644
--- a/llvm/utils/lit/lit/TestTimes.py
+++ b/llvm/utils/lit/lit/TestTimes.py
@@ -28,7 +28,7 @@ def record_test_times(tests, lit_config):
         # used as an actual path to a filesystem API, therefore we use '/' as
         # the canonical separator so that Unix and Windows machines can share
         # timing data.
-        times_by_suite[t.suite.exec_root]['/'.join(t.path_in_suite)] = t.result.elapsed
+        times_by_suite[t.suite.exec_root]['/'.join(t.path_in_suite)] = time
 
     for s, value in times_by_suite.items():
         try:

diff  --git a/llvm/utils/lit/tests/Inputs/reorder/.lit_test_times.txt b/llvm/utils/lit/tests/Inputs/reorder/.lit_test_times.txt
index 73d55debc6052..227a4381db002 100644
--- a/llvm/utils/lit/tests/Inputs/reorder/.lit_test_times.txt
+++ b/llvm/utils/lit/tests/Inputs/reorder/.lit_test_times.txt
@@ -1,4 +1,5 @@
 42.0 not-executed.txt
 3.0 subdir/ccc.txt
 2.0 bbb.txt
+-0.2 fff.txt
 0.1 aaa.txt

diff  --git a/llvm/utils/lit/tests/Inputs/reorder/fff.txt b/llvm/utils/lit/tests/Inputs/reorder/fff.txt
new file mode 100644
index 0000000000000..49932c3006e15
--- /dev/null
+++ b/llvm/utils/lit/tests/Inputs/reorder/fff.txt
@@ -0,0 +1 @@
+# RUN: false

diff  --git a/llvm/utils/lit/tests/reorder.py b/llvm/utils/lit/tests/reorder.py
index f911de21ed316..b4d7beb6e54ee 100644
--- a/llvm/utils/lit/tests/reorder.py
+++ b/llvm/utils/lit/tests/reorder.py
@@ -1,7 +1,7 @@
 ## Check that we can reorder test runs.
 
 # RUN: cp %{inputs}/reorder/.lit_test_times.txt %{inputs}/reorder/.lit_test_times.txt.orig
-# RUN: %{lit-no-order-opt} %{inputs}/reorder > %t.out
+# RUN: not %{lit-no-order-opt} %{inputs}/reorder > %t.out
 # RUN: cp %{inputs}/reorder/.lit_test_times.txt %{inputs}/reorder/.lit_test_times.txt.new
 # RUN: cp %{inputs}/reorder/.lit_test_times.txt.orig %{inputs}/reorder/.lit_test_times.txt
 # RUN: not 
diff  %{inputs}/reorder/.lit_test_times.txt.new %{inputs}/reorder/.lit_test_times.txt.orig
@@ -12,10 +12,12 @@
 # TIMES: not-executed.txt
 # TIMES-NEXT: subdir/ccc.txt
 # TIMES-NEXT: bbb.txt
+# TIMES-NEXT: -{{.*}} fff.txt
 # TIMES-NEXT: aaa.txt
 # TIMES-NEXT: new-test.txt
 
-# CHECK:     -- Testing: 4 tests, 1 workers --
+# CHECK:     -- Testing: 5 tests, 1 workers --
+# CHECK-NEXT: FAIL: reorder :: fff.txt
 # CHECK-NEXT: PASS: reorder :: subdir/ccc.txt
 # CHECK-NEXT: PASS: reorder :: bbb.txt
 # CHECK-NEXT: PASS: reorder :: aaa.txt