[PATCH] D130570: [Lit] Sort previously failed tests early

[Dave Green via Phabricator via llvm-commits]

dmgreen created this revision.
dmgreen added reviewers: davezarzycki, jhenderson, yln, mehdi_amini, thopre.
Herald added a subscriber: delcypher.
Herald added a project: All.
dmgreen requested review of this revision.
Herald added a project: LLVM.

D98179 <https://reviews.llvm.org/D98179> added a mechanism to sort tests by test time to run slow tests early, increasing potential parallelism. It also added a feature where negative tests would be marked as negative, allowing subsequent test runs to run them earlier. Unfortunately it never actually stored the negative time though, even if all the other code seemed to be inplace to sort them early. Luckily the fix seems simple.


https://reviews.llvm.org/D130570

Files:
  llvm/utils/lit/lit/TestTimes.py
  llvm/utils/lit/tests/Inputs/reorder/.lit_test_times.txt
  llvm/utils/lit/tests/Inputs/reorder/fff.txt
  llvm/utils/lit/tests/reorder.py


Index: llvm/utils/lit/tests/reorder.py
===================================================================
--- llvm/utils/lit/tests/reorder.py
+++ llvm/utils/lit/tests/reorder.py
@@ -1,7 +1,7 @@
 ## Check that we can reorder test runs.
 
 # RUN: cp %{inputs}/reorder/.lit_test_times.txt %{inputs}/reorder/.lit_test_times.txt.orig
-# RUN: %{lit-no-order-opt} %{inputs}/reorder > %t.out
+# RUN: not %{lit-no-order-opt} %{inputs}/reorder > %t.out
 # RUN: cp %{inputs}/reorder/.lit_test_times.txt %{inputs}/reorder/.lit_test_times.txt.new
 # RUN: cp %{inputs}/reorder/.lit_test_times.txt.orig %{inputs}/reorder/.lit_test_times.txt
 # RUN: not diff %{inputs}/reorder/.lit_test_times.txt.new %{inputs}/reorder/.lit_test_times.txt.orig
@@ -12,10 +12,12 @@
 # TIMES: not-executed.txt
 # TIMES-NEXT: subdir/ccc.txt
 # TIMES-NEXT: bbb.txt
+# TIMES-NEXT: -{{.*}} fff.txt
 # TIMES-NEXT: aaa.txt
 # TIMES-NEXT: new-test.txt
 
-# CHECK:     -- Testing: 4 tests, 1 workers --
+# CHECK:     -- Testing: 5 tests, 1 workers --
+# CHECK:      FAIL: reorder :: fff.txt
 # CHECK-NEXT: PASS: reorder :: subdir/ccc.txt
 # CHECK-NEXT: PASS: reorder :: bbb.txt
 # CHECK-NEXT: PASS: reorder :: aaa.txt
Index: llvm/utils/lit/tests/Inputs/reorder/fff.txt
===================================================================
--- /dev/null
+++ llvm/utils/lit/tests/Inputs/reorder/fff.txt
@@ -0,0 +1 @@
+# RUN: false
Index: llvm/utils/lit/tests/Inputs/reorder/.lit_test_times.txt
===================================================================
--- llvm/utils/lit/tests/Inputs/reorder/.lit_test_times.txt
+++ llvm/utils/lit/tests/Inputs/reorder/.lit_test_times.txt
@@ -1,4 +1,5 @@
 42.0 not-executed.txt
 3.0 subdir/ccc.txt
 2.0 bbb.txt
+-0.2 fff.txt
 0.1 aaa.txt
Index: llvm/utils/lit/lit/TestTimes.py
===================================================================
--- llvm/utils/lit/lit/TestTimes.py
+++ llvm/utils/lit/lit/TestTimes.py
@@ -28,7 +28,7 @@
         # used as an actual path to a filesystem API, therefore we use '/' as
         # the canonical separator so that Unix and Windows machines can share
         # timing data.
-        times_by_suite[t.suite.exec_root]['/'.join(t.path_in_suite)] = t.result.elapsed
+        times_by_suite[t.suite.exec_root]['/'.join(t.path_in_suite)] = time
 
     for s, value in times_by_suite.items():
         try:


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D130570.447669.patch
Type: text/x-patch
Size: 2341 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20220726/925ddd3a/attachment.bin>