[PATCH] D121004: [lit] Fix failed tests to be recorded to .lit_test_times.txt

[Sebastian Grabowski via Phabricator via llvm-commits]

picsel2 created this revision.
picsel2 added a reviewer: davezarzycki.
Herald added a subscriber: delcypher.
Herald added a project: All.
picsel2 requested review of this revision.
Herald added a project: LLVM.
Herald added a subscriber: llvm-commits.

Currently lit "forgets" that a test has failed, which the breaks the smart order option (probably the incremental option, too). This patch fixes the recording of failing tests by saving the negative time in .lit_test_times.txt.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D121004

Files:
  llvm/utils/lit/lit/TestTimes.py
  llvm/utils/lit/tests/Inputs/save-failed/fail.txt
  llvm/utils/lit/tests/Inputs/save-failed/lit.cfg
  llvm/utils/lit/tests/Inputs/save-failed/pass.txt
  llvm/utils/lit/tests/save-failed.py


Index: llvm/utils/lit/tests/save-failed.py
===================================================================
--- /dev/null
+++ llvm/utils/lit/tests/save-failed.py
@@ -0,0 +1,7 @@
+# Check that failed tests are saved with negative test time to indicate failure
+
+# RUN: not %{lit} %{inputs}/save-failed
+# RUN: FileCheck %s < %{inputs}/save-failed/.lit_test_times.txt
+
+# CHECK-DAG: {{^[^-].*}} pass.txt
+# CHECK-DAG: {{^-.*}} fail.txt
Index: llvm/utils/lit/tests/Inputs/save-failed/pass.txt
===================================================================
--- /dev/null
+++ llvm/utils/lit/tests/Inputs/save-failed/pass.txt
@@ -0,0 +1 @@
+# RUN: true
Index: llvm/utils/lit/tests/Inputs/save-failed/lit.cfg
===================================================================
--- /dev/null
+++ llvm/utils/lit/tests/Inputs/save-failed/lit.cfg
@@ -0,0 +1,6 @@
+import lit.formats
+config.name = 'save-failed'
+config.suffixes = ['.txt']
+config.test_format = lit.formats.ShTest()
+config.test_source_root = None
+config.test_exec_root = None
Index: llvm/utils/lit/tests/Inputs/save-failed/fail.txt
===================================================================
--- /dev/null
+++ llvm/utils/lit/tests/Inputs/save-failed/fail.txt
@@ -0,0 +1 @@
+# RUN: false
Index: llvm/utils/lit/lit/TestTimes.py
===================================================================
--- llvm/utils/lit/lit/TestTimes.py
+++ llvm/utils/lit/lit/TestTimes.py
@@ -28,7 +28,7 @@
         # used as an actual path to a filesystem API, therefore we use '/' as
         # the canonical separator so that Unix and Windows machines can share
         # timing data.
-        times_by_suite[t.suite.exec_root]['/'.join(t.path_in_suite)] = t.result.elapsed
+        times_by_suite[t.suite.exec_root]['/'.join(t.path_in_suite)] = time
 
     for s, value in times_by_suite.items():
         try:


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D121004.413022.patch
Type: text/x-patch
Size: 1869 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20220304/cfe248e0/attachment.bin>