[llvm] [llvm-lit] Ignore malformed `.lit_test_times` entries (PR #191305)

[via llvm-commits]

llvmbot wrote:


<!--LLVM PR SUMMARY COMMENT-->

@llvm/pr-subscribers-testing-tools

Author: Jonathan Thackray (jthackray)

<details>
<summary>Changes</summary>

When running `llvm-lit`, I sometimes hit a traceback, because a
`.lit_test_times.txt` file has got corrupted (not sure how).
However, it's non-obvious what the issue is (you just get a traceback),
so I've fixed this as follows:

`read_test_times()` currently assumes every line in .lit_test_times.txt
contains a floating-point time followed by a test path. If the file
contains a blank line, a line without a path, or a non-numeric time,
lit raises during discovery instead of skipping the bad entry.

Fix this by parsing each line defensively. Split once, ignore lines
that do not produce both fields, and ignore entries whose time cannot
be parsed as a float. This keeps the existing behavior for valid lines
while making malformed timing files non-fatal.

Add a new `malformed-test-times.py` regression test and a checked-in
lit_test_times fixture containing both valid and malformed entries. The
test copies that fixture into `.lit_test_times.txt`, checks that lit still
discovers and runs both tests, and verifies that the valid timing entries
still order `b.txt` before `a.txt`.

---
Full diff: https://github.com/llvm/llvm-project/pull/191305.diff


6 Files Affected:

- (modified) llvm/utils/lit/lit/TestTimes.py (+8-2) 
- (added) llvm/utils/lit/tests/Inputs/malformed-test-times/a.txt (+1) 
- (added) llvm/utils/lit/tests/Inputs/malformed-test-times/b.txt (+1) 
- (added) llvm/utils/lit/tests/Inputs/malformed-test-times/lit.cfg (+7) 
- (added) llvm/utils/lit/tests/Inputs/malformed-test-times/lit_test_times (+5) 
- (added) llvm/utils/lit/tests/malformed-test-times.py (+12) 


``````````diff
diff --git a/llvm/utils/lit/lit/TestTimes.py b/llvm/utils/lit/lit/TestTimes.py
index 7ec1a1b97b717..79ac5c3120080 100644
--- a/llvm/utils/lit/lit/TestTimes.py
+++ b/llvm/utils/lit/lit/TestTimes.py
@@ -9,8 +9,14 @@ def read_test_times(suite):
     if os.path.exists(test_times_file):
         with open(test_times_file, "r") as time_file:
             for line in time_file:
-                time, path = line.split(maxsplit=1)
-                test_times[path.strip("\n")] = float(time)
+                fields = line.split(maxsplit=1)
+                if len(fields) != 2:
+                    continue
+                time, path = fields
+                try:
+                    test_times[path.strip("\n")] = float(time)
+                except ValueError:
+                    continue
     return test_times
 
 
diff --git a/llvm/utils/lit/tests/Inputs/malformed-test-times/a.txt b/llvm/utils/lit/tests/Inputs/malformed-test-times/a.txt
new file mode 100644
index 0000000000000..b80b60b7a2794
--- /dev/null
+++ b/llvm/utils/lit/tests/Inputs/malformed-test-times/a.txt
@@ -0,0 +1 @@
+# RUN: true
diff --git a/llvm/utils/lit/tests/Inputs/malformed-test-times/b.txt b/llvm/utils/lit/tests/Inputs/malformed-test-times/b.txt
new file mode 100644
index 0000000000000..b80b60b7a2794
--- /dev/null
+++ b/llvm/utils/lit/tests/Inputs/malformed-test-times/b.txt
@@ -0,0 +1 @@
+# RUN: true
diff --git a/llvm/utils/lit/tests/Inputs/malformed-test-times/lit.cfg b/llvm/utils/lit/tests/Inputs/malformed-test-times/lit.cfg
new file mode 100644
index 0000000000000..0558595a8c305
--- /dev/null
+++ b/llvm/utils/lit/tests/Inputs/malformed-test-times/lit.cfg
@@ -0,0 +1,7 @@
+import lit.formats
+
+config.name = "malformed-test-times"
+config.suffixes = [".txt"]
+config.test_format = lit.formats.ShTest()
+config.test_source_root = None
+config.test_exec_root = None
diff --git a/llvm/utils/lit/tests/Inputs/malformed-test-times/lit_test_times b/llvm/utils/lit/tests/Inputs/malformed-test-times/lit_test_times
new file mode 100644
index 0000000000000..28c0b7da7aba0
--- /dev/null
+++ b/llvm/utils/lit/tests/Inputs/malformed-test-times/lit_test_times
@@ -0,0 +1,5 @@
+4.798174e-03 b.txt
+missing-path
+
+not-a-float ignored.txt
+1.412153e-03 a.txt
diff --git a/llvm/utils/lit/tests/malformed-test-times.py b/llvm/utils/lit/tests/malformed-test-times.py
new file mode 100644
index 0000000000000..9426e4d684401
--- /dev/null
+++ b/llvm/utils/lit/tests/malformed-test-times.py
@@ -0,0 +1,12 @@
+## Check that malformed .lit_test_times.txt lines do not crash discovery.
+##
+## The valid timing entries should still be honored for smart ordering.
+
+# RUN: cp %{inputs}/malformed-test-times/lit_test_times %{inputs}/malformed-test-times/.lit_test_times.txt
+# RUN: %{lit-no-order-opt} %{inputs}/malformed-test-times > %t.out
+# RUN: FileCheck < %t.out %s
+
+# CHECK: -- Testing: 2 tests, 1 workers --
+# CHECK-NEXT: PASS: malformed-test-times :: b.txt
+# CHECK-NEXT: PASS: malformed-test-times :: a.txt
+# CHECK: Passed: 2

``````````

</details>


https://github.com/llvm/llvm-project/pull/191305