[llvm] [Util] Only run --update-tests functions on failing tests (PR #155148)

LLVM Continuous Integration via llvm-commits llvm-commits at lists.llvm.org
Mon Aug 25 14:17:38 PDT 2025


llvm-ci wrote:

LLVM Buildbot has detected a new failure on builder `ml-opt-dev-x86-64` running on `ml-opt-dev-x86-64-b2` while building `llvm` at step 6 "test-build-unified-tree-check-all".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/137/builds/24222

<details>
<summary>Here is the relevant piece of the build log for the reference</summary>

```
Step 6 (test-build-unified-tree-check-all) failure: test (failure)
******************** TEST 'lit :: pass-test-update.py' FAILED ********************
Exit Code: 1

Command Output (stdout):
--
# RUN: at line 1
env -u FILECHECK_OPTS "/usr/bin/python3" /b/ml-opt-dev-x86-64-b1/llvm-project/llvm/utils/lit/lit.py -j1 --order=lexical --update-tests --ignore-fail -v /b/ml-opt-dev-x86-64-b1/build/utils/lit/tests/Inputs/pass-test-update | FileCheck /b/ml-opt-dev-x86-64-b1/build/utils/lit/tests/pass-test-update.py --implicit-check-not Exception
# executed command: env -u FILECHECK_OPTS /usr/bin/python3 /b/ml-opt-dev-x86-64-b1/llvm-project/llvm/utils/lit/lit.py -j1 --order=lexical --update-tests --ignore-fail -v /b/ml-opt-dev-x86-64-b1/build/utils/lit/tests/Inputs/pass-test-update
# .---command stderr------------
# | 
# | Exiting with status 0 instead of 1 because '--ignore-fail' was specified.
# `-----------------------------
# executed command: FileCheck /b/ml-opt-dev-x86-64-b1/build/utils/lit/tests/pass-test-update.py --implicit-check-not Exception
# .---command stderr------------
# | /b/ml-opt-dev-x86-64-b1/build/utils/lit/tests/pass-test-update.py:16:15: error: CHECK-NEXT: is not on the line after the previous match
# | # CHECK-NEXT: File "{{.*}}/should_not_run.py", line {{.*}}, in should_not_run
# |               ^
# | <stdin>:16:2: note: 'next' match was here
# |  File "/b/ml-opt-dev-x86-64-b1/build/utils/lit/tests/Inputs/pass-test-update/should_not_run.py", line 2, in should_not_run
# |  ^
# | <stdin>:14:44: note: previous match ended here
# |  update_output = test_updater(result, test)
# |                                            ^
# | <stdin>:15:1: note: non-matching line after previous match is here
# |  ^^^^^^^^^^^^^^^^^^^^^^^^^^
# | ^
# | 
# | Input file: <stdin>
# | Check file: /b/ml-opt-dev-x86-64-b1/build/utils/lit/tests/pass-test-update.py
# | 
# | -dump-input=help explains the following input dump.
# | 
# | Input was:
# | <<<<<<
# |          .
# |          .
# |          .
# |         11: Exception occurred in test updater: 
# |         12: Traceback (most recent call last): 
# |         13:  File "/b/ml-opt-dev-x86-64-b1/llvm-project/llvm/utils/lit/lit/TestRunner.py", line 1216, in executeScriptInternal 
# |         14:  update_output = test_updater(result, test) 
# |         15:  ^^^^^^^^^^^^^^^^^^^^^^^^^^ 
# |         16:  File "/b/ml-opt-dev-x86-64-b1/build/utils/lit/tests/Inputs/pass-test-update/should_not_run.py", line 2, in should_not_run 
# | next:16      !~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  error: match on wrong line
# |         17:  raise Exception("this test updater should only run on failure") 
# |         18: Exception: this test updater should only run on failure 
# |         19:  
# |         20: ******************** 
# |         21: PASS: pass-test-update :: pass-silent.test (2 of 5) 
# |          .
...

```

</details>

https://github.com/llvm/llvm-project/pull/155148


More information about the llvm-commits mailing list