[llvm] [UpdateTestChecks] Don't fail silently when conflicting CHECK lines means no checks are generated for some functions (PR #159321)
via llvm-commits
llvm-commits at lists.llvm.org
Wed Sep 17 04:07:18 PDT 2025
llvmbot wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-testing-tools
Author: Alex Bradbury (asb)
<details>
<summary>Changes</summary>
There is a warning that triggers if you (for instance) run `update_llc_test_checks.py` on an input where _all_ functions have conflicting check lines and so no checks are generated. However, there are no warnings emitted at all for the case where some functions have non-conflicting check lines but others don't. This is a source of frustration because running update_llc_test_checks can result in all check lines being removed when such a conflict exists with no warning, meaning we have to be extra vigilant inspecting the diff. I've also personally wasted time tracking down the source of the dropped lines assuming that update_test_checks would emit a warning in such cases.
This change adds logic to emit warnings on a function-by-function basis for any RUN that has conflicting prefixes meaning no output is generated. This subsumes the previous warning for when _all_ functions conflict.
---
CC @<!-- -->topperc @<!-- -->preames as this came up on a recent RISC-V LLVM sync call.
Note to reviewers: I am fairly happy this change addresses the issue I'm seeing for update_llc_testing_checks. I believe (hope!) I've coded it defensively enough that it shouldn't generate false warnings, but I would of course appreciate more eyes on this.
---
Full diff: https://github.com/llvm/llvm-project/pull/159321.diff
5 Files Affected:
- (added) llvm/test/tools/UpdateTestChecks/update_llc_test_checks/Inputs/conflicting-prefixes.ll (+16)
- (added) llvm/test/tools/UpdateTestChecks/update_llc_test_checks/Inputs/conflicting-prefixes.ll.expected (+21)
- (added) llvm/test/tools/UpdateTestChecks/update_llc_test_checks/conflicting-prefixes.test (+7)
- (modified) llvm/test/tools/UpdateTestChecks/update_llc_test_checks/prefix-never-matches.test (+1-1)
- (modified) llvm/utils/UpdateTestChecks/common.py (+45-20)
``````````diff
diff --git a/llvm/test/tools/UpdateTestChecks/update_llc_test_checks/Inputs/conflicting-prefixes.ll b/llvm/test/tools/UpdateTestChecks/update_llc_test_checks/Inputs/conflicting-prefixes.ll
new file mode 100644
index 0000000000000..fdc53951d6bb0
--- /dev/null
+++ b/llvm/test/tools/UpdateTestChecks/update_llc_test_checks/Inputs/conflicting-prefixes.ll
@@ -0,0 +1,16 @@
+; RUN: sed 's/RETVAL/1/g' %s | llc -mtriple=riscv32 \
+; RUN: | FileCheck -check-prefixes=CHECK,CHECKA %s
+; RUN: sed 's/RETVAL/2/g' %s | llc -mtriple=riscv32 \
+; RUN: | FileCheck -check-prefixes=CHECK,CHECKA %s
+; RUN: sed 's/RETVAL/3/g' %s | llc -mtriple=riscv32 \
+; RUN: | FileCheck -check-prefixes=CHECK,CHECKB %s
+; RUN: sed 's/RETVAL/4/g' %s | llc -mtriple=riscv32 \
+; RUN: | FileCheck -check-prefixes=CHECK,CHECKB %s
+
+define i32 @foo() {
+ ret i32 RETVAL
+}
+
+define i32 @bar() {
+ ret i32 100
+}
diff --git a/llvm/test/tools/UpdateTestChecks/update_llc_test_checks/Inputs/conflicting-prefixes.ll.expected b/llvm/test/tools/UpdateTestChecks/update_llc_test_checks/Inputs/conflicting-prefixes.ll.expected
new file mode 100644
index 0000000000000..b3cad11e2ec1d
--- /dev/null
+++ b/llvm/test/tools/UpdateTestChecks/update_llc_test_checks/Inputs/conflicting-prefixes.ll.expected
@@ -0,0 +1,21 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --no-generate-body-for-unused-prefixes
+; RUN: sed 's/RETVAL/1/g' %s | llc -mtriple=riscv32 \
+; RUN: | FileCheck -check-prefixes=CHECK,CHECKA %s
+; RUN: sed 's/RETVAL/2/g' %s | llc -mtriple=riscv32 \
+; RUN: | FileCheck -check-prefixes=CHECK,CHECKA %s
+; RUN: sed 's/RETVAL/3/g' %s | llc -mtriple=riscv32 \
+; RUN: | FileCheck -check-prefixes=CHECK,CHECKB %s
+; RUN: sed 's/RETVAL/4/g' %s | llc -mtriple=riscv32 \
+; RUN: | FileCheck -check-prefixes=CHECK,CHECKB %s
+
+define i32 @foo() {
+ ret i32 RETVAL
+}
+
+define i32 @bar() {
+; CHECK-LABEL: bar:
+; CHECK: # %bb.0:
+; CHECK-NEXT: li a0, 100
+; CHECK-NEXT: ret
+ ret i32 100
+}
diff --git a/llvm/test/tools/UpdateTestChecks/update_llc_test_checks/conflicting-prefixes.test b/llvm/test/tools/UpdateTestChecks/update_llc_test_checks/conflicting-prefixes.test
new file mode 100644
index 0000000000000..c9a6a7268d0b3
--- /dev/null
+++ b/llvm/test/tools/UpdateTestChecks/update_llc_test_checks/conflicting-prefixes.test
@@ -0,0 +1,7 @@
+# REQUIRES: riscv-registered-target
+
+# RUN: cp -f %S/Inputs/conflicting-prefixes.ll %t.ll
+# RUN: %update_llc_test_checks --no-generate-body-for-unused-prefixes %t.ll 2>&1 | FileCheck %s
+# RUN: diff -u %S/Inputs/conflicting-prefixes.ll.expected %t.ll
+
+# CHECK: WARNING: For function 'foo', the following RUN lines will not generate checks due to conflicting output
diff --git a/llvm/test/tools/UpdateTestChecks/update_llc_test_checks/prefix-never-matches.test b/llvm/test/tools/UpdateTestChecks/update_llc_test_checks/prefix-never-matches.test
index 2e75148addd84..90ae70bda64d9 100644
--- a/llvm/test/tools/UpdateTestChecks/update_llc_test_checks/prefix-never-matches.test
+++ b/llvm/test/tools/UpdateTestChecks/update_llc_test_checks/prefix-never-matches.test
@@ -4,5 +4,5 @@
# RUN: %update_llc_test_checks --no-generate-body-for-unused-prefixes %t.ll 2>&1 | FileCheck %s
# RUN: FileCheck --input-file=%t.ll %s --check-prefix=OUTPUT
-# CHECK: WARNING: Prefix A had conflicting output
+# CHECK: WARNING: For function 'fold_v2i64', the following RUN lines will not generate checks due to conflicting output
# OUTPUT-NOT: A:
diff --git a/llvm/utils/UpdateTestChecks/common.py b/llvm/utils/UpdateTestChecks/common.py
index 1c795afa9e700..c1cb195852a79 100644
--- a/llvm/utils/UpdateTestChecks/common.py
+++ b/llvm/utils/UpdateTestChecks/common.py
@@ -882,6 +882,7 @@ def __str__(self):
class FunctionTestBuilder:
def __init__(self, run_list, flags, scrubber_args, path, ginfo):
+ self._run_list = run_list
self._verbose = flags.verbose
self._record_args = flags.function_signature
self._check_attributes = flags.check_attributes
@@ -918,14 +919,52 @@ def __init__(self, run_list, flags, scrubber_args, path, ginfo):
self._global_var_dict.update({prefix: dict()})
def finish_and_get_func_dict(self):
- for prefix in self.get_failed_prefixes():
- warn(
- "Prefix %s had conflicting output from different RUN lines for all functions in test %s"
- % (
- prefix,
- self._path,
+ all_funcs = set()
+ for prefix in self._func_dict:
+ all_funcs.update(self._func_dict[prefix].keys())
+
+ warnings_to_print = collections.defaultdict(list)
+ for func in sorted(list(all_funcs)):
+ for i, run_info in enumerate(self._run_list):
+ prefixes = run_info[0]
+ if not prefixes:
+ continue
+
+ # Check if this RUN line produces this function at all.
+ run_contains_func = True
+ for p in prefixes:
+ if func not in self._func_dict.get(p, {}):
+ run_contains_func = False
+ break
+ if not run_contains_func:
+ continue
+
+ # Check if this RUN line can print any checks for this
+ # function. It can't if all of its prefixes have conflicting
+ # (None) output.
+ can_print_for_this_run = False
+ for p in prefixes:
+ if self._func_dict[p].get(func) is not None:
+ can_print_for_this_run = True
+ break
+
+ if not can_print_for_this_run:
+ warnings_to_print[func].append((i, prefixes))
+
+ for func, warning_info in warnings_to_print.items():
+ conflict_strs = []
+ for run_index, prefixes in warning_info:
+ conflict_strs.append(
+ "RUN #{} (prefixes: {})".format(
+ run_index + 1, ", ".join(prefixes)
+ )
)
+ warn(
+ "For function '{}', the following RUN lines will not generate checks due to conflicting output: {}".format(
+ func, ", ".join(conflict_strs)),
+ test_file=self._path,
)
+
return self._func_dict
def func_order(self):
@@ -1078,20 +1117,6 @@ def processed_prefixes(self, prefixes):
"""
self._processed_prefixes.update(prefixes)
- def get_failed_prefixes(self):
- # This returns the list of those prefixes that failed to match any function,
- # because there were conflicting bodies produced by different RUN lines, in
- # all instances of the prefix.
- for prefix in self._func_dict:
- if self._func_dict[prefix] and (
- not [
- fct
- for fct in self._func_dict[prefix]
- if self._func_dict[prefix][fct] is not None
- ]
- ):
- yield prefix
-
##### Generator of LLVM IR CHECK lines
``````````
</details>
https://github.com/llvm/llvm-project/pull/159321
More information about the llvm-commits
mailing list