[clang] [llvm] [Utils] Add new --update-tests flag to llvm-lit (PR #108425)
via cfe-commits
cfe-commits at lists.llvm.org
Thu Sep 12 10:07:45 PDT 2024
llvmbot wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-clang
Author: Henrik G. Olsson (hnrklssn)
<details>
<summary>Changes</summary>
This adds a flag to lit for detecting and updating failing tests when possible to do so automatically. The flag uses a plugin architecture where config files can add additional auto-updaters for the types of tests in the test suite. When a test fails with `--update-tests` enabled lit passes the test RUN invocation and output to each registered test updater until one of them signals that it updated the test (or all test updaters have been run). As such it is the responsibility of the test updater to only update tests where it is reasonably certain that it will actually fix the test, or come close to doing so.
Initially adds support for UpdateVerifyTests and UpdateTestChecks. The flag is currently only implemented for lit's internal shell, so `--update-tests` implies `LIT_USE_INTERNAL_SHELL=1`.
Builds on work in #<!-- -->97369
Fixes #<!-- -->81320
---
Patch is 47.68 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/108425.diff
50 Files Affected:
- (modified) clang/test/Sema/lit.local.cfg (+12)
- (modified) clang/test/lit.cfg.py (+9)
- (added) clang/test/utils/update-verify-tests/Inputs/duplicate-diag.c (+10)
- (added) clang/test/utils/update-verify-tests/Inputs/duplicate-diag.c.expected (+10)
- (added) clang/test/utils/update-verify-tests/Inputs/infer-indentation.c (+10)
- (added) clang/test/utils/update-verify-tests/Inputs/infer-indentation.c.expected (+13)
- (added) clang/test/utils/update-verify-tests/Inputs/leave-existing-diags.c (+13)
- (added) clang/test/utils/update-verify-tests/Inputs/leave-existing-diags.c.expected (+14)
- (added) clang/test/utils/update-verify-tests/Inputs/multiple-errors.c (+8)
- (added) clang/test/utils/update-verify-tests/Inputs/multiple-errors.c.expected (+11)
- (added) clang/test/utils/update-verify-tests/Inputs/multiple-missing-errors-same-line.c (+10)
- (added) clang/test/utils/update-verify-tests/Inputs/multiple-missing-errors-same-line.c.expected (+15)
- (added) clang/test/utils/update-verify-tests/Inputs/no-checks.c (+5)
- (added) clang/test/utils/update-verify-tests/Inputs/no-checks.c.expected (+6)
- (added) clang/test/utils/update-verify-tests/Inputs/no-diags.c (+7)
- (added) clang/test/utils/update-verify-tests/Inputs/no-diags.c.expected (+7)
- (added) clang/test/utils/update-verify-tests/Inputs/no-expected-diags.c (+6)
- (added) clang/test/utils/update-verify-tests/Inputs/no-expected-diags.c.expected (+6)
- (added) clang/test/utils/update-verify-tests/Inputs/non-default-prefix.c (+7)
- (added) clang/test/utils/update-verify-tests/Inputs/non-default-prefix.c.expected (+7)
- (added) clang/test/utils/update-verify-tests/Inputs/update-same-line.c (+6)
- (added) clang/test/utils/update-verify-tests/Inputs/update-same-line.c.expected (+6)
- (added) clang/test/utils/update-verify-tests/Inputs/update-single-check.c (+6)
- (added) clang/test/utils/update-verify-tests/Inputs/update-single-check.c.expected (+6)
- (added) clang/test/utils/update-verify-tests/LitTests/.gitignore (+4)
- (added) clang/test/utils/update-verify-tests/LitTests/lit.cfg (+31)
- (added) clang/test/utils/update-verify-tests/duplicate-diag.test (+4)
- (added) clang/test/utils/update-verify-tests/infer-indentation.test (+3)
- (added) clang/test/utils/update-verify-tests/leave-existing-diags.test (+4)
- (added) clang/test/utils/update-verify-tests/lit-plugin.test (+5)
- (added) clang/test/utils/update-verify-tests/lit.local.cfg (+36)
- (added) clang/test/utils/update-verify-tests/multiple-errors.test (+3)
- (added) clang/test/utils/update-verify-tests/multiple-missing-errors-same-line.test (+3)
- (added) clang/test/utils/update-verify-tests/no-checks.test (+3)
- (added) clang/test/utils/update-verify-tests/no-diags.test (+4)
- (added) clang/test/utils/update-verify-tests/no-expected-diags.test (+4)
- (added) clang/test/utils/update-verify-tests/non-default-prefix.test (+4)
- (added) clang/test/utils/update-verify-tests/update-same-line.test (+4)
- (added) clang/test/utils/update-verify-tests/update-single-check.test (+3)
- (added) clang/utils/UpdateVerifyTests/__init__.py (+1)
- (added) clang/utils/UpdateVerifyTests/core.py (+452)
- (added) clang/utils/UpdateVerifyTests/litplugin.py (+35)
- (added) clang/utils/update-verify-tests.py (+41)
- (modified) llvm/test/lit.cfg.py (+9)
- (modified) llvm/utils/lit/lit/LitConfig.py (+3)
- (modified) llvm/utils/lit/lit/TestRunner.py (+12)
- (modified) llvm/utils/lit/lit/cl_arguments.py (+6)
- (modified) llvm/utils/lit/lit/llvm/config.py (+5)
- (modified) llvm/utils/lit/lit/main.py (+1)
- (modified) llvm/utils/update_any_test_checks.py (+46-3)
``````````diff
diff --git a/clang/test/Sema/lit.local.cfg b/clang/test/Sema/lit.local.cfg
index baf1b39ef238cd..b385fd92098a8e 100644
--- a/clang/test/Sema/lit.local.cfg
+++ b/clang/test/Sema/lit.local.cfg
@@ -2,3 +2,15 @@ config.substitutions = list(config.substitutions)
config.substitutions.insert(
0, (r"%clang\b", """*** Do not use the driver in Sema tests. ***""")
)
+
+if lit_config.update_tests:
+ import sys
+ import os
+
+ curdir = os.path.dirname(os.path.realpath(__file__))
+ testdir = os.path.dirname(curdir)
+ clangdir = os.path.dirname(testdir)
+ utilspath = os.path.join(clangdir, "utils")
+ sys.path.append(utilspath)
+ from UpdateVerifyTests.litplugin import verify_test_updater
+ lit_config.test_updaters.append(verify_test_updater)
diff --git a/clang/test/lit.cfg.py b/clang/test/lit.cfg.py
index 92a3361ce672e2..c965d3167fe8eb 100644
--- a/clang/test/lit.cfg.py
+++ b/clang/test/lit.cfg.py
@@ -362,3 +362,12 @@ def calculate_arch_features(arch_string):
# possibly be present in system and user configuration files, so disable
# default configs for the test runs.
config.environment["CLANG_NO_DEFAULT_CONFIG"] = "1"
+
+if lit_config.update_tests:
+ import sys
+ import os
+
+ utilspath = os.path.join(config.llvm_src_root, "utils")
+ sys.path.append(utilspath)
+ from update_any_test_checks import utc_lit_plugin
+ lit_config.test_updaters.append(utc_lit_plugin)
diff --git a/clang/test/utils/update-verify-tests/Inputs/duplicate-diag.c b/clang/test/utils/update-verify-tests/Inputs/duplicate-diag.c
new file mode 100644
index 00000000000000..d4a92eb4a7874a
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/Inputs/duplicate-diag.c
@@ -0,0 +1,10 @@
+// RUN: %clang_cc1 -verify %s
+// RUN: diff %s %s.expected
+void foo() {
+ // expected-error at +1{{use of undeclared identifier 'a'}}
+ a = 2; a = 2;
+ b = 2; b = 2;
+ // expected-error at +1 3{{use of undeclared identifier 'c'}}
+ c = 2; c = 2;
+ // expected-error 2{{asdf}}
+}
diff --git a/clang/test/utils/update-verify-tests/Inputs/duplicate-diag.c.expected b/clang/test/utils/update-verify-tests/Inputs/duplicate-diag.c.expected
new file mode 100644
index 00000000000000..5fd72709e487f5
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/Inputs/duplicate-diag.c.expected
@@ -0,0 +1,10 @@
+// RUN: %clang_cc1 -verify %s
+// RUN: diff %s %s.expected
+void foo() {
+ // expected-error at +1 2{{use of undeclared identifier 'a'}}
+ a = 2; a = 2;
+ // expected-error at +1 2{{use of undeclared identifier 'b'}}
+ b = 2; b = 2;
+ // expected-error at +1 2{{use of undeclared identifier 'c'}}
+ c = 2; c = 2;
+}
diff --git a/clang/test/utils/update-verify-tests/Inputs/infer-indentation.c b/clang/test/utils/update-verify-tests/Inputs/infer-indentation.c
new file mode 100644
index 00000000000000..9a58244d92e066
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/Inputs/infer-indentation.c
@@ -0,0 +1,10 @@
+// RUN: %clang_cc1 -verify %s
+// RUN: diff %s %s.expected
+void foo() {
+ // expected-error at +1 2 {{use of undeclared identifier 'a'}}
+ a = 2; a = 2; b = 2; b = 2; c = 2;
+ // expected-error at +1 2 {{asdf}}
+ d = 2;
+ e = 2; f = 2; // expected-error 2 {{use of undeclared identifier 'e'}}
+}
+
diff --git a/clang/test/utils/update-verify-tests/Inputs/infer-indentation.c.expected b/clang/test/utils/update-verify-tests/Inputs/infer-indentation.c.expected
new file mode 100644
index 00000000000000..127efb003b1883
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/Inputs/infer-indentation.c.expected
@@ -0,0 +1,13 @@
+// RUN: %clang_cc1 -verify %s
+// RUN: diff %s %s.expected
+void foo() {
+ // expected-error at +3 {{use of undeclared identifier 'c'}}
+ // expected-error at +2 2 {{use of undeclared identifier 'b'}}
+ // expected-error at +1 2 {{use of undeclared identifier 'a'}}
+ a = 2; a = 2; b = 2; b = 2; c = 2;
+ // expected-error at +1 {{use of undeclared identifier 'd'}}
+ d = 2;
+ // expected-error at +1 {{use of undeclared identifier 'f'}}
+ e = 2; f = 2; // expected-error {{use of undeclared identifier 'e'}}
+}
+
diff --git a/clang/test/utils/update-verify-tests/Inputs/leave-existing-diags.c b/clang/test/utils/update-verify-tests/Inputs/leave-existing-diags.c
new file mode 100644
index 00000000000000..075d6c80fd4c36
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/Inputs/leave-existing-diags.c
@@ -0,0 +1,13 @@
+// RUN: %clang_cc1 -verify %s
+// RUN: diff %s %s.expected
+void foo() {
+ a = 2;
+ // expected-error at -1{{use of undeclared identifier 'a'}}
+ b = 2;// expected-error{{use of undeclared identifier 'b'}}
+ c = 2;
+ // expected-error at 7{{use of undeclared identifier 'c'}}
+ d = 2; // expected-error-re{{use of {{.*}} identifier 'd'}}
+
+ e = 2; // error to trigger mismatch
+}
+
diff --git a/clang/test/utils/update-verify-tests/Inputs/leave-existing-diags.c.expected b/clang/test/utils/update-verify-tests/Inputs/leave-existing-diags.c.expected
new file mode 100644
index 00000000000000..778ebd4ff4c7b9
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/Inputs/leave-existing-diags.c.expected
@@ -0,0 +1,14 @@
+// RUN: %clang_cc1 -verify %s
+// RUN: diff %s %s.expected
+void foo() {
+ a = 2;
+ // expected-error at -1{{use of undeclared identifier 'a'}}
+ b = 2;// expected-error{{use of undeclared identifier 'b'}}
+ c = 2;
+ // expected-error at 7{{use of undeclared identifier 'c'}}
+ d = 2; // expected-error-re{{use of {{.*}} identifier 'd'}}
+
+ // expected-error at +1{{use of undeclared identifier 'e'}}
+ e = 2; // error to trigger mismatch
+}
+
diff --git a/clang/test/utils/update-verify-tests/Inputs/multiple-errors.c b/clang/test/utils/update-verify-tests/Inputs/multiple-errors.c
new file mode 100644
index 00000000000000..d3fa93ec0e8221
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/Inputs/multiple-errors.c
@@ -0,0 +1,8 @@
+// RUN: %clang_cc1 -verify %s
+// RUN: diff %s %s.expected
+void foo() {
+ a = 2;
+ b = 2;
+
+ c = 2;
+}
diff --git a/clang/test/utils/update-verify-tests/Inputs/multiple-errors.c.expected b/clang/test/utils/update-verify-tests/Inputs/multiple-errors.c.expected
new file mode 100644
index 00000000000000..aa0cf22c8e8a7b
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/Inputs/multiple-errors.c.expected
@@ -0,0 +1,11 @@
+// RUN: %clang_cc1 -verify %s
+// RUN: diff %s %s.expected
+void foo() {
+ // expected-error at +1{{use of undeclared identifier 'a'}}
+ a = 2;
+ // expected-error at +1{{use of undeclared identifier 'b'}}
+ b = 2;
+
+ // expected-error at +1{{use of undeclared identifier 'c'}}
+ c = 2;
+}
diff --git a/clang/test/utils/update-verify-tests/Inputs/multiple-missing-errors-same-line.c b/clang/test/utils/update-verify-tests/Inputs/multiple-missing-errors-same-line.c
new file mode 100644
index 00000000000000..9c5104e693ae57
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/Inputs/multiple-missing-errors-same-line.c
@@ -0,0 +1,10 @@
+// RUN: %clang_cc1 -verify %s
+// RUN: diff %s %s.expected
+void foo() {
+ a = 2; b = 2; c = 2;
+}
+
+void bar() {
+ x = 2; y = 2; z = 2;
+ // expected-error at -1{{use of undeclared identifier 'x'}}
+}
diff --git a/clang/test/utils/update-verify-tests/Inputs/multiple-missing-errors-same-line.c.expected b/clang/test/utils/update-verify-tests/Inputs/multiple-missing-errors-same-line.c.expected
new file mode 100644
index 00000000000000..01076f76318997
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/Inputs/multiple-missing-errors-same-line.c.expected
@@ -0,0 +1,15 @@
+// RUN: %clang_cc1 -verify %s
+// RUN: diff %s %s.expected
+void foo() {
+ // expected-error at +3{{use of undeclared identifier 'c'}}
+ // expected-error at +2{{use of undeclared identifier 'b'}}
+ // expected-error at +1{{use of undeclared identifier 'a'}}
+ a = 2; b = 2; c = 2;
+}
+
+void bar() {
+ x = 2; y = 2; z = 2;
+ // expected-error at -1{{use of undeclared identifier 'x'}}
+ // expected-error at -2{{use of undeclared identifier 'y'}}
+ // expected-error at -3{{use of undeclared identifier 'z'}}
+}
diff --git a/clang/test/utils/update-verify-tests/Inputs/no-checks.c b/clang/test/utils/update-verify-tests/Inputs/no-checks.c
new file mode 100644
index 00000000000000..019698122c3f2b
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/Inputs/no-checks.c
@@ -0,0 +1,5 @@
+// RUN: %clang_cc1 -verify %s
+// RUN: diff %s %s.expected
+void foo() {
+ bar = 2;
+}
diff --git a/clang/test/utils/update-verify-tests/Inputs/no-checks.c.expected b/clang/test/utils/update-verify-tests/Inputs/no-checks.c.expected
new file mode 100644
index 00000000000000..42c98d4f77b6ce
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/Inputs/no-checks.c.expected
@@ -0,0 +1,6 @@
+// RUN: %clang_cc1 -verify %s
+// RUN: diff %s %s.expected
+void foo() {
+ // expected-error at +1{{use of undeclared identifier 'bar'}}
+ bar = 2;
+}
diff --git a/clang/test/utils/update-verify-tests/Inputs/no-diags.c b/clang/test/utils/update-verify-tests/Inputs/no-diags.c
new file mode 100644
index 00000000000000..fddc2ad661fd2d
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/Inputs/no-diags.c
@@ -0,0 +1,7 @@
+// RUN: %clang_cc1 -verify %s
+// RUN: diff %s %s.expected
+void foo() {
+ // expected-error at +1{{asdf}}
+ int a = 2;
+}
+
diff --git a/clang/test/utils/update-verify-tests/Inputs/no-diags.c.expected b/clang/test/utils/update-verify-tests/Inputs/no-diags.c.expected
new file mode 100644
index 00000000000000..acb02badc5655a
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/Inputs/no-diags.c.expected
@@ -0,0 +1,7 @@
+// expected-no-diagnostics
+// RUN: %clang_cc1 -verify %s
+// RUN: diff %s %s.expected
+void foo() {
+ int a = 2;
+}
+
diff --git a/clang/test/utils/update-verify-tests/Inputs/no-expected-diags.c b/clang/test/utils/update-verify-tests/Inputs/no-expected-diags.c
new file mode 100644
index 00000000000000..e02b63dcd7a21b
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/Inputs/no-expected-diags.c
@@ -0,0 +1,6 @@
+// RUN: %clang_cc1 -verify %s
+// RUN: diff %s %s.expected
+// expected-no-diagnostics
+void foo() {
+ a = 2;
+}
diff --git a/clang/test/utils/update-verify-tests/Inputs/no-expected-diags.c.expected b/clang/test/utils/update-verify-tests/Inputs/no-expected-diags.c.expected
new file mode 100644
index 00000000000000..7fd85f2e064e28
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/Inputs/no-expected-diags.c.expected
@@ -0,0 +1,6 @@
+// RUN: %clang_cc1 -verify %s
+// RUN: diff %s %s.expected
+void foo() {
+ // expected-error at +1{{use of undeclared identifier 'a'}}
+ a = 2;
+}
diff --git a/clang/test/utils/update-verify-tests/Inputs/non-default-prefix.c b/clang/test/utils/update-verify-tests/Inputs/non-default-prefix.c
new file mode 100644
index 00000000000000..b7a34e2dfa20d4
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/Inputs/non-default-prefix.c
@@ -0,0 +1,7 @@
+// RUN: %clang_cc1 -verify=check %s
+// RUN: diff %s %s.expected
+void foo() {
+ a = 2; // check-error{{asdf}}
+ // expected-error at -1{ignored}}
+}
+
diff --git a/clang/test/utils/update-verify-tests/Inputs/non-default-prefix.c.expected b/clang/test/utils/update-verify-tests/Inputs/non-default-prefix.c.expected
new file mode 100644
index 00000000000000..087a6be5ea4d2b
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/Inputs/non-default-prefix.c.expected
@@ -0,0 +1,7 @@
+// RUN: %clang_cc1 -verify=check %s
+// RUN: diff %s %s.expected
+void foo() {
+ a = 2; // check-error{{use of undeclared identifier 'a'}}
+ // expected-error at -1{ignored}}
+}
+
diff --git a/clang/test/utils/update-verify-tests/Inputs/update-same-line.c b/clang/test/utils/update-verify-tests/Inputs/update-same-line.c
new file mode 100644
index 00000000000000..e448800324e373
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/Inputs/update-same-line.c
@@ -0,0 +1,6 @@
+// RUN: %clang_cc1 -verify %s
+// RUN: diff %s %s.expected
+void foo() {
+ bar = 2; // expected-error {{asdf}}
+}
+
diff --git a/clang/test/utils/update-verify-tests/Inputs/update-same-line.c.expected b/clang/test/utils/update-verify-tests/Inputs/update-same-line.c.expected
new file mode 100644
index 00000000000000..ba4387a445fe08
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/Inputs/update-same-line.c.expected
@@ -0,0 +1,6 @@
+// RUN: %clang_cc1 -verify %s
+// RUN: diff %s %s.expected
+void foo() {
+ bar = 2; // expected-error {{use of undeclared identifier 'bar'}}
+}
+
diff --git a/clang/test/utils/update-verify-tests/Inputs/update-single-check.c b/clang/test/utils/update-verify-tests/Inputs/update-single-check.c
new file mode 100644
index 00000000000000..e093a8c2ddddbb
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/Inputs/update-single-check.c
@@ -0,0 +1,6 @@
+// RUN: %clang_cc1 -verify %s
+// RUN: diff %s %s.expected
+void foo() {
+ // expected-error at +1{{asdf}}
+ bar = 2;
+}
diff --git a/clang/test/utils/update-verify-tests/Inputs/update-single-check.c.expected b/clang/test/utils/update-verify-tests/Inputs/update-single-check.c.expected
new file mode 100644
index 00000000000000..42c98d4f77b6ce
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/Inputs/update-single-check.c.expected
@@ -0,0 +1,6 @@
+// RUN: %clang_cc1 -verify %s
+// RUN: diff %s %s.expected
+void foo() {
+ // expected-error at +1{{use of undeclared identifier 'bar'}}
+ bar = 2;
+}
diff --git a/clang/test/utils/update-verify-tests/LitTests/.gitignore b/clang/test/utils/update-verify-tests/LitTests/.gitignore
new file mode 100644
index 00000000000000..307a6d636e7a28
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/LitTests/.gitignore
@@ -0,0 +1,4 @@
+*.c
+*.c.expected
+.lit_test_times.txt
+Output
diff --git a/clang/test/utils/update-verify-tests/LitTests/lit.cfg b/clang/test/utils/update-verify-tests/LitTests/lit.cfg
new file mode 100644
index 00000000000000..dbb22f13ef1303
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/LitTests/lit.cfg
@@ -0,0 +1,31 @@
+import lit.formats
+import lit.llvm
+lit.llvm.initialize(lit_config, config)
+from lit.llvm import llvm_config
+from os.path import dirname
+
+config.name = "Lit Update Verify Tests"
+config.test_format = lit.formats.ShTest(not llvm_config.use_lit_shell)
+config.suffixes = [
+ ".c",
+]
+config.test_source_root = dirname(__file__)
+config.target_triple = None
+config.host_triple = None
+
+config.llvm_tools_dir = lit_config.path[0] # --path is explicitly passed by lit-plugin.test
+llvm_config.use_default_substitutions()
+llvm_config.use_clang()
+config.substitutions.append(("%clang_cc1", "%clang -cc1"))
+
+if lit_config.update_tests:
+ import sys
+ import os
+
+ config.clang_src_dir = dirname(dirname(dirname(dirname(config.test_source_root))))
+ utilspath = os.path.join(config.clang_src_dir, "utils")
+ sys.path.append(utilspath)
+ from UpdateVerifyTests.litplugin import verify_test_updater
+ # normally we'd append to the existing list, but when testing
+ # verify_test_updater we don't want diff_test_updater to accidentally interfere
+ lit_config.test_updaters = [verify_test_updater]
diff --git a/clang/test/utils/update-verify-tests/duplicate-diag.test b/clang/test/utils/update-verify-tests/duplicate-diag.test
new file mode 100644
index 00000000000000..3163ce46199c3f
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/duplicate-diag.test
@@ -0,0 +1,4 @@
+# RUN: cp %S/Inputs/duplicate-diag.c %t.c && not %clang_cc1 -verify %t.c 2>&1 | %update-verify-tests
+# RUN: diff -u %S/Inputs/duplicate-diag.c.expected %t.c
+# RUN: %clang_cc1 -verify %t.c
+
diff --git a/clang/test/utils/update-verify-tests/infer-indentation.test b/clang/test/utils/update-verify-tests/infer-indentation.test
new file mode 100644
index 00000000000000..6ba2f5d9d505bf
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/infer-indentation.test
@@ -0,0 +1,3 @@
+# RUN: cp %S/Inputs/infer-indentation.c %t.c && not %clang_cc1 -verify %t.c 2>&1 | %update-verify-tests
+# RUN: diff -u %S/Inputs/infer-indentation.c.expected %t.c
+# RUN: %clang_cc1 -verify %t.c
diff --git a/clang/test/utils/update-verify-tests/leave-existing-diags.test b/clang/test/utils/update-verify-tests/leave-existing-diags.test
new file mode 100644
index 00000000000000..cde690ef715a67
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/leave-existing-diags.test
@@ -0,0 +1,4 @@
+# RUN: cp %S/Inputs/leave-existing-diags.c %t.c && not %clang_cc1 -verify %t.c 2>&1 | %update-verify-tests
+# RUN: diff -u %S/Inputs/leave-existing-diags.c.expected %t.c
+# RUN: %clang_cc1 -verify %t.c
+
diff --git a/clang/test/utils/update-verify-tests/lit-plugin.test b/clang/test/utils/update-verify-tests/lit-plugin.test
new file mode 100644
index 00000000000000..f3bbf5ec36bfb5
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/lit-plugin.test
@@ -0,0 +1,5 @@
+# RUN: cp %S/Inputs/*.c %S/LitTests
+# RUN: cp %S/Inputs/*.c.expected %S/LitTests
+# RUN: not %{lit} %S/LitTests
+# RUN: not %{lit} %S/LitTests --update-tests
+# RUN: %{lit} %S/LitTests
diff --git a/clang/test/utils/update-verify-tests/lit.local.cfg b/clang/test/utils/update-verify-tests/lit.local.cfg
new file mode 100644
index 00000000000000..281c5be37a983a
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/lit.local.cfg
@@ -0,0 +1,36 @@
+import lit.util
+
+# python 2.7 backwards compatibility
+try:
+ from shlex import quote as shell_quote
+except ImportError:
+ from pipes import quote as shell_quote
+
+if config.standalone_build:
+ # These tests require the update-verify-tests.py script from the clang
+ # source tree, so skip these tests if we are doing standalone builds.
+ config.unsupported = True
+else:
+ config.suffixes = [".test"]
+ config.excludes.add("LitTests")
+
+ script_path = os.path.join(
+ config.clang_src_dir, "utils", "update-verify-tests.py"
+ )
+ python = shell_quote(config.python_executable)
+ config.substitutions.append(
+ (
+ "%update-verify-tests",
+ "%s %s" % (python, shell_quote(script_path)),
+ )
+ )
+ config.substitutions.append(("%{lit}", "%{lit-no-order-opt} --order=lexical"))
+
+ lit_path = os.path.join(config.llvm_tools_dir, "llvm-lit")
+ lit_path = os.path.abspath(lit_path)
+ config.substitutions.append(
+ (
+ "%{lit-no-order-opt}",
+ f"{python} {lit_path} -j1 --path {config.llvm_tools_dir}"
+ )
+)
diff --git a/clang/test/utils/update-verify-tests/multiple-errors.test b/clang/test/utils/update-verify-tests/multiple-errors.test
new file mode 100644
index 00000000000000..1332ef365dc863
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/multiple-errors.test
@@ -0,0 +1,3 @@
+# RUN: cp %S/Inputs/multiple-errors.c %t.c && not %clang_cc1 -verify %t.c 2>&1 | %update-verify-tests
+# RUN: diff -u %S/Inputs/multiple-errors.c.expected %t.c
+# RUN: %clang_cc1 -verify %t.c
diff --git a/clang/test/utils/update-verify-tests/multiple-missing-errors-same-line.test b/clang/test/utils/update-verify-tests/multiple-missing-errors-same-line.test
new file mode 100644
index 00000000000000..a9c21cd77e192b
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/multiple-missing-errors-same-line.test
@@ -0,0 +1,3 @@
+# RUN: cp %S/Inputs/multiple-missing-errors-same-line.c %t.c && not %clang_cc1 -verify %t.c 2>&1 | %update-verify-tests
+# RUN: diff -u %S/Inputs/multiple-missing-errors-same-line.c.expected %t.c
+# RUN: %clang_cc1 -verify %t.c
diff --git a/clang/test/utils/update-verify-tests/no-checks.test b/clang/test/utils/update-verify-tests/no-checks.test
new file mode 100644
index 00000000000000..f6ea91fa552be4
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/no-checks.test
@@ -0,0 +1,3 @@
+# RUN: cp %S/Inputs/no-checks.c %t.c && not %clang_cc1 -verify %t.c 2>&1 | %update-verify-tests
+# RUN: diff -u %S/Inputs/no-checks.c.expected %t.c
+# RUN: %clang_cc1 -verify %t.c
diff --git a/clang/test/utils/update-verify-tests/no-diags.test b/clang/test/utils/update-verify-tests/no-diags.test
new file mode 100644
index 00000000000000..464fe8894253b6
--- /dev/null
+++ b/clang/test/utils/update-verify-tests/no-diags.test
@@ -0,0 +1,4 @@
+# RUN: cp %S/Inputs/no-diag...
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/108425
More information about the cfe-commits
mailing list