[llvm] 6cecd3c - [lit] Protect full test suite from FILECHECK_OPTS
Joel E. Denny via llvm-commits
llvm-commits at lists.llvm.org
Wed Nov 6 13:26:38 PST 2019
Author: Joel E. Denny
Date: 2019-11-06T16:25:25-05:00
New Revision: 6cecd3c3dbef48eca6c4cf2dcc2df3290ab91488
URL: https://github.com/llvm/llvm-project/commit/6cecd3c3dbef48eca6c4cf2dcc2df3290ab91488
DIFF: https://github.com/llvm/llvm-project/commit/6cecd3c3dbef48eca6c4cf2dcc2df3290ab91488.diff
LOG: [lit] Protect full test suite from FILECHECK_OPTS
lit's test suite calls lit multiple times for various sample test
suites. `FILECHECK_OPTS` is safe for FileCheck calls in lit's test
suite. It's not safe for FileCheck calls in the sample test suites,
whose output affects the results of lit's test suite.
Without this patch, only one such sample test suite is protected from
`FILECHECK_OPTS`, and currently `shtest-shell.py` breaks with
`FILECHECK_OPTS=-vv`. Moreover, it's hard to predict the future,
especially false passes. Thus, this patch protects all existing and
future sample test suites from `FILECHECK_OPTS` (and the deprecated
`FILECHECK_DUMP_INPUT_ON_FAILURE`).
Reviewed By: probinson
Differential Revision: https://reviews.llvm.org/D65156
Added:
Modified:
llvm/utils/lit/tests/lit.cfg
llvm/utils/lit/tests/shtest-run-at-line.py
Removed:
################################################################################
diff --git a/llvm/utils/lit/tests/lit.cfg b/llvm/utils/lit/tests/lit.cfg
index 6205615f0139..ba9cb4da9cfd 100644
--- a/llvm/utils/lit/tests/lit.cfg
+++ b/llvm/utils/lit/tests/lit.cfg
@@ -52,10 +52,17 @@ for attribute in ('llvm_tools_dir', 'lit_tools_dir'):
if directory:
llvm_config.with_environment('PATH', directory, append_path=True)
+# This test suite calls %{lit} to test lit's behavior for the sample test
+# suites in %{inputs}. This test suite's results are then determined in part
+# by %{lit}'s textual output, which includes the output of FileCheck calls
+# within %{inputs}'s test suites. Thus, %{lit} clears environment variables
+# that can affect FileCheck's output.
config.substitutions.append(('%{inputs}', os.path.join(
config.test_source_root, 'Inputs')))
-config.substitutions.append(('%{lit}', "%%{python} %s" % (
- os.path.join(lit_path, 'lit.py'),)))
+config.substitutions.append(('%{lit}',
+ "{env} %{{python}} {lit}".format(
+ env="env -u FILECHECK_OPTS -u FILECHECK_DUMP_INPUT_ON_FAILURE",
+ lit=os.path.join(lit_path, 'lit.py'))))
config.substitutions.append(('%{python}', '"%s"' % (sys.executable)))
# Enable coverage.py reporting, assuming the coverage module has been installed
diff --git a/llvm/utils/lit/tests/shtest-run-at-line.py b/llvm/utils/lit/tests/shtest-run-at-line.py
index 7e5d53b8e26c..cd0e08137ee5 100644
--- a/llvm/utils/lit/tests/shtest-run-at-line.py
+++ b/llvm/utils/lit/tests/shtest-run-at-line.py
@@ -1,7 +1,7 @@
# Check that -vv makes the line number of the failing RUN command clear.
# (-v is actually sufficient in the case of the internal shell.)
#
-# RUN: env -u FILECHECK_OPTS not %{lit} -j 1 -vv %{inputs}/shtest-run-at-line > %t.out
+# RUN: not %{lit} -j 1 -vv %{inputs}/shtest-run-at-line > %t.out
# RUN: FileCheck --input-file %t.out %s
#
# END.
More information about the llvm-commits
mailing list