[PATCH] D65156: [lit] Protect full test suite from FILECHECK_OPTS
Joel E. Denny via Phabricator via llvm-commits
llvm-commits at lists.llvm.org
Tue Jul 23 10:01:55 PDT 2019
jdenny created this revision.
jdenny added reviewers: probinson, delcypher, thopre.
Herald added a project: LLVM.
jdenny added a comment.
This patch was extracted from D65121 <https://reviews.llvm.org/D65121>.
lit's test suite calls lit multiple times for various sample test
suites. `FILECHECK_OPTS` is safe for FileCheck calls in lit's test
suite. It's not safe for FileCheck calls in the sample test suites,
whose output affects the results of lit's test suite.
Without this patch, only one such sample test suite is protected from
`FILECHECK_OPTS`, and I admit I haven't discovered other cases for
which I can produce false failures using `FILECHECK_OPTS`. However,
it's hard to predict the future, especially false passes. Thus, this
patch protects all existing and future sample test suites from
`FILECHECK_OPTS` (and the deprecated
`FILECHECK_DUMP_INPUT_ON_FAILURE`).
Repository:
rG LLVM Github Monorepo
https://reviews.llvm.org/D65156
Files:
llvm/utils/lit/tests/lit.cfg
llvm/utils/lit/tests/shtest-run-at-line.py
Index: llvm/utils/lit/tests/shtest-run-at-line.py
===================================================================
--- llvm/utils/lit/tests/shtest-run-at-line.py
+++ llvm/utils/lit/tests/shtest-run-at-line.py
@@ -1,7 +1,7 @@
# Check that -vv makes the line number of the failing RUN command clear.
# (-v is actually sufficient in the case of the internal shell.)
#
-# RUN: env -u FILECHECK_OPTS not %{lit} -j 1 -vv %{inputs}/shtest-run-at-line > %t.out
+# RUN: not %{lit} -j 1 -vv %{inputs}/shtest-run-at-line > %t.out
# RUN: FileCheck --input-file %t.out %s
#
# END.
Index: llvm/utils/lit/tests/lit.cfg
===================================================================
--- llvm/utils/lit/tests/lit.cfg
+++ llvm/utils/lit/tests/lit.cfg
@@ -43,10 +43,17 @@
if directory:
llvm_config.with_environment('PATH', directory, append_path=True)
+# This test suite calls %{lit} to test lit's behavior for the sample test
+# suites in %{inputs}. This test suite's results are then determined in part
+# by %{lit}'s textual output, which includes the output of FileCheck calls
+# within %{inputs}'s test suites. Thus, %{lit} clears environment variables
+# that can affect FileCheck's output.
config.substitutions.append(('%{inputs}', os.path.join(
config.test_source_root, 'Inputs')))
-config.substitutions.append(('%{lit}', "%%{python} %s" % (
- os.path.join(lit_path, 'lit.py'),)))
+config.substitutions.append(('%{lit}',
+ "{env} %{{python}} {lit}".format(
+ env="env -u FILECHECK_OPTS -u FILECHECK_DUMP_INPUT_ON_FAILURE",
+ lit=os.path.join(lit_path, 'lit.py'))))
config.substitutions.append(('%{python}', '"%s"' % (sys.executable)))
# Enable coverage.py reporting, assuming the coverage module has been installed
-------------- next part --------------
A non-text attachment was scrubbed...
Name: D65156.211324.patch
Type: text/x-patch
Size: 1766 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20190723/d1100ca0/attachment.bin>
More information about the llvm-commits
mailing list