[PATCH] D29684: [lit] Add callback support for modifying test script

[Dominic Chen via Phabricator via llvm-commits]

ddcc created this revision.

Adds a callback field to TestConfig, and a "ShTest" callback in TestRunner to allow direct manipulation of command-line arguments.

This mechanism is proposed to be used by the Clang static analyzer in https://reviews.llvm.org/D28952 for checking if an additional constraint solver backend is available, and
if so, executing each test automatically with each constraint solver backend, without adding additional lines to each testcase.


https://reviews.llvm.org/D29684

Files:
  docs/TestingGuide.rst
  utils/lit/lit/TestRunner.py
  utils/lit/lit/TestingConfig.py
  utils/lit/tests/Inputs/shtest-callback/lit.cfg
  utils/lit/tests/Inputs/shtest-callback/lit.local.cfg
  utils/lit/tests/Inputs/shtest-callback/shtest.txt
  utils/lit/tests/shtest-callback.py


Index: utils/lit/tests/shtest-callback.py
===================================================================
--- /dev/null
+++ utils/lit/tests/shtest-callback.py
@@ -0,0 +1,8 @@
+# Test callbacks are executed correctly.
+
+# RUN: %{lit} -j 1 -v %{inputs}/callback > %t.out
+# RUN: FileCheck < %t.out %s
+
+# CHECK: -- Testing:
+
+# CHECK: PASS: callback :: shtest.txt
Index: utils/lit/tests/Inputs/shtest-callback/shtest.txt
===================================================================
--- /dev/null
+++ utils/lit/tests/Inputs/shtest-callback/shtest.txt
@@ -0,0 +1 @@
+# RUN: %callback1
Index: utils/lit/tests/Inputs/shtest-callback/lit.local.cfg
===================================================================
--- /dev/null
+++ utils/lit/tests/Inputs/shtest-callback/lit.local.cfg
@@ -0,0 +1,7 @@
+import lit.TestRunner
+
+def callback(test, cmd):
+    return lit.TestRunner.applySubstitutions(cmd, [('%callback2', 'true')])
+
+config.callbacks["ShTest"] = callback
+
Index: utils/lit/tests/Inputs/shtest-callback/lit.cfg
===================================================================
--- /dev/null
+++ utils/lit/tests/Inputs/shtest-callback/lit.cfg
@@ -0,0 +1,8 @@
+import lit.formats
+config.name = 'shtest-callback'
+config.substitutions = [('%callback1', '%callback2')]
+config.suffixes = ['.txt']
+config.test_format = lit.formats.ShTest()
+config.test_source_root = None
+config.test_exec_root = None
+config.target_triple = 'x86_64-unknown-unknown'
Index: utils/lit/lit/TestingConfig.py
===================================================================
--- utils/lit/lit/TestingConfig.py
+++ utils/lit/lit/TestingConfig.py
@@ -106,7 +106,7 @@
                  environment, substitutions, unsupported,
                  test_exec_root, test_source_root, excludes,
                  available_features, pipefail, limit_to_features = [],
-                 is_early = False, parallelism_group = ""):
+                 is_early = False, parallelism_group = "", callbacks = {}):
         self.parent = parent
         self.name = str(name)
         self.suffixes = set(suffixes)
@@ -126,6 +126,7 @@
         # Whether the suite should be tested early in a given run.
         self.is_early = bool(is_early)
         self.parallelism_group = parallelism_group
+        self.callbacks = callbacks
 
     def finish(self, litConfig):
         """finish() - Finish this config object, after loading is complete."""
@@ -143,6 +144,7 @@
             # files. Should we distinguish them?
             self.test_source_root = str(self.test_source_root)
         self.excludes = set(self.excludes)
+        self.callbacks = dict(self.callbacks)
 
     @property
     def root(self):
Index: utils/lit/lit/TestRunner.py
===================================================================
--- utils/lit/lit/TestRunner.py
+++ utils/lit/lit/TestRunner.py
@@ -1045,6 +1045,8 @@
     if hasattr(test.config, 'test_retry_attempts'):
         attempts += test.config.test_retry_attempts
     for i in range(attempts):
+        if "ShTest" in test.config.callbacks:
+            script = test.config.callbacks["ShTest"](test, script)
         res = _runShTest(test, litConfig, useExternalSh, script, tmpBase)
         if res.code != Test.FAIL:
             break
Index: docs/TestingGuide.rst
===================================================================
--- docs/TestingGuide.rst
+++ docs/TestingGuide.rst
@@ -506,6 +506,9 @@
 ``%clang_cc1``
    Invokes the Clang frontend.
 
+``%clang_analyze_cc1``
+   Invokes the Clang frontend with the static analyzer.
+
 ``%itanium_abi_triple``, ``%ms_abi_triple``
    These substitutions can be used to get the current target triple adjusted to
    the desired ABI. For example, if the test suite is running with the


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D29684.87518.patch
Type: text/x-patch
Size: 3766 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20170207/04746ef8/attachment.bin>