[PATCH] D96594: [lit] Add "early_tests" config option

[David Zarzycki via Phabricator via llvm-commits]

davezarzycki updated this revision to Diff 323952.
davezarzycki added a comment.

I believe I have addressed all of the feedback to date.

As to having a "run early" hint within the test files, I did look into that. For better and for worse, `lit` parses the test files as a part of the execution path, which is long after `lit` has sorted the tests. Changing this would be a nontrivial redesign of `lit` and it would put the parsing of the test suite (as opposed to mere discovery and sorting) on the critical path before tests are run. This approach is simpler and far less controversial.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D96594/new/

https://reviews.llvm.org/D96594

Files:
  llvm/docs/CommandGuide/lit.rst
  llvm/utils/lit/lit/Test.py
  llvm/utils/lit/lit/TestingConfig.py
  llvm/utils/lit/tests/Inputs/early-tests/aaa.txt
  llvm/utils/lit/tests/Inputs/early-tests/bbb.txt
  llvm/utils/lit/tests/Inputs/early-tests/lit.cfg
  llvm/utils/lit/tests/Inputs/early-tests/subdir/ccc.txt
  llvm/utils/lit/tests/early-tests.py


Index: llvm/utils/lit/tests/early-tests.py
===================================================================
--- /dev/null
+++ llvm/utils/lit/tests/early-tests.py
@@ -0,0 +1,9 @@
+## Check that we can run tests early.
+
+# RUN: %{lit} -j1 %{inputs}/early-tests | FileCheck %s
+
+# CHECK:     -- Testing: 3 tests, 1 workers --
+# CHECK-NEXT: PASS: early-tests :: subdir/ccc.txt
+# CHECK-NEXT: PASS: early-tests :: aaa.txt
+# CHECK-NEXT: PASS: early-tests :: bbb.txt
+# CHECK:     Passed: 3
Index: llvm/utils/lit/tests/Inputs/early-tests/subdir/ccc.txt
===================================================================
--- /dev/null
+++ llvm/utils/lit/tests/Inputs/early-tests/subdir/ccc.txt
@@ -0,0 +1 @@
+# RUN: true
Index: llvm/utils/lit/tests/Inputs/early-tests/lit.cfg
===================================================================
--- /dev/null
+++ llvm/utils/lit/tests/Inputs/early-tests/lit.cfg
@@ -0,0 +1,7 @@
+import lit.formats
+config.name = 'early-tests'
+config.suffixes = ['.txt']
+config.test_format = lit.formats.ShTest()
+config.test_source_root = None
+config.test_exec_root = None
+config.early_tests = { "subdir/ccc.txt" }
Index: llvm/utils/lit/tests/Inputs/early-tests/bbb.txt
===================================================================
--- /dev/null
+++ llvm/utils/lit/tests/Inputs/early-tests/bbb.txt
@@ -0,0 +1 @@
+# RUN: true
Index: llvm/utils/lit/tests/Inputs/early-tests/aaa.txt
===================================================================
--- /dev/null
+++ llvm/utils/lit/tests/Inputs/early-tests/aaa.txt
@@ -0,0 +1 @@
+# RUN: true
Index: llvm/utils/lit/lit/TestingConfig.py
===================================================================
--- llvm/utils/lit/lit/TestingConfig.py
+++ llvm/utils/lit/lit/TestingConfig.py
@@ -124,6 +124,8 @@
         self.limit_to_features = set(limit_to_features)
         # Whether the suite should be tested early in a given run.
         self.is_early = bool(is_early)
+        # List of tests to run early.
+        self.early_tests = {}
         self.parallelism_group = parallelism_group
         self._recursiveExpansionLimit = None
 
Index: llvm/utils/lit/lit/Test.py
===================================================================
--- llvm/utils/lit/lit/Test.py
+++ llvm/utils/lit/lit/Test.py
@@ -404,4 +404,6 @@
         This can be used for test suites with long running tests to maximize
         parallelism or where it is desirable to surface their failures early.
         """
+        if '/'.join(self.path_in_suite) in self.suite.config.early_tests:
+            return True
         return self.suite.config.is_early
Index: llvm/docs/CommandGuide/lit.rst
===================================================================
--- llvm/docs/CommandGuide/lit.rst
+++ llvm/docs/CommandGuide/lit.rst
@@ -402,6 +402,13 @@
  **root** The root configuration.  This is the top-most :program:`lit` configuration in
  the project.
 
+ **is_early** Whether the test suite as a whole should be given a head start
+ before other test suites run.
+
+ **early_tests** An explicit set of '/' separated test paths that should be
+ given a head start before other tests run. For example, the top five or so
+ slowest tests. See also: `--time-tests`
+
  **pipefail** Normally a test using a shell pipe fails if any of the commands
  on the pipe fail. If this is not desired, setting this variable to false
  makes the test fail only if the last command in the pipe fails.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D96594.323952.patch
Type: text/x-patch
Size: 3459 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20210216/9f90b102/attachment.bin>