[PATCH] D96594: [lit] Add "slowest_tests" config option

[David Zarzycki via Phabricator via llvm-commits]

davezarzycki updated this revision to Diff 323560.
davezarzycki added a comment.

I think I have addressed all of the feedback to date. Thank you @jdenny

Please note, the `/` separator is harmless and we need some kind of separator. It doesn't affect any filesystem APIs, just test sorting.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D96594/new/

https://reviews.llvm.org/D96594

Files:
  llvm/docs/CommandGuide/lit.rst
  llvm/utils/lit/lit/Test.py
  llvm/utils/lit/lit/TestingConfig.py
  llvm/utils/lit/tests/Inputs/early-tests/lit.cfg
  llvm/utils/lit/tests/Inputs/early-tests/test1.txt
  llvm/utils/lit/tests/Inputs/early-tests/test2.txt
  llvm/utils/lit/tests/early-tests.py


Index: llvm/utils/lit/tests/early-tests.py
===================================================================
--- /dev/null
+++ llvm/utils/lit/tests/early-tests.py
@@ -0,0 +1,8 @@
+# Check that we can run tests early.
+
+# RUN: %{lit} -j1 %{inputs}/early-tests | FileCheck %s
+
+# CHECK:     -- Testing: 2 tests, 1 workers --
+# CHECK-NEXT: PASS: early-tests :: test2.txt
+# CHECK-NEXT: PASS: early-tests :: test1.txt
+# CHECK:     Passed: 2
Index: llvm/utils/lit/tests/Inputs/early-tests/test2.txt
===================================================================
--- /dev/null
+++ llvm/utils/lit/tests/Inputs/early-tests/test2.txt
@@ -0,0 +1 @@
+# RUN: true
Index: llvm/utils/lit/tests/Inputs/early-tests/test1.txt
===================================================================
--- /dev/null
+++ llvm/utils/lit/tests/Inputs/early-tests/test1.txt
@@ -0,0 +1 @@
+# RUN: true
Index: llvm/utils/lit/tests/Inputs/early-tests/lit.cfg
===================================================================
--- /dev/null
+++ llvm/utils/lit/tests/Inputs/early-tests/lit.cfg
@@ -0,0 +1,7 @@
+import lit.formats
+config.name = 'early-tests'
+config.suffixes = ['.txt']
+config.test_format = lit.formats.ShTest()
+config.test_source_root = None
+config.test_exec_root = None
+config.early_tests = { "test2.txt" }
Index: llvm/utils/lit/lit/TestingConfig.py
===================================================================
--- llvm/utils/lit/lit/TestingConfig.py
+++ llvm/utils/lit/lit/TestingConfig.py
@@ -124,6 +124,8 @@
         self.limit_to_features = set(limit_to_features)
         # Whether the suite should be tested early in a given run.
         self.is_early = bool(is_early)
+        # List of slowest tests to run early.
+        self.early_tests = {}
         self.parallelism_group = parallelism_group
         self._recursiveExpansionLimit = None
 
Index: llvm/utils/lit/lit/Test.py
===================================================================
--- llvm/utils/lit/lit/Test.py
+++ llvm/utils/lit/lit/Test.py
@@ -404,4 +404,6 @@
         This can be used for test suites with long running tests to maximize
         parallelism or where it is desirable to surface their failures early.
         """
+        if '/'.join(self.path_in_suite) in self.suite.config.early_tests:
+            return True
         return self.suite.config.is_early
Index: llvm/docs/CommandGuide/lit.rst
===================================================================
--- llvm/docs/CommandGuide/lit.rst
+++ llvm/docs/CommandGuide/lit.rst
@@ -387,6 +387,13 @@
  **root** The root configuration.  This is the top-most :program:`lit` configuration in
  the project.
 
+ **is_early** Whether the test suite as a whole should be given a head start
+ before other test suites run.
+
+ **early_tests** An explicit set of '/' separated test paths that should be
+ given a head start before other tests run. For example, the top five or so
+ slowest tests. See also: `--time-tests`
+
  **pipefail** Normally a test using a shell pipe fails if any of the commands
  on the pipe fail. If this is not desired, setting this variable to false
  makes the test fail only if the last command in the pipe fails.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D96594.323560.patch
Type: text/x-patch
Size: 3192 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20210213/541b565d/attachment.bin>