[PATCH] D97373: [lit] Introduce LIT_WORKERS environment variable

[Jan Svoboda via Phabricator via llvm-commits]

jansvoboda11 created this revision.
jansvoboda11 added reviewers: Bigcheese, dexonsmith, yln.
Herald added a subscriber: delcypher.
jansvoboda11 requested review of this revision.
Herald added a project: LLVM.
Herald added a subscriber: llvm-commits.

There is no way to specify the number of workers when invoking lit inderectly (e.g. when building targets similar "check-clang"). Lit does not see the `-j N` target passed to the build tool (e.g. `ninja`).

This patch introduces the `LIT_WORKERS` environment variable, which works around this. Now, the number of cores for building dependencies of a lit test and runing the lit test can be both parametrized:
`LIT_WORKERS=N ninja check-clang -j M`

When invoking lit directly, the value specified via `LIT_WORKERS` can be overwritten by the `-j` argument.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D97373

Files:
  llvm/docs/CommandGuide/lit.rst
  llvm/utils/lit/lit/cl_arguments.py
  llvm/utils/lit/tests/Inputs/lit-workers/lit.cfg
  llvm/utils/lit/tests/Inputs/lit-workers/test1.txt
  llvm/utils/lit/tests/Inputs/lit-workers/test2.txt
  llvm/utils/lit/tests/Inputs/lit-workers/test3.txt
  llvm/utils/lit/tests/Inputs/lit-workers/test4.txt
  llvm/utils/lit/tests/lit-workers.py


Index: llvm/utils/lit/tests/lit-workers.py
===================================================================
--- /dev/null
+++ llvm/utils/lit/tests/lit-workers.py
@@ -0,0 +1,12 @@
+# Check that the number of workers can be specified via the `-j` argument.
+# RUN:                   %{lit} -s %{inputs}/lit-workers -j 4 | FileCheck %s
+
+# Check that the number of workers can be specified via the `LIT_WORKERS`
+# environment variable.
+# RUN: env LIT_WORKERS=4 %{lit} -s %{inputs}/lit-workers      | FileCheck %s
+
+# Check that the number of workers specified via the `LIT_WORKERS` environment
+# variable can be overwritten by the `-j` argument.
+# RUN: env LIT_WORKERS=2 %{lit} -s %{inputs}/lit-workers -j 4 | FileCheck %s
+
+# CHECK: 4 workers
Index: llvm/utils/lit/tests/Inputs/lit-workers/test4.txt
===================================================================
--- /dev/null
+++ llvm/utils/lit/tests/Inputs/lit-workers/test4.txt
@@ -0,0 +1 @@
+# RUN: true
Index: llvm/utils/lit/tests/Inputs/lit-workers/test3.txt
===================================================================
--- /dev/null
+++ llvm/utils/lit/tests/Inputs/lit-workers/test3.txt
@@ -0,0 +1 @@
+# RUN: true
Index: llvm/utils/lit/tests/Inputs/lit-workers/test2.txt
===================================================================
--- /dev/null
+++ llvm/utils/lit/tests/Inputs/lit-workers/test2.txt
@@ -0,0 +1 @@
+# RUN: true
Index: llvm/utils/lit/tests/Inputs/lit-workers/test1.txt
===================================================================
--- /dev/null
+++ llvm/utils/lit/tests/Inputs/lit-workers/test1.txt
@@ -0,0 +1 @@
+# RUN: true
Index: llvm/utils/lit/tests/Inputs/lit-workers/lit.cfg
===================================================================
--- /dev/null
+++ llvm/utils/lit/tests/Inputs/lit-workers/lit.cfg
@@ -0,0 +1,6 @@
+import lit.formats
+config.name = 'lit-workers'
+config.suffixes = ['.txt']
+config.test_format = lit.formats.ShTest()
+config.test_source_root = None
+config.test_exec_root = None
Index: llvm/utils/lit/lit/cl_arguments.py
===================================================================
--- llvm/utils/lit/lit/cl_arguments.py
+++ llvm/utils/lit/lit/cl_arguments.py
@@ -30,7 +30,7 @@
             metavar="N",
             help="Number of workers used for testing",
             type=_positive_int,
-            default=lit.util.usable_core_count())
+            default=os.environ.get("LIT_WORKERS", lit.util.usable_core_count()))
     parser.add_argument("--config-prefix",
             dest="configPrefix",
             metavar="NAME",
Index: llvm/docs/CommandGuide/lit.rst
===================================================================
--- llvm/docs/CommandGuide/lit.rst
+++ llvm/docs/CommandGuide/lit.rst
@@ -56,7 +56,9 @@
 .. option:: -j N, --workers=N
 
  Run ``N`` tests in parallel.  By default, this is automatically chosen to
- match the number of detected available CPUs.
+ match the number of detected available CPUs. The environment variable
+ ``LIT_WORKERS`` can be also used in place of this option, which is especially
+ useful in environments where the call to ``lit`` is issued indirectly.
 
 .. option:: --config-prefix=NAME
 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D97373.326031.patch
Type: text/x-patch
Size: 3192 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20210224/e423a416/attachment.bin>