[PATCH] D83069: [lit] warn if explicitly specified test won't be run indirectly

[ben via Phabricator via llvm-commits]

bd1976llvm created this revision.
bd1976llvm added reviewers: ddunbar, rnk, yln, thakis, pcc, chapuni.
Herald added subscribers: llvm-commits, delcypher.
Herald added a project: LLVM.

It's relatively easy to write a lit test that won't actually be run. I did in: https://reviews.llvm.org/D82567

This patch adds a warning to avoid users doing that.


https://reviews.llvm.org/D83069

Files:
  llvm/utils/lit/lit/discovery.py
  llvm/utils/lit/tests/discovery.py


Index: llvm/utils/lit/tests/discovery.py
===================================================================
--- llvm/utils/lit/tests/discovery.py
+++ llvm/utils/lit/tests/discovery.py
@@ -134,6 +134,14 @@
 # CHECK-ASEXEC-EXACT-TEST: -- Available Tests --
 # CHECK-ASEXEC-EXACT-TEST: top-level-suite :: subdir/test-three
 
+# Check warning emitted when exact test name given will
+# not be run when specified indirectly.
+#
+# RUN: %{lit} \
+# RUN:     %{inputs}/discovery/test.not-txt -j 1 2>%t.err
+# RUN: FileCheck --check-prefix=CHECK-WARN-EXACT-TEST-INDIRECT < %t.err %s
+#
+# CHECK-WARN-EXACT-TEST-INDIRECT: warning: 'top-level-suite :: test.not-txt' would not be run indirectly
 
 # Check that we don't recurse infinitely when loading an site specific test
 # suite located inside the test source root.
Index: llvm/utils/lit/lit/discovery.py
===================================================================
--- llvm/utils/lit/lit/discovery.py
+++ llvm/utils/lit/lit/discovery.py
@@ -149,8 +149,26 @@
 
     # Check if the user named a test directly.
     if not os.path.isdir(source_path):
-        lc = getLocalConfig(ts, path_in_suite[:-1], litConfig, localConfigCache)
-        yield Test.Test(ts, path_in_suite, lc)
+        test_dir_in_suite = path_in_suite[:-1]
+        lc = getLocalConfig(ts, test_dir_in_suite, litConfig, localConfigCache)
+        test = Test.Test(ts, path_in_suite, lc)
+
+        # Issue a warning if the specified test would not be run
+        # if the user had specified the containing directory instead.
+        # This helps to avoid writing tests which are not executed.
+        if lc.test_format is not None:
+            found = False
+            for res in lc.test_format.getTestsInDirectory(ts, test_dir_in_suite,
+                                                          litConfig, lc):
+                if test.getFullName() == res.getFullName():
+                    found = True
+                    break
+            if not found:            
+                litConfig.warning(
+                    '%r would not be run indirectly: change name or LIT config'
+                    % test.getFullName())
+
+        yield test
         return
 
     # Otherwise we have a directory to search for tests, start by getting the


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D83069.275178.patch
Type: text/x-patch
Size: 2280 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20200702/dc3499bf/attachment.bin>