[libc-commits] [libc] [libc] Improve lit test discovery and execution (PR #192993)

[via libc-commits]

llvmbot wrote:


<!--LLVM PR SUMMARY COMMENT-->

@llvm/pr-subscribers-libc

Author: Jeff Bailey (kaladron)

<details>
<summary>Changes</summary>

Updated LibcTest to support cross-compilation and custom test directories:

* Passed localConfig to _isTestExecutable to check for emulator.
* Skipped host execution permission check if emulator is set.
* Supported tests in libc.test.shared and libc.test.utils.
* Handled fallback if @<!-- -->BINARY@ is missing in test command template.
* Removed UnitTest from excludes in lit site config.

---
Full diff: https://github.com/llvm/llvm-project/pull/192993.diff


2 Files Affected:

- (modified) libc/test/lit.site.cfg.py.in (+2-2) 
- (modified) libc/utils/libctest/format.py (+17-4) 


``````````diff
diff --git a/libc/test/lit.site.cfg.py.in b/libc/test/lit.site.cfg.py.in
index 7727f16956d73..3668a491cd05c 100644
--- a/libc/test/lit.site.cfg.py.in
+++ b/libc/test/lit.site.cfg.py.in
@@ -12,7 +12,7 @@ config.libc_crosscompiling_emulator = path(r"@CMAKE_CROSSCOMPILING_EMULATOR@")
 
 # If no explicit test command is set, use the cross-compiling emulator.
 if not config.libc_test_cmd and config.libc_crosscompiling_emulator:
-    config.libc_test_cmd = config.libc_crosscompiling_emulator + " @BINARY@"
+    config.libc_test_cmd = config.libc_crosscompiling_emulator
 
 # Add libc's utils directory to the path so we can import the test format.
 site.addsitedir(os.path.join(config.libc_src_root, "utils"))
@@ -26,7 +26,7 @@ config.name = "libc"
 config.test_format = libctest.LibcTest()
 
 # excludes: A list of directories to exclude from the testsuite.
-config.excludes = ["Inputs", "CMakeLists.txt", "README.txt", "LICENSE.txt", "UnitTest"]
+config.excludes = ["Inputs", "CMakeLists.txt", "README.txt", "LICENSE.txt"]
 
 # test_source_root: The root path where tests are located.
 # test_exec_root: The root path where test executables are built.
diff --git a/libc/utils/libctest/format.py b/libc/utils/libctest/format.py
index c8280fd1b5561..25152c1ce6906 100644
--- a/libc/utils/libctest/format.py
+++ b/libc/utils/libctest/format.py
@@ -60,11 +60,11 @@ def getTestsInDirectory(self, testSuite, path_in_suite, litConfig, localConfig):
             filepath = os.path.join(source_path, filename)
 
             # Match our test executable pattern
-            if self._isTestExecutable(filename, filepath):
+            if self._isTestExecutable(filename, filepath, localConfig):
                 # Create a test with the executable name
                 yield lit.Test.Test(testSuite, path_in_suite + (filename,), localConfig)
 
-    def _isTestExecutable(self, filename, filepath):
+    def _isTestExecutable(self, filename, filepath, localConfig):
         """
         Check if a file is a libc test executable we should run.
 
@@ -90,11 +90,20 @@ def _isTestExecutable(self, filename, filepath):
                 return False
         elif test_name.startswith("libc.test.integration."):
             pass  # Accept all integration tests ending in .__build__
+        elif test_name.startswith("libc.test.shared."):
+            pass  # Accept all shared tests ending in .__build__
+        elif test_name.startswith("libc.test.utils."):
+            pass  # Accept all utils tests ending in .__build__
         else:
             return False
         if not os.path.isfile(filepath):
             return False
-        if not kIsWindows and not os.access(filepath, os.X_OK):
+        # GPU binaries are not host-executable but run via an emulator, so ignore X_OK if emulator is set.
+        if (
+            not kIsWindows
+            and not os.access(filepath, os.X_OK)
+            and not getattr(localConfig, "libc_crosscompiling_emulator", None)
+        ):
             return False
         return True
 
@@ -150,7 +159,11 @@ def execute(self, test, litConfig):
 
         test_cmd_template = getattr(test.config, "libc_test_cmd", "")
         if test_cmd_template:
-            test_cmd = test_cmd_template.replace("@BINARY@", test_path)
+            # Fallback to appending the binary path if @BINARY@ placeholder is missing.
+            if "@BINARY@" in test_cmd_template:
+                test_cmd = test_cmd_template.replace("@BINARY@", test_path)
+            else:
+                test_cmd = test_cmd_template + " " + test_path
             cmd_args = shlex.split(test_cmd)
             if not cmd_args:
                 cmd_args = [test_path]

``````````

</details>


https://github.com/llvm/llvm-project/pull/192993