[Lldb-commits] [lldb] [lldb][test] Don't run libc++ API tests without a locally built libc++ (PR #162657)

[via lldb-commits]

llvmbot wrote:


<!--LLVM PR SUMMARY COMMENT-->

@llvm/pr-subscribers-lldb

Author: Michael Buch (Michael137)

<details>
<summary>Changes</summary>

API tests in the `libc++` category will try their best to build against a locally built libc++. If none exists, the `Makefile.rules` currently fall back to using the system libc++.

The issue with falling back to the system libc++ is that we are now potentially not testing what we intended to. But we also can't rely on certain libc++ features being available that the tests are trying to use. On Apple platforms this is a configuration error (because libc++ is the only stdlib supported), but we can't make it an error on Linux because a user might want to run the API tests with libstdc++.

The Ubunutu 22.04 bots on the Apple fork are failing to run following tests are failing:
* `TestLibcxxInternalsRecognizer.py`
* `TestDataFormatterStdRangesRefView.py` because the system stdlib doesn't have `std::ranges` support yet. And the tests just fail to build. Building libc++ on those bots is also not possible because the system compiler is too old (and the Apple fork builds all the subprojects standalone, so it requires the system compiler).

This patch marks tests in the `libc++` category as `UNSUPPORTED` if no local libc++ is available.

The downside is that we will inevitably lose coverage on bots that were running these tests without a local libc++. Arguably those weren't really testing the right thing. But for vendors with LLDB forks it might have been useful to at least know that the tests on the fork don't fail against the system libc++.

rdar://136231390

---
Full diff: https://github.com/llvm/llvm-project/pull/162657.diff


1 Files Affected:

- (modified) lldb/packages/Python/lldbsuite/test/dotest.py (+4) 


``````````diff
diff --git a/lldb/packages/Python/lldbsuite/test/dotest.py b/lldb/packages/Python/lldbsuite/test/dotest.py
index 2966ac04227cb..1d474a2180d22 100644
--- a/lldb/packages/Python/lldbsuite/test/dotest.py
+++ b/lldb/packages/Python/lldbsuite/test/dotest.py
@@ -789,6 +789,10 @@ def canRunLibcxxTests():
         return True, "libc++ always present"
 
     if platform == "linux":
+        if not configuration.libcxx_include_dir or not configuration.libcxx_library_dir:
+            return False, "API tests require a locally built libc++."
+
+        # Make sure -stdlib=libc++ works since that's how the tests will be built.
         with temp_file.OnDiskTempFile() as f:
             cmd = [configuration.compiler, "-xc++", "-stdlib=libc++", "-o", f.path, "-"]
             p = subprocess.Popen(

``````````

</details>


https://github.com/llvm/llvm-project/pull/162657