[Lldb-commits] [lldb] [lldb][test] Don't run simulator tests in parallel to stabilize them (PR #201870)

[via lldb-commits]

llvmorg-github-actions[bot] wrote:


<!--LLVM PR SUMMARY COMMENT-->

@llvm/pr-subscribers-lldb

Author: Raphael Isemann (Teemperor)

<details>
<summary>Changes</summary>

TestSimulatorPlatform is a notoriously flaky test on macOS. However, it seems the test failures only happen when its exectuion overlaps with the execution of TestAppleSimulatorOSType.py (which also interacts with the simulator). Somehow one simulator test seems to break the other one, but it's not exactly clear how they are inteferring with each other.

This patch ensures these two tests never run in parallel to avoid these kind of issues. It creates a parallelism_group in lit for apple-simulator and then puts both tests into that group.

TestAppleSimulatorOSType.py had to be moved to its own directory for this. It shared its directory with a a lot of other unrelated tests and the lit config applies to the whole directory.

---
Full diff: https://github.com/llvm/llvm-project/pull/201870.diff


6 Files Affected:

- (modified) lldb/test/API/lit.cfg.py (+4) 
- (added) lldb/test/API/macosx/simulator/lit.local.cfg (+1) 
- (added) lldb/test/API/tools/lldb-server/apple-simulator/Makefile (+5) 
- (renamed) lldb/test/API/tools/lldb-server/apple-simulator/TestAppleSimulatorOSType.py () 
- (added) lldb/test/API/tools/lldb-server/apple-simulator/lit.local.cfg (+1) 
- (added) lldb/test/API/tools/lldb-server/apple-simulator/main.cpp (+19) 


``````````diff
diff --git a/lldb/test/API/lit.cfg.py b/lldb/test/API/lit.cfg.py
index 2662a77199641..d1a28a842371b 100644
--- a/lldb/test/API/lit.cfg.py
+++ b/lldb/test/API/lit.cfg.py
@@ -194,6 +194,10 @@ def delete_module_cache(path):
     else:
         lit_config.error("Unknown simulator id '{}'".format(lldb_use_simulator))
 
+# Simulator tests can interfer with each other when they access the same device
+# kind, so prevent them from running at the same time.
+lit_config.parallelism_groups["apple-simulator"] = 1
+
 # Set a default per-test timeout of 10 minutes. Setting a timeout per test
 # requires that killProcessAndChildren() is supported on the platform and
 # lit complains if the value is set but it is not supported.
diff --git a/lldb/test/API/macosx/simulator/lit.local.cfg b/lldb/test/API/macosx/simulator/lit.local.cfg
new file mode 100644
index 0000000000000..864a447c3caac
--- /dev/null
+++ b/lldb/test/API/macosx/simulator/lit.local.cfg
@@ -0,0 +1 @@
+config.parallelism_group = "apple-simulator"
diff --git a/lldb/test/API/tools/lldb-server/apple-simulator/Makefile b/lldb/test/API/tools/lldb-server/apple-simulator/Makefile
new file mode 100644
index 0000000000000..2eba9e4c435f2
--- /dev/null
+++ b/lldb/test/API/tools/lldb-server/apple-simulator/Makefile
@@ -0,0 +1,5 @@
+CXX_SOURCES := main.cpp
+ENABLE_THREADS := YES
+MAKE_DSYM := NO
+
+include Makefile.rules
diff --git a/lldb/test/API/tools/lldb-server/TestAppleSimulatorOSType.py b/lldb/test/API/tools/lldb-server/apple-simulator/TestAppleSimulatorOSType.py
similarity index 100%
rename from lldb/test/API/tools/lldb-server/TestAppleSimulatorOSType.py
rename to lldb/test/API/tools/lldb-server/apple-simulator/TestAppleSimulatorOSType.py
diff --git a/lldb/test/API/tools/lldb-server/apple-simulator/lit.local.cfg b/lldb/test/API/tools/lldb-server/apple-simulator/lit.local.cfg
new file mode 100644
index 0000000000000..864a447c3caac
--- /dev/null
+++ b/lldb/test/API/tools/lldb-server/apple-simulator/lit.local.cfg
@@ -0,0 +1 @@
+config.parallelism_group = "apple-simulator"
diff --git a/lldb/test/API/tools/lldb-server/apple-simulator/main.cpp b/lldb/test/API/tools/lldb-server/apple-simulator/main.cpp
new file mode 100644
index 0000000000000..5bba4c7457a1d
--- /dev/null
+++ b/lldb/test/API/tools/lldb-server/apple-simulator/main.cpp
@@ -0,0 +1,19 @@
+#include <chrono>
+#include <cstdio>
+#include <cstdlib>
+#include <cstring>
+#include <thread>
+#include <unistd.h>
+
+int main(int argc, char **argv) {
+  for (int i = 1; i < argc; ++i) {
+    if (std::strcmp(argv[i], "print-pid") == 0) {
+      std::fprintf(stderr, "PID: %d\n", getpid());
+      std::fflush(stderr);
+    } else if (std::strncmp(argv[i], "sleep:", 6) == 0) {
+      int seconds = std::atoi(argv[i] + 6);
+      std::this_thread::sleep_for(std::chrono::seconds(seconds));
+    }
+  }
+  return 0;
+}

``````````

</details>


https://github.com/llvm/llvm-project/pull/201870