[Lldb-commits] [lldb] [lldb][test] Don't run simulator tests in parallel to stabilize them (PR #201870)

[Raphael Isemann via lldb-commits]

https://github.com/Teemperor created https://github.com/llvm/llvm-project/pull/201870

TestSimulatorPlatform is a notoriously flaky test on macOS. However, it seems the test failures only happen when its exectuion overlaps with the execution of TestAppleSimulatorOSType.py (which also interacts with the simulator). Somehow one simulator test seems to break the other one, but it's not exactly clear how they are inteferring with each other.

This patch ensures these two tests never run in parallel to avoid these kind of issues. It creates a parallelism_group in lit for apple-simulator and then puts both tests into that group.

TestAppleSimulatorOSType.py had to be moved to its own directory for this. It shared its directory with a a lot of other unrelated tests and the lit config applies to the whole directory.

>From e27f74965bdd677cd321adaf743aab510e69e21a Mon Sep 17 00:00:00 2001
From: Raphael Isemann <rise at apple.com>
Date: Fri, 5 Jun 2026 16:45:05 +0100
Subject: [PATCH] [lldb][test] Don't run simulator tests in parallel to
 stabilize them

TestSimulatorPlatform is a notoriously flaky test on macOS. However,
it seems the test failures only happen when its exectuion overlaps with
the execution of TestAppleSimulatorOSType.py (which also interacts
with the simulator). Somehow one simulator test seems to break the
other one, but it's not exactly clear how they are inteferring with
each other.

This patch ensures these two tests never run in parallel to avoid
these kind of issues. It creates a parallelism_group in lit for
apple-simulator and then puts both tests into that group.

TestAppleSimulatorOSType.py had to be moved to its own directory
for this. It shared its directory with a a lot of other unrelated
tests and the lit config applies to the whole directory.
---
 lldb/test/API/lit.cfg.py                      |  4 ++++
 lldb/test/API/macosx/simulator/lit.local.cfg  |  1 +
 .../lldb-server/apple-simulator/Makefile      |  5 +++++
 .../TestAppleSimulatorOSType.py               |  0
 .../lldb-server/apple-simulator/lit.local.cfg |  1 +
 .../lldb-server/apple-simulator/main.cpp      | 19 +++++++++++++++++++
 6 files changed, 30 insertions(+)
 create mode 100644 lldb/test/API/macosx/simulator/lit.local.cfg
 create mode 100644 lldb/test/API/tools/lldb-server/apple-simulator/Makefile
 rename lldb/test/API/tools/lldb-server/{ => apple-simulator}/TestAppleSimulatorOSType.py (100%)
 create mode 100644 lldb/test/API/tools/lldb-server/apple-simulator/lit.local.cfg
 create mode 100644 lldb/test/API/tools/lldb-server/apple-simulator/main.cpp

diff --git a/lldb/test/API/lit.cfg.py b/lldb/test/API/lit.cfg.py
index 2662a77199641..d1a28a842371b 100644
--- a/lldb/test/API/lit.cfg.py
+++ b/lldb/test/API/lit.cfg.py
@@ -194,6 +194,10 @@ def delete_module_cache(path):
     else:
         lit_config.error("Unknown simulator id '{}'".format(lldb_use_simulator))
 
+# Simulator tests can interfer with each other when they access the same device
+# kind, so prevent them from running at the same time.
+lit_config.parallelism_groups["apple-simulator"] = 1
+
 # Set a default per-test timeout of 10 minutes. Setting a timeout per test
 # requires that killProcessAndChildren() is supported on the platform and
 # lit complains if the value is set but it is not supported.
diff --git a/lldb/test/API/macosx/simulator/lit.local.cfg b/lldb/test/API/macosx/simulator/lit.local.cfg
new file mode 100644
index 0000000000000..864a447c3caac
--- /dev/null
+++ b/lldb/test/API/macosx/simulator/lit.local.cfg
@@ -0,0 +1 @@
+config.parallelism_group = "apple-simulator"
diff --git a/lldb/test/API/tools/lldb-server/apple-simulator/Makefile b/lldb/test/API/tools/lldb-server/apple-simulator/Makefile
new file mode 100644
index 0000000000000..2eba9e4c435f2
--- /dev/null
+++ b/lldb/test/API/tools/lldb-server/apple-simulator/Makefile
@@ -0,0 +1,5 @@
+CXX_SOURCES := main.cpp
+ENABLE_THREADS := YES
+MAKE_DSYM := NO
+
+include Makefile.rules
diff --git a/lldb/test/API/tools/lldb-server/TestAppleSimulatorOSType.py b/lldb/test/API/tools/lldb-server/apple-simulator/TestAppleSimulatorOSType.py
similarity index 100%
rename from lldb/test/API/tools/lldb-server/TestAppleSimulatorOSType.py
rename to lldb/test/API/tools/lldb-server/apple-simulator/TestAppleSimulatorOSType.py
diff --git a/lldb/test/API/tools/lldb-server/apple-simulator/lit.local.cfg b/lldb/test/API/tools/lldb-server/apple-simulator/lit.local.cfg
new file mode 100644
index 0000000000000..864a447c3caac
--- /dev/null
+++ b/lldb/test/API/tools/lldb-server/apple-simulator/lit.local.cfg
@@ -0,0 +1 @@
+config.parallelism_group = "apple-simulator"
diff --git a/lldb/test/API/tools/lldb-server/apple-simulator/main.cpp b/lldb/test/API/tools/lldb-server/apple-simulator/main.cpp
new file mode 100644
index 0000000000000..5bba4c7457a1d
--- /dev/null
+++ b/lldb/test/API/tools/lldb-server/apple-simulator/main.cpp
@@ -0,0 +1,19 @@
+#include <chrono>
+#include <cstdio>
+#include <cstdlib>
+#include <cstring>
+#include <thread>
+#include <unistd.h>
+
+int main(int argc, char **argv) {
+  for (int i = 1; i < argc; ++i) {
+    if (std::strcmp(argv[i], "print-pid") == 0) {
+      std::fprintf(stderr, "PID: %d\n", getpid());
+      std::fflush(stderr);
+    } else if (std::strncmp(argv[i], "sleep:", 6) == 0) {
+      int seconds = std::atoi(argv[i] + 6);
+      std::this_thread::sleep_for(std::chrono::seconds(seconds));
+    }
+  }
+  return 0;
+}