[llvm] [orc-rt] Add regression test-tool infrastructure (PR #208398)

[Lang Hames via llvm-commits]

https://github.com/lhames created https://github.com/llvm/llvm-project/pull/208398

Set up the infrastructure for regression tests that build and run a helper tool, in preparation for testing the logging backends.

- Add test/tools/ for test-support binaries, with a first tool, orc-rt-smoke-check, wired into ORC_RT_TEST_DEPS and onto lit's PATH.
- Add test/regression/smoke-check.test, which runs the tool and matches its output with FileCheck, exercising the tool-build and lit plumbing end to end.

>From 31805e092da5a9a05cb60a3ea6547b9e6f36ae7c Mon Sep 17 00:00:00 2001
From: Lang Hames <lhames at gmail.com>
Date: Thu, 9 Jul 2026 17:30:35 +1000
Subject: [PATCH] [orc-rt] Add regression test-tool infrastructure

Set up the infrastructure for regression tests that build and run a
helper tool, in preparation for testing the logging backends.

- Add test/tools/ for test-support binaries, with a first tool,
  orc-rt-smoke-check, wired into ORC_RT_TEST_DEPS and onto lit's PATH.
- Add test/regression/smoke-check.test, which runs the tool and matches
  its output with FileCheck, exercising the tool-build and lit plumbing
  end to end.
---
 orc-rt/test/CMakeLists.txt               |  4 ++++
 orc-rt/test/regression/lit.cfg.py        |  6 ++++++
 orc-rt/test/regression/smoke-check.test  |  5 +++++
 orc-rt/test/tools/CMakeLists.txt         |  4 ++++
 orc-rt/test/tools/orc-rt-smoke-check.cpp | 21 +++++++++++++++++++++
 5 files changed, 40 insertions(+)
 create mode 100644 orc-rt/test/regression/smoke-check.test
 create mode 100644 orc-rt/test/tools/CMakeLists.txt
 create mode 100644 orc-rt/test/tools/orc-rt-smoke-check.cpp

diff --git a/orc-rt/test/CMakeLists.txt b/orc-rt/test/CMakeLists.txt
index 12cac0c4dc3a3..f9dd6a3b0a8cd 100644
--- a/orc-rt/test/CMakeLists.txt
+++ b/orc-rt/test/CMakeLists.txt
@@ -8,8 +8,12 @@ if (ORC_RT_LLVM_TOOLS_AVAILABLE)
     ${CMAKE_CURRENT_SOURCE_DIR}/regression/lit.cfg.py
   )
 
+  # Build the regression test-support tools.
+  add_subdirectory(tools)
+
   list(APPEND ORC_RT_TEST_DEPS
     orc-executor
+    orc-rt-smoke-check
   )
 
   add_custom_target(orc-rt-test-depends DEPENDS ${ORC_RT_TEST_DEPS})
diff --git a/orc-rt/test/regression/lit.cfg.py b/orc-rt/test/regression/lit.cfg.py
index 002d1cb99371d..c7b5d8a99890a 100644
--- a/orc-rt/test/regression/lit.cfg.py
+++ b/orc-rt/test/regression/lit.cfg.py
@@ -21,4 +21,10 @@
     os.path.join(config.orc_rt_obj_root, "tools", "orc-executor"),
     append_path=True)
 
+# Regression test-support tools live under test/tools.
+llvm_config.with_environment(
+    "PATH",
+    os.path.join(config.orc_rt_obj_root, "test", "tools"),
+    append_path=True)
+
 llvm_config.use_default_substitutions()
diff --git a/orc-rt/test/regression/smoke-check.test b/orc-rt/test/regression/smoke-check.test
new file mode 100644
index 0000000000000..5db5c2411a5c2
--- /dev/null
+++ b/orc-rt/test/regression/smoke-check.test
@@ -0,0 +1,5 @@
+# Smoke-check the regression test-tool infrastructure: that a tool under
+# test/tools is built, is on lit's PATH, and that its output can be matched.
+RUN: orc-rt-smoke-check | FileCheck %s
+
+CHECK: smoke check
diff --git a/orc-rt/test/tools/CMakeLists.txt b/orc-rt/test/tools/CMakeLists.txt
new file mode 100644
index 0000000000000..64708d58d53ef
--- /dev/null
+++ b/orc-rt/test/tools/CMakeLists.txt
@@ -0,0 +1,4 @@
+# Tools used by the ORC-RT regression tests. These are test-support binaries
+# built into the test tree; they are not installed with the runtime.
+
+add_executable(orc-rt-smoke-check orc-rt-smoke-check.cpp)
diff --git a/orc-rt/test/tools/orc-rt-smoke-check.cpp b/orc-rt/test/tools/orc-rt-smoke-check.cpp
new file mode 100644
index 0000000000000..0e0f1b0630970
--- /dev/null
+++ b/orc-rt/test/tools/orc-rt-smoke-check.cpp
@@ -0,0 +1,21 @@
+//===- orc-rt-smoke-check.cpp ---------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// A minimal regression-test tool. It exists only to smoke-check that the ORC
+// runtime regression test-tool infrastructure works end to end: that a tool
+// under test/tools is built, placed where lit can find it, and that its output
+// can be matched by a regression test.
+//
+//===----------------------------------------------------------------------===//
+
+#include <cstdio>
+
+int main() {
+  std::puts("smoke check");
+  return 0;
+}