[clang] [lld] [llvm] [WIP][clang][lld] Dispatch registered subtools through LLVMToolSession (PR #222531)
Anutosh Bhat via cfe-commits
cfe-commits at lists.llvm.org
Thu Sep 10 00:38:54 PDT 2026
https://github.com/anutosh491 created https://github.com/llvm/llvm-project/pull/222531
Please Read: [RFC: Embeddable LLVM tool drivers for long-lived hosts](https://discourse.llvm.org/t/rfc-embeddable-llvm-tool-drivers-for-long-lived-hosts/91754) and [RFC section](https://discourse.llvm.org/t/rfc-embeddable-llvm-tool-drivers-for-long-lived-hosts/91754#p-367966-why-initllvm-ownership-matters-7)
Depends on [#221996](https://github.com/llvm/llvm-project/pull/221996) (which is the first commit here)
This is the second PR in the embeddable-tools series. It is based on #221996, which introduces `LLVMToolSession`: a long-lived LLVM host that owns LLVM initialization and a registry of callable tools.
[WasmBolt](https://github.com/anutosh491/WasmBolt) (try [here](https://anutosh21.github.io/WasmBolt/)) can currently run Clang and `wasm-ld` as separate in-process commands:
```bash
clang++ --target=wasm32-unknown-emscripten -c add.cpp -o add.o
clang++ --target=wasm32-unknown-emscripten -c main.cpp -o main.o
wasm-ld -shared add.o main.o -o project.wasm
```
The goal of this patch is to support the natural Clang driver invocation:
```
clang++ --target=wasm32-unknown-emscripten add.cpp main.cpp -o project.wasm
```
The resulting execution model is:
```
one LLVMToolSession
│
└── clang_main
├── cc1(add.cpp)
├── cc1(main.cpp)
└── lld_main(add.o, main.o)
│
└── project.wasm
```
Clang still constructs its normal compilation jobs. Before spawning a subprocess, it checks whether the requested tool is registered and owned by the surrounding LLVMToolSession. Eligible jobs are called through their registered entry points; other commands retain the normal subprocess path. A browser host can select this policy once when creating the session.
The ownership check also prevents a registered tool such as ld from accidentally replacing an unrelated system executable such as /usr/bin/ld.
LLD requires one additional lifetime adjustment. Its normal standalone path can terminate early and rely on process exit for cleanup. When invoked through an LLVMToolSession, it instead follows its reusable cleanup path and returns control to Clang.
The tests exercise two Clang frontend jobs followed by an in-process Wasm link and verify that the resulting WebAssembly module is valid. Existing native subprocess behavior and the LLD-as-a-library tests continue to pass.
A longer-term WasmBolt goal is to expose this through JupyterLite terminal (https://github.com/jupyterlite/terminal)
- clone a repository into the browser filesystem (my colleagues are working on libgit for wasm)
- build it use cmake and make (while involving clang & wasm-ld and the compiler & linker)
- use the build in the Jupyterlite kernel like xeus-cpp
Tagging @vgvassilev @mcbarton and @aganea would might be interested.
>From 1ce2fb65646da3593a7ae2e2901cafb32344696b Mon Sep 17 00:00:00 2001
From: anutosh491 <andersonbhat491 at gmail.com>
Date: Tue, 8 Sep 2026 18:42:12 +0530
Subject: [PATCH 1/2] [Support] Add LLVMToolSession for in-process tool
invocation
---
llvm/include/llvm/Support/LLVMDriver.h | 67 +++++++++-
llvm/lib/Support/CMakeLists.txt | 1 +
llvm/lib/Support/LLVMToolSession.cpp | 119 ++++++++++++++++++
.../tools/llvm-driver/session-dispatch.test | 7 ++
llvm/tools/llvm-driver/llvm-driver.cpp | 66 +++-------
llvm/unittests/Support/CMakeLists.txt | 1 +
.../Support/LLVMToolSession/CMakeLists.txt | 20 +++
.../LLVMToolSession/LLVMToolSessionTest.cpp | 65 ++++++++++
8 files changed, 298 insertions(+), 48 deletions(-)
create mode 100644 llvm/lib/Support/LLVMToolSession.cpp
create mode 100644 llvm/test/tools/llvm-driver/session-dispatch.test
create mode 100644 llvm/unittests/Support/LLVMToolSession/CMakeLists.txt
create mode 100644 llvm/unittests/Support/LLVMToolSession/LLVMToolSessionTest.cpp
diff --git a/llvm/include/llvm/Support/LLVMDriver.h b/llvm/include/llvm/Support/LLVMDriver.h
index 0b2e265d50b42..48122bfaceb7c 100644
--- a/llvm/include/llvm/Support/LLVMDriver.h
+++ b/llvm/include/llvm/Support/LLVMDriver.h
@@ -9,9 +9,35 @@
#ifndef LLVM_SUPPORT_LLVMDRIVER_H
#define LLVM_SUPPORT_LLVMDRIVER_H
+#include "llvm/ADT/ArrayRef.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/Compiler.h"
+#include "llvm/Support/ErrorOr.h"
+
+#include <memory>
+
namespace llvm {
-struct ToolContext {
+class LLVMToolSession;
+class ToolContext;
+
+using ToolMainFn = int (*)(int, char **, const ToolContext &);
+
+/// An LLVM command-line tool that can be invoked without creating a process.
+struct CallableTool {
+ StringRef Name;
+ ToolMainFn Main;
+
+ explicit operator bool() const { return Main != nullptr; }
+};
+
+/// Describes how a tool was invoked and provides access to its host session.
+class ToolContext {
+ LLVMToolSession *Session = nullptr;
+
+ friend class LLVMToolSession;
+
+public:
const char *Path;
const char *PrependArg;
// PrependArg will be added unconditionally by the llvm-driver, but
@@ -20,6 +46,45 @@ struct ToolContext {
// point to the llvm-driver executable, where PrependArg will be needed to
// invoke the correct tool.
bool NeedsPrependArg;
+
+ ToolContext(const char *Path, const char *PrependArg, bool NeedsPrependArg)
+ : Path(Path), PrependArg(PrependArg), NeedsPrependArg(NeedsPrependArg) {}
+
+ /// Finds a tool registered with the session that owns this context.
+ LLVM_ABI ErrorOr<CallableTool> getCallableTool(StringRef Name) const;
+
+ /// Invokes another tool registered with the same host session.
+ LLVM_ABI int callTool(ArrayRef<const char *> Args) const;
+};
+
+/// Owns LLVM process initialization and an in-process tool registry.
+///
+/// A long-lived host constructs one session and uses it for every embedded
+/// tool invocation. The individual tools borrow a ToolContext and therefore do
+/// not initialize or shut down LLVM themselves.
+class LLVM_ABI LLVMToolSession {
+public:
+ LLVMToolSession(int &Argc, char **&Argv, ArrayRef<CallableTool> Tools,
+ bool InstallPipeSignalExitHandler = true,
+ bool NeedsPOSIXUtilitySignalHandling = false);
+ ~LLVMToolSession();
+
+ LLVMToolSession(const LLVMToolSession &) = delete;
+ LLVMToolSession &operator=(const LLVMToolSession &) = delete;
+
+ /// Invokes the tool named by Args[0]. Args may instead contain a
+ /// process-style argv beginning with the session executable or an LLVM
+ /// multicall name.
+ int callTool(ArrayRef<const char *> Args);
+
+private:
+ struct Impl;
+ std::unique_ptr<Impl> PImpl;
+
+ ErrorOr<CallableTool> findTool(StringRef Name) const;
+ ToolContext makeContext(StringRef InvokedName);
+
+ friend class ToolContext;
};
} // namespace llvm
diff --git a/llvm/lib/Support/CMakeLists.txt b/llvm/lib/Support/CMakeLists.txt
index c85c784993e20..a834fef252d70 100644
--- a/llvm/lib/Support/CMakeLists.txt
+++ b/llvm/lib/Support/CMakeLists.txt
@@ -222,6 +222,7 @@ add_llvm_component_library(LLVMSupport
LineIterator.cpp
Locale.cpp
LockFileManager.cpp
+ LLVMToolSession.cpp
ManagedStatic.cpp
MathExtras.cpp
MemAlloc.cpp
diff --git a/llvm/lib/Support/LLVMToolSession.cpp b/llvm/lib/Support/LLVMToolSession.cpp
new file mode 100644
index 0000000000000..94f596a9d8903
--- /dev/null
+++ b/llvm/lib/Support/LLVMToolSession.cpp
@@ -0,0 +1,119 @@
+//===-- LLVMToolSession.cpp ----------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/Support/LLVMDriver.h"
+
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/StringExtras.h"
+#include "llvm/Support/InitLLVM.h"
+#include "llvm/Support/Path.h"
+
+#include <string>
+#include <system_error>
+#include <utility>
+#include <vector>
+
+using namespace llvm;
+
+namespace {
+
+bool matchesToolName(StringRef RegisteredName, StringRef InvokedName) {
+ StringRef Stem = sys::path::stem(InvokedName);
+ auto Matches = [RegisteredName](StringRef Candidate) {
+ size_t Position = Candidate.rfind_insensitive(RegisteredName);
+ return Position != StringRef::npos &&
+ (Position + RegisteredName.size() == Candidate.size() ||
+ !llvm::isAlnum(Candidate[Position + RegisteredName.size()]));
+ };
+ return Matches(Stem) || Matches(sys::path::filename(InvokedName));
+}
+
+bool isMulticallName(StringRef Name) { return matchesToolName("llvm", Name); }
+
+} // namespace
+
+struct LLVMToolSession::Impl {
+ InitLLVM Initialization;
+ std::string ExecutablePath;
+ std::vector<std::pair<std::string, ToolMainFn>> Tools;
+
+ Impl(int &Argc, char **&Argv, ArrayRef<CallableTool> RegisteredTools,
+ bool InstallPipeSignalExitHandler, bool NeedsPOSIXUtilitySignalHandling)
+ : Initialization(Argc, Argv, InstallPipeSignalExitHandler,
+ NeedsPOSIXUtilitySignalHandling),
+ ExecutablePath(Argv[0]) {
+ Tools.reserve(RegisteredTools.size());
+ for (const CallableTool &Tool : RegisteredTools)
+ Tools.emplace_back(Tool.Name.str(), Tool.Main);
+ }
+};
+
+LLVMToolSession::LLVMToolSession(int &Argc, char **&Argv,
+ ArrayRef<CallableTool> Tools,
+ bool InstallPipeSignalExitHandler,
+ bool NeedsPOSIXUtilitySignalHandling)
+ : PImpl(std::make_unique<Impl>(Argc, Argv, Tools,
+ InstallPipeSignalExitHandler,
+ NeedsPOSIXUtilitySignalHandling)) {}
+
+LLVMToolSession::~LLVMToolSession() = default;
+
+ErrorOr<CallableTool> LLVMToolSession::findTool(StringRef Name) const {
+ for (const auto &[RegisteredName, Main] : PImpl->Tools)
+ if (matchesToolName(RegisteredName, Name))
+ return CallableTool{RegisteredName, Main};
+ return make_error_code(std::errc::no_such_file_or_directory);
+}
+
+ToolContext LLVMToolSession::makeContext(StringRef InvokedName) {
+ bool NeedsPrependArg = !matchesToolName(InvokedName, PImpl->ExecutablePath);
+ ToolContext Context(PImpl->ExecutablePath.c_str(), InvokedName.data(),
+ NeedsPrependArg);
+ Context.Session = this;
+ return Context;
+}
+
+int LLVMToolSession::callTool(ArrayRef<const char *> Args) {
+ if (Args.empty())
+ return -1;
+
+ StringRef InvokedName = Args.front();
+ ErrorOr<CallableTool> Tool = findTool(InvokedName);
+ if (!Tool) {
+ if (InvokedName != PImpl->ExecutablePath && !isMulticallName(InvokedName))
+ return -1;
+ Args = Args.drop_front();
+ if (Args.empty())
+ return -1;
+ InvokedName = Args.front();
+ Tool = findTool(InvokedName);
+ }
+
+ if (!Tool)
+ return -1;
+
+ ToolContext Context = makeContext(InvokedName);
+ SmallVector<char *, 16> MutableArgs;
+ MutableArgs.reserve(Args.size() + 1);
+ for (const char *Arg : Args)
+ MutableArgs.push_back(const_cast<char *>(Arg));
+ MutableArgs.push_back(nullptr);
+ return Tool->Main(Args.size(), MutableArgs.data(), Context);
+}
+
+ErrorOr<CallableTool> ToolContext::getCallableTool(StringRef Name) const {
+ if (!Session)
+ return make_error_code(std::errc::operation_not_permitted);
+ return Session->findTool(Name);
+}
+
+int ToolContext::callTool(ArrayRef<const char *> Args) const {
+ if (!Session)
+ return -1;
+ return Session->callTool(Args);
+}
diff --git a/llvm/test/tools/llvm-driver/session-dispatch.test b/llvm/test/tools/llvm-driver/session-dispatch.test
new file mode 100644
index 0000000000000..6ac93be6ce431
--- /dev/null
+++ b/llvm/test/tools/llvm-driver/session-dispatch.test
@@ -0,0 +1,7 @@
+# REQUIRES: llvm-driver
+
+## Exercise a real LLVM tool entry point through LLVMToolSession rather than
+## only testing the registry with synthetic callbacks.
+# RUN: %llvm cxxfilt _Z3foov | FileCheck %s
+
+# CHECK: foo()
diff --git a/llvm/tools/llvm-driver/llvm-driver.cpp b/llvm/tools/llvm-driver/llvm-driver.cpp
index 14ce162faee46..865d6d432d473 100644
--- a/llvm/tools/llvm-driver/llvm-driver.cpp
+++ b/llvm/tools/llvm-driver/llvm-driver.cpp
@@ -6,14 +6,11 @@
//
//===----------------------------------------------------------------------===//
-#include "llvm/ADT/StringExtras.h"
+#include "llvm/ADT/SmallVector.h"
#include "llvm/ADT/StringRef.h"
-#include "llvm/Support/CommandLine.h"
-#include "llvm/Support/ErrorHandling.h"
-#include "llvm/Support/InitLLVM.h"
#include "llvm/Support/LLVMDriver.h"
#include "llvm/Support/Path.h"
-#include "llvm/Support/WithColor.h"
+#include "llvm/Support/raw_ostream.h"
using namespace llvm;
@@ -36,51 +33,26 @@ static void printHelpMessage() {
<< "OPTIONS:\n\n --help - Display this message\n";
}
-static int findTool(int Argc, char **Argv, const char *Argv0) {
- if (!Argc) {
- printHelpMessage();
- return 1;
- }
+int main(int Argc, char **Argv) {
+ const CallableTool Tools[] = {
+#define LLVM_DRIVER_TOOL(tool, entry) {tool, entry##_main},
+#include "LLVMDriverTools.def"
+ };
- StringRef ToolName = Argv[0];
+ LLVMToolSession Session(Argc, Argv, Tools);
- if (ToolName == "--help") {
+ StringRef Stem = sys::path::stem(Argv[0]);
+ if (Stem.equals_insensitive("llvm") &&
+ (Argc == 1 || (Argc == 2 && StringRef(Argv[1]) == "--help"))) {
printHelpMessage();
- return 0;
+ return Argc == 1 ? 1 : 0;
}
- StringRef Stem = sys::path::stem(ToolName);
- auto Is = [=](StringRef Tool) {
- auto IsImpl = [=](StringRef Stem) {
- auto I = Stem.rfind_insensitive(Tool);
- return I != StringRef::npos && (I + Tool.size() == Stem.size() ||
- !llvm::isAlnum(Stem[I + Tool.size()]));
- };
- for (StringRef S : {Stem, sys::path::filename(ToolName)})
- if (IsImpl(S))
- return true;
- return false;
- };
-
- auto MakeDriverArgs = [=]() -> llvm::ToolContext {
- if (ToolName != Argv0)
- return {Argv0, ToolName.data(), true};
- return {Argv0, sys::path::filename(Argv0).data(), false};
- };
-
-#define LLVM_DRIVER_TOOL(tool, entry) \
- if (Is(tool)) \
- return entry##_main(Argc, Argv, MakeDriverArgs());
-#include "LLVMDriverTools.def"
-
- if (Is("llvm") || Argv0 == Argv[0])
- return findTool(Argc - 1, Argv + 1, Argv0);
-
- printHelpMessage();
- return 1;
-}
-
-int main(int Argc, char **Argv) {
- llvm::InitLLVM X(Argc, Argv);
- return findTool(Argc, Argv, Argv[0]);
+ SmallVector<const char *, 16> Args(Argv, Argv + Argc);
+ int Result = Session.callTool(Args);
+ if (Result == -1) {
+ printHelpMessage();
+ return 1;
+ }
+ return Result;
}
diff --git a/llvm/unittests/Support/CMakeLists.txt b/llvm/unittests/Support/CMakeLists.txt
index cc88a6c5670ca..f9732ff4e0e4b 100644
--- a/llvm/unittests/Support/CMakeLists.txt
+++ b/llvm/unittests/Support/CMakeLists.txt
@@ -162,3 +162,4 @@ if(NOT LLVM_INTEGRATED_CRT_ALLOC)
endif()
add_subdirectory(CommandLineInit)
+add_subdirectory(LLVMToolSession)
diff --git a/llvm/unittests/Support/LLVMToolSession/CMakeLists.txt b/llvm/unittests/Support/LLVMToolSession/CMakeLists.txt
new file mode 100644
index 0000000000000..2d343295fb191
--- /dev/null
+++ b/llvm/unittests/Support/LLVMToolSession/CMakeLists.txt
@@ -0,0 +1,20 @@
+set(test_name LLVMToolSessionTests)
+set(test_suite UnitTests)
+
+# This test supplies its own main() so a single LLVMToolSession can own
+# InitLLVM for the complete test process.
+if (NOT LLVM_BUILD_TESTS)
+ set(EXCLUDE_FROM_ALL ON)
+endif()
+
+list(APPEND LLVM_LINK_COMPONENTS Support)
+
+add_llvm_executable(${test_name}
+ IGNORE_EXTERNALIZE_DEBUGINFO NO_INSTALL_RPATH
+ LLVMToolSessionTest.cpp)
+
+target_link_libraries(${test_name} PRIVATE llvm_gtest)
+add_dependencies(${test_suite} ${test_name})
+
+set(outdir ${CMAKE_CURRENT_BINARY_DIR}/${CMAKE_CFG_INTDIR})
+set_output_directory(${test_name} BINARY_DIR ${outdir} LIBRARY_DIR ${outdir})
diff --git a/llvm/unittests/Support/LLVMToolSession/LLVMToolSessionTest.cpp b/llvm/unittests/Support/LLVMToolSession/LLVMToolSessionTest.cpp
new file mode 100644
index 0000000000000..a8a12f3a7e4d1
--- /dev/null
+++ b/llvm/unittests/Support/LLVMToolSession/LLVMToolSessionTest.cpp
@@ -0,0 +1,65 @@
+//===- LLVMToolSessionTest.cpp -------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/Support/LLVMDriver.h"
+#include "gtest/gtest.h"
+
+#include <memory>
+
+using namespace llvm;
+
+namespace {
+
+std::unique_ptr<LLVMToolSession> Session;
+unsigned CompilerCalls;
+unsigned LinkerCalls;
+
+int linkerMain(int Argc, char **Argv, const ToolContext &Context) {
+ ++LinkerCalls;
+ EXPECT_EQ(Argc, 3);
+ EXPECT_STREQ(Argv[0], "wasm-ld");
+ EXPECT_TRUE(Context.getCallableTool("clang"));
+ return 0;
+}
+
+int compilerMain(int Argc, char **Argv, const ToolContext &Context) {
+ ++CompilerCalls;
+ EXPECT_EQ(Argc, 2);
+ EXPECT_STREQ(Argv[0], "clang");
+ const char *LinkArgs[] = {"wasm-ld", Argv[1], "out.wasm"};
+ return Context.callTool(LinkArgs);
+}
+
+TEST(LLVMToolSessionTest, SupportsSequentialNestedToolCalls) {
+ const char *First[] = {"clang", "first.cpp"};
+ const char *Second[] = {"clang", "second.cpp"};
+
+ EXPECT_EQ(Session->callTool(First), 0);
+ EXPECT_EQ(Session->callTool(Second), 0);
+ EXPECT_EQ(CompilerCalls, 2u);
+ EXPECT_EQ(LinkerCalls, 2u);
+}
+
+TEST(LLVMToolSessionTest, ReportsUnknownTools) {
+ const char *Args[] = {"not-an-llvm-tool"};
+ EXPECT_EQ(Session->callTool(Args), -1);
+}
+
+} // namespace
+
+int main(int Argc, char **Argv) {
+ const CallableTool Tools[] = {
+ {"clang", compilerMain},
+ {"wasm-ld", linkerMain},
+ };
+ Session = std::make_unique<LLVMToolSession>(Argc, Argv, Tools);
+ testing::InitGoogleTest(&Argc, Argv);
+ int Result = RUN_ALL_TESTS();
+ Session.reset();
+ return Result;
+}
>From 05097d159768993d252d8268ff5b8e19e70a3196 Mon Sep 17 00:00:00 2001
From: anutosh491 <andersonbhat491 at gmail.com>
Date: Thu, 10 Sep 2026 12:22:00 +0530
Subject: [PATCH 2/2] [clang][lld] Dispatch registered subtools through
LLVMToolSession
---
clang/include/clang/Driver/Job.h | 14 ++++++++
clang/lib/Driver/Job.cpp | 11 ++++++
clang/test/Driver/in-process-wasm-link.c | 25 ++++++++++++++
clang/tools/driver/driver.cpp | 9 +++++
lld/tools/lld/lld.cpp | 16 ++++++---
llvm/include/llvm/Support/LLVMDriver.h | 23 +++++++++++--
llvm/lib/Support/LLVMToolSession.cpp | 34 ++++++++++++++-----
.../LLVMToolSession/LLVMToolSessionTest.cpp | 7 +++-
8 files changed, 122 insertions(+), 17 deletions(-)
create mode 100644 clang/test/Driver/in-process-wasm-link.c
diff --git a/clang/include/clang/Driver/Job.h b/clang/include/clang/Driver/Job.h
index 03779be5b5a6a..c0b128852da28 100644
--- a/clang/include/clang/Driver/Job.h
+++ b/clang/include/clang/Driver/Job.h
@@ -24,6 +24,10 @@
#include <utility>
#include <vector>
+namespace llvm {
+class ToolContext;
+}
+
namespace clang {
namespace driver {
@@ -151,6 +155,9 @@ class Command {
/// Information on executable run provided by OS.
mutable std::optional<llvm::sys::ProcessStatistics> ProcStat;
+ /// The host context used to invoke this command without spawning a process.
+ const llvm::ToolContext *InProcessToolContext = nullptr;
+
/// The bound architecture for this command (e.g. "arm64", "gfx90a").
std::string BoundArchStr;
@@ -238,6 +245,13 @@ class Command {
void replaceExecutable(const char *Exe) { Executable = Exe; }
+ /// Execute this command through a tool registered with the given host
+ /// context instead of starting a subprocess.
+ void setInProcessToolContext(const llvm::ToolContext &Context) {
+ InProcessToolContext = &Context;
+ InProcess = true;
+ }
+
const char *getExecutable() const { return Executable; }
const llvm::opt::ArgStringList &getArguments() const { return Arguments; }
diff --git a/clang/lib/Driver/Job.cpp b/clang/lib/Driver/Job.cpp
index da7a1f2e07e90..415da03f86669 100644
--- a/clang/lib/Driver/Job.cpp
+++ b/clang/lib/Driver/Job.cpp
@@ -22,6 +22,7 @@
#include "llvm/Support/CrashRecoveryContext.h"
#include "llvm/Support/FileSystem.h"
#include "llvm/Support/IOSandbox.h"
+#include "llvm/Support/LLVMDriver.h"
#include "llvm/Support/Path.h"
#include "llvm/Support/PrettyStackTrace.h"
#include "llvm/Support/Program.h"
@@ -205,6 +206,9 @@ rewriteIncludes(const llvm::ArrayRef<const char *> &Args, size_t Idx,
void Command::Print(raw_ostream &OS, const char *Terminator, bool Quote,
CrashReportInfo *CrashInfo) const {
+ if (InProcessToolContext)
+ OS << " (in-process)\n";
+
// Always quote the exe.
OS << ' ';
llvm::sys::printArg(OS, Executable, /*Quote=*/true);
@@ -367,6 +371,13 @@ int Command::Execute(ArrayRef<std::optional<StringRef>> Redirects,
auto Args = llvm::toStringRefArray(Argv.data());
+ if (InProcessToolContext) {
+ if (ExecutionFailed)
+ *ExecutionFailed = false;
+ return InProcessToolContext->callTool(
+ ArrayRef<const char *>(Argv).drop_back());
+ }
+
// Use Job-specific redirect files if they are present.
if (!RedirectFiles.empty()) {
std::vector<std::optional<StringRef>> RedirectFilesOptional;
diff --git a/clang/test/Driver/in-process-wasm-link.c b/clang/test/Driver/in-process-wasm-link.c
new file mode 100644
index 0000000000000..3735dc6af423c
--- /dev/null
+++ b/clang/test/Driver/in-process-wasm-link.c
@@ -0,0 +1,25 @@
+// REQUIRES: llvm-driver, lld, webassembly-registered-target
+
+// The multicall driver registers Clang and LLD in one LLVMToolSession. Verify
+// that Clang discovers wasm-ld in that session rather than spawning it.
+// RUN: split-file %s %t
+// RUN: %clang --target=wasm32-unknown-unknown -nostdlib -fuse-ld=lld \
+// RUN: -Wl,--no-entry -### %t/add.c %t/sub.c 2>&1 \
+// RUN: | FileCheck %s --check-prefix=PRINT
+// PRINT: {{.*}}clang{{.*}}-cc1
+// PRINT: {{.*}}clang{{.*}}-cc1
+// PRINT: (in-process)
+// PRINT-NEXT: {{.*}}wasm-ld
+
+// Also exercise multiple integrated cc1 jobs followed by the in-process link.
+// RUN: %clang --target=wasm32-unknown-unknown -nostdlib -fuse-ld=lld \
+// RUN: -Wl,--no-entry -Wl,--export=add -Wl,--export=sub \
+// RUN: %t/add.c %t/sub.c -o %t.wasm
+// RUN: llvm-readobj --file-headers %t.wasm | FileCheck %s --check-prefix=WASM
+// WASM: Format: WASM
+
+//--- add.c
+int add(int lhs, int rhs) { return lhs + rhs; }
+
+//--- sub.c
+int sub(int lhs, int rhs) { return lhs - rhs; }
diff --git a/clang/tools/driver/driver.cpp b/clang/tools/driver/driver.cpp
index d4d913a8977a4..a947459c51d6a 100644
--- a/clang/tools/driver/driver.cpp
+++ b/clang/tools/driver/driver.cpp
@@ -387,6 +387,15 @@ int clang_main(int Argc, char **Argv, const llvm::ToolContext &ToolContext) {
std::unique_ptr<Compilation> C(TheDriver.BuildCompilation(Args));
+ // Prefer subtools owned by the embedding host over spawning subprocesses.
+ // Integrated cc1 jobs already have their own in-process execution path.
+ for (Command &Job : C->getJobs()) {
+ if (Job.InProcess)
+ continue;
+ if (ToolContext.canExecuteInProcess(Job.getExecutable()))
+ Job.setInProcessToolContext(ToolContext);
+ }
+
Driver::ReproLevel ReproLevel = Driver::ReproLevel::OnCrash;
if (Arg *A = C->getArgs().getLastArg(options::OPT_gen_reproducer_eq)) {
auto Level =
diff --git a/lld/tools/lld/lld.cpp b/lld/tools/lld/lld.cpp
index d6800fa1eea4b..4f07675c0513e 100644
--- a/lld/tools/lld/lld.cpp
+++ b/lld/tools/lld/lld.cpp
@@ -72,7 +72,7 @@ LLD_HAS_DRIVER(mingw)
LLD_HAS_DRIVER(macho)
LLD_HAS_DRIVER(wasm)
-int lld_main(int argc, char **argv, const llvm::ToolContext &) {
+int lld_main(int argc, char **argv, const llvm::ToolContext &ToolContext) {
sys::Process::UseANSIEscapeCodes(true);
if (::getenv("FORCE_LLD_DIAGNOSTICS_CRASH")) {
@@ -83,9 +83,12 @@ int lld_main(int argc, char **argv, const llvm::ToolContext &) {
ArrayRef<const char *> args(argv, argv + argc);
- // Not running in lit tests, just take the shortest codepath with global
- // exception handling and no memory cleanup on exit.
- if (!inTestVerbosity()) {
+ unsigned Iterations = inTestVerbosity();
+
+ // Standalone LLD can take the shortest code path and let the process reclaim
+ // its resources. An in-process invocation must instead clean up before
+ // returning to its host.
+ if (!ToolContext.isInProcess() && !Iterations) {
int r =
lld::unsafeLldMain(args, llvm::outs(), llvm::errs(), LLD_ALL_DRIVERS,
/*exitEarly=*/true);
@@ -95,7 +98,10 @@ int lld_main(int argc, char **argv, const llvm::ToolContext &) {
std::optional<int> mainRet;
CrashRecoveryContext::Enable();
- for (unsigned i = inTestVerbosity(); i > 0; --i) {
+ if (!Iterations)
+ Iterations = 1;
+
+ for (unsigned i = Iterations; i > 0; --i) {
// Disable stdout/stderr for all iterations but the last one.
inTestOutputDisabled = (i != 1);
diff --git a/llvm/include/llvm/Support/LLVMDriver.h b/llvm/include/llvm/Support/LLVMDriver.h
index 48122bfaceb7c..cc5faf9e38623 100644
--- a/llvm/include/llvm/Support/LLVMDriver.h
+++ b/llvm/include/llvm/Support/LLVMDriver.h
@@ -31,6 +31,17 @@ struct CallableTool {
explicit operator bool() const { return Main != nullptr; }
};
+/// Configures process-wide behavior owned by an LLVM tool session.
+struct LLVMToolSessionOptions {
+ /// Prefer a registered in-process tool even when its executable is not an
+ /// alias of the session executable. This is useful for hosts whose tools do
+ /// not exist as separate files, such as browser applications.
+ bool PreferInProcessTools = false;
+
+ bool InstallPipeSignalExitHandler = true;
+ bool NeedsPOSIXUtilitySignalHandling = false;
+};
+
/// Describes how a tool was invoked and provides access to its host session.
class ToolContext {
LLVMToolSession *Session = nullptr;
@@ -55,6 +66,14 @@ class ToolContext {
/// Invokes another tool registered with the same host session.
LLVM_ABI int callTool(ArrayRef<const char *> Args) const;
+
+ /// Returns true when Executable names a registered tool owned by this host.
+ /// A tool is owned when it is an alias of the session executable or when the
+ /// session explicitly prefers its registered in-process tools.
+ LLVM_ABI bool canExecuteInProcess(StringRef Executable) const;
+
+ /// Returns true when this invocation is owned by a long-lived tool session.
+ bool isInProcess() const { return Session != nullptr; }
};
/// Owns LLVM process initialization and an in-process tool registry.
@@ -65,8 +84,7 @@ class ToolContext {
class LLVM_ABI LLVMToolSession {
public:
LLVMToolSession(int &Argc, char **&Argv, ArrayRef<CallableTool> Tools,
- bool InstallPipeSignalExitHandler = true,
- bool NeedsPOSIXUtilitySignalHandling = false);
+ LLVMToolSessionOptions Options = {});
~LLVMToolSession();
LLVMToolSession(const LLVMToolSession &) = delete;
@@ -82,6 +100,7 @@ class LLVM_ABI LLVMToolSession {
std::unique_ptr<Impl> PImpl;
ErrorOr<CallableTool> findTool(StringRef Name) const;
+ bool canExecuteInProcess(StringRef Executable) const;
ToolContext makeContext(StringRef InvokedName);
friend class ToolContext;
diff --git a/llvm/lib/Support/LLVMToolSession.cpp b/llvm/lib/Support/LLVMToolSession.cpp
index 94f596a9d8903..a3ce51a4f1688 100644
--- a/llvm/lib/Support/LLVMToolSession.cpp
+++ b/llvm/lib/Support/LLVMToolSession.cpp
@@ -10,6 +10,7 @@
#include "llvm/ADT/SmallVector.h"
#include "llvm/ADT/StringExtras.h"
+#include "llvm/Support/FileSystem.h"
#include "llvm/Support/InitLLVM.h"
#include "llvm/Support/Path.h"
@@ -41,12 +42,14 @@ struct LLVMToolSession::Impl {
InitLLVM Initialization;
std::string ExecutablePath;
std::vector<std::pair<std::string, ToolMainFn>> Tools;
+ bool PreferInProcessTools;
Impl(int &Argc, char **&Argv, ArrayRef<CallableTool> RegisteredTools,
- bool InstallPipeSignalExitHandler, bool NeedsPOSIXUtilitySignalHandling)
- : Initialization(Argc, Argv, InstallPipeSignalExitHandler,
- NeedsPOSIXUtilitySignalHandling),
- ExecutablePath(Argv[0]) {
+ LLVMToolSessionOptions Options)
+ : Initialization(Argc, Argv, Options.InstallPipeSignalExitHandler,
+ Options.NeedsPOSIXUtilitySignalHandling),
+ ExecutablePath(Argv[0]),
+ PreferInProcessTools(Options.PreferInProcessTools) {
Tools.reserve(RegisteredTools.size());
for (const CallableTool &Tool : RegisteredTools)
Tools.emplace_back(Tool.Name.str(), Tool.Main);
@@ -55,11 +58,8 @@ struct LLVMToolSession::Impl {
LLVMToolSession::LLVMToolSession(int &Argc, char **&Argv,
ArrayRef<CallableTool> Tools,
- bool InstallPipeSignalExitHandler,
- bool NeedsPOSIXUtilitySignalHandling)
- : PImpl(std::make_unique<Impl>(Argc, Argv, Tools,
- InstallPipeSignalExitHandler,
- NeedsPOSIXUtilitySignalHandling)) {}
+ LLVMToolSessionOptions Options)
+ : PImpl(std::make_unique<Impl>(Argc, Argv, Tools, Options)) {}
LLVMToolSession::~LLVMToolSession() = default;
@@ -70,6 +70,18 @@ ErrorOr<CallableTool> LLVMToolSession::findTool(StringRef Name) const {
return make_error_code(std::errc::no_such_file_or_directory);
}
+bool LLVMToolSession::canExecuteInProcess(StringRef Executable) const {
+ if (!findTool(Executable))
+ return false;
+ if (PImpl->PreferInProcessTools)
+ return true;
+
+ bool IsSessionExecutable = false;
+ return !sys::fs::equivalent(PImpl->ExecutablePath, Executable,
+ IsSessionExecutable) &&
+ IsSessionExecutable;
+}
+
ToolContext LLVMToolSession::makeContext(StringRef InvokedName) {
bool NeedsPrependArg = !matchesToolName(InvokedName, PImpl->ExecutablePath);
ToolContext Context(PImpl->ExecutablePath.c_str(), InvokedName.data(),
@@ -117,3 +129,7 @@ int ToolContext::callTool(ArrayRef<const char *> Args) const {
return -1;
return Session->callTool(Args);
}
+
+bool ToolContext::canExecuteInProcess(StringRef Executable) const {
+ return Session && Session->canExecuteInProcess(Executable);
+}
diff --git a/llvm/unittests/Support/LLVMToolSession/LLVMToolSessionTest.cpp b/llvm/unittests/Support/LLVMToolSession/LLVMToolSessionTest.cpp
index a8a12f3a7e4d1..279c2600e1891 100644
--- a/llvm/unittests/Support/LLVMToolSession/LLVMToolSessionTest.cpp
+++ b/llvm/unittests/Support/LLVMToolSession/LLVMToolSessionTest.cpp
@@ -23,7 +23,10 @@ int linkerMain(int Argc, char **Argv, const ToolContext &Context) {
++LinkerCalls;
EXPECT_EQ(Argc, 3);
EXPECT_STREQ(Argv[0], "wasm-ld");
+ EXPECT_TRUE(Context.isInProcess());
EXPECT_TRUE(Context.getCallableTool("clang"));
+ EXPECT_TRUE(Context.canExecuteInProcess("clang"));
+ EXPECT_FALSE(Context.canExecuteInProcess("not-an-llvm-tool"));
return 0;
}
@@ -57,7 +60,9 @@ int main(int Argc, char **Argv) {
{"clang", compilerMain},
{"wasm-ld", linkerMain},
};
- Session = std::make_unique<LLVMToolSession>(Argc, Argv, Tools);
+ LLVMToolSessionOptions Options;
+ Options.PreferInProcessTools = true;
+ Session = std::make_unique<LLVMToolSession>(Argc, Argv, Tools, Options);
testing::InitGoogleTest(&Argc, Argv);
int Result = RUN_ALL_TESTS();
Session.reset();
More information about the cfe-commits
mailing list