[clang] [clang-repl] Rework layering of incremental executors. (PR #175448)
Vassil Vassilev via cfe-commits
cfe-commits at lists.llvm.org
Sun Jan 11 09:54:01 PST 2026
https://github.com/vgvassilev created https://github.com/llvm/llvm-project/pull/175448
The original Interpreter implementation had a hard dependency on ORC and grew organically with the addition of out-of-process JIT support. This tightly coupled the Interpreter to a specific execution engine and leaked ORC-specific assumptions (runtime layout, symbol lookup, exception model) into higher layers.
The WebAssembly integration demonstrated that incremental execution can be implemented without ORC, exposing the need for a cleaner abstraction boundary.
This change introduces an IncrementalExecutor interface and moves ORC-based execution behind a concrete implementation. The Interpreter now depends only on the abstract executor, improving layering and encapsulation.
In addition, the Interpreter can be configured with user-provided incremental executor implementations, enabling ORC-independent execution, easier testing, and future extensions without modifying the core Interpreter.
>From 4a5e6cc32fec5b485d44cff4c8f9ea44725d58ce Mon Sep 17 00:00:00 2001
From: Vassil Vassilev <v.g.vassilev at gmail.com>
Date: Sun, 11 Jan 2026 17:46:51 +0000
Subject: [PATCH] [clang-repl] Rework layering of incremental executors.
The original Interpreter implementation had a hard dependency on ORC and grew
organically with the addition of out-of-process JIT support. This tightly
coupled the Interpreter to a specific execution engine and leaked ORC-specific
assumptions (runtime layout, symbol lookup, exception model) into higher
layers.
The WebAssembly integration demonstrated that incremental execution can be
implemented without ORC, exposing the need for a cleaner abstraction boundary.
This change introduces an IncrementalExecutor interface and moves ORC-based
execution behind a concrete implementation. The Interpreter now depends only
on the abstract executor, improving layering and encapsulation.
In addition, the Interpreter can be configured with user-provided incremental
executor implementations, enabling ORC-independent execution, easier testing,
and future extensions without modifying the core Interpreter.
---
.../clang/Interpreter/IncrementalExecutor.h | 95 +++++
clang/include/clang/Interpreter/Interpreter.h | 54 +--
clang/lib/Interpreter/CMakeLists.txt | 1 +
clang/lib/Interpreter/IncrementalExecutor.cpp | 350 +++++++++++++-----
clang/lib/Interpreter/IncrementalExecutor.h | 90 -----
clang/lib/Interpreter/Interpreter.cpp | 265 +------------
.../Interpreter/OrcIncrementalExecutor.cpp | 121 ++++++
.../lib/Interpreter/OrcIncrementalExecutor.h | 71 ++++
clang/lib/Interpreter/Wasm.cpp | 5 +-
clang/lib/Interpreter/Wasm.h | 6 +-
clang/tools/clang-repl/ClangRepl.cpp | 1 +
.../Interpreter/InterpreterExtensionsTest.cpp | 71 +++-
.../OutOfProcessInterpreterTests.cpp | 6 +-
13 files changed, 631 insertions(+), 505 deletions(-)
create mode 100644 clang/include/clang/Interpreter/IncrementalExecutor.h
delete mode 100644 clang/lib/Interpreter/IncrementalExecutor.h
create mode 100644 clang/lib/Interpreter/OrcIncrementalExecutor.cpp
create mode 100644 clang/lib/Interpreter/OrcIncrementalExecutor.h
diff --git a/clang/include/clang/Interpreter/IncrementalExecutor.h b/clang/include/clang/Interpreter/IncrementalExecutor.h
new file mode 100644
index 0000000000000..5dfc02e46e6ce
--- /dev/null
+++ b/clang/include/clang/Interpreter/IncrementalExecutor.h
@@ -0,0 +1,95 @@
+//===--- IncrementalExecutor.h - Base Incremental Execution -----*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file implements the base class that performs incremental code execution.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_LIB_INTERPRETER_INCREMENTALEXECUTOR_H
+#define LLVM_CLANG_LIB_INTERPRETER_INCREMENTALEXECUTOR_H
+
+#include "llvm/Support/CodeGen.h"
+#include "llvm/Support/Error.h"
+
+namespace llvm {
+namespace orc {
+class ExecutorAddr;
+class LLJITBuilder;
+class ThreadSafeContext;
+} // namespace orc
+} // namespace llvm
+
+namespace clang {
+class IncrementalExecutor;
+class TargetInfo;
+namespace driver {
+class Compilation;
+} // namespace driver
+
+// FIXME: Consider deriving from the LLJITBuilder into a common interpreter
+// creation configuraion class.
+class IncrementalExecutorBuilder {
+public:
+ /// Indicates whether out-of-process JIT execution is enabled.
+ bool IsOutOfProcess = false;
+ /// Path to the out-of-process JIT executor.
+ std::string OOPExecutor = "";
+ std::string OOPExecutorConnect = "";
+ /// Indicates whether to use shared memory for communication.
+ bool UseSharedMemory = false;
+ /// Representing the slab allocation size for memory management in kb.
+ unsigned SlabAllocateSize = 0;
+ /// Path to the ORC runtime library.
+ std::string OrcRuntimePath = "";
+ /// PID of the out-of-process JIT executor.
+ uint32_t ExecutorPID = 0;
+ /// Custom lambda to be executed inside child process/executor
+ std::function<void()> CustomizeFork = nullptr;
+ /// An optional code model to provide to the JITTargetMachineBuilder
+ std::optional<llvm::CodeModel::Model> CM = std::nullopt;
+ /// An optional external IncrementalExecutor
+ std::unique_ptr<IncrementalExecutor> IE;
+ /// An optional external orc jit builder
+ std::unique_ptr<llvm::orc::LLJITBuilder> JITBuilder;
+ /// A default callback that can be used in the IncrementalCompilerBuilder to
+ /// retrieve the path to the orc runtime.
+ std::function<llvm::Error(const driver::Compilation &)>
+ UpdateOrcRuntimePathCB = [this](const driver::Compilation &C) {
+ return UpdateOrcRuntimePath(C);
+ };
+
+ ~IncrementalExecutorBuilder();
+
+ llvm::Expected<std::unique_ptr<IncrementalExecutor>>
+ create(llvm::orc::ThreadSafeContext &TSC, const clang::TargetInfo &TI);
+
+private:
+ llvm::Error UpdateOrcRuntimePath(const driver::Compilation &C);
+};
+
+struct PartialTranslationUnit;
+
+class IncrementalExecutor {
+public:
+ enum SymbolNameKind { IRName, LinkerName };
+
+ virtual ~IncrementalExecutor() = default;
+
+ virtual llvm::Error addModule(PartialTranslationUnit &PTU) = 0;
+ virtual llvm::Error removeModule(PartialTranslationUnit &PTU) = 0;
+ virtual llvm::Error runCtors() const = 0;
+ virtual llvm::Error cleanUp() = 0;
+
+ virtual llvm::Expected<llvm::orc::ExecutorAddr>
+ getSymbolAddress(llvm::StringRef Name, SymbolNameKind NameKind) const = 0;
+ virtual llvm::Error LoadDynamicLibrary(const char *name) = 0;
+};
+
+} // namespace clang
+
+#endif // LLVM_CLANG_LIB_INTERPRETER_INCREMENTALEXECUTOR_H
diff --git a/clang/include/clang/Interpreter/Interpreter.h b/clang/include/clang/Interpreter/Interpreter.h
index c4ddfb067be0f..4af5a55e6860e 100644
--- a/clang/include/clang/Interpreter/Interpreter.h
+++ b/clang/include/clang/Interpreter/Interpreter.h
@@ -15,6 +15,7 @@
#define LLVM_CLANG_INTERPRETER_INTERPRETER_H
#include "clang/AST/GlobalDecl.h"
+#include "clang/Interpreter/IncrementalExecutor.h"
#include "clang/Interpreter/PartialTranslationUnit.h"
#include "clang/Interpreter/Value.h"
@@ -29,8 +30,6 @@
namespace llvm {
namespace orc {
-class LLJIT;
-class LLJITBuilder;
class ThreadSafeContext;
} // namespace orc
} // namespace llvm
@@ -44,7 +43,6 @@ class Compilation;
class CompilerInstance;
class CXXRecordDecl;
class Decl;
-class IncrementalExecutor;
class IncrementalParser;
class IncrementalCUDADeviceParser;
@@ -93,42 +91,6 @@ class IncrementalCompilerBuilder {
std::optional<std::function<DriverCompilationFn>> CompilationCB;
};
-// FIXME: Consider deriving from the LLJITBuilder into a common interpreter
-// creation configuraion class.
-class IncrementalExecutorBuilder {
-public:
- /// Indicates whether out-of-process JIT execution is enabled.
- bool IsOutOfProcess = false;
- /// Path to the out-of-process JIT executor.
- std::string OOPExecutor = "";
- std::string OOPExecutorConnect = "";
- /// Indicates whether to use shared memory for communication.
- bool UseSharedMemory = false;
- /// Representing the slab allocation size for memory management in kb.
- unsigned SlabAllocateSize = 0;
- /// Path to the ORC runtime library.
- std::string OrcRuntimePath = "";
- /// PID of the out-of-process JIT executor.
- uint32_t ExecutorPID = 0;
- /// Custom lambda to be executed inside child process/executor
- std::function<void()> CustomizeFork = nullptr;
- /// An optional code model to provide to the JITTargetMachineBuilder
- std::optional<llvm::CodeModel::Model> CM = std::nullopt;
- std::function<llvm::Error(const driver::Compilation &)>
- UpdateOrcRuntimePathCB = [this](const driver::Compilation &C) {
- return UpdateOrcRuntimePath(C);
- };
-
- ~IncrementalExecutorBuilder();
-
- llvm::Expected<std::unique_ptr<IncrementalExecutor>>
- create(llvm::orc::ThreadSafeContext &TSC,
- llvm::orc::LLJITBuilder &JITBuilder);
-
-private:
- llvm::Error UpdateOrcRuntimePath(const driver::Compilation &C);
-};
-
class IncrementalAction;
class InProcessPrintingASTConsumer;
@@ -168,9 +130,8 @@ class Interpreter {
protected:
// Derived classes can use an extended interface of the Interpreter.
Interpreter(std::unique_ptr<CompilerInstance> Instance, llvm::Error &Err,
- std::unique_ptr<llvm::orc::LLJITBuilder> JITBuilder = nullptr,
- std::unique_ptr<clang::ASTConsumer> Consumer = nullptr,
- std::unique_ptr<IncrementalExecutorBuilder> IEB = nullptr);
+ std::unique_ptr<IncrementalExecutorBuilder> IEB = nullptr,
+ std::unique_ptr<clang::ASTConsumer> Consumer = nullptr);
// Create the internal IncrementalExecutor, or re-create it after calling
// ResetExecutor().
@@ -178,7 +139,7 @@ class Interpreter {
// Delete the internal IncrementalExecutor. This causes a hard shutdown of the
// JIT engine. In particular, it doesn't run cleanup or destructors.
- void ResetExecutor();
+ void ResetExecutor() { IncrExecutor.reset(); }
public:
virtual ~Interpreter();
@@ -188,15 +149,12 @@ class Interpreter {
static llvm::Expected<std::unique_ptr<Interpreter>>
createWithCUDA(std::unique_ptr<CompilerInstance> CI,
std::unique_ptr<CompilerInstance> DCI);
- static llvm::Expected<std::unique_ptr<llvm::orc::LLJITBuilder>>
- createLLJITBuilder(std::unique_ptr<llvm::orc::ExecutorProcessControl> EPC,
- llvm::StringRef OrcRuntimePath);
const ASTContext &getASTContext() const;
ASTContext &getASTContext();
const CompilerInstance *getCompilerInstance() const;
CompilerInstance *getCompilerInstance();
- llvm::Expected<llvm::orc::LLJIT &> getExecutionEngine();
+ llvm::Expected<IncrementalExecutor &> getExecutionEngine();
llvm::Expected<PartialTranslationUnit &> Parse(llvm::StringRef Code);
llvm::Error Execute(PartialTranslationUnit &T);
@@ -234,8 +192,6 @@ class Interpreter {
std::array<Expr *, 4> ValuePrintingInfo = {0};
- std::unique_ptr<llvm::orc::LLJITBuilder> JITBuilder;
-
std::unique_ptr<IncrementalExecutorBuilder> IncrExecutorBuilder;
/// @}
diff --git a/clang/lib/Interpreter/CMakeLists.txt b/clang/lib/Interpreter/CMakeLists.txt
index 9a597146b2fc4..01d3295d1ac30 100644
--- a/clang/lib/Interpreter/CMakeLists.txt
+++ b/clang/lib/Interpreter/CMakeLists.txt
@@ -24,6 +24,7 @@ add_clang_library(clangInterpreter
CodeCompletion.cpp
IncrementalAction.cpp
IncrementalExecutor.cpp
+ OrcIncrementalExecutor.cpp
IncrementalParser.cpp
Interpreter.cpp
InterpreterValuePrinter.cpp
diff --git a/clang/lib/Interpreter/IncrementalExecutor.cpp b/clang/lib/Interpreter/IncrementalExecutor.cpp
index f0069c4924f7a..282138e5c0ab2 100644
--- a/clang/lib/Interpreter/IncrementalExecutor.cpp
+++ b/clang/lib/Interpreter/IncrementalExecutor.cpp
@@ -6,56 +6,75 @@
//
//===----------------------------------------------------------------------===//
//
-// This file implements the class which performs incremental code execution.
+// This has the implementation of the base facilities for incremental execution.
//
//===----------------------------------------------------------------------===//
-#include "IncrementalExecutor.h"
+#include "clang/Interpreter/IncrementalExecutor.h"
+#include "OrcIncrementalExecutor.h"
+#ifdef __EMSCRIPTEN__
+#include "Wasm.h"
+#endif // __EMSCRIPTEN__
#include "clang/Basic/TargetInfo.h"
-#include "clang/Basic/TargetOptions.h"
-#include "clang/Interpreter/PartialTranslationUnit.h"
-#include "llvm/ADT/StringExtras.h"
-#include "llvm/ExecutionEngine/ExecutionEngine.h"
-#include "llvm/ExecutionEngine/Orc/CompileUtils.h"
+#include "clang/Driver/Compilation.h"
+#include "clang/Driver/Driver.h"
+#include "clang/Driver/ToolChain.h"
+
+#include "llvm/ADT/SmallString.h"
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/ADT/Twine.h"
+
+#include "llvm/ExecutionEngine/JITLink/JITLinkMemoryManager.h"
#include "llvm/ExecutionEngine/Orc/Debugging/DebuggerSupport.h"
#include "llvm/ExecutionEngine/Orc/EPCDynamicLibrarySearchGenerator.h"
#include "llvm/ExecutionEngine/Orc/ExecutionUtils.h"
-#include "llvm/ExecutionEngine/Orc/IRCompileLayer.h"
+#include "llvm/ExecutionEngine/Orc/ExecutorProcessControl.h"
#include "llvm/ExecutionEngine/Orc/JITTargetMachineBuilder.h"
#include "llvm/ExecutionEngine/Orc/LLJIT.h"
#include "llvm/ExecutionEngine/Orc/MapperJITLinkMemoryManager.h"
-#include "llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h"
#include "llvm/ExecutionEngine/Orc/Shared/OrcRTBridge.h"
#include "llvm/ExecutionEngine/Orc/Shared/SimpleRemoteEPCUtils.h"
-#include "llvm/ExecutionEngine/Orc/TargetProcess/JITLoaderGDB.h"
-#include "llvm/ExecutionEngine/SectionMemoryManager.h"
-#include "llvm/IR/Module.h"
+#include "llvm/ExecutionEngine/Orc/SimpleRemoteEPC.h"
+
+#include "llvm/Support/Error.h"
#include "llvm/Support/FileSystem.h"
-#include "llvm/Support/ManagedStatic.h"
+#include "llvm/Support/FormatVariadic.h"
#include "llvm/Support/Path.h"
-#include "llvm/Support/TargetSelect.h"
+#include "llvm/Support/raw_ostream.h"
+
#include "llvm/TargetParser/Host.h"
+#include <array>
+#include <functional>
+#include <memory>
+#include <optional>
+#include <string>
+#include <utility>
+
#ifdef LLVM_ON_UNIX
#include <netdb.h>
#include <netinet/in.h>
#include <sys/socket.h>
#include <unistd.h>
-#endif // LLVM_ON_UNIX
-
-// Force linking some of the runtimes that helps attaching to a debugger.
-LLVM_ATTRIBUTE_USED void linkComponents() {
- llvm::errs() << (void *)&llvm_orc_registerJITLoaderGDBAllocAction;
-}
+#endif
namespace clang {
-IncrementalExecutor::IncrementalExecutor(llvm::orc::ThreadSafeContext &TSC)
- : TSCtx(TSC) {}
+IncrementalExecutorBuilder::~IncrementalExecutorBuilder() = default;
-llvm::Expected<std::unique_ptr<llvm::orc::LLJITBuilder>>
-IncrementalExecutor::createDefaultJITBuilder(
- llvm::orc::JITTargetMachineBuilder JTMB) {
+static llvm::Expected<llvm::orc::JITTargetMachineBuilder>
+createJITTargetMachineBuilder(const llvm::Triple &TT) {
+ if (TT.getTriple() == llvm::sys::getProcessTriple())
+ // This fails immediately if the target backend is not registered
+ return llvm::orc::JITTargetMachineBuilder::detectHost();
+
+ // If the target backend is not registered, LLJITBuilder::create() will fail
+ return llvm::orc::JITTargetMachineBuilder(TT);
+}
+
+static llvm::Expected<std::unique_ptr<llvm::orc::LLJITBuilder>>
+createDefaultJITBuilder(llvm::orc::JITTargetMachineBuilder JTMB) {
auto JITBuilder = std::make_unique<llvm::orc::LLJITBuilder>();
JITBuilder->setJITTargetMachineBuilder(std::move(JTMB));
JITBuilder->setPrePlatformSetup([](llvm::orc::LLJIT &J) {
@@ -67,71 +86,6 @@ IncrementalExecutor::createDefaultJITBuilder(
return std::move(JITBuilder);
}
-IncrementalExecutor::IncrementalExecutor(llvm::orc::ThreadSafeContext &TSC,
- llvm::orc::LLJITBuilder &JITBuilder,
- llvm::Error &Err)
- : TSCtx(TSC) {
- using namespace llvm::orc;
- llvm::ErrorAsOutParameter EAO(&Err);
-
- if (auto JitOrErr = JITBuilder.create())
- Jit = std::move(*JitOrErr);
- else {
- Err = JitOrErr.takeError();
- return;
- }
-}
-
-IncrementalExecutor::~IncrementalExecutor() {}
-
-llvm::Error IncrementalExecutor::addModule(PartialTranslationUnit &PTU) {
- llvm::orc::ResourceTrackerSP RT =
- Jit->getMainJITDylib().createResourceTracker();
- ResourceTrackers[&PTU] = RT;
-
- return Jit->addIRModule(RT, {std::move(PTU.TheModule), TSCtx});
-}
-
-llvm::Error IncrementalExecutor::removeModule(PartialTranslationUnit &PTU) {
-
- llvm::orc::ResourceTrackerSP RT = std::move(ResourceTrackers[&PTU]);
- if (!RT)
- return llvm::Error::success();
-
- ResourceTrackers.erase(&PTU);
- if (llvm::Error Err = RT->remove())
- return Err;
- return llvm::Error::success();
-}
-
-// Clean up the JIT instance.
-llvm::Error IncrementalExecutor::cleanUp() {
- // This calls the global dtors of registered modules.
- return Jit->deinitialize(Jit->getMainJITDylib());
-}
-
-llvm::Error IncrementalExecutor::runCtors() const {
- return Jit->initialize(Jit->getMainJITDylib());
-}
-
-llvm::Expected<llvm::orc::ExecutorAddr>
-IncrementalExecutor::getSymbolAddress(llvm::StringRef Name,
- SymbolNameKind NameKind) const {
- using namespace llvm::orc;
- auto SO = makeJITDylibSearchOrder({&Jit->getMainJITDylib(),
- Jit->getPlatformJITDylib().get(),
- Jit->getProcessSymbolsJITDylib().get()});
-
- ExecutionSession &ES = Jit->getExecutionSession();
-
- auto SymOrErr =
- ES.lookup(SO, (NameKind == LinkerName) ? ES.intern(Name)
- : Jit->mangleAndIntern(Name));
- if (auto Err = SymOrErr.takeError())
- return std::move(Err);
- return SymOrErr->getAddress();
-}
-
Expected<std::unique_ptr<llvm::jitlink::JITLinkMemoryManager>>
createSharedMemoryManager(llvm::orc::SimpleRemoteEPC &SREPC,
unsigned SlabAllocateSize) {
@@ -165,11 +119,10 @@ createSharedMemoryManager(llvm::orc::SimpleRemoteEPC &SREPC,
llvm::orc::SharedMemoryMapper>(SlabSize, SREPC, SAs);
}
-llvm::Expected<std::pair<std::unique_ptr<llvm::orc::SimpleRemoteEPC>, uint32_t>>
-IncrementalExecutor::launchExecutor(llvm::StringRef ExecutablePath,
- bool UseSharedMemory,
- unsigned SlabAllocateSize,
- std::function<void()> CustomizeFork) {
+static llvm::Expected<
+ std::pair<std::unique_ptr<llvm::orc::SimpleRemoteEPC>, uint32_t>>
+launchExecutor(llvm::StringRef ExecutablePath, bool UseSharedMemory,
+ unsigned SlabAllocateSize, std::function<void()> CustomizeFork) {
#ifndef LLVM_ON_UNIX
// FIXME: Add support for Windows.
return llvm::make_error<llvm::StringError>(
@@ -302,10 +255,9 @@ static Expected<int> connectTCPSocketImpl(std::string Host,
return SockFD;
}
-llvm::Expected<std::unique_ptr<llvm::orc::SimpleRemoteEPC>>
-IncrementalExecutor::connectTCPSocket(llvm::StringRef NetworkAddress,
- bool UseSharedMemory,
- unsigned SlabAllocateSize) {
+static llvm::Expected<std::unique_ptr<llvm::orc::SimpleRemoteEPC>>
+connectTCPSocket(llvm::StringRef NetworkAddress, bool UseSharedMemory,
+ unsigned SlabAllocateSize) {
#ifndef LLVM_ON_UNIX
// FIXME: Add TCP support for Windows.
return llvm::make_error<llvm::StringError>(
@@ -357,4 +309,202 @@ IncrementalExecutor::connectTCPSocket(llvm::StringRef NetworkAddress,
}
#endif // _WIN32
-} // namespace clang
+static llvm::Expected<std::unique_ptr<llvm::orc::LLJITBuilder>>
+createLLJITBuilder(std::unique_ptr<llvm::orc::ExecutorProcessControl> EPC,
+ llvm::StringRef OrcRuntimePath) {
+ auto JTMB = createJITTargetMachineBuilder(EPC->getTargetTriple());
+ if (!JTMB)
+ return JTMB.takeError();
+ auto JB = createDefaultJITBuilder(std::move(*JTMB));
+ if (!JB)
+ return JB.takeError();
+
+ (*JB)->setExecutorProcessControl(std::move(EPC));
+ (*JB)->setPlatformSetUp(
+ llvm::orc::ExecutorNativePlatform(OrcRuntimePath.str()));
+
+ return std::move(*JB);
+}
+
+static llvm::Expected<
+ std::pair<std::unique_ptr<llvm::orc::LLJITBuilder>, uint32_t>>
+outOfProcessJITBuilder(const IncrementalExecutorBuilder &IncrExecutorBuilder) {
+ std::unique_ptr<llvm::orc::ExecutorProcessControl> EPC;
+ uint32_t childPid = -1;
+ if (!IncrExecutorBuilder.OOPExecutor.empty()) {
+ // Launch an out-of-process executor locally in a child process.
+ auto ResultOrErr = launchExecutor(IncrExecutorBuilder.OOPExecutor,
+ IncrExecutorBuilder.UseSharedMemory,
+ IncrExecutorBuilder.SlabAllocateSize,
+ IncrExecutorBuilder.CustomizeFork);
+ if (!ResultOrErr)
+ return ResultOrErr.takeError();
+ childPid = ResultOrErr->second;
+ auto EPCOrErr = std::move(ResultOrErr->first);
+ EPC = std::move(EPCOrErr);
+ } else if (IncrExecutorBuilder.OOPExecutorConnect != "") {
+#if LLVM_ON_UNIX && LLVM_ENABLE_THREADS
+ auto EPCOrErr = connectTCPSocket(IncrExecutorBuilder.OOPExecutorConnect,
+ IncrExecutorBuilder.UseSharedMemory,
+ IncrExecutorBuilder.SlabAllocateSize);
+ if (!EPCOrErr)
+ return EPCOrErr.takeError();
+ EPC = std::move(*EPCOrErr);
+#else
+ return llvm::make_error<llvm::StringError>(
+ "Out-of-process JIT over TCP is not supported on this platform",
+ std::error_code());
+#endif
+ }
+
+ std::unique_ptr<llvm::orc::LLJITBuilder> JB;
+ if (EPC) {
+ auto JBOrErr =
+ createLLJITBuilder(std::move(EPC), IncrExecutorBuilder.OrcRuntimePath);
+ if (!JBOrErr)
+ return JBOrErr.takeError();
+ JB = std::move(*JBOrErr);
+ }
+
+ return std::make_pair(std::move(JB), childPid);
+}
+
+llvm::Expected<std::unique_ptr<IncrementalExecutor>>
+IncrementalExecutorBuilder::create(llvm::orc::ThreadSafeContext &TSC,
+ const clang::TargetInfo &TI) {
+ if (IE)
+ return std::move(IE);
+ llvm::Triple TT = TI.getTriple();
+ if (!TT.isOSWindows() && IsOutOfProcess) {
+ if (!JITBuilder) {
+ auto ResOrErr = outOfProcessJITBuilder(*this);
+ if (!ResOrErr)
+ return ResOrErr.takeError();
+ JITBuilder = std::move(ResOrErr->first);
+ ExecutorPID = ResOrErr->second;
+ }
+ if (!JITBuilder)
+ return llvm::make_error<llvm::StringError>(
+ "Operation failed. No LLJITBuilder for out-of-process JIT",
+ std::error_code());
+ }
+
+ if (!JITBuilder) {
+ auto JTMB = createJITTargetMachineBuilder(TT);
+ if (!JTMB)
+ return JTMB.takeError();
+ if (CM)
+ JTMB->setCodeModel(CM);
+ auto JB = createDefaultJITBuilder(std::move(*JTMB));
+ if (!JB)
+ return JB.takeError();
+ JITBuilder = std::move(*JB);
+ }
+
+ llvm::Error Err = llvm::Error::success();
+ std::unique_ptr<IncrementalExecutor> Executor;
+#ifdef __EMSCRIPTEN__
+ Executor = std::make_unique<WasmIncrementalExecutor>(TSC);
+#else
+ Executor = std::make_unique<OrcIncrementalExecutor>(TSC, *JITBuilder, Err);
+#endif
+
+ if (Err)
+ return std::move(Err);
+
+ return std::move(Executor);
+}
+
+llvm::Error IncrementalExecutorBuilder::UpdateOrcRuntimePath(
+ const clang::driver::Compilation &C) {
+ if (!IsOutOfProcess)
+ return llvm::Error::success();
+
+ static constexpr std::array<const char *, 3> OrcRTLibNames = {
+ "liborc_rt.a",
+ "liborc_rt_osx.a",
+ "liborc_rt-x86_64.a",
+ };
+
+ auto findInDir = [&](llvm::StringRef Base) -> std::optional<std::string> {
+ if (Base.empty() || !llvm::sys::fs::exists(Base))
+ return std::nullopt;
+ for (const char *LibName : OrcRTLibNames) {
+ llvm::SmallString<256> Candidate(Base);
+ llvm::sys::path::append(Candidate, LibName);
+ if (llvm::sys::fs::exists(Candidate))
+ return std::string(Candidate.str());
+ }
+ return std::nullopt;
+ };
+
+ const clang::driver::Driver &D = C.getDriver();
+ const clang::driver::ToolChain &TC = C.getDefaultToolChain();
+ llvm::SmallVector<std::string, 8> triedPaths;
+
+ llvm::SmallString<256> Resource(D.ResourceDir);
+ if (llvm::sys::fs::exists(Resource)) {
+ // Ask the ToolChain for its runtime paths first (most authoritative).
+ for (auto RuntimePath :
+ {TC.getRuntimePath(), std::make_optional(TC.getCompilerRTPath())}) {
+ if (RuntimePath) {
+ if (auto Found = findInDir(*RuntimePath)) {
+ OrcRuntimePath = *Found;
+ return llvm::Error::success();
+ }
+ triedPaths.emplace_back(*RuntimePath);
+ }
+ }
+
+ // Check ResourceDir and ResourceDir/lib
+ for (auto P : {Resource.str().str(), (Resource + "/lib").str()}) {
+ if (auto F = findInDir(P)) {
+ OrcRuntimePath = *F;
+ return llvm::Error::success();
+ }
+ triedPaths.emplace_back(P);
+ }
+ } else {
+ // The binary was misplaced. Generic Backward Search (Climbing the tree)
+ // This allows unit tests in tools/clang/unittests to find the real lib/
+ llvm::SmallString<256> Cursor = Resource;
+ // ResourceDir-derived locations
+ llvm::StringRef Version = llvm::sys::path::filename(Resource);
+ llvm::StringRef OSName = TC.getOSLibName();
+ while (llvm::sys::path::has_parent_path(Cursor)) {
+ Cursor = llvm::sys::path::parent_path(Cursor).str();
+ // At each level, try standard relative layouts
+ for (auto Rel :
+ {(llvm::Twine("lib/clang/") + Version + "/lib/" + OSName).str(),
+ (llvm::Twine("lib/clang/") + Version + "/lib").str(),
+ (llvm::Twine("lib/") + OSName).str(), std::string("lib/clang")}) {
+ llvm::SmallString<256> Candidate = Cursor;
+ llvm::sys::path::append(Candidate, Rel);
+ if (auto F = findInDir(Candidate)) {
+ OrcRuntimePath = *F;
+ return llvm::Error::success();
+ }
+ triedPaths.emplace_back(std::string(Candidate.str()));
+ }
+ // Stop if we hit the root or go too far (safety check)
+ if (triedPaths.size() > 32)
+ break;
+ }
+ }
+
+ // Build a helpful error string if everything failed.
+ std::string Joined;
+ for (size_t i = 0; i < triedPaths.size(); ++i) {
+ if (i)
+ Joined += ", ";
+ Joined += triedPaths[i];
+ }
+ if (Joined.empty())
+ Joined = "<no candidate paths available>";
+
+ return llvm::make_error<llvm::StringError>(
+ llvm::formatv("OrcRuntime library not found in: {0}", Joined).str(),
+ llvm::inconvertibleErrorCode());
+}
+
+} // end namespace clang
diff --git a/clang/lib/Interpreter/IncrementalExecutor.h b/clang/lib/Interpreter/IncrementalExecutor.h
deleted file mode 100644
index c11c99b9bff8d..0000000000000
--- a/clang/lib/Interpreter/IncrementalExecutor.h
+++ /dev/null
@@ -1,90 +0,0 @@
-//===--- IncrementalExecutor.h - Incremental Execution ----------*- C++ -*-===//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-//
-// This file implements the class which performs incremental code execution.
-//
-//===----------------------------------------------------------------------===//
-
-#ifndef LLVM_CLANG_LIB_INTERPRETER_INCREMENTALEXECUTOR_H
-#define LLVM_CLANG_LIB_INTERPRETER_INCREMENTALEXECUTOR_H
-
-#include "clang/Interpreter/Interpreter.h"
-#include "llvm/ADT/DenseMap.h"
-#include "llvm/ADT/StringRef.h"
-#include "llvm/ExecutionEngine/Orc/Core.h"
-#include "llvm/ExecutionEngine/Orc/ExecutionUtils.h"
-#include "llvm/ExecutionEngine/Orc/Layer.h"
-#include "llvm/ExecutionEngine/Orc/Shared/ExecutorAddress.h"
-#include "llvm/ExecutionEngine/Orc/SimpleRemoteEPC.h"
-#include "llvm/Support/Error.h"
-
-#include <cstdint>
-#include <memory>
-#include <string>
-
-namespace llvm {
-class Error;
-namespace orc {
-class JITTargetMachineBuilder;
-class LLJIT;
-class LLJITBuilder;
-class ThreadSafeContext;
-} // namespace orc
-} // namespace llvm
-
-namespace clang {
-
-struct PartialTranslationUnit;
-class TargetInfo;
-
-class IncrementalExecutor {
- using CtorDtorIterator = llvm::orc::CtorDtorIterator;
- std::unique_ptr<llvm::orc::LLJIT> Jit;
- llvm::orc::ThreadSafeContext &TSCtx;
-
- llvm::DenseMap<const PartialTranslationUnit *, llvm::orc::ResourceTrackerSP>
- ResourceTrackers;
-
-protected:
- IncrementalExecutor(llvm::orc::ThreadSafeContext &TSC);
-
-public:
- enum SymbolNameKind { IRName, LinkerName };
-
- IncrementalExecutor(llvm::orc::ThreadSafeContext &TSC,
- llvm::orc::LLJITBuilder &JITBuilder, llvm::Error &Err);
- virtual ~IncrementalExecutor();
-
- virtual llvm::Error addModule(PartialTranslationUnit &PTU);
- virtual llvm::Error removeModule(PartialTranslationUnit &PTU);
- virtual llvm::Error runCtors() const;
- virtual llvm::Error cleanUp();
- virtual llvm::Expected<llvm::orc::ExecutorAddr>
- getSymbolAddress(llvm::StringRef Name, SymbolNameKind NameKind) const;
-
- llvm::orc::LLJIT &GetExecutionEngine() { return *Jit; }
-
- static llvm::Expected<std::unique_ptr<llvm::orc::LLJITBuilder>>
- createDefaultJITBuilder(llvm::orc::JITTargetMachineBuilder JTMB);
-
- static llvm::Expected<
- std::pair<std::unique_ptr<llvm::orc::SimpleRemoteEPC>, uint32_t>>
- launchExecutor(llvm::StringRef ExecutablePath, bool UseSharedMemory,
- unsigned SlabAllocateSize,
- std::function<void()> CustomizeFork = nullptr);
-
-#if LLVM_ON_UNIX && LLVM_ENABLE_THREADS
- static llvm::Expected<std::unique_ptr<llvm::orc::SimpleRemoteEPC>>
- connectTCPSocket(llvm::StringRef NetworkAddress, bool UseSharedMemory,
- unsigned SlabAllocateSize);
-#endif
-};
-
-} // end namespace clang
-
-#endif // LLVM_CLANG_LIB_INTERPRETER_INCREMENTALEXECUTOR_H
diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp
index f69c57fe48001..41f725415cd1c 100644
--- a/clang/lib/Interpreter/Interpreter.cpp
+++ b/clang/lib/Interpreter/Interpreter.cpp
@@ -13,14 +13,8 @@
#include "DeviceOffload.h"
#include "IncrementalAction.h"
-#include "IncrementalExecutor.h"
#include "IncrementalParser.h"
#include "InterpreterUtils.h"
-#include "llvm/Support/VirtualFileSystem.h"
-#ifdef __EMSCRIPTEN__
-#include "Wasm.h"
-#include <dlfcn.h>
-#endif // __EMSCRIPTEN__
#include "clang/AST/ASTConsumer.h"
#include "clang/AST/ASTContext.h"
@@ -39,6 +33,7 @@
#include "clang/Frontend/MultiplexConsumer.h"
#include "clang/Frontend/TextDiagnosticBuffer.h"
#include "clang/FrontendTool/Utils.h"
+#include "clang/Interpreter/IncrementalExecutor.h"
#include "clang/Interpreter/Interpreter.h"
#include "clang/Interpreter/Value.h"
#include "clang/Lex/PreprocessorOptions.h"
@@ -52,6 +47,7 @@
#include "llvm/IR/Module.h"
#include "llvm/Support/Errc.h"
#include "llvm/Support/ErrorHandling.h"
+#include "llvm/Support/VirtualFileSystem.h"
#include "llvm/Support/raw_ostream.h"
#include "llvm/TargetParser/Host.h"
#include "llvm/Transforms/Utils/Cloning.h" // for CloneModule
@@ -251,123 +247,11 @@ IncrementalCompilerBuilder::CreateCudaHost() {
return IncrementalCompilerBuilder::createCuda(false);
}
-IncrementalExecutorBuilder::~IncrementalExecutorBuilder() = default;
-
-llvm::Expected<std::unique_ptr<IncrementalExecutor>>
-IncrementalExecutorBuilder::create(llvm::orc::ThreadSafeContext &TSC,
- llvm::orc::LLJITBuilder &JITBuilder) {
- llvm::Error Err = llvm::Error::success();
- std::unique_ptr<IncrementalExecutor> Executor;
-#ifdef __EMSCRIPTEN__
- Executor = std::make_unique<WasmIncrementalExecutor>(TSC);
-#else
- Executor = std::make_unique<IncrementalExecutor>(TSC, JITBuilder, Err);
-#endif
-
- if (Err)
- return std::move(Err);
-
- return std::move(Executor);
-}
-
-llvm::Error IncrementalExecutorBuilder::UpdateOrcRuntimePath(
- const clang::driver::Compilation &C) {
- if (!IsOutOfProcess)
- return llvm::Error::success();
-
- static constexpr std::array<const char *, 3> OrcRTLibNames = {
- "liborc_rt.a",
- "liborc_rt_osx.a",
- "liborc_rt-x86_64.a",
- };
-
- auto findInDir = [&](llvm::StringRef Base) -> std::optional<std::string> {
- if (Base.empty() || !llvm::sys::fs::exists(Base))
- return std::nullopt;
- for (const char *LibName : OrcRTLibNames) {
- llvm::SmallString<256> Candidate(Base);
- llvm::sys::path::append(Candidate, LibName);
- if (llvm::sys::fs::exists(Candidate))
- return std::string(Candidate.str());
- }
- return std::nullopt;
- };
-
- const clang::driver::Driver &D = C.getDriver();
- const clang::driver::ToolChain &TC = C.getDefaultToolChain();
- llvm::SmallVector<std::string, 8> triedPaths;
-
- llvm::SmallString<256> Resource(D.ResourceDir);
- if (llvm::sys::fs::exists(Resource)) {
- // Ask the ToolChain for its runtime paths first (most authoritative).
- for (auto RuntimePath :
- {TC.getRuntimePath(), std::make_optional(TC.getCompilerRTPath())}) {
- if (RuntimePath) {
- if (auto Found = findInDir(*RuntimePath)) {
- OrcRuntimePath = *Found;
- return llvm::Error::success();
- }
- triedPaths.emplace_back(*RuntimePath);
- }
- }
-
- // Check ResourceDir and ResourceDir/lib
- for (auto P : {Resource.str().str(), (Resource + "/lib").str()}) {
- if (auto F = findInDir(P)) {
- OrcRuntimePath = *F;
- return llvm::Error::success();
- }
- triedPaths.emplace_back(P);
- }
- } else {
- // The binary was misplaced. Generic Backward Search (Climbing the tree)
- // This allows unit tests in tools/clang/unittests to find the real lib/
- llvm::SmallString<256> Cursor = Resource;
- // ResourceDir-derived locations
- llvm::StringRef Version = llvm::sys::path::filename(Resource);
- llvm::StringRef OSName = TC.getOSLibName();
- while (llvm::sys::path::has_parent_path(Cursor)) {
- Cursor = llvm::sys::path::parent_path(Cursor).str();
- // At each level, try standard relative layouts
- for (auto Rel :
- {(llvm::Twine("lib/clang/") + Version + "/lib/" + OSName).str(),
- (llvm::Twine("lib/clang/") + Version + "/lib").str(),
- (llvm::Twine("lib/") + OSName).str(), std::string("lib/clang")}) {
- llvm::SmallString<256> Candidate = Cursor;
- llvm::sys::path::append(Candidate, Rel);
- if (auto F = findInDir(Candidate)) {
- OrcRuntimePath = *F;
- return llvm::Error::success();
- }
- triedPaths.emplace_back(std::string(Candidate.str()));
- }
- // Stop if we hit the root or go too far (safety check)
- if (triedPaths.size() > 32)
- break;
- }
- }
-
- // Build a helpful error string if everything failed.
- std::string Joined;
- for (size_t i = 0; i < triedPaths.size(); ++i) {
- if (i)
- Joined += ", ";
- Joined += triedPaths[i];
- }
- if (Joined.empty())
- Joined = "<no candidate paths available>";
-
- return llvm::make_error<llvm::StringError>(
- llvm::formatv("OrcRuntime library not found in: {0}", Joined).str(),
- llvm::inconvertibleErrorCode());
-}
-
Interpreter::Interpreter(std::unique_ptr<CompilerInstance> Instance,
llvm::Error &ErrOut,
- std::unique_ptr<llvm::orc::LLJITBuilder> JITBuilder,
- std::unique_ptr<clang::ASTConsumer> Consumer,
- std::unique_ptr<IncrementalExecutorBuilder> IEB)
- : JITBuilder(std::move(JITBuilder)), IncrExecutorBuilder(std::move(IEB)) {
+ std::unique_ptr<IncrementalExecutorBuilder> IEB,
+ std::unique_ptr<clang::ASTConsumer> Consumer)
+ : IncrExecutorBuilder(std::move(IEB)) {
CI = std::move(Instance);
llvm::ErrorAsOutParameter EAO(&ErrOut);
auto LLVMCtx = std::make_unique<llvm::LLVMContext>();
@@ -463,58 +347,13 @@ const char *const Runtimes = R"(
EXTERN_C void __clang_Interpreter_SetValueNoAlloc(void *This, void *OutVal, void *OpaqueType, ...);
)";
-static llvm::Expected<
- std::pair<std::unique_ptr<llvm::orc::LLJITBuilder>, uint32_t>>
-outOfProcessJITBuilder(const IncrementalExecutorBuilder &IncrExecutorBuilder) {
- std::unique_ptr<llvm::orc::ExecutorProcessControl> EPC;
- uint32_t childPid = -1;
- if (!IncrExecutorBuilder.OOPExecutor.empty()) {
- // Launch an out-of-process executor locally in a child process.
- auto ResultOrErr = IncrementalExecutor::launchExecutor(
- IncrExecutorBuilder.OOPExecutor, IncrExecutorBuilder.UseSharedMemory,
- IncrExecutorBuilder.SlabAllocateSize,
- IncrExecutorBuilder.CustomizeFork);
- if (!ResultOrErr)
- return ResultOrErr.takeError();
- childPid = ResultOrErr->second;
- auto EPCOrErr = std::move(ResultOrErr->first);
- EPC = std::move(EPCOrErr);
- } else if (IncrExecutorBuilder.OOPExecutorConnect != "") {
-#if LLVM_ON_UNIX && LLVM_ENABLE_THREADS
- auto EPCOrErr = IncrementalExecutor::connectTCPSocket(
- IncrExecutorBuilder.OOPExecutorConnect,
- IncrExecutorBuilder.UseSharedMemory,
- IncrExecutorBuilder.SlabAllocateSize);
- if (!EPCOrErr)
- return EPCOrErr.takeError();
- EPC = std::move(*EPCOrErr);
-#else
- return llvm::make_error<llvm::StringError>(
- "Out-of-process JIT over TCP is not supported on this platform",
- std::error_code());
-#endif
- }
-
- std::unique_ptr<llvm::orc::LLJITBuilder> JB;
- if (EPC) {
- auto JBOrErr = clang::Interpreter::createLLJITBuilder(
- std::move(EPC), IncrExecutorBuilder.OrcRuntimePath);
- if (!JBOrErr)
- return JBOrErr.takeError();
- JB = std::move(*JBOrErr);
- }
-
- return std::make_pair(std::move(JB), childPid);
-}
-
llvm::Expected<std::unique_ptr<Interpreter>> Interpreter::create(
std::unique_ptr<CompilerInstance> CI,
std::unique_ptr<IncrementalExecutorBuilder> IEB /*=nullptr*/) {
llvm::Error Err = llvm::Error::success();
- std::unique_ptr<llvm::orc::LLJITBuilder> JB;
auto Interp = std::unique_ptr<Interpreter>(new Interpreter(
- std::move(CI), Err, std::move(JB), /*Consumer=*/nullptr, std::move(IEB)));
+ std::move(CI), Err, std::move(IEB), /*Consumer=*/nullptr));
if (auto E = std::move(Err))
return std::move(E);
@@ -579,13 +418,13 @@ const CompilerInstance *Interpreter::getCompilerInstance() const {
return const_cast<Interpreter *>(this)->getCompilerInstance();
}
-llvm::Expected<llvm::orc::LLJIT &> Interpreter::getExecutionEngine() {
+llvm::Expected<IncrementalExecutor &> Interpreter::getExecutionEngine() {
if (!IncrExecutor) {
if (auto Err = CreateExecutor())
return std::move(Err);
}
- return IncrExecutor->GetExecutionEngine();
+ return *IncrExecutor.get();
}
ASTContext &Interpreter::getASTContext() {
@@ -646,35 +485,6 @@ Interpreter::Parse(llvm::StringRef Code) {
return LastPTU;
}
-static llvm::Expected<llvm::orc::JITTargetMachineBuilder>
-createJITTargetMachineBuilder(const std::string &TT) {
- if (TT == llvm::sys::getProcessTriple())
- // This fails immediately if the target backend is not registered
- return llvm::orc::JITTargetMachineBuilder::detectHost();
-
- // If the target backend is not registered, LLJITBuilder::create() will fail
- return llvm::orc::JITTargetMachineBuilder(llvm::Triple(TT));
-}
-
-llvm::Expected<std::unique_ptr<llvm::orc::LLJITBuilder>>
-Interpreter::createLLJITBuilder(
- std::unique_ptr<llvm::orc::ExecutorProcessControl> EPC,
- llvm::StringRef OrcRuntimePath) {
- const std::string &TT = EPC->getTargetTriple().getTriple();
- auto JTMB = createJITTargetMachineBuilder(TT);
- if (!JTMB)
- return JTMB.takeError();
- auto JB = IncrementalExecutor::createDefaultJITBuilder(std::move(*JTMB));
- if (!JB)
- return JB.takeError();
-
- (*JB)->setExecutorProcessControl(std::move(EPC));
- (*JB)->setPlatformSetUp(
- llvm::orc::ExecutorNativePlatform(OrcRuntimePath.str()));
-
- return std::move(*JB);
-}
-
llvm::Error Interpreter::CreateExecutor() {
if (IncrExecutor)
return llvm::make_error<llvm::StringError>("Operation failed. "
@@ -685,48 +495,16 @@ llvm::Error Interpreter::CreateExecutor() {
"No code generator available",
std::error_code());
- const std::string &TT = getCompilerInstance()->getTargetOpts().Triple;
- llvm::Triple TargetTriple(TT);
- bool IsWindowsTarget = TargetTriple.isOSWindows();
-
if (!IncrExecutorBuilder)
IncrExecutorBuilder = std::make_unique<IncrementalExecutorBuilder>();
- if (!IsWindowsTarget && IncrExecutorBuilder->IsOutOfProcess) {
- if (!JITBuilder) {
- auto ResOrErr = outOfProcessJITBuilder(*IncrExecutorBuilder);
- if (!ResOrErr)
- return ResOrErr.takeError();
- JITBuilder = std::move(ResOrErr->first);
- IncrExecutorBuilder->ExecutorPID = ResOrErr->second;
- }
- if (!JITBuilder)
- return llvm::make_error<llvm::StringError>(
- "Operation failed. No LLJITBuilder for out-of-process JIT",
- std::error_code());
- }
-
- if (!JITBuilder) {
- auto JTMB = createJITTargetMachineBuilder(TT);
- if (!JTMB)
- return JTMB.takeError();
- if (IncrExecutorBuilder->CM)
- JTMB->setCodeModel(IncrExecutorBuilder->CM);
- auto JB = IncrementalExecutor::createDefaultJITBuilder(std::move(*JTMB));
- if (!JB)
- return JB.takeError();
- JITBuilder = std::move(*JB);
- }
-
- auto ExecutorOrErr = IncrExecutorBuilder->create(*TSCtx, *JITBuilder);
+ auto ExecutorOrErr = IncrExecutorBuilder->create(*TSCtx, CI->getTarget());
if (ExecutorOrErr)
IncrExecutor = std::move(*ExecutorOrErr);
return ExecutorOrErr.takeError();
}
-void Interpreter::ResetExecutor() { IncrExecutor.reset(); }
-
llvm::Error Interpreter::Execute(PartialTranslationUnit &T) {
assert(T.TheModule);
LLVM_DEBUG(
@@ -827,29 +605,6 @@ llvm::Error Interpreter::Undo(unsigned N) {
}
llvm::Error Interpreter::LoadDynamicLibrary(const char *name) {
-#ifdef __EMSCRIPTEN__
- void *handle = dlopen(name, RTLD_NOW | RTLD_GLOBAL);
- if (!handle) {
- llvm::errs() << dlerror() << '\n';
- return llvm::make_error<llvm::StringError>("Failed to load dynamic library",
- llvm::inconvertibleErrorCode());
- }
-#else
- auto EE = getExecutionEngine();
- if (!EE)
- return EE.takeError();
-
- if (llvm::Expected<
- std::unique_ptr<llvm::orc::EPCDynamicLibrarySearchGenerator>>
- DLSG = llvm::orc::EPCDynamicLibrarySearchGenerator::Load(
- EE->getExecutionSession(), name))
- // FIXME: Eventually we should put each library in its own JITDylib and
- // turn off process symbols by default.
- EE->getProcessSymbolsJITDylib()->addGenerator(std::move(*DLSG));
- else
- return DLSG.takeError();
-#endif
-
- return llvm::Error::success();
+ return getExecutionEngine()->LoadDynamicLibrary(name);
}
} // end namespace clang
diff --git a/clang/lib/Interpreter/OrcIncrementalExecutor.cpp b/clang/lib/Interpreter/OrcIncrementalExecutor.cpp
new file mode 100644
index 0000000000000..5b214a987fd0a
--- /dev/null
+++ b/clang/lib/Interpreter/OrcIncrementalExecutor.cpp
@@ -0,0 +1,121 @@
+//===--- OrcIncrementalExecutor.cpp - Orc Incremental Execution -*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file implements an Orc-based incremental code execution.
+//
+//===----------------------------------------------------------------------===//
+
+#include "OrcIncrementalExecutor.h"
+#include "clang/Interpreter/PartialTranslationUnit.h"
+
+#include "llvm/ExecutionEngine/Orc/EPCDynamicLibrarySearchGenerator.h"
+#include "llvm/ExecutionEngine/Orc/ExecutionUtils.h"
+#include "llvm/ExecutionEngine/Orc/JITTargetMachineBuilder.h"
+#include "llvm/ExecutionEngine/Orc/LLJIT.h"
+#include "llvm/ExecutionEngine/Orc/Shared/OrcRTBridge.h"
+#include "llvm/ExecutionEngine/Orc/Shared/SimpleRemoteEPCUtils.h"
+#include "llvm/ExecutionEngine/Orc/TargetProcess/JITLoaderGDB.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/raw_ostream.h"
+
+#ifdef LLVM_ON_UNIX
+#include <netdb.h>
+#include <netinet/in.h>
+#include <sys/socket.h>
+#include <unistd.h>
+#endif // LLVM_ON_UNIX
+
+// Force linking some of the runtimes that helps attaching to a debugger.
+LLVM_ATTRIBUTE_USED void linkComponents() {
+ llvm::errs() << (void *)&llvm_orc_registerJITLoaderGDBAllocAction;
+}
+
+namespace clang {
+OrcIncrementalExecutor::OrcIncrementalExecutor(
+ llvm::orc::ThreadSafeContext &TSC)
+ : TSCtx(TSC) {}
+
+OrcIncrementalExecutor::OrcIncrementalExecutor(
+ llvm::orc::ThreadSafeContext &TSC, llvm::orc::LLJITBuilder &JITBuilder,
+ llvm::Error &Err)
+ : TSCtx(TSC) {
+ using namespace llvm::orc;
+ llvm::ErrorAsOutParameter EAO(&Err);
+
+ if (auto JitOrErr = JITBuilder.create())
+ Jit = std::move(*JitOrErr);
+ else {
+ Err = JitOrErr.takeError();
+ return;
+ }
+}
+
+OrcIncrementalExecutor::~OrcIncrementalExecutor() {}
+
+llvm::Error OrcIncrementalExecutor::addModule(PartialTranslationUnit &PTU) {
+ llvm::orc::ResourceTrackerSP RT =
+ Jit->getMainJITDylib().createResourceTracker();
+ ResourceTrackers[&PTU] = RT;
+
+ return Jit->addIRModule(RT, {std::move(PTU.TheModule), TSCtx});
+}
+
+llvm::Error OrcIncrementalExecutor::removeModule(PartialTranslationUnit &PTU) {
+
+ llvm::orc::ResourceTrackerSP RT = std::move(ResourceTrackers[&PTU]);
+ if (!RT)
+ return llvm::Error::success();
+
+ ResourceTrackers.erase(&PTU);
+ if (llvm::Error Err = RT->remove())
+ return Err;
+ return llvm::Error::success();
+}
+
+// Clean up the JIT instance.
+llvm::Error OrcIncrementalExecutor::cleanUp() {
+ // This calls the global dtors of registered modules.
+ return Jit->deinitialize(Jit->getMainJITDylib());
+}
+
+llvm::Error OrcIncrementalExecutor::runCtors() const {
+ return Jit->initialize(Jit->getMainJITDylib());
+}
+
+llvm::Expected<llvm::orc::ExecutorAddr>
+OrcIncrementalExecutor::getSymbolAddress(llvm::StringRef Name,
+ SymbolNameKind NameKind) const {
+ using namespace llvm::orc;
+ auto SO = makeJITDylibSearchOrder({&Jit->getMainJITDylib(),
+ Jit->getPlatformJITDylib().get(),
+ Jit->getProcessSymbolsJITDylib().get()});
+
+ ExecutionSession &ES = Jit->getExecutionSession();
+
+ auto SymOrErr = ES.lookup(SO, (NameKind == SymbolNameKind::LinkerName)
+ ? ES.intern(Name)
+ : Jit->mangleAndIntern(Name));
+ if (auto Err = SymOrErr.takeError())
+ return std::move(Err);
+ return SymOrErr->getAddress();
+}
+
+llvm::Error OrcIncrementalExecutor::LoadDynamicLibrary(const char *name) {
+ // FIXME: Eventually we should put each library in its own JITDylib and
+ // turn off process symbols by default.
+ llvm::orc::ExecutionSession &ES = Jit->getExecutionSession();
+ auto DLSGOrErr = llvm::orc::EPCDynamicLibrarySearchGenerator::Load(ES, name);
+ if (auto Err = DLSGOrErr.takeError())
+ return Err;
+
+ Jit->getProcessSymbolsJITDylib()->addGenerator(std::move(*DLSGOrErr));
+
+ return llvm::Error::success();
+}
+
+} // namespace clang
diff --git a/clang/lib/Interpreter/OrcIncrementalExecutor.h b/clang/lib/Interpreter/OrcIncrementalExecutor.h
new file mode 100644
index 0000000000000..ff648d6dab5af
--- /dev/null
+++ b/clang/lib/Interpreter/OrcIncrementalExecutor.h
@@ -0,0 +1,71 @@
+//===--- OrcIncrementalExecutor.h - Orc Incremental Execution ---*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file implements an Orc-based incremental code execution.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_LIB_INTERPRETER_ORCINCREMENTALEXECUTOR_H
+#define LLVM_CLANG_LIB_INTERPRETER_ORCINCREMENTALEXECUTOR_H
+
+#include "clang/Interpreter/IncrementalExecutor.h"
+
+#include "llvm/ADT/DenseMap.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/ExecutionEngine/Orc/Core.h"
+#include "llvm/ExecutionEngine/Orc/ExecutionUtils.h"
+#include "llvm/ExecutionEngine/Orc/Layer.h"
+#include "llvm/ExecutionEngine/Orc/Shared/ExecutorAddress.h"
+#include "llvm/ExecutionEngine/Orc/SimpleRemoteEPC.h"
+#include "llvm/Support/Error.h"
+
+#include <cstdint>
+#include <memory>
+
+namespace llvm {
+class Error;
+namespace orc {
+class JITTargetMachineBuilder;
+class LLJIT;
+class LLJITBuilder;
+class ThreadSafeContext;
+} // namespace orc
+} // namespace llvm
+
+namespace clang {
+
+struct PartialTranslationUnit;
+
+class OrcIncrementalExecutor : public IncrementalExecutor {
+ std::unique_ptr<llvm::orc::LLJIT> Jit;
+ llvm::orc::ThreadSafeContext &TSCtx;
+
+ llvm::DenseMap<const PartialTranslationUnit *, llvm::orc::ResourceTrackerSP>
+ ResourceTrackers;
+
+protected:
+ OrcIncrementalExecutor(llvm::orc::ThreadSafeContext &TSC);
+
+public:
+ OrcIncrementalExecutor(llvm::orc::ThreadSafeContext &TSC,
+ llvm::orc::LLJITBuilder &JITBuilder, llvm::Error &Err);
+ ~OrcIncrementalExecutor() override;
+
+ llvm::Error addModule(PartialTranslationUnit &PTU) override;
+ llvm::Error removeModule(PartialTranslationUnit &PTU) override;
+ llvm::Error runCtors() const override;
+ llvm::Error cleanUp() override;
+ llvm::Expected<llvm::orc::ExecutorAddr>
+ getSymbolAddress(llvm::StringRef Name,
+ SymbolNameKind NameKind) const override;
+ llvm::Error LoadDynamicLibrary(const char *name) override;
+};
+
+} // end namespace clang
+
+#endif // LLVM_CLANG_LIB_INTERPRETER_ORCINCREMENTALEXECUTOR_H
diff --git a/clang/lib/Interpreter/Wasm.cpp b/clang/lib/Interpreter/Wasm.cpp
index 0543a3504c9a2..39c1f9639913a 100644
--- a/clang/lib/Interpreter/Wasm.cpp
+++ b/clang/lib/Interpreter/Wasm.cpp
@@ -11,7 +11,6 @@
//===----------------------------------------------------------------------===//
#include "Wasm.h"
-#include "IncrementalExecutor.h"
#include <llvm/IR/LegacyPassManager.h>
#include <llvm/IR/Module.h>
@@ -62,6 +61,8 @@ WasmIncrementalExecutor::WasmIncrementalExecutor(
llvm::orc::ThreadSafeContext &TSC)
: IncrementalExecutor(TSC) {}
+WasmIncrementalExecutor::~WasmIncrementalExecutor() = default;
+
llvm::Error WasmIncrementalExecutor::addModule(PartialTranslationUnit &PTU) {
std::string ErrorString;
@@ -157,6 +158,6 @@ WasmIncrementalExecutor::getSymbolAddress(llvm::StringRef Name,
return llvm::orc::ExecutorAddr::fromPtr(Sym);
}
-WasmIncrementalExecutor::~WasmIncrementalExecutor() = default;
+llvm::Error WasmIncrementalExecutor::LoadDynamicLibrary(const char *name) {
} // namespace clang
diff --git a/clang/lib/Interpreter/Wasm.h b/clang/lib/Interpreter/Wasm.h
index 9a752934e3185..3122e564ae31f 100644
--- a/clang/lib/Interpreter/Wasm.h
+++ b/clang/lib/Interpreter/Wasm.h
@@ -17,13 +17,14 @@
#error "This requires emscripten."
#endif // __EMSCRIPTEN__
-#include "IncrementalExecutor.h"
+#include "clang/Interpreter/IncrementalExecutor.h"
namespace clang {
class WasmIncrementalExecutor : public IncrementalExecutor {
public:
WasmIncrementalExecutor(llvm::orc::ThreadSafeContext &TSC);
+ ~WasmIncrementalExecutor() override;
llvm::Error addModule(PartialTranslationUnit &PTU) override;
llvm::Error removeModule(PartialTranslationUnit &PTU) override;
@@ -32,8 +33,7 @@ class WasmIncrementalExecutor : public IncrementalExecutor {
llvm::Expected<llvm::orc::ExecutorAddr>
getSymbolAddress(llvm::StringRef Name,
SymbolNameKind NameKind) const override;
-
- ~WasmIncrementalExecutor() override;
+ llvm::Error LoadDynamicLibrary(const char *name) override;
};
} // namespace clang
diff --git a/clang/tools/clang-repl/ClangRepl.cpp b/clang/tools/clang-repl/ClangRepl.cpp
index 95786d688b76e..c9873540a5d66 100644
--- a/clang/tools/clang-repl/ClangRepl.cpp
+++ b/clang/tools/clang-repl/ClangRepl.cpp
@@ -16,6 +16,7 @@
#include "clang/Config/config.h"
#include "clang/Frontend/CompilerInstance.h"
#include "clang/Interpreter/CodeCompletion.h"
+#include "clang/Interpreter/IncrementalExecutor.h"
#include "clang/Interpreter/Interpreter.h"
#include "clang/Lex/Preprocessor.h"
#include "clang/Sema/Sema.h"
diff --git a/clang/unittests/Interpreter/InterpreterExtensionsTest.cpp b/clang/unittests/Interpreter/InterpreterExtensionsTest.cpp
index 07a4a224f16bc..13825f2cbdddf 100644
--- a/clang/unittests/Interpreter/InterpreterExtensionsTest.cpp
+++ b/clang/unittests/Interpreter/InterpreterExtensionsTest.cpp
@@ -72,8 +72,8 @@ class CustomJBInterpreter : public Interpreter {
public:
CustomJBInterpreter(std::unique_ptr<CompilerInstance> CI, llvm::Error &ErrOut,
- std::unique_ptr<llvm::orc::LLJITBuilder> JB)
- : Interpreter(std::move(CI), ErrOut, std::move(JB)) {}
+ std::unique_ptr<clang::IncrementalExecutorBuilder> IEB)
+ : Interpreter(std::move(CI), ErrOut, std::move(IEB)) {}
~CustomJBInterpreter() override {
// Skip cleanUp() because it would trigger LLJIT default dtors
@@ -124,9 +124,10 @@ TEST_F(InterpreterExtensionsTest, CustomCrossJIT) {
JIT = &J;
return llvm::Error::success();
});
-
+ auto IEB = std::make_unique<IncrementalExecutorBuilder>();
+ IEB->JITBuilder = std::move(JB);
llvm::Error ErrOut = llvm::Error::success();
- CustomJBInterpreter Interp(std::move(CI), ErrOut, std::move(JB));
+ CustomJBInterpreter Interp(std::move(CI), ErrOut, std::move(IEB));
cantFail(std::move(ErrOut));
EXPECT_EQ(0U, Objs.size());
@@ -137,4 +138,66 @@ TEST_F(InterpreterExtensionsTest, CustomCrossJIT) {
EXPECT_EQ(1U, Objs.size());
}
+TEST_F(InterpreterExtensionsTest, CustomIncrementalExecutor) {
+ struct RecordingIncrementalExecutor : public clang::IncrementalExecutor {
+ mutable unsigned RanCtors = false;
+ unsigned Added = 0;
+ unsigned Removed = 0;
+
+ // Default ctor is fine; builder.create() will just return this IE.
+ RecordingIncrementalExecutor() = default;
+
+ llvm::Error addModule(clang::PartialTranslationUnit &PTU) override {
+ Added++;
+ return llvm::Error::success();
+ }
+
+ llvm::Error removeModule(clang::PartialTranslationUnit &PTU) override {
+ Removed++;
+ return llvm::Error::success();
+ }
+
+ llvm::Error runCtors() const override {
+ RanCtors++;
+ return llvm::Error::success();
+ }
+
+ llvm::Error cleanUp() override { return llvm::Error::success(); }
+
+ llvm::Expected<llvm::orc::ExecutorAddr>
+ getSymbolAddress(llvm::StringRef /*Name*/,
+ SymbolNameKind /*NameKind*/) const override {
+ // Return an error here; test doesn't need a real address.
+ return llvm::make_error<llvm::StringError>(
+ "not implemented in test", llvm::inconvertibleErrorCode());
+ }
+
+ llvm::Error LoadDynamicLibrary(const char * /*name*/) override {
+ return llvm::Error::success();
+ }
+ };
+
+ // Prepare a builder that hands out our recording executor.
+ auto B = std::make_unique<IncrementalExecutorBuilder>();
+ B->IE = std::make_unique<RecordingIncrementalExecutor>();
+
+ IncrementalCompilerBuilder CB;
+ auto CI = cantFail(CB.CreateCpp());
+
+ auto I = cantFail(Interpreter::create(std::move(CI), std::move(B)));
+ ASSERT_TRUE(I);
+
+ const auto &Rec = static_cast<RecordingIncrementalExecutor &>(
+ cantFail(I->getExecutionEngine()));
+ unsigned NumInitAdded = Rec.Added;
+ unsigned NumInitRanCtors = Rec.RanCtors;
+ unsigned NumInitRemoved = Rec.Removed;
+
+ cantFail(I->ParseAndExecute("int a = 1;"));
+
+ EXPECT_TRUE(Rec.Added == NumInitAdded + 1);
+ EXPECT_TRUE(Rec.RanCtors == NumInitRanCtors + 1);
+ EXPECT_TRUE(Rec.Removed == NumInitRemoved);
+}
+
} // end anonymous namespace
diff --git a/clang/unittests/Interpreter/OutOfProcessInterpreterTests.cpp b/clang/unittests/Interpreter/OutOfProcessInterpreterTests.cpp
index 225d6c8c66cab..54866eba8c6d0 100644
--- a/clang/unittests/Interpreter/OutOfProcessInterpreterTests.cpp
+++ b/clang/unittests/Interpreter/OutOfProcessInterpreterTests.cpp
@@ -12,19 +12,21 @@
//===----------------------------------------------------------------------===//
#include "InterpreterTestFixture.h"
+
#include "clang/AST/Decl.h"
#include "clang/AST/DeclGroup.h"
#include "clang/AST/Mangle.h"
-#include "clang/Basic/Version.h"
-#include "clang/Config/config.h"
#include "clang/Frontend/CompilerInstance.h"
#include "clang/Frontend/TextDiagnosticPrinter.h"
+#include "clang/Interpreter/IncrementalExecutor.h"
#include "clang/Interpreter/Interpreter.h"
#include "clang/Interpreter/Value.h"
#include "clang/Sema/Lookup.h"
#include "clang/Sema/Sema.h"
+
#include "llvm/Support/Error.h"
#include "llvm/TargetParser/Host.h"
+
#include "gmock/gmock.h"
#include "gtest/gtest.h"
More information about the cfe-commits
mailing list