<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hi Lang,<div class=""><br class=""></div><div class="">It looks like a test you added in this commit is failing on macOS: <a href="http://lab.llvm.org:8080/green/job/clang-stage2-cmake-RgSan/6977" class="">http://lab.llvm.org:8080/green/job/clang-stage2-cmake-RgSan/6977</a></div><div class=""><br class=""></div><div class=""><div class="">Can you please take a look?</div><div class=""><br class=""></div><div class="">Thanks!</div><div class="">Erik<div><blockquote type="cite" class=""></blockquote></div></div></div><div class=""><br class=""></div><div class=""><pre style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); word-wrap: break-word; white-space: pre-wrap;" class="">******************** TEST 'LLVM :: ExecutionEngine/OrcLazy/objc-minimal.ll' FAILED ********************
Script:
--
: 'RUN: at line 2'; /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/lli -jit-kind=orc-lazy /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm-project/llvm/test/ExecutionEngine/OrcLazy/objc-minimal.ll
--
Exit Code: 1
Command Output (stderr):
--
=================================================================
==96779==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 80 byte(s) in 1 object(s) allocated from:
#0 0x1127fbef2 in wrap_calloc+0xa2 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x45ef2)
#1 0x7fff5addac5d in allocateBuckets(unsigned int)+0x1d (libobjc.A.dylib:x86_64h+0x9c5d)
#2 0x7fff5addabec in cache_t::reallocate(unsigned int, unsigned int)+0x2a (libobjc.A.dylib:x86_64h+0x9bec)
#3 0x7fff5adda3a8 in cache_fill+0xc2 (libobjc.A.dylib:x86_64h+0x93a8)
#4 0x7fff5add9b94 in lookUpImpOrForward+0x269 (libobjc.A.dylib:x86_64h+0x8b94)
#5 0x7fff5add9493 in _objc_msgSend_uncached+0x43 (libobjc.A.dylib:x86_64h+0x8493)
#6 0x1166f702a (<unknown module>)
#7 0x10a89cd22 in runOrcLazyJIT(char const*) lli.cpp:991
#8 0x10a890b79 in main lli.cpp:409
#9 0x7fff5beb5084 in start+0x0 (libdyld.dylib:x86_64+0x17084)
Direct leak of 80 byte(s) in 1 object(s) allocated from:
#0 0x1127fbef2 in wrap_calloc+0xa2 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x45ef2)
#1 0x7fff5addac5d in allocateBuckets(unsigned int)+0x1d (libobjc.A.dylib:x86_64h+0x9c5d)
#2 0x7fff5addabec in cache_t::reallocate(unsigned int, unsigned int)+0x2a (libobjc.A.dylib:x86_64h+0x9bec)
#3 0x7fff5adda3a8 in cache_fill+0xc2 (libobjc.A.dylib:x86_64h+0x93a8)
#4 0x7fff5add9b94 in lookUpImpOrForward+0x269 (libobjc.A.dylib:x86_64h+0x8b94)
#5 0x7fff5add9493 in _objc_msgSend_uncached+0x43 (libobjc.A.dylib:x86_64h+0x8493)
#6 0x1166f701b (<unknown module>)
#7 0x10a89cd22 in runOrcLazyJIT(char const*) lli.cpp:991
#8 0x10a890b79 in main lli.cpp:409
#9 0x7fff5beb5084 in start+0x0 (libdyld.dylib:x86_64+0x17084)
Direct leak of 64 byte(s) in 1 object(s) allocated from:
#0 0x1127fbef2 in wrap_calloc+0xa2 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x45ef2)
#1 0x7fff5add6510 in realizeClass(objc_class*)+0x4c (libobjc.A.dylib:x86_64h+0x5510)
#2 0x7fff5add65e2 in realizeClass(objc_class*)+0x11e (libobjc.A.dylib:x86_64h+0x55e2)
#3 0x7fff5adee3f5 in objc_readClassPair+0x76 (libobjc.A.dylib:x86_64h+0x1d3f5)
#4 0x10be4620c in llvm::orc::MachOJITDylibInitializers::registerObjCClasses() const MachOPlatform.cpp:133
#5 0x10be246a7 in (anonymous namespace)::MachOPlatformSupport::initialize(llvm::orc::JITDylib&) LLJIT.cpp:525
#6 0x10a8a73f7 in llvm::orc::LLJIT::initialize(llvm::orc::JITDylib&) LLJIT.h:141
#7 0x10a89c514 in runOrcLazyJIT(char const*) lli.cpp:975
#8 0x10a890b79 in main lli.cpp:409
#9 0x7fff5beb5084 in start+0x0 (libdyld.dylib:x86_64+0x17084)
Direct leak of 64 byte(s) in 1 object(s) allocated from:
#0 0x1127fbef2 in wrap_calloc+0xa2 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x45ef2)
#1 0x7fff5add6510 in realizeClass(objc_class*)+0x4c (libobjc.A.dylib:x86_64h+0x5510)
#2 0x7fff5adee3f5 in objc_readClassPair+0x76 (libobjc.A.dylib:x86_64h+0x1d3f5)
#3 0x10be4620c in llvm::orc::MachOJITDylibInitializers::registerObjCClasses() const MachOPlatform.cpp:133
#4 0x10be246a7 in (anonymous namespace)::MachOPlatformSupport::initialize(llvm::orc::JITDylib&) LLJIT.cpp:525
#5 0x10a8a73f7 in llvm::orc::LLJIT::initialize(llvm::orc::JITDylib&) LLJIT.h:141
#6 0x10a89c514 in runOrcLazyJIT(char const*) lli.cpp:975
#7 0x10a890b79 in main lli.cpp:409
#8 0x7fff5beb5084 in start+0x0 (libdyld.dylib:x86_64+0x17084)
Direct leak of 16 byte(s) in 1 object(s) allocated from:
#0 0x1127fbef2 in wrap_calloc+0xa2 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x45ef2)
#1 0x7fff5add883d in class_createInstance+0x52 (libobjc.A.dylib:x86_64h+0x783d)
#2 0x7fff5addc595 in _objc_rootAlloc+0x2c (libobjc.A.dylib:x86_64h+0xb595)
#3 0x1166f701b (<unknown module>)
#4 0x10a89cd22 in runOrcLazyJIT(char const*) lli.cpp:991
#5 0x10a890b79 in main lli.cpp:409
#6 0x7fff5beb5084 in start+0x0 (libdyld.dylib:x86_64+0x17084)
SUMMARY: AddressSanitizer: 304 byte(s) leaked in 5 allocation(s)
</pre></div><div class=""><div><blockquote type="cite" class=""><div class="">On Feb 19, 2020, at 2:03 PM, Lang Hames via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org" class="">llvm-commits@lists.llvm.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class=""><br class="">Author: Lang Hames<br class="">Date: 2020-02-19T13:59:32-08:00<br class="">New Revision: 85fb997659b55101b58e3545733e581bf4ed9cf2<br class=""><br class="">URL: <a href="https://github.com/llvm/llvm-project/commit/85fb997659b55101b58e3545733e581bf4ed9cf2" class="">https://github.com/llvm/llvm-project/commit/85fb997659b55101b58e3545733e581bf4ed9cf2</a><br class="">DIFF: <a href="https://github.com/llvm/llvm-project/commit/85fb997659b55101b58e3545733e581bf4ed9cf2.diff" class="">https://github.com/llvm/llvm-project/commit/85fb997659b55101b58e3545733e581bf4ed9cf2.diff</a><br class=""><br class="">LOG: [ORC] Add generic initializer/deinitializer support.<br class=""><br class="">Initializers and deinitializers are used to implement C++ static constructors<br class="">and destructors, runtime registration for some languages (e.g. with the<br class="">Objective-C runtime for Objective-C/C++ code) and other tasks that would<br class="">typically be performed when a shared-object/dylib is loaded or unloaded by a<br class="">statically compiled program.<br class=""><br class="">MCJIT and ORC have historically provided limited support for discovering and<br class="">running initializers/deinitializers by scanning the llvm.global_ctors and<br class="">llvm.global_dtors variables and recording the functions to be run. This approach<br class="">suffers from several drawbacks: (1) It only works for IR inputs, not for object<br class="">files (including cached JIT'd objects). (2) It only works for initializers<br class="">described by llvm.global_ctors and llvm.global_dtors, however not all<br class="">initializers are described in this way (Objective-C, for example, describes<br class="">initializers via specially named metadata sections). (3) To make the<br class="">initializer/deinitializer functions described by llvm.global_ctors and<br class="">llvm.global_dtors searchable they must be promoted to extern linkage, polluting<br class="">the JIT symbol table (extra care must be taken to ensure this promotion does<br class="">not result in symbol name clashes).<br class=""><br class="">This patch introduces several interdependent changes to ORCv2 to support the<br class="">construction of new initialization schemes, and includes an implementation of a<br class="">backwards-compatible llvm.global_ctor/llvm.global_dtor scanning scheme, and a<br class="">MachO specific scheme that handles Objective-C runtime registration (if the<br class="">Objective-C runtime is available) enabling execution of LLVM IR compiled from<br class="">Objective-C and Swift.<br class=""><br class="">The major changes included in this patch are:<br class=""><br class="">(1) The MaterializationUnit and MaterializationResponsibility classes are<br class="">extended to describe an optional "initializer" symbol for the module (see the<br class="">getInitializerSymbol method on each class). The presence or absence of this<br class="">symbol indicates whether the module contains any initializers or<br class="">deinitializers. The initializer symbol otherwise behaves like any other:<br class="">searching for it triggers materialization.<br class=""><br class="">(2) A new Platform interface is introduced in llvm/ExecutionEngine/Orc/Core.h<br class="">which provides the following callback interface:<br class=""><br class=""> - Error setupJITDylib(JITDylib &JD): Can be used to install standard symbols<br class=""> in JITDylibs upon creation. E.g. __dso_handle.<br class=""><br class=""> - Error notifyAdding(JITDylib &JD, const MaterializationUnit &MU): Generally<br class=""> used to record initializer symbols.<br class=""><br class=""> - Error notifyRemoving(JITDylib &JD, VModuleKey K): Used to notify a platform<br class=""> that a module is being removed.<br class=""><br class=""> Platform implementations can use these callbacks to track outstanding<br class="">initializers and implement a platform-specific approach for executing them. For<br class="">example, the MachOPlatform installs a plugin in the JIT linker to scan for both<br class="">__mod_inits sections (for C++ static constructors) and ObjC metadata sections.<br class="">If discovered, these are processed in the usual platform order: Objective-C<br class="">registration is carried out first, then static initializers are executed,<br class="">ensuring that calls to Objective-C from static initializers will be safe.<br class=""><br class="">This patch updates LLJIT to use the new scheme for initialization. Two<br class="">LLJIT::PlatformSupport classes are implemented: A GenericIR platform and a MachO<br class="">platform. The GenericIR platform implements a modified version of the previous<br class="">llvm.global-ctor scraping scheme to provide support for Windows and<br class="">Linux. LLJIT's MachO platform uses the MachOPlatform class to provide MachO<br class="">specific initialization as described above.<br class=""><br class="">Reviewers: sgraenitz, dblaikie<br class=""><br class="">Subscribers: mgorny, hiraditya, mgrang, ributzka, llvm-commits<br class=""><br class="">Tags: #llvm<br class=""><br class="">Differential Revision: <a href="https://reviews.llvm.org/D74300" class="">https://reviews.llvm.org/D74300</a><br class=""><br class="">Added: <br class=""> llvm/include/llvm/ExecutionEngine/Orc/MachOPlatform.h<br class=""> llvm/include/llvm/ExecutionEngine/Orc/Mangling.h<br class=""> llvm/lib/ExecutionEngine/Orc/MachOPlatform.cpp<br class=""> llvm/lib/ExecutionEngine/Orc/Mangling.cpp<br class=""> llvm/test/ExecutionEngine/OrcLazy/objc-minimal.ll<br class=""><br class="">Modified: <br class=""> llvm/examples/Kaleidoscope/BuildingAJIT/Chapter1/KaleidoscopeJIT.h<br class=""> llvm/examples/Kaleidoscope/BuildingAJIT/Chapter2/KaleidoscopeJIT.h<br class=""> llvm/include/llvm/ExecutionEngine/JITLink/JITLink.h<br class=""> llvm/include/llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h<br class=""> llvm/include/llvm/ExecutionEngine/Orc/CompileUtils.h<br class=""> llvm/include/llvm/ExecutionEngine/Orc/Core.h<br class=""> llvm/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h<br class=""> llvm/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h<br class=""> llvm/include/llvm/ExecutionEngine/Orc/IRTransformLayer.h<br class=""> llvm/include/llvm/ExecutionEngine/Orc/IndirectionUtils.h<br class=""> llvm/include/llvm/ExecutionEngine/Orc/LLJIT.h<br class=""> llvm/include/llvm/ExecutionEngine/Orc/Layer.h<br class=""> llvm/include/llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h<br class=""> llvm/include/llvm/ExecutionEngine/Orc/SymbolStringPool.h<br class=""> llvm/lib/ExecutionEngine/Orc/CMakeLists.txt<br class=""> llvm/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp<br class=""> llvm/lib/ExecutionEngine/Orc/CompileUtils.cpp<br class=""> llvm/lib/ExecutionEngine/Orc/Core.cpp<br class=""> llvm/lib/ExecutionEngine/Orc/ExecutionUtils.cpp<br class=""> llvm/lib/ExecutionEngine/Orc/IndirectionUtils.cpp<br class=""> llvm/lib/ExecutionEngine/Orc/LLJIT.cpp<br class=""> llvm/lib/ExecutionEngine/Orc/Layer.cpp<br class=""> llvm/lib/ExecutionEngine/Orc/LazyReexports.cpp<br class=""> llvm/lib/ExecutionEngine/Orc/ObjectLinkingLayer.cpp<br class=""> llvm/lib/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.cpp<br class=""> llvm/tools/lli/lli.cpp<br class=""> llvm/tools/llvm-jitlink/llvm-jitlink.cpp<br class=""> llvm/tools/llvm-jitlink/llvm-jitlink.h<br class=""> llvm/unittests/ExecutionEngine/Orc/CoreAPIsTest.cpp<br class=""> llvm/unittests/ExecutionEngine/Orc/OrcTestCommon.h<br class=""> llvm/unittests/ExecutionEngine/Orc/RTDyldObjectLinkingLayerTest.cpp<br class=""><br class="">Removed: <br class=""><br class=""><br class=""><br class="">################################################################################<br class="">diff --git a/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter1/KaleidoscopeJIT.h b/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter1/KaleidoscopeJIT.h<br class="">index b7404baf1ff0..adb6be436660 100644<br class="">--- a/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter1/KaleidoscopeJIT.h<br class="">+++ b/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter1/KaleidoscopeJIT.h<br class="">@@ -49,7 +49,7 @@ class KaleidoscopeJIT {<br class=""> std::make_unique<ConcurrentIRCompiler>(std::move(JTMB))),<br class=""> DL(std::move(DL)), Mangle(ES, this->DL),<br class=""> Ctx(std::make_unique<LLVMContext>()),<br class="">- MainJD(ES.createJITDylib("<main>")) {<br class="">+ MainJD(ES.createBareJITDylib("<main>")) {<br class=""> MainJD.addGenerator(<br class=""> cantFail(DynamicLibrarySearchGenerator::GetForCurrentProcess(<br class=""> DL.getGlobalPrefix())));<br class=""><br class="">diff --git a/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter2/KaleidoscopeJIT.h b/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter2/KaleidoscopeJIT.h<br class="">index efb19349e3e1..28649ffe4910 100644<br class="">--- a/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter2/KaleidoscopeJIT.h<br class="">+++ b/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter2/KaleidoscopeJIT.h<br class="">@@ -55,7 +55,7 @@ class KaleidoscopeJIT {<br class=""> std::make_unique<ConcurrentIRCompiler>(std::move(JTMB))),<br class=""> OptimizeLayer(ES, CompileLayer, optimizeModule), DL(std::move(DL)),<br class=""> Mangle(ES, this->DL), Ctx(std::make_unique<LLVMContext>()),<br class="">- MainJD(ES.createJITDylib("<main>")) {<br class="">+ MainJD(ES.createBareJITDylib("<main>")) {<br class=""> MainJD.addGenerator(<br class=""> cantFail(DynamicLibrarySearchGenerator::GetForCurrentProcess(<br class=""> DL.getGlobalPrefix())));<br class=""><br class="">diff --git a/llvm/include/llvm/ExecutionEngine/JITLink/JITLink.h b/llvm/include/llvm/ExecutionEngine/JITLink/JITLink.h<br class="">index 1a8f5cacbcd4..d673a89386da 100644<br class="">--- a/llvm/include/llvm/ExecutionEngine/JITLink/JITLink.h<br class="">+++ b/llvm/include/llvm/ExecutionEngine/JITLink/JITLink.h<br class="">@@ -488,6 +488,8 @@ class Symbol {<br class=""><br class=""> /// Set the visibility for this Symbol.<br class=""> void setScope(Scope S) {<br class="">+ assert((!Name.empty() || S == Scope::Local) &&<br class="">+ "Can not set anonymous symbol to non-local scope");<br class=""> assert((S == Scope::Default || Base->isDefined() || Base->isAbsolute()) &&<br class=""> "Invalid visibility for symbol type");<br class=""> this->S = static_cast<uint8_t>(S);<br class=""><br class="">diff --git a/llvm/include/llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h b/llvm/include/llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h<br class="">index 65295f97e2c6..ef251ac302df 100644<br class="">--- a/llvm/include/llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h<br class="">+++ b/llvm/include/llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h<br class="">@@ -94,6 +94,7 @@ class CompileOnDemandLayer : public IRLayer {<br class=""><br class=""> /// Sets the ImplSymbolMap<br class=""> void setImplMap(ImplSymbolMap *Imp);<br class="">+<br class=""> /// Emits the given module. This should not be called by clients: it will be<br class=""> /// called by the JIT when a definition added via the add method is requested.<br class=""> void emit(MaterializationResponsibility R, ThreadSafeModule TSM) override;<br class=""><br class="">diff --git a/llvm/include/llvm/ExecutionEngine/Orc/CompileUtils.h b/llvm/include/llvm/ExecutionEngine/Orc/CompileUtils.h<br class="">index 218afda1b546..8376d163d57a 100644<br class="">--- a/llvm/include/llvm/ExecutionEngine/Orc/CompileUtils.h<br class="">+++ b/llvm/include/llvm/ExecutionEngine/Orc/CompileUtils.h<br class="">@@ -30,7 +30,7 @@ namespace orc {<br class=""><br class=""> class JITTargetMachineBuilder;<br class=""><br class="">-IRMaterializationUnit::ManglingOptions<br class="">+IRSymbolMapper::ManglingOptions<br class=""> irManglingOptionsFromTargetOptions(const TargetOptions &Opts);<br class=""><br class=""> /// Simple compile functor: Takes a single IR module and returns an ObjectFile.<br class="">@@ -52,7 +52,7 @@ class SimpleCompiler : public IRCompileLayer::IRCompiler {<br class=""> Expected<CompileResult> operator()(Module &M) override;<br class=""><br class=""> private:<br class="">- IRMaterializationUnit::ManglingOptions<br class="">+ IRSymbolMapper::ManglingOptions<br class=""> manglingOptionsForTargetMachine(const TargetMachine &TM);<br class=""><br class=""> CompileResult tryToLoadFromObjectCache(const Module &M);<br class=""><br class="">diff --git a/llvm/include/llvm/ExecutionEngine/Orc/Core.h b/llvm/include/llvm/ExecutionEngine/Orc/Core.h<br class="">index 1a55801b9d87..94600f2286b8 100644<br class="">--- a/llvm/include/llvm/ExecutionEngine/Orc/Core.h<br class="">+++ b/llvm/include/llvm/ExecutionEngine/Orc/Core.h<br class="">@@ -14,11 +14,11 @@<br class=""> #define LLVM_EXECUTIONENGINE_ORC_CORE_H<br class=""><br class=""> #include "llvm/ADT/BitmaskEnum.h"<br class="">+#include "llvm/ADT/DenseSet.h"<br class=""> #include "llvm/ADT/FunctionExtras.h"<br class=""> #include "llvm/ExecutionEngine/JITSymbol.h"<br class=""> #include "llvm/ExecutionEngine/Orc/SymbolStringPool.h"<br class=""> #include "llvm/ExecutionEngine/OrcV1Deprecation.h"<br class="">-#include "llvm/IR/Module.h"<br class=""> #include "llvm/Support/Debug.h"<br class=""><br class=""> #include <memory><br class="">@@ -456,6 +456,11 @@ class MaterializationResponsibility {<br class=""> /// before using.<br class=""> const SymbolFlagsMap &getSymbols() const { return SymbolFlags; }<br class=""><br class="">+ /// Returns the initialization pseudo-symbol, if any. This symbol will also<br class="">+ /// be present in the SymbolFlagsMap for this MaterializationResponsibility<br class="">+ /// object.<br class="">+ const SymbolStringPtr &getInitializerSymbol() const { return InitSymbol; }<br class="">+<br class=""> /// Returns the names of any symbols covered by this<br class=""> /// MaterializationResponsibility object that have queries pending. This<br class=""> /// information can be used to return responsibility for unrequested symbols<br class="">@@ -532,10 +537,15 @@ class MaterializationResponsibility {<br class=""> /// Create a MaterializationResponsibility for the given JITDylib and<br class=""> /// initial symbols.<br class=""> MaterializationResponsibility(JITDylib &JD, SymbolFlagsMap SymbolFlags,<br class="">- VModuleKey K);<br class="">+ SymbolStringPtr InitSymbol, VModuleKey K)<br class="">+ : JD(JD), SymbolFlags(std::move(SymbolFlags)),<br class="">+ InitSymbol(std::move(InitSymbol)), K(std::move(K)) {<br class="">+ assert(!this->SymbolFlags.empty() && "Materializing nothing?");<br class="">+ }<br class=""><br class=""> JITDylib &JD;<br class=""> SymbolFlagsMap SymbolFlags;<br class="">+ SymbolStringPtr InitSymbol;<br class=""> VModuleKey K;<br class=""> };<br class=""><br class="">@@ -549,8 +559,13 @@ class MaterializationResponsibility {<br class=""> /// stronger definition is added or already present.<br class=""> class MaterializationUnit {<br class=""> public:<br class="">- MaterializationUnit(SymbolFlagsMap InitalSymbolFlags, VModuleKey K)<br class="">- : SymbolFlags(std::move(InitalSymbolFlags)), K(std::move(K)) {}<br class="">+ MaterializationUnit(SymbolFlagsMap InitalSymbolFlags,<br class="">+ SymbolStringPtr InitSymbol, VModuleKey K)<br class="">+ : SymbolFlags(std::move(InitalSymbolFlags)),<br class="">+ InitSymbol(std::move(InitSymbol)), K(std::move(K)) {<br class="">+ assert((!this->InitSymbol || this->SymbolFlags.count(this->InitSymbol)) &&<br class="">+ "If set, InitSymbol should appear in InitialSymbolFlags map");<br class="">+ }<br class=""><br class=""> virtual ~MaterializationUnit() {}<br class=""><br class="">@@ -561,12 +576,15 @@ class MaterializationUnit {<br class=""> /// Return the set of symbols that this source provides.<br class=""> const SymbolFlagsMap &getSymbols() const { return SymbolFlags; }<br class=""><br class="">+ /// Returns the initialization symbol for this MaterializationUnit (if any).<br class="">+ const SymbolStringPtr &getInitializerSymbol() const { return InitSymbol; }<br class="">+<br class=""> /// Called by materialization dispatchers (see<br class=""> /// ExecutionSession::DispatchMaterializationFunction) to trigger<br class=""> /// materialization of this MaterializationUnit.<br class=""> void doMaterialize(JITDylib &JD) {<br class="">- materialize(MaterializationResponsibility(JD, std::move(SymbolFlags),<br class="">- std::move(K)));<br class="">+ materialize(MaterializationResponsibility(<br class="">+ JD, std::move(SymbolFlags), std::move(InitSymbol), std::move(K)));<br class=""> }<br class=""><br class=""> /// Called by JITDylibs to notify MaterializationUnits that the given symbol<br class="">@@ -578,6 +596,7 @@ class MaterializationUnit {<br class=""><br class=""> protected:<br class=""> SymbolFlagsMap SymbolFlags;<br class="">+ SymbolStringPtr InitSymbol;<br class=""> VModuleKey K;<br class=""><br class=""> private:<br class="">@@ -774,6 +793,7 @@ class AsynchronousSymbolQuery {<br class=""> class JITDylib {<br class=""> friend class AsynchronousSymbolQuery;<br class=""> friend class ExecutionSession;<br class="">+ friend class Platform;<br class=""> friend class MaterializationResponsibility;<br class=""> public:<br class=""> /// Definition generators can be attached to JITDylibs to generate new<br class="">@@ -1054,6 +1074,35 @@ class JITDylib {<br class=""> JITDylibSearchOrder SearchOrder;<br class=""> };<br class=""><br class="">+/// Platforms set up standard symbols and mediate interactions between dynamic<br class="">+/// initializers (e.g. C++ static constructors) and ExecutionSession state.<br class="">+/// Note that Platforms do not automatically run initializers: clients are still<br class="">+/// responsible for doing this.<br class="">+class Platform {<br class="">+public:<br class="">+ virtual ~Platform();<br class="">+<br class="">+ /// This method will be called outside the session lock each time a JITDylib<br class="">+ /// is created (unless it is created with EmptyJITDylib set) to allow the<br class="">+ /// Platform to install any JITDylib specific standard symbols (e.g<br class="">+ /// __dso_handle).<br class="">+ virtual Error setupJITDylib(JITDylib &JD) = 0;<br class="">+<br class="">+ /// This method will be called under the ExecutionSession lock each time a<br class="">+ /// MaterializationUnit is added to a JITDylib.<br class="">+ virtual Error notifyAdding(JITDylib &JD, const MaterializationUnit &MU) = 0;<br class="">+<br class="">+ /// This method will be called under the ExecutionSession lock when a<br class="">+ /// VModuleKey is removed.<br class="">+ virtual Error notifyRemoving(JITDylib &JD, VModuleKey K) = 0;<br class="">+<br class="">+ /// A utility function for looking up initializer symbols. Performs a blocking<br class="">+ /// lookup for the given symbols in each of the given JITDylibs.<br class="">+ static Expected<DenseMap<JITDylib *, SymbolMap>><br class="">+ lookupInitSymbols(ExecutionSession &ES,<br class="">+ const DenseMap<JITDylib *, SymbolLookupSet> &InitSyms);<br class="">+};<br class="">+<br class=""> /// An ExecutionSession represents a running JIT program.<br class=""> class ExecutionSession {<br class=""> // FIXME: Remove this when we remove the old ORC layers.<br class="">@@ -1078,6 +1127,13 @@ class ExecutionSession {<br class=""> /// Returns a shared_ptr to the SymbolStringPool for this ExecutionSession.<br class=""> std::shared_ptr<SymbolStringPool> getSymbolStringPool() const { return SSP; }<br class=""><br class="">+ /// Set the Platform for this ExecutionSession.<br class="">+ void setPlatform(std::unique_ptr<Platform> P) { this->P = std::move(P); }<br class="">+<br class="">+ /// Get the Platform for this session.<br class="">+ /// Will return null if no Platform has been set for this ExecutionSession.<br class="">+ Platform *getPlatform() { return P.get(); }<br class="">+<br class=""> /// Run the given lambda with the session mutex locked.<br class=""> template <typename Func> decltype(auto) runSessionLocked(Func &&F) {<br class=""> std::lock_guard<std::recursive_mutex> Lock(SessionMutex);<br class="">@@ -1088,12 +1144,26 @@ class ExecutionSession {<br class=""> /// Ownership of JITDylib remains within Execution Session<br class=""> JITDylib *getJITDylibByName(StringRef Name);<br class=""><br class="">+ /// Add a new bare JITDylib to this ExecutionSession.<br class="">+ ///<br class="">+ /// The JITDylib Name is required to be unique. Clients should verify that<br class="">+ /// names are not being re-used (E.g. by calling getJITDylibByName) if names<br class="">+ /// are based on user input.<br class="">+ ///<br class="">+ /// This call does not install any library code or symbols into the newly<br class="">+ /// created JITDylib. The client is responsible for all configuration.<br class="">+ JITDylib &createBareJITDylib(std::string Name);<br class="">+<br class=""> /// Add a new JITDylib to this ExecutionSession.<br class=""> ///<br class=""> /// The JITDylib Name is required to be unique. Clients should verify that<br class=""> /// names are not being re-used (e.g. by calling getJITDylibByName) if names<br class=""> /// are based on user input.<br class="">- JITDylib &createJITDylib(std::string Name);<br class="">+ ///<br class="">+ /// If a Platform is attached then Platform::setupJITDylib will be called to<br class="">+ /// install standard platform symbols (e.g. standard library interposes).<br class="">+ /// If no Platform is attached this call is equivalent to createBareJITDylib.<br class="">+ Expected<JITDylib &> createJITDylib(std::string Name);<br class=""><br class=""> /// Allocate a module key for a new module to add to the JIT.<br class=""> VModuleKey allocateVModule() {<br class="">@@ -1177,20 +1247,23 @@ class ExecutionSession {<br class=""> /// Convenience version of blocking lookup.<br class=""> /// Searches each of the JITDylibs in the search order in turn for the given<br class=""> /// symbol.<br class="">- Expected<JITEvaluatedSymbol> lookup(const JITDylibSearchOrder &SearchOrder,<br class="">- SymbolStringPtr Symbol);<br class="">+ Expected<JITEvaluatedSymbol><br class="">+ lookup(const JITDylibSearchOrder &SearchOrder, SymbolStringPtr Symbol,<br class="">+ SymbolState RequiredState = SymbolState::Ready);<br class=""><br class=""> /// Convenience version of blocking lookup.<br class=""> /// Searches each of the JITDylibs in the search order in turn for the given<br class=""> /// symbol. The search will not find non-exported symbols.<br class="">- Expected<JITEvaluatedSymbol> lookup(ArrayRef<JITDylib *> SearchOrder,<br class="">- SymbolStringPtr Symbol);<br class="">+ Expected<JITEvaluatedSymbol><br class="">+ lookup(ArrayRef<JITDylib *> SearchOrder, SymbolStringPtr Symbol,<br class="">+ SymbolState RequiredState = SymbolState::Ready);<br class=""><br class=""> /// Convenience version of blocking lookup.<br class=""> /// Searches each of the JITDylibs in the search order in turn for the given<br class=""> /// symbol. The search will not find non-exported symbols.<br class="">- Expected<JITEvaluatedSymbol> lookup(ArrayRef<JITDylib *> SearchOrder,<br class="">- StringRef Symbol);<br class="">+ Expected<JITEvaluatedSymbol><br class="">+ lookup(ArrayRef<JITDylib *> SearchOrder, StringRef Symbol,<br class="">+ SymbolState RequiredState = SymbolState::Ready);<br class=""><br class=""> /// Materialize the given unit.<br class=""> void dispatchMaterialization(JITDylib &JD,<br class="">@@ -1221,6 +1294,7 @@ class ExecutionSession {<br class=""><br class=""> mutable std::recursive_mutex SessionMutex;<br class=""> std::shared_ptr<SymbolStringPool> SSP;<br class="">+ std::unique_ptr<Platform> P;<br class=""> VModuleKey LastKey = 0;<br class=""> ErrorReporter ReportError = logErrorsToStdErr;<br class=""> DispatchMaterializationFunction DispatchMaterialization =<br class="">@@ -1256,6 +1330,11 @@ Error JITDylib::define(std::unique_ptr<MaterializationUnitType> &&MU) {<br class=""> if (auto Err = defineImpl(*MU))<br class=""> return Err;<br class=""><br class="">+ if (auto *P = ES.getPlatform()) {<br class="">+ if (auto Err = P->notifyAdding(*this, *MU))<br class="">+ return Err;<br class="">+ }<br class="">+<br class=""> /// defineImpl succeeded.<br class=""> auto UMI = std::make_shared<UnmaterializedInfo>(std::move(MU));<br class=""> for (auto &KV : UMI->MU->getSymbols())<br class="">@@ -1273,6 +1352,11 @@ Error JITDylib::define(std::unique_ptr<MaterializationUnitType> &MU) {<br class=""> if (auto Err = defineImpl(*MU))<br class=""> return Err;<br class=""><br class="">+ if (auto *P = ES.getPlatform()) {<br class="">+ if (auto Err = P->notifyAdding(*this, *MU))<br class="">+ return Err;<br class="">+ }<br class="">+<br class=""> /// defineImpl succeeded.<br class=""> auto UMI = std::make_shared<UnmaterializedInfo>(std::move(MU));<br class=""> for (auto &KV : UMI->MU->getSymbols())<br class="">@@ -1305,18 +1389,6 @@ class ReexportsGenerator : public JITDylib::DefinitionGenerator {<br class=""> SymbolPredicate Allow;<br class=""> };<br class=""><br class="">-/// Mangles symbol names then uniques them in the context of an<br class="">-/// ExecutionSession.<br class="">-class MangleAndInterner {<br class="">-public:<br class="">- MangleAndInterner(ExecutionSession &ES, const DataLayout &DL);<br class="">- SymbolStringPtr operator()(StringRef Name);<br class="">-<br class="">-private:<br class="">- ExecutionSession &ES;<br class="">- const DataLayout &DL;<br class="">-};<br class="">-<br class=""> } // End namespace orc<br class=""> } // End namespace llvm<br class=""><br class=""><br class="">diff --git a/llvm/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h b/llvm/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h<br class="">index f7255c5af845..49ab0d628b0b 100644<br class="">--- a/llvm/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h<br class="">+++ b/llvm/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h<br class="">@@ -17,6 +17,7 @@<br class=""> #include "llvm/ADT/iterator_range.h"<br class=""> #include "llvm/ExecutionEngine/JITSymbol.h"<br class=""> #include "llvm/ExecutionEngine/Orc/Core.h"<br class="">+#include "llvm/ExecutionEngine/Orc/Mangling.h"<br class=""> #include "llvm/ExecutionEngine/Orc/OrcError.h"<br class=""> #include "llvm/ExecutionEngine/RuntimeDyld.h"<br class=""> #include "llvm/Object/Archive.h"<br class="">@@ -104,6 +105,53 @@ iterator_range<CtorDtorIterator> getConstructors(const Module &M);<br class=""> /// array.<br class=""> iterator_range<CtorDtorIterator> getDestructors(const Module &M);<br class=""><br class="">+/// This iterator provides a convenient way to iterate over GlobalValues that<br class="">+/// have initialization effects.<br class="">+class StaticInitGVIterator {<br class="">+public:<br class="">+ StaticInitGVIterator() = default;<br class="">+<br class="">+ StaticInitGVIterator(Module &M)<br class="">+ : I(M.global_values().begin()), E(M.global_values().end()),<br class="">+ ObjFmt(Triple(M.getTargetTriple()).getObjectFormat()) {<br class="">+ if (I != E) {<br class="">+ if (!isStaticInitGlobal(*I))<br class="">+ moveToNextStaticInitGlobal();<br class="">+ } else<br class="">+ I = E = Module::global_value_iterator();<br class="">+ }<br class="">+<br class="">+ bool operator==(const StaticInitGVIterator &O) const { return I == O.I; }<br class="">+ bool operator!=(const StaticInitGVIterator &O) const { return I != O.I; }<br class="">+<br class="">+ StaticInitGVIterator &operator++() {<br class="">+ assert(I != E && "Increment past end of range");<br class="">+ moveToNextStaticInitGlobal();<br class="">+ return *this;<br class="">+ }<br class="">+<br class="">+ GlobalValue &operator*() { return *I; }<br class="">+<br class="">+private:<br class="">+ bool isStaticInitGlobal(GlobalValue &GV);<br class="">+ void moveToNextStaticInitGlobal() {<br class="">+ ++I;<br class="">+ while (I != E && !isStaticInitGlobal(*I))<br class="">+ ++I;<br class="">+ if (I == E)<br class="">+ I = E = Module::global_value_iterator();<br class="">+ }<br class="">+<br class="">+ Module::global_value_iterator I, E;<br class="">+ Triple::ObjectFormatType ObjFmt;<br class="">+};<br class="">+<br class="">+/// Create an iterator range over the GlobalValues that contribute to static<br class="">+/// initialization.<br class="">+inline iterator_range<StaticInitGVIterator> getStaticInitGVs(Module &M) {<br class="">+ return make_range(StaticInitGVIterator(M), StaticInitGVIterator());<br class="">+}<br class="">+<br class=""> /// Convenience class for recording constructor/destructor names for<br class=""> /// later execution.<br class=""> template <typename JITLayerT><br class="">@@ -246,6 +294,22 @@ class LocalCXXRuntimeOverrides : public LocalCXXRuntimeOverridesBase {<br class=""> Error enable(JITDylib &JD, MangleAndInterner &Mangler);<br class=""> };<br class=""><br class="">+/// An interface for Itanium __cxa_atexit interposer implementations.<br class="">+class ItaniumCXAAtExitSupport {<br class="">+public:<br class="">+ struct AtExitRecord {<br class="">+ void (*F)(void *);<br class="">+ void *Ctx;<br class="">+ };<br class="">+<br class="">+ void registerAtExit(void (*F)(void *), void *Ctx, void *DSOHandle);<br class="">+ void runAtExits(void *DSOHandle);<br class="">+<br class="">+private:<br class="">+ std::mutex AtExitsMutex;<br class="">+ DenseMap<void *, std::vector<AtExitRecord>> AtExitRecords;<br class="">+};<br class="">+<br class=""> /// A utility class to expose symbols found via dlsym to the JIT.<br class=""> ///<br class=""> /// If an instance of this class is attached to a JITDylib as a fallback<br class=""><br class="">diff --git a/llvm/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h b/llvm/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h<br class="">index bb8270fe80a3..eb74d283f043 100644<br class="">--- a/llvm/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h<br class="">+++ b/llvm/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h<br class="">@@ -31,18 +31,18 @@ class IRCompileLayer : public IRLayer {<br class=""> public:<br class=""> class IRCompiler {<br class=""> public:<br class="">- IRCompiler(IRMaterializationUnit::ManglingOptions MO) : MO(std::move(MO)) {}<br class="">+ IRCompiler(IRSymbolMapper::ManglingOptions MO) : MO(std::move(MO)) {}<br class=""> virtual ~IRCompiler();<br class="">- const IRMaterializationUnit::ManglingOptions &getManglingOptions() const {<br class="">+ const IRSymbolMapper::ManglingOptions &getManglingOptions() const {<br class=""> return MO;<br class=""> }<br class=""> virtual Expected<std::unique_ptr<MemoryBuffer>> operator()(Module &M) = 0;<br class=""><br class=""> protected:<br class="">- IRMaterializationUnit::ManglingOptions &manglingOptions() { return MO; }<br class="">+ IRSymbolMapper::ManglingOptions &manglingOptions() { return MO; }<br class=""><br class=""> private:<br class="">- IRMaterializationUnit::ManglingOptions MO;<br class="">+ IRSymbolMapper::ManglingOptions MO;<br class=""> };<br class=""><br class=""> using NotifyCompiledFunction =<br class="">@@ -61,7 +61,7 @@ class IRCompileLayer : public IRLayer {<br class=""> mutable std::mutex IRLayerMutex;<br class=""> ObjectLayer &BaseLayer;<br class=""> std::unique_ptr<IRCompiler> Compile;<br class="">- const IRMaterializationUnit::ManglingOptions *ManglingOpts;<br class="">+ const IRSymbolMapper::ManglingOptions *ManglingOpts;<br class=""> NotifyCompiledFunction NotifyCompiled = NotifyCompiledFunction();<br class=""> };<br class=""><br class=""><br class="">diff --git a/llvm/include/llvm/ExecutionEngine/Orc/IRTransformLayer.h b/llvm/include/llvm/ExecutionEngine/Orc/IRTransformLayer.h<br class="">index b71e5b339711..296d74ae6b86 100644<br class="">--- a/llvm/include/llvm/ExecutionEngine/Orc/IRTransformLayer.h<br class="">+++ b/llvm/include/llvm/ExecutionEngine/Orc/IRTransformLayer.h<br class="">@@ -28,7 +28,7 @@ namespace orc {<br class=""> class IRTransformLayer : public IRLayer {<br class=""> public:<br class=""> using TransformFunction = std::function<Expected<ThreadSafeModule>(<br class="">- ThreadSafeModule, const MaterializationResponsibility &R)>;<br class="">+ ThreadSafeModule, MaterializationResponsibility &R)>;<br class=""><br class=""> IRTransformLayer(ExecutionSession &ES, IRLayer &BaseLayer,<br class=""> TransformFunction Transform = identityTransform);<br class="">@@ -39,9 +39,8 @@ class IRTransformLayer : public IRLayer {<br class=""><br class=""> void emit(MaterializationResponsibility R, ThreadSafeModule TSM) override;<br class=""><br class="">- static ThreadSafeModule<br class="">- identityTransform(ThreadSafeModule TSM,<br class="">- const MaterializationResponsibility &R) {<br class="">+ static ThreadSafeModule identityTransform(ThreadSafeModule TSM,<br class="">+ MaterializationResponsibility &R) {<br class=""> return TSM;<br class=""> }<br class=""><br class=""><br class="">diff --git a/llvm/include/llvm/ExecutionEngine/Orc/IndirectionUtils.h b/llvm/include/llvm/ExecutionEngine/Orc/IndirectionUtils.h<br class="">index a9ab3a630a64..880c8ff44c5f 100644<br class="">--- a/llvm/include/llvm/ExecutionEngine/Orc/IndirectionUtils.h<br class="">+++ b/llvm/include/llvm/ExecutionEngine/Orc/IndirectionUtils.h<br class="">@@ -201,7 +201,7 @@ class JITCompileCallbackManager {<br class=""> ExecutionSession &ES,<br class=""> JITTargetAddress ErrorHandlerAddress)<br class=""> : TP(std::move(TP)), ES(ES),<br class="">- CallbacksJD(ES.createJITDylib("<Callbacks>")),<br class="">+ CallbacksJD(ES.createBareJITDylib("<Callbacks>")),<br class=""> ErrorHandlerAddress(ErrorHandlerAddress) {}<br class=""><br class=""> void setTrampolinePool(std::unique_ptr<TrampolinePool> TP) {<br class=""><br class="">diff --git a/llvm/include/llvm/ExecutionEngine/Orc/LLJIT.h b/llvm/include/llvm/ExecutionEngine/Orc/LLJIT.h<br class="">index 3374a29f04df..c0b5012f801f 100644<br class="">--- a/llvm/include/llvm/ExecutionEngine/Orc/LLJIT.h<br class="">+++ b/llvm/include/llvm/ExecutionEngine/Orc/LLJIT.h<br class="">@@ -35,7 +35,23 @@ class LLLazyJITBuilderState;<br class=""> class LLJIT {<br class=""> template <typename, typename, typename> friend class LLJITBuilderSetters;<br class=""><br class="">+ friend void setUpGenericLLVMIRPlatform(LLJIT &J);<br class="">+<br class=""> public:<br class="">+ /// Initializer support for LLJIT.<br class="">+ class PlatformSupport {<br class="">+ public:<br class="">+ virtual ~PlatformSupport();<br class="">+<br class="">+ virtual Error initialize(JITDylib &JD) = 0;<br class="">+<br class="">+ virtual Error deinitialize(JITDylib &JD) = 0;<br class="">+<br class="">+ protected:<br class="">+ static void setInitTransform(LLJIT &J,<br class="">+ IRTransformLayer::TransformFunction T);<br class="">+ };<br class="">+<br class=""> static Expected<std::unique_ptr<LLJIT>> Create(LLJITBuilderState &S);<br class=""><br class=""> /// Destruct this instance. If a multi-threaded instance, waits for all<br class="">@@ -52,7 +68,7 @@ class LLJIT {<br class=""> const DataLayout &getDataLayout() const { return DL; }<br class=""><br class=""> /// Returns a reference to the JITDylib representing the JIT'd main program.<br class="">- JITDylib &getMainJITDylib() { return Main; }<br class="">+ JITDylib &getMainJITDylib() { return *Main; }<br class=""><br class=""> /// Returns the JITDylib with the given name, or nullptr if no JITDylib with<br class=""> /// that name exists.<br class="">@@ -66,7 +82,7 @@ class LLJIT {<br class=""> /// input or elsewhere in the environment then the client should check<br class=""> /// (e.g. by calling getJITDylibByName) that the given name is not already in<br class=""> /// use.<br class="">- JITDylib &createJITDylib(std::string Name) {<br class="">+ Expected<JITDylib &> createJITDylib(std::string Name) {<br class=""> return ES->createJITDylib(std::move(Name));<br class=""> }<br class=""><br class="">@@ -78,7 +94,7 @@ class LLJIT {<br class=""><br class=""> /// Adds an IR module to the Main JITDylib.<br class=""> Error addIRModule(ThreadSafeModule TSM) {<br class="">- return addIRModule(Main, std::move(TSM));<br class="">+ return addIRModule(*Main, std::move(TSM));<br class=""> }<br class=""><br class=""> /// Adds an object file to the given JITDylib.<br class="">@@ -86,7 +102,7 @@ class LLJIT {<br class=""><br class=""> /// Adds an object file to the given JITDylib.<br class=""> Error addObjectFile(std::unique_ptr<MemoryBuffer> Obj) {<br class="">- return addObjectFile(Main, std::move(Obj));<br class="">+ return addObjectFile(*Main, std::move(Obj));<br class=""> }<br class=""><br class=""> /// Look up a symbol in JITDylib JD by the symbol's linker-mangled name (to<br class="">@@ -98,7 +114,7 @@ class LLJIT {<br class=""> /// (to look up symbols based on their IR name use the lookup function<br class=""> /// instead).<br class=""> Expected<JITEvaluatedSymbol> lookupLinkerMangled(StringRef Name) {<br class="">- return lookupLinkerMangled(Main, Name);<br class="">+ return lookupLinkerMangled(*Main, Name);<br class=""> }<br class=""><br class=""> /// Look up a symbol in JITDylib JD based on its IR symbol name.<br class="">@@ -108,14 +124,28 @@ class LLJIT {<br class=""><br class=""> /// Look up a symbol in the main JITDylib based on its IR symbol name.<br class=""> Expected<JITEvaluatedSymbol> lookup(StringRef UnmangledName) {<br class="">- return lookup(Main, UnmangledName);<br class="">+ return lookup(*Main, UnmangledName);<br class=""> }<br class=""><br class="">- /// Runs all not-yet-run static constructors.<br class="">- Error runConstructors() { return CtorRunner.run(); }<br class="">+ /// Set the PlatformSupport instance.<br class="">+ void setPlatformSupport(std::unique_ptr<PlatformSupport> PS) {<br class="">+ this->PS = std::move(PS);<br class="">+ }<br class=""><br class="">- /// Runs all not-yet-run static destructors.<br class="">- Error runDestructors() { return DtorRunner.run(); }<br class="">+ /// Get the PlatformSupport instance.<br class="">+ PlatformSupport *getPlatformSupport() { return PS.get(); }<br class="">+<br class="">+ /// Run the initializers for the given JITDylib.<br class="">+ Error initialize(JITDylib &JD) {<br class="">+ assert(PS && "PlatformSupport must be set to run initializers.");<br class="">+ return PS->initialize(JD);<br class="">+ }<br class="">+<br class="">+ /// Run the deinitializers for the given JITDylib.<br class="">+ Error deinitialize(JITDylib &JD) {<br class="">+ assert(PS && "PlatformSupport must be set to run initializers.");<br class="">+ return PS->deinitialize(JD);<br class="">+ }<br class=""><br class=""> /// Returns a reference to the ObjLinkingLayer<br class=""> ObjectLayer &getObjLinkingLayer() { return *ObjLinkingLayer; }<br class="">@@ -126,6 +156,9 @@ class LLJIT {<br class=""> /// Returns a reference to the IR transform layer.<br class=""> IRTransformLayer &getIRTransformLayer() { return *TransformLayer; }<br class=""><br class="">+ /// Returns a reference to the IR compile layer.<br class="">+ IRCompileLayer &getIRCompileLayer() { return *CompileLayer; }<br class="">+<br class=""> protected:<br class=""> static std::unique_ptr<ObjectLayer><br class=""> createObjectLinkingLayer(LLJITBuilderState &S, ExecutionSession &ES);<br class="">@@ -143,7 +176,9 @@ class LLJIT {<br class=""> void recordCtorDtors(Module &M);<br class=""><br class=""> std::unique_ptr<ExecutionSession> ES;<br class="">- JITDylib &Main;<br class="">+ std::unique_ptr<PlatformSupport> PS;<br class="">+<br class="">+ JITDylib *Main = nullptr;<br class=""><br class=""> DataLayout DL;<br class=""> Triple TT;<br class="">@@ -153,8 +188,7 @@ class LLJIT {<br class=""> ObjectTransformLayer ObjTransformLayer;<br class=""> std::unique_ptr<IRCompileLayer> CompileLayer;<br class=""> std::unique_ptr<IRTransformLayer> TransformLayer;<br class="">-<br class="">- CtorDtorRunner CtorRunner, DtorRunner;<br class="">+ std::unique_ptr<IRTransformLayer> InitHelperTransformLayer;<br class=""> };<br class=""><br class=""> /// An extended version of LLJIT that supports lazy function-at-a-time<br class="">@@ -175,7 +209,7 @@ class LLLazyJIT : public LLJIT {<br class=""><br class=""> /// Add a module to be lazily compiled to the main JITDylib.<br class=""> Error addLazyIRModule(ThreadSafeModule M) {<br class="">- return addLazyIRModule(Main, std::move(M));<br class="">+ return addLazyIRModule(*Main, std::move(M));<br class=""> }<br class=""><br class=""> private:<br class="">@@ -196,10 +230,14 @@ class LLJITBuilderState {<br class=""> std::function<Expected<std::unique_ptr<IRCompileLayer::IRCompiler>>(<br class=""> JITTargetMachineBuilder JTMB)>;<br class=""><br class="">+ using PlatformSetupFunction = std::function<Error(LLJIT &J)>;<br class="">+<br class=""> std::unique_ptr<ExecutionSession> ES;<br class=""> Optional<JITTargetMachineBuilder> JTMB;<br class="">+ Optional<DataLayout> DL;<br class=""> ObjectLinkingLayerCreator CreateObjectLinkingLayer;<br class=""> CompileFunctionCreator CreateCompileFunction;<br class="">+ PlatformSetupFunction SetUpPlatform;<br class=""> unsigned NumCompileThreads = 0;<br class=""><br class=""> /// Called prior to JIT class construcion to fix up defaults.<br class="">@@ -224,6 +262,13 @@ class LLJITBuilderSetters {<br class=""> return impl().JTMB;<br class=""> }<br class=""><br class="">+ /// Set a DataLayout for this instance. If no data layout is specified then<br class="">+ /// the target's default data layout will be used.<br class="">+ SetterImpl &setDataLayout(Optional<DataLayout> DL) {<br class="">+ impl().DL = std::move(DL);<br class="">+ return impl();<br class="">+ }<br class="">+<br class=""> /// Set an ObjectLinkingLayer creation function.<br class=""> ///<br class=""> /// If this method is not called, a default creation function will be used<br class="">@@ -246,6 +291,16 @@ class LLJITBuilderSetters {<br class=""> return impl();<br class=""> }<br class=""><br class="">+ /// Set up an PlatformSetupFunction.<br class="">+ ///<br class="">+ /// If this method is not called then setUpGenericLLVMIRPlatform<br class="">+ /// will be used to configure the JIT's platform support.<br class="">+ SetterImpl &<br class="">+ setPlatformSetUp(LLJITBuilderState::PlatformSetupFunction SetUpPlatform) {<br class="">+ impl().SetUpPlatform = std::move(SetUpPlatform);<br class="">+ return impl();<br class="">+ }<br class="">+<br class=""> /// Set the number of compile threads to use.<br class=""> ///<br class=""> /// If set to zero, compilation will be performed on the execution thread when<br class="">@@ -334,6 +389,26 @@ class LLLazyJITBuilder<br class=""> public LLLazyJITBuilderSetters<LLLazyJIT, LLLazyJITBuilder,<br class=""> LLLazyJITBuilderState> {};<br class=""><br class="">+/// Configure the LLJIT instance to scrape modules for llvm.global_ctors and<br class="">+/// llvm.global_dtors variables and (if present) build initialization and<br class="">+/// deinitialization functions. Platform specific initialization configurations<br class="">+/// should be preferred where available.<br class="">+void setUpGenericLLVMIRPlatform(LLJIT &J);<br class="">+<br class="">+/// Configure the LLJIT instance to use MachOPlatform support.<br class="">+///<br class="">+/// Warning: MachOPlatform *requires* that LLJIT be configured to use<br class="">+/// ObjectLinkingLayer (default on platforms supported by JITLink). If<br class="">+/// MachOPlatform is used with RTDyldObjectLinkingLayer it will result in<br class="">+/// undefined behavior).<br class="">+///<br class="">+/// MachOPlatform installs an ObjectLinkingLayer plugin to scrape initializers<br class="">+/// from the __mod_inits section. It also provides interposes for the dlfcn<br class="">+/// functions (dlopen, dlclose, dlsym, dlerror) that work for JITDylibs as<br class="">+/// well as regular libraries (JITDylibs will be preferenced, so make sure<br class="">+/// your JITDylib names do not shadow any real library paths).<br class="">+Error setUpMachOPlatform(LLJIT &J);<br class="">+<br class=""> } // End namespace orc<br class=""> } // End namespace llvm<br class=""><br class=""><br class="">diff --git a/llvm/include/llvm/ExecutionEngine/Orc/Layer.h b/llvm/include/llvm/ExecutionEngine/Orc/Layer.h<br class="">index 95e32b2431a0..e843d0f56245 100644<br class="">--- a/llvm/include/llvm/ExecutionEngine/Orc/Layer.h<br class="">+++ b/llvm/include/llvm/ExecutionEngine/Orc/Layer.h<br class="">@@ -14,6 +14,7 @@<br class=""> #define LLVM_EXECUTIONENGINE_ORC_LAYER_H<br class=""><br class=""> #include "llvm/ExecutionEngine/Orc/Core.h"<br class="">+#include "llvm/ExecutionEngine/Orc/Mangling.h"<br class=""> #include "llvm/ExecutionEngine/Orc/ThreadSafeModule.h"<br class=""> #include "llvm/IR/Module.h"<br class=""> #include "llvm/Support/MemoryBuffer.h"<br class="">@@ -27,15 +28,12 @@ namespace orc {<br class=""> /// their linkage is changed to available-externally.<br class=""> class IRMaterializationUnit : public MaterializationUnit {<br class=""> public:<br class="">- struct ManglingOptions {<br class="">- bool EmulatedTLS = false;<br class="">- };<br class="">-<br class=""> using SymbolNameToDefinitionMap = std::map<SymbolStringPtr, GlobalValue *>;<br class=""><br class=""> /// Create an IRMaterializationLayer. Scans the module to build the<br class=""> /// SymbolFlags and SymbolToDefinition maps.<br class="">- IRMaterializationUnit(ExecutionSession &ES, const ManglingOptions &MO,<br class="">+ IRMaterializationUnit(ExecutionSession &ES,<br class="">+ const IRSymbolMapper::ManglingOptions &MO,<br class=""> ThreadSafeModule TSM, VModuleKey K);<br class=""><br class=""> /// Create an IRMaterializationLayer from a module, and pre-existing<br class="">@@ -44,12 +42,13 @@ class IRMaterializationUnit : public MaterializationUnit {<br class=""> /// This constructor is useful for delegating work from one<br class=""> /// IRMaterializationUnit to another.<br class=""> IRMaterializationUnit(ThreadSafeModule TSM, VModuleKey K,<br class="">- SymbolFlagsMap SymbolFlags,<br class="">+ SymbolFlagsMap SymbolFlags, SymbolStringPtr InitSymbol,<br class=""> SymbolNameToDefinitionMap SymbolToDefinition);<br class=""><br class=""> /// Return the ModuleIdentifier as the name for this MaterializationUnit.<br class=""> StringRef getName() const override;<br class=""><br class="">+ /// Return a reference to the contained ThreadSafeModule.<br class=""> const ThreadSafeModule &getModule() const { return TSM; }<br class=""><br class=""> protected:<br class="">@@ -57,14 +56,16 @@ class IRMaterializationUnit : public MaterializationUnit {<br class=""> SymbolNameToDefinitionMap SymbolToDefinition;<br class=""><br class=""> private:<br class="">+ static SymbolStringPtr getInitSymbol(ExecutionSession &ES,<br class="">+ const ThreadSafeModule &TSM);<br class="">+<br class=""> void discard(const JITDylib &JD, const SymbolStringPtr &Name) override;<br class=""> };<br class=""><br class=""> /// Interface for layers that accept LLVM IR.<br class=""> class IRLayer {<br class=""> public:<br class="">- IRLayer(ExecutionSession &ES,<br class="">- const IRMaterializationUnit::ManglingOptions *&MO)<br class="">+ IRLayer(ExecutionSession &ES, const IRSymbolMapper::ManglingOptions *&MO)<br class=""> : ES(ES), MO(MO) {}<br class=""><br class=""> virtual ~IRLayer();<br class="">@@ -73,7 +74,7 @@ class IRLayer {<br class=""> ExecutionSession &getExecutionSession() { return ES; }<br class=""><br class=""> /// Get the mangling options for this layer.<br class="">- const IRMaterializationUnit::ManglingOptions *&getManglingOptions() const {<br class="">+ const IRSymbolMapper::ManglingOptions *&getManglingOptions() const {<br class=""> return MO;<br class=""> }<br class=""><br class="">@@ -104,14 +105,15 @@ class IRLayer {<br class=""> private:<br class=""> bool CloneToNewContextOnEmit = false;<br class=""> ExecutionSession &ES;<br class="">- const IRMaterializationUnit::ManglingOptions *&MO;<br class="">+ const IRSymbolMapper::ManglingOptions *&MO;<br class=""> };<br class=""><br class=""> /// MaterializationUnit that materializes modules by calling the 'emit' method<br class=""> /// on the given IRLayer.<br class=""> class BasicIRLayerMaterializationUnit : public IRMaterializationUnit {<br class=""> public:<br class="">- BasicIRLayerMaterializationUnit(IRLayer &L, const ManglingOptions &MO,<br class="">+ BasicIRLayerMaterializationUnit(IRLayer &L,<br class="">+ const IRSymbolMapper::ManglingOptions &MO,<br class=""> ThreadSafeModule TSM, VModuleKey K);<br class=""><br class=""> private:<br class="">@@ -153,7 +155,8 @@ class BasicObjectLayerMaterializationUnit : public MaterializationUnit {<br class=""><br class=""> BasicObjectLayerMaterializationUnit(ObjectLayer &L, VModuleKey K,<br class=""> std::unique_ptr<MemoryBuffer> O,<br class="">- SymbolFlagsMap SymbolFlags);<br class="">+ SymbolFlagsMap SymbolFlags,<br class="">+ SymbolStringPtr InitSymbol);<br class=""><br class=""> /// Return the buffer's identifier as the name for this MaterializationUnit.<br class=""> StringRef getName() const override;<br class="">@@ -167,12 +170,6 @@ class BasicObjectLayerMaterializationUnit : public MaterializationUnit {<br class=""> std::unique_ptr<MemoryBuffer> O;<br class=""> };<br class=""><br class="">-/// Returns a SymbolFlagsMap for the object file represented by the given<br class="">-/// buffer, or an error if the buffer does not contain a valid object file.<br class="">-// FIXME: Maybe move to Core.h?<br class="">-Expected<SymbolFlagsMap> getObjectSymbolFlags(ExecutionSession &ES,<br class="">- MemoryBufferRef ObjBuffer);<br class="">-<br class=""> } // End namespace orc<br class=""> } // End namespace llvm<br class=""><br class=""><br class="">diff --git a/llvm/include/llvm/ExecutionEngine/Orc/MachOPlatform.h b/llvm/include/llvm/ExecutionEngine/Orc/MachOPlatform.h<br class="">new file mode 100644<br class="">index 000000000000..92c1a04921b8<br class="">--- /dev/null<br class="">+++ b/llvm/include/llvm/ExecutionEngine/Orc/MachOPlatform.h<br class="">@@ -0,0 +1,147 @@<br class="">+//===-- MachOPlatform.h - Utilities for executing MachO in Orc --*- C++ -*-===//<br class="">+//<br class="">+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.<br class="">+// See <a href="https://llvm.org/LICENSE.txt" class="">https://llvm.org/LICENSE.txt</a> for license information.<br class="">+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception<br class="">+//<br class="">+//===----------------------------------------------------------------------===//<br class="">+//<br class="">+// Utilities for executing JIT'd MachO in Orc.<br class="">+//<br class="">+//===----------------------------------------------------------------------===//<br class="">+<br class="">+#ifndef LLVM_EXECUTIONENGINE_ORC_MACHOPLATFORM_H<br class="">+#define LLVM_EXECUTIONENGINE_ORC_MACHOPLATFORM_H<br class="">+<br class="">+#include "llvm/ADT/StringRef.h"<br class="">+#include "llvm/ExecutionEngine/Orc/Core.h"<br class="">+#include "llvm/ExecutionEngine/Orc/ExecutionUtils.h"<br class="">+#include "llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h"<br class="">+<br class="">+#include <future><br class="">+#include <thread><br class="">+#include <vector><br class="">+<br class="">+namespace llvm {<br class="">+namespace orc {<br class="">+<br class="">+/// Enable registration of JIT'd ObjC classes and selectors.<br class="">+Error enableObjCRegistration(const char *PathToLibObjC);<br class="">+bool objCRegistrationEnabled();<br class="">+<br class="">+class MachOJITDylibInitializers {<br class="">+public:<br class="">+ struct SectionExtent {<br class="">+ SectionExtent() = default;<br class="">+ SectionExtent(JITTargetAddress Address, uint64_t NumPtrs)<br class="">+ : Address(Address), NumPtrs(NumPtrs) {}<br class="">+ JITTargetAddress Address = 0;<br class="">+ uint64_t NumPtrs = 0;<br class="">+ };<br class="">+<br class="">+ void setObjCImageInfoAddr(JITTargetAddress ObjCImageInfoAddr) {<br class="">+ this->ObjCImageInfoAddr = ObjCImageInfoAddr;<br class="">+ }<br class="">+<br class="">+ void addModInitsSection(SectionExtent ModInit) {<br class="">+ ModInitSections.push_back(std::move(ModInit));<br class="">+ }<br class="">+<br class="">+ void addObjCSelRefsSection(SectionExtent ObjCSelRefs) {<br class="">+ ObjCSelRefsSections.push_back(std::move(ObjCSelRefs));<br class="">+ }<br class="">+<br class="">+ void addObjCClassListSection(SectionExtent ObjCClassList) {<br class="">+ ObjCClassListSections.push_back(std::move(ObjCClassList));<br class="">+ }<br class="">+<br class="">+ void runModInits() const;<br class="">+ void registerObjCSelectors() const;<br class="">+ Error registerObjCClasses() const;<br class="">+<br class="">+ void dump() const;<br class="">+<br class="">+private:<br class="">+ using RawPointerSectionList = std::vector<SectionExtent>;<br class="">+<br class="">+ JITTargetAddress ObjCImageInfoAddr;<br class="">+ RawPointerSectionList ModInitSections;<br class="">+ RawPointerSectionList ObjCSelRefsSections;<br class="">+ RawPointerSectionList ObjCClassListSections;<br class="">+};<br class="">+<br class="">+class MachOJITDylibDeinitializers {};<br class="">+<br class="">+/// Mediates between MachO initialization and ExecutionSession state.<br class="">+class MachOPlatform : public Platform {<br class="">+public:<br class="">+ using InitializerSequence =<br class="">+ std::vector<std::pair<JITDylib *, MachOJITDylibInitializers>>;<br class="">+<br class="">+ using DeinitializerSequence =<br class="">+ std::vector<std::pair<JITDylib *, MachOJITDylibDeinitializers>>;<br class="">+<br class="">+ MachOPlatform(ExecutionSession &ES, ObjectLinkingLayer &ObjLinkingLayer,<br class="">+ std::unique_ptr<MemoryBuffer> StandardSymbolsObject);<br class="">+<br class="">+ ExecutionSession &getExecutionSession() const { return ES; }<br class="">+<br class="">+ Error setupJITDylib(JITDylib &JD) override;<br class="">+ Error notifyAdding(JITDylib &JD, const MaterializationUnit &MU) override;<br class="">+ Error notifyRemoving(JITDylib &JD, VModuleKey K) override;<br class="">+<br class="">+ Expected<InitializerSequence> getInitializerSequence(JITDylib &JD);<br class="">+<br class="">+ Expected<DeinitializerSequence> getDeinitializerSequence(JITDylib &JD);<br class="">+<br class="">+private:<br class="">+ // This ObjectLinkingLayer plugin scans JITLink graphs for __mod_init_func,<br class="">+ // __objc_classlist and __sel_ref sections and records their extents so that<br class="">+ // they can be run in the target process.<br class="">+ class InitScraperPlugin : public ObjectLinkingLayer::Plugin {<br class="">+ public:<br class="">+ InitScraperPlugin(MachOPlatform &MP) : MP(MP) {}<br class="">+<br class="">+ void modifyPassConfig(MaterializationResponsibility &MR, const Triple &TT,<br class="">+ jitlink::PassConfiguration &Config) override;<br class="">+<br class="">+ LocalDependenciesMap getSyntheticSymbolLocalDependencies(<br class="">+ MaterializationResponsibility &MR) override;<br class="">+<br class="">+ private:<br class="">+ using InitSymbolDepMap =<br class="">+ DenseMap<MaterializationResponsibility *, JITLinkSymbolVector>;<br class="">+<br class="">+ void preserveInitSectionIfPresent(JITLinkSymbolVector &Syms,<br class="">+ jitlink::LinkGraph &G,<br class="">+ StringRef SectionName);<br class="">+<br class="">+ Error processObjCImageInfo(jitlink::LinkGraph &G,<br class="">+ MaterializationResponsibility &MR);<br class="">+<br class="">+ std::mutex InitScraperMutex;<br class="">+ MachOPlatform &MP;<br class="">+ DenseMap<JITDylib *, std::pair<uint32_t, uint32_t>> ObjCImageInfos;<br class="">+ InitSymbolDepMap InitSymbolDeps;<br class="">+ };<br class="">+<br class="">+ static std::vector<JITDylib *> getDFSLinkOrder(JITDylib &JD);<br class="">+<br class="">+ void registerInitInfo(JITDylib &JD, JITTargetAddress ObjCImageInfoAddr,<br class="">+ MachOJITDylibInitializers::SectionExtent ModInits,<br class="">+ MachOJITDylibInitializers::SectionExtent ObjCSelRefs,<br class="">+ MachOJITDylibInitializers::SectionExtent ObjCClassList);<br class="">+<br class="">+ std::mutex PlatformMutex;<br class="">+ ExecutionSession &ES;<br class="">+ ObjectLinkingLayer &ObjLinkingLayer;<br class="">+ std::unique_ptr<MemoryBuffer> StandardSymbolsObject;<br class="">+<br class="">+ DenseMap<JITDylib *, SymbolLookupSet> RegisteredInitSymbols;<br class="">+ DenseMap<JITDylib *, MachOJITDylibInitializers> InitSeqs;<br class="">+};<br class="">+<br class="">+} // end namespace orc<br class="">+} // end namespace llvm<br class="">+<br class="">+#endif // LLVM_EXECUTIONENGINE_ORC_MACHOPLATFORM_H<br class=""><br class="">diff --git a/llvm/include/llvm/ExecutionEngine/Orc/Mangling.h b/llvm/include/llvm/ExecutionEngine/Orc/Mangling.h<br class="">new file mode 100644<br class="">index 000000000000..e0f770a601fb<br class="">--- /dev/null<br class="">+++ b/llvm/include/llvm/ExecutionEngine/Orc/Mangling.h<br class="">@@ -0,0 +1,66 @@<br class="">+//===------ Mangling.h -- Name Mangling Utilities for ORC -------*- C++ -*-===//<br class="">+//<br class="">+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.<br class="">+// See <a href="https://llvm.org/LICENSE.txt" class="">https://llvm.org/LICENSE.txt</a> for license information.<br class="">+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception<br class="">+//<br class="">+//===----------------------------------------------------------------------===//<br class="">+//<br class="">+// Name mangling utilities for ORC.<br class="">+//<br class="">+//===----------------------------------------------------------------------===//<br class="">+<br class="">+#ifndef LLVM_EXECUTIONENGINE_ORC_MANGLING_H<br class="">+#define LLVM_EXECUTIONENGINE_ORC_MANGLING_H<br class="">+<br class="">+#include "llvm/ExecutionEngine/Orc/Core.h"<br class="">+#include "llvm/ExecutionEngine/Orc/ThreadSafeModule.h"<br class="">+#include "llvm/IR/Module.h"<br class="">+#include "llvm/Support/MemoryBuffer.h"<br class="">+<br class="">+namespace llvm {<br class="">+namespace orc {<br class="">+<br class="">+/// Mangles symbol names then uniques them in the context of an<br class="">+/// ExecutionSession.<br class="">+class MangleAndInterner {<br class="">+public:<br class="">+ MangleAndInterner(ExecutionSession &ES, const DataLayout &DL);<br class="">+ SymbolStringPtr operator()(StringRef Name);<br class="">+<br class="">+private:<br class="">+ ExecutionSession &ES;<br class="">+ const DataLayout &DL;<br class="">+};<br class="">+<br class="">+/// Maps IR global values to their linker symbol names / flags.<br class="">+///<br class="">+/// This utility can be used when adding new IR globals in the JIT.<br class="">+class IRSymbolMapper {<br class="">+public:<br class="">+ struct ManglingOptions {<br class="">+ bool EmulatedTLS = false;<br class="">+ };<br class="">+<br class="">+ using SymbolNameToDefinitionMap = std::map<SymbolStringPtr, GlobalValue *>;<br class="">+<br class="">+ /// Add mangled symbols for the given GlobalValues to SymbolFlags.<br class="">+ /// If a SymbolToDefinitionMap pointer is supplied then it will be populated<br class="">+ /// with Name-to-GlobalValue* mappings. Note that this mapping is not<br class="">+ /// necessarily one-to-one: thread-local GlobalValues, for example, may<br class="">+ /// produce more than one symbol, in which case the map will contain duplicate<br class="">+ /// values.<br class="">+ static void add(ExecutionSession &ES, const ManglingOptions &MO,<br class="">+ ArrayRef<GlobalValue *> GVs, SymbolFlagsMap &SymbolFlags,<br class="">+ SymbolNameToDefinitionMap *SymbolToDefinition = nullptr);<br class="">+};<br class="">+<br class="">+/// Returns a SymbolFlagsMap for the object file represented by the given<br class="">+/// buffer, or an error if the buffer does not contain a valid object file.<br class="">+Expected<std::pair<SymbolFlagsMap, SymbolStringPtr>><br class="">+getObjectSymbolInfo(ExecutionSession &ES, MemoryBufferRef ObjBuffer);<br class="">+<br class="">+} // End namespace orc<br class="">+} // End namespace llvm<br class="">+<br class="">+#endif // LLVM_EXECUTIONENGINE_ORC_MANGLING_H<br class=""><br class="">diff --git a/llvm/include/llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h b/llvm/include/llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h<br class="">index 50d25f18891e..1b4c03bd2cfd 100644<br class="">--- a/llvm/include/llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h<br class="">+++ b/llvm/include/llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h<br class="">@@ -35,6 +35,7 @@ namespace llvm {<br class=""><br class=""> namespace jitlink {<br class=""> class EHFrameRegistrar;<br class="">+class Symbol;<br class=""> } // namespace jitlink<br class=""><br class=""> namespace object {<br class="">@@ -59,10 +60,14 @@ class ObjectLinkingLayer : public ObjectLayer {<br class=""> /// configured.<br class=""> class Plugin {<br class=""> public:<br class="">+ using JITLinkSymbolVector = std::vector<const jitlink::Symbol *>;<br class="">+ using LocalDependenciesMap = DenseMap<SymbolStringPtr, JITLinkSymbolVector>;<br class="">+<br class=""> virtual ~Plugin();<br class=""> virtual void modifyPassConfig(MaterializationResponsibility &MR,<br class=""> const Triple &TT,<br class=""> jitlink::PassConfiguration &Config) {}<br class="">+<br class=""> virtual void notifyLoaded(MaterializationResponsibility &MR) {}<br class=""> virtual Error notifyEmitted(MaterializationResponsibility &MR) {<br class=""> return Error::success();<br class="">@@ -71,6 +76,15 @@ class ObjectLinkingLayer : public ObjectLayer {<br class=""> return Error::success();<br class=""> }<br class=""> virtual Error notifyRemovingAllModules() { return Error::success(); }<br class="">+<br class="">+ /// Return any dependencies that synthetic symbols (e.g. init symbols)<br class="">+ /// have on locally scoped jitlink::Symbols. This is used by the<br class="">+ /// ObjectLinkingLayer to update the dependencies for the synthetic<br class="">+ /// symbols.<br class="">+ virtual LocalDependenciesMap<br class="">+ getSyntheticSymbolLocalDependencies(MaterializationResponsibility &MR) {<br class="">+ return LocalDependenciesMap();<br class="">+ }<br class=""> };<br class=""><br class=""> using ReturnObjectBufferFunction =<br class=""><br class="">diff --git a/llvm/include/llvm/ExecutionEngine/Orc/SymbolStringPool.h b/llvm/include/llvm/ExecutionEngine/Orc/SymbolStringPool.h<br class="">index c354f6c3559c..c8462dc3d2c8 100644<br class="">--- a/llvm/include/llvm/ExecutionEngine/Orc/SymbolStringPool.h<br class="">+++ b/llvm/include/llvm/ExecutionEngine/Orc/SymbolStringPool.h<br class="">@@ -53,6 +53,7 @@ class SymbolStringPtr {<br class=""><br class=""> public:<br class=""> SymbolStringPtr() = default;<br class="">+ SymbolStringPtr(nullptr_t) {}<br class=""> SymbolStringPtr(const SymbolStringPtr &Other)<br class=""> : S(Other.S) {<br class=""> if (isRealPoolEntry(S))<br class="">@@ -85,6 +86,8 @@ class SymbolStringPtr {<br class=""> --S->getValue();<br class=""> }<br class=""><br class="">+ explicit operator bool() const { return S; }<br class="">+<br class=""> StringRef operator*() const { return S->first(); }<br class=""><br class=""> friend bool operator==(const SymbolStringPtr &LHS,<br class=""><br class="">diff --git a/llvm/lib/ExecutionEngine/Orc/CMakeLists.txt b/llvm/lib/ExecutionEngine/Orc/CMakeLists.txt<br class="">index e3a7a67c8a0e..7eb2742d7b53 100644<br class="">--- a/llvm/lib/ExecutionEngine/Orc/CMakeLists.txt<br class="">+++ b/llvm/lib/ExecutionEngine/Orc/CMakeLists.txt<br class="">@@ -12,6 +12,8 @@ add_llvm_component_library(LLVMOrcJIT<br class=""> Legacy.cpp<br class=""> Layer.cpp<br class=""> LLJIT.cpp<br class="">+ MachOPlatform.cpp<br class="">+ Mangling.cpp<br class=""> NullResolver.cpp<br class=""> ObjectLinkingLayer.cpp<br class=""> ObjectTransformLayer.cpp<br class=""><br class="">diff --git a/llvm/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp b/llvm/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp<br class="">index 29d18b6e4d7b..f22ae01a2080 100644<br class="">--- a/llvm/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp<br class="">+++ b/llvm/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp<br class="">@@ -7,6 +7,7 @@<br class=""> //===----------------------------------------------------------------------===//<br class=""><br class=""> #include "llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h"<br class="">+#include "llvm/ExecutionEngine/Orc/ExecutionUtils.h"<br class=""> #include "llvm/IR/Mangler.h"<br class=""> #include "llvm/IR/Module.h"<br class=""><br class="">@@ -68,18 +69,18 @@ namespace orc {<br class=""> class PartitioningIRMaterializationUnit : public IRMaterializationUnit {<br class=""> public:<br class=""> PartitioningIRMaterializationUnit(ExecutionSession &ES,<br class="">- const ManglingOptions &MO,<br class="">+ const IRSymbolMapper::ManglingOptions &MO,<br class=""> ThreadSafeModule TSM, VModuleKey K,<br class=""> CompileOnDemandLayer &Parent)<br class=""> : IRMaterializationUnit(ES, MO, std::move(TSM), std::move(K)),<br class=""> Parent(Parent) {}<br class=""><br class=""> PartitioningIRMaterializationUnit(<br class="">- ThreadSafeModule TSM, SymbolFlagsMap SymbolFlags,<br class="">- SymbolNameToDefinitionMap SymbolToDefinition,<br class="">+ ThreadSafeModule TSM, VModuleKey K, SymbolFlagsMap SymbolFlags,<br class="">+ SymbolStringPtr InitSymbol, SymbolNameToDefinitionMap SymbolToDefinition,<br class=""> CompileOnDemandLayer &Parent)<br class=""> : IRMaterializationUnit(std::move(TSM), std::move(K),<br class="">- std::move(SymbolFlags),<br class="">+ std::move(SymbolFlags), std::move(InitSymbol),<br class=""> std::move(SymbolToDefinition)),<br class=""> Parent(Parent) {}<br class=""><br class="">@@ -172,21 +173,23 @@ CompileOnDemandLayer::getPerDylibResources(JITDylib &TargetD) {<br class=""> auto I = DylibResources.find(&TargetD);<br class=""> if (I == DylibResources.end()) {<br class=""> auto &ImplD =<br class="">- getExecutionSession().createJITDylib(TargetD.getName() + ".impl");<br class="">+ getExecutionSession().createBareJITDylib(TargetD.getName() + ".impl");<br class="">+ JITDylibSearchOrder NewSearchOrder;<br class=""> TargetD.withSearchOrderDo(<br class=""> [&](const JITDylibSearchOrder &TargetSearchOrder) {<br class="">- auto NewSearchOrder = TargetSearchOrder;<br class="">- assert(<br class="">- !NewSearchOrder.empty() &&<br class="">- NewSearchOrder.front().first == &TargetD &&<br class="">- NewSearchOrder.front().second ==<br class="">- JITDylibLookupFlags::MatchAllSymbols &&<br class="">- "TargetD must be at the front of its own search order and match "<br class="">- "non-exported symbol");<br class="">- NewSearchOrder.insert(std::next(NewSearchOrder.begin()),<br class="">- {&ImplD, JITDylibLookupFlags::MatchAllSymbols});<br class="">- ImplD.setSearchOrder(std::move(NewSearchOrder), false);<br class="">+ NewSearchOrder = TargetSearchOrder;<br class=""> });<br class="">+<br class="">+ assert(<br class="">+ !NewSearchOrder.empty() && NewSearchOrder.front().first == &TargetD &&<br class="">+ NewSearchOrder.front().second == JITDylibLookupFlags::MatchAllSymbols &&<br class="">+ "TargetD must be at the front of its own search order and match "<br class="">+ "non-exported symbol");<br class="">+ NewSearchOrder.insert(std::next(NewSearchOrder.begin()),<br class="">+ {&ImplD, JITDylibLookupFlags::MatchAllSymbols});<br class="">+ ImplD.setSearchOrder(NewSearchOrder, false);<br class="">+ TargetD.setSearchOrder(std::move(NewSearchOrder), false);<br class="">+<br class=""> PerDylibResources PDR(ImplD, BuildIndirectStubsManager());<br class=""> I = DylibResources.insert(std::make_pair(&TargetD, std::move(PDR))).first;<br class=""> }<br class="">@@ -251,8 +254,15 @@ void CompileOnDemandLayer::emitPartition(<br class=""> auto &ES = getExecutionSession();<br class=""> GlobalValueSet RequestedGVs;<br class=""> for (auto &Name : R.getRequestedSymbols()) {<br class="">- assert(Defs.count(Name) && "No definition for symbol");<br class="">- RequestedGVs.insert(Defs[Name]);<br class="">+ if (Name == R.getInitializerSymbol())<br class="">+ TSM.withModuleDo([&](Module &M) {<br class="">+ for (auto &GV : getStaticInitGVs(M))<br class="">+ RequestedGVs.insert(&GV);<br class="">+ });<br class="">+ else {<br class="">+ assert(Defs.count(Name) && "No definition for symbol");<br class="">+ RequestedGVs.insert(Defs[Name]);<br class="">+ }<br class=""> }<br class=""><br class=""> /// Perform partitioning with the context lock held, since the partition<br class="">@@ -272,7 +282,8 @@ void CompileOnDemandLayer::emitPartition(<br class=""> // If the partition is empty, return the whole module to the symbol table.<br class=""> if (GVsToExtract->empty()) {<br class=""> R.replace(std::make_unique<PartitioningIRMaterializationUnit>(<br class="">- std::move(TSM), R.getSymbols(), std::move(Defs), *this));<br class="">+ std::move(TSM), R.getVModuleKey(), R.getSymbols(),<br class="">+ R.getInitializerSymbol(), std::move(Defs), *this));<br class=""> return;<br class=""> }<br class=""><br class=""><br class="">diff --git a/llvm/lib/ExecutionEngine/Orc/CompileUtils.cpp b/llvm/lib/ExecutionEngine/Orc/CompileUtils.cpp<br class="">index 160e5ba50311..f8efed15edea 100644<br class="">--- a/llvm/lib/ExecutionEngine/Orc/CompileUtils.cpp<br class="">+++ b/llvm/lib/ExecutionEngine/Orc/CompileUtils.cpp<br class="">@@ -24,9 +24,9 @@<br class=""> namespace llvm {<br class=""> namespace orc {<br class=""><br class="">-IRMaterializationUnit::ManglingOptions<br class="">+IRSymbolMapper::ManglingOptions<br class=""> irManglingOptionsFromTargetOptions(const TargetOptions &Opts) {<br class="">- IRMaterializationUnit::ManglingOptions MO;<br class="">+ IRSymbolMapper::ManglingOptions MO;<br class=""><br class=""> MO.EmulatedTLS = Opts.EmulatedTLS;<br class=""><br class=""><br class="">diff --git a/llvm/lib/ExecutionEngine/Orc/Core.cpp b/llvm/lib/ExecutionEngine/Orc/Core.cpp<br class="">index a7af34a21926..6a0df10f7c3f 100644<br class="">--- a/llvm/lib/ExecutionEngine/Orc/Core.cpp<br class="">+++ b/llvm/lib/ExecutionEngine/Orc/Core.cpp<br class="">@@ -11,7 +11,6 @@<br class=""> #include "llvm/ADT/STLExtras.h"<br class=""> #include "llvm/Config/llvm-config.h"<br class=""> #include "llvm/ExecutionEngine/Orc/OrcError.h"<br class="">-#include "llvm/IR/Mangler.h"<br class=""> #include "llvm/Support/CommandLine.h"<br class=""> #include "llvm/Support/Debug.h"<br class=""> #include "llvm/Support/Format.h"<br class="">@@ -421,12 +420,6 @@ void AsynchronousSymbolQuery::detach() {<br class=""> QueryRegistrations.clear();<br class=""> }<br class=""><br class="">-MaterializationResponsibility::MaterializationResponsibility(<br class="">- JITDylib &JD, SymbolFlagsMap SymbolFlags, VModuleKey K)<br class="">- : JD(JD), SymbolFlags(std::move(SymbolFlags)), K(std::move(K)) {<br class="">- assert(!this->SymbolFlags.empty() && "Materializing nothing?");<br class="">-}<br class="">-<br class=""> MaterializationResponsibility::~MaterializationResponsibility() {<br class=""> assert(SymbolFlags.empty() &&<br class=""> "All symbols should have been explicitly materialized or failed");<br class="">@@ -501,9 +494,13 @@ void MaterializationResponsibility::failMaterialization() {<br class=""><br class=""> void MaterializationResponsibility::replace(<br class=""> std::unique_ptr<MaterializationUnit> MU) {<br class="">+<br class=""> for (auto &KV : MU->getSymbols())<br class=""> SymbolFlags.erase(KV.first);<br class=""><br class="">+ if (MU->getInitializerSymbol() == InitSymbol)<br class="">+ InitSymbol = nullptr;<br class="">+<br class=""> LLVM_DEBUG(JD.getExecutionSession().runSessionLocked([&]() {<br class=""> dbgs() << "In " << JD.getName() << " replacing symbols with " << *MU<br class=""> << "\n";<br class="">@@ -519,6 +516,7 @@ MaterializationResponsibility::delegate(const SymbolNameSet &Symbols,<br class=""> if (NewKey == VModuleKey())<br class=""> NewKey = K;<br class=""><br class="">+ SymbolStringPtr DelegatedInitSymbol;<br class=""> SymbolFlagsMap DelegatedFlags;<br class=""><br class=""> for (auto &Name : Symbols) {<br class="">@@ -528,10 +526,14 @@ MaterializationResponsibility::delegate(const SymbolNameSet &Symbols,<br class=""> "instance");<br class=""><br class=""> DelegatedFlags[Name] = std::move(I->second);<br class="">+ if (Name == InitSymbol)<br class="">+ std::swap(InitSymbol, DelegatedInitSymbol);<br class="">+<br class=""> SymbolFlags.erase(I);<br class=""> }<br class=""><br class=""> return MaterializationResponsibility(JD, std::move(DelegatedFlags),<br class="">+ std::move(DelegatedInitSymbol),<br class=""> std::move(NewKey));<br class=""> }<br class=""><br class="">@@ -550,7 +552,7 @@ void MaterializationResponsibility::addDependenciesForAll(<br class=""><br class=""> AbsoluteSymbolsMaterializationUnit::AbsoluteSymbolsMaterializationUnit(<br class=""> SymbolMap Symbols, VModuleKey K)<br class="">- : MaterializationUnit(extractFlags(Symbols), std::move(K)),<br class="">+ : MaterializationUnit(extractFlags(Symbols), nullptr, std::move(K)),<br class=""> Symbols(std::move(Symbols)) {}<br class=""><br class=""> StringRef AbsoluteSymbolsMaterializationUnit::getName() const {<br class="">@@ -581,7 +583,7 @@ AbsoluteSymbolsMaterializationUnit::extractFlags(const SymbolMap &Symbols) {<br class=""> ReExportsMaterializationUnit::ReExportsMaterializationUnit(<br class=""> JITDylib *SourceJD, JITDylibLookupFlags SourceJDLookupFlags,<br class=""> SymbolAliasMap Aliases, VModuleKey K)<br class="">- : MaterializationUnit(extractFlags(Aliases), std::move(K)),<br class="">+ : MaterializationUnit(extractFlags(Aliases), nullptr, std::move(K)),<br class=""> SourceJD(SourceJD), SourceJDLookupFlags(SourceJDLookupFlags),<br class=""> Aliases(std::move(Aliases)) {}<br class=""><br class="">@@ -972,7 +974,7 @@ void JITDylib::addDependencies(const SymbolStringPtr &Name,<br class=""> // Assert that this symbol exists and has not reached the ready state<br class=""> // already.<br class=""> assert(OtherSymI != OtherJITDylib.Symbols.end() &&<br class="">- (OtherSymI->second.getState() != SymbolState::Ready &&<br class="">+ (OtherSymI->second.getState() < SymbolState::Ready &&<br class=""> "Dependency on emitted/ready symbol"));<br class=""> #endif<br class=""><br class="">@@ -1101,6 +1103,7 @@ Error JITDylib::resolve(const SymbolMap &Resolved) {<br class=""> Error JITDylib::emit(const SymbolFlagsMap &Emitted) {<br class=""> AsynchronousSymbolQuerySet CompletedQueries;<br class=""> SymbolNameSet SymbolsInErrorState;<br class="">+ DenseMap<JITDylib *, SymbolNameVector> ReadySymbols;<br class=""><br class=""> ES.runSessionLocked([&, this]() {<br class=""> std::vector<SymbolTable::iterator> Worklist;<br class="">@@ -1145,6 +1148,7 @@ Error JITDylib::emit(const SymbolFlagsMap &Emitted) {<br class=""> // dependencies) then notify any pending queries.<br class=""> for (auto &KV : MI.Dependants) {<br class=""> auto &DependantJD = *KV.first;<br class="">+ auto &DependantJDReadySymbols = ReadySymbols[&DependantJD];<br class=""> for (auto &DependantName : KV.second) {<br class=""> auto DependantMII =<br class=""> DependantJD.MaterializingInfos.find(DependantName);<br class="">@@ -1184,6 +1188,7 @@ Error JITDylib::emit(const SymbolFlagsMap &Emitted) {<br class=""> // Since this dependant is now ready, we erase its MaterializingInfo<br class=""> // and update its materializing state.<br class=""> DependantSymEntry.setState(SymbolState::Ready);<br class="">+ DependantJDReadySymbols.push_back(DependantName);<br class=""><br class=""> for (auto &Q : DependantMI.takeQueriesMeeting(SymbolState::Ready)) {<br class=""> Q->notifySymbolMetRequiredState(<br class="">@@ -1192,22 +1197,21 @@ Error JITDylib::emit(const SymbolFlagsMap &Emitted) {<br class=""> CompletedQueries.insert(Q);<br class=""> Q->removeQueryDependence(DependantJD, DependantName);<br class=""> }<br class="">-<br class="">- DependantJD.MaterializingInfos.erase(DependantMII);<br class=""> }<br class=""> }<br class=""> }<br class=""><br class="">+ auto &ThisJDReadySymbols = ReadySymbols[this];<br class=""> MI.Dependants.clear();<br class=""> if (MI.UnemittedDependencies.empty()) {<br class=""> SymI->second.setState(SymbolState::Ready);<br class="">+ ThisJDReadySymbols.push_back(Name);<br class=""> for (auto &Q : MI.takeQueriesMeeting(SymbolState::Ready)) {<br class=""> Q->notifySymbolMetRequiredState(Name, SymI->second.getSymbol());<br class=""> if (Q->isComplete())<br class=""> CompletedQueries.insert(Q);<br class=""> Q->removeQueryDependence(*this, Name);<br class=""> }<br class="">- MaterializingInfos.erase(MII);<br class=""> }<br class=""> }<br class=""> });<br class="">@@ -1882,6 +1886,57 @@ void JITDylib::transferEmittedNodeDependencies(<br class=""> }<br class=""> }<br class=""><br class="">+Platform::~Platform() {}<br class="">+<br class="">+Expected<DenseMap<JITDylib *, SymbolMap>> Platform::lookupInitSymbols(<br class="">+ ExecutionSession &ES,<br class="">+ const DenseMap<JITDylib *, SymbolLookupSet> &InitSyms) {<br class="">+<br class="">+ DenseMap<JITDylib *, SymbolMap> CompoundResult;<br class="">+ Error CompoundErr = Error::success();<br class="">+ std::mutex LookupMutex;<br class="">+ std::condition_variable CV;<br class="">+ uint64_t Count = InitSyms.size();<br class="">+<br class="">+ LLVM_DEBUG({<br class="">+ dbgs() << "Issuing init-symbol lookup:\n";<br class="">+ for (auto &KV : InitSyms)<br class="">+ dbgs() << " " << KV.first->getName() << ": " << KV.second << "\n";<br class="">+ });<br class="">+<br class="">+ for (auto &KV : InitSyms) {<br class="">+ auto *JD = KV.first;<br class="">+ auto Names = std::move(KV.second);<br class="">+ ES.lookup(<br class="">+ LookupKind::Static,<br class="">+ JITDylibSearchOrder({{JD, JITDylibLookupFlags::MatchAllSymbols}}),<br class="">+ std::move(Names), SymbolState::Ready,<br class="">+ [&, JD](Expected<SymbolMap> Result) {<br class="">+ {<br class="">+ std::lock_guard<std::mutex> Lock(LookupMutex);<br class="">+ --Count;<br class="">+ if (Result) {<br class="">+ assert(!CompoundResult.count(JD) &&<br class="">+ "Duplicate JITDylib in lookup?");<br class="">+ CompoundResult[JD] = std::move(*Result);<br class="">+ } else<br class="">+ CompoundErr =<br class="">+ joinErrors(std::move(CompoundErr), Result.takeError());<br class="">+ }<br class="">+ CV.notify_one();<br class="">+ },<br class="">+ NoDependenciesToRegister);<br class="">+ }<br class="">+<br class="">+ std::unique_lock<std::mutex> Lock(LookupMutex);<br class="">+ CV.wait(Lock, [&] { return Count == 0 || CompoundErr; });<br class="">+<br class="">+ if (CompoundErr)<br class="">+ return std::move(CompoundErr);<br class="">+<br class="">+ return std::move(CompoundResult);<br class="">+}<br class="">+<br class=""> ExecutionSession::ExecutionSession(std::shared_ptr<SymbolStringPool> SSP)<br class=""> : SSP(SSP ? std::move(SSP) : std::make_shared<SymbolStringPool>()) {<br class=""> }<br class="">@@ -1895,7 +1950,7 @@ JITDylib *ExecutionSession::getJITDylibByName(StringRef Name) {<br class=""> });<br class=""> }<br class=""><br class="">-JITDylib &ExecutionSession::createJITDylib(std::string Name) {<br class="">+JITDylib &ExecutionSession::createBareJITDylib(std::string Name) {<br class=""> assert(!getJITDylibByName(Name) && "JITDylib with that name already exists");<br class=""> return runSessionLocked([&, this]() -> JITDylib & {<br class=""> JDs.push_back(<br class="">@@ -1904,6 +1959,14 @@ JITDylib &ExecutionSession::createJITDylib(std::string Name) {<br class=""> });<br class=""> }<br class=""><br class="">+Expected<JITDylib &> ExecutionSession::createJITDylib(std::string Name) {<br class="">+ auto &JD = createBareJITDylib(Name);<br class="">+ if (P)<br class="">+ if (auto Err = P->setupJITDylib(JD))<br class="">+ return std::move(Err);<br class="">+ return JD;<br class="">+}<br class="">+<br class=""> void ExecutionSession::legacyFailQuery(AsynchronousSymbolQuery &Q, Error Err) {<br class=""> assert(!!Err && "Error should be in failure state");<br class=""><br class="">@@ -2144,11 +2207,11 @@ ExecutionSession::lookup(const JITDylibSearchOrder &SearchOrder,<br class=""><br class=""> Expected<JITEvaluatedSymbol><br class=""> ExecutionSession::lookup(const JITDylibSearchOrder &SearchOrder,<br class="">- SymbolStringPtr Name) {<br class="">+ SymbolStringPtr Name, SymbolState RequiredState) {<br class=""> SymbolLookupSet Names({Name});<br class=""><br class=""> if (auto ResultMap = lookup(SearchOrder, std::move(Names), LookupKind::Static,<br class="">- SymbolState::Ready, NoDependenciesToRegister)) {<br class="">+ RequiredState, NoDependenciesToRegister)) {<br class=""> assert(ResultMap->size() == 1 && "Unexpected number of results");<br class=""> assert(ResultMap->count(Name) && "Missing result for symbol");<br class=""> return std::move(ResultMap->begin()->second);<br class="">@@ -2157,14 +2220,15 @@ ExecutionSession::lookup(const JITDylibSearchOrder &SearchOrder,<br class=""> }<br class=""><br class=""> Expected<JITEvaluatedSymbol><br class="">-ExecutionSession::lookup(ArrayRef<JITDylib *> SearchOrder,<br class="">- SymbolStringPtr Name) {<br class="">- return lookup(makeJITDylibSearchOrder(SearchOrder), Name);<br class="">+ExecutionSession::lookup(ArrayRef<JITDylib *> SearchOrder, SymbolStringPtr Name,<br class="">+ SymbolState RequiredState) {<br class="">+ return lookup(makeJITDylibSearchOrder(SearchOrder), Name, RequiredState);<br class=""> }<br class=""><br class=""> Expected<JITEvaluatedSymbol><br class="">-ExecutionSession::lookup(ArrayRef<JITDylib *> SearchOrder, StringRef Name) {<br class="">- return lookup(SearchOrder, intern(Name));<br class="">+ExecutionSession::lookup(ArrayRef<JITDylib *> SearchOrder, StringRef Name,<br class="">+ SymbolState RequiredState) {<br class="">+ return lookup(SearchOrder, intern(Name), RequiredState);<br class=""> }<br class=""><br class=""> void ExecutionSession::dump(raw_ostream &OS) {<br class="">@@ -2195,17 +2259,5 @@ void ExecutionSession::runOutstandingMUs() {<br class=""> }<br class=""> }<br class=""><br class="">-MangleAndInterner::MangleAndInterner(ExecutionSession &ES, const DataLayout &DL)<br class="">- : ES(ES), DL(DL) {}<br class="">-<br class="">-SymbolStringPtr MangleAndInterner::operator()(StringRef Name) {<br class="">- std::string MangledName;<br class="">- {<br class="">- raw_string_ostream MangledNameStream(MangledName);<br class="">- Mangler::getNameWithPrefix(MangledNameStream, Name, DL);<br class="">- }<br class="">- return ES.intern(MangledName);<br class="">-}<br class="">-<br class=""> } // End namespace orc.<br class=""> } // End namespace llvm.<br class=""><br class="">diff --git a/llvm/lib/ExecutionEngine/Orc/ExecutionUtils.cpp b/llvm/lib/ExecutionEngine/Orc/ExecutionUtils.cpp<br class="">index 3d97fe9eeab1..a98445a2295e 100644<br class="">--- a/llvm/lib/ExecutionEngine/Orc/ExecutionUtils.cpp<br class="">+++ b/llvm/lib/ExecutionEngine/Orc/ExecutionUtils.cpp<br class="">@@ -113,6 +113,26 @@ iterator_range<CtorDtorIterator> getDestructors(const Module &M) {<br class=""> CtorDtorIterator(DtorsList, true));<br class=""> }<br class=""><br class="">+bool StaticInitGVIterator::isStaticInitGlobal(GlobalValue &GV) {<br class="">+ if (GV.isDeclaration())<br class="">+ return false;<br class="">+<br class="">+ if (GV.hasName() && (GV.getName() == "llvm.global_ctors" ||<br class="">+ GV.getName() == "llvm.global_dtors"))<br class="">+ return true;<br class="">+<br class="">+ if (ObjFmt == Triple::MachO) {<br class="">+ // FIXME: These section checks are too strict: We should match first and<br class="">+ // second word split by comma.<br class="">+ if (GV.hasSection() &&<br class="">+ (GV.getSection().startswith("__DATA,__objc_classlist") ||<br class="">+ GV.getSection().startswith("__DATA,__objc_selrefs")))<br class="">+ return true;<br class="">+ }<br class="">+<br class="">+ return false;<br class="">+}<br class="">+<br class=""> void CtorDtorRunner::add(iterator_range<CtorDtorIterator> CtorDtors) {<br class=""> if (CtorDtors.empty())<br class=""> return;<br class="">@@ -198,6 +218,30 @@ Error LocalCXXRuntimeOverrides::enable(JITDylib &JD,<br class=""> return JD.define(absoluteSymbols(std::move(RuntimeInterposes)));<br class=""> }<br class=""><br class="">+void ItaniumCXAAtExitSupport::registerAtExit(void (*F)(void *), void *Ctx,<br class="">+ void *DSOHandle) {<br class="">+ std::lock_guard<std::mutex> Lock(AtExitsMutex);<br class="">+ AtExitRecords[DSOHandle].push_back({F, Ctx});<br class="">+}<br class="">+<br class="">+void ItaniumCXAAtExitSupport::runAtExits(void *DSOHandle) {<br class="">+ std::vector<AtExitRecord> AtExitsToRun;<br class="">+<br class="">+ {<br class="">+ std::lock_guard<std::mutex> Lock(AtExitsMutex);<br class="">+ auto I = AtExitRecords.find(DSOHandle);<br class="">+ if (I != AtExitRecords.end()) {<br class="">+ AtExitsToRun = std::move(I->second);<br class="">+ AtExitRecords.erase(I);<br class="">+ }<br class="">+ }<br class="">+<br class="">+ while (!AtExitsToRun.empty()) {<br class="">+ AtExitsToRun.back().F(AtExitsToRun.back().Ctx);<br class="">+ AtExitsToRun.pop_back();<br class="">+ }<br class="">+}<br class="">+<br class=""> DynamicLibrarySearchGenerator::DynamicLibrarySearchGenerator(<br class=""> sys::DynamicLibrary Dylib, char GlobalPrefix, SymbolPredicate Allow)<br class=""> : Dylib(std::move(Dylib)), Allow(std::move(Allow)),<br class=""><br class="">diff --git a/llvm/lib/ExecutionEngine/Orc/IndirectionUtils.cpp b/llvm/lib/ExecutionEngine/Orc/IndirectionUtils.cpp<br class="">index 1ac9a58aeaef..1e991d8f58f2 100644<br class="">--- a/llvm/lib/ExecutionEngine/Orc/IndirectionUtils.cpp<br class="">+++ b/llvm/lib/ExecutionEngine/Orc/IndirectionUtils.cpp<br class="">@@ -28,7 +28,7 @@ class CompileCallbackMaterializationUnit : public orc::MaterializationUnit {<br class=""> CompileCallbackMaterializationUnit(SymbolStringPtr Name,<br class=""> CompileFunction Compile, VModuleKey K)<br class=""> : MaterializationUnit(SymbolFlagsMap({{Name, JITSymbolFlags::Exported}}),<br class="">- std::move(K)),<br class="">+ nullptr, std::move(K)),<br class=""> Name(std::move(Name)), Compile(std::move(Compile)) {}<br class=""><br class=""> StringRef getName() const override { return "<Compile Callbacks>"; }<br class=""><br class="">diff --git a/llvm/lib/ExecutionEngine/Orc/LLJIT.cpp b/llvm/lib/ExecutionEngine/Orc/LLJIT.cpp<br class="">index 4218ca4e481f..5b2f86823355 100644<br class="">--- a/llvm/lib/ExecutionEngine/Orc/LLJIT.cpp<br class="">+++ b/llvm/lib/ExecutionEngine/Orc/LLJIT.cpp<br class="">@@ -8,15 +8,827 @@<br class=""><br class=""> #include "llvm/ExecutionEngine/Orc/LLJIT.h"<br class=""> #include "llvm/ExecutionEngine/JITLink/JITLinkMemoryManager.h"<br class="">+#include "llvm/ExecutionEngine/Orc/MachOPlatform.h"<br class=""> #include "llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h"<br class=""> #include "llvm/ExecutionEngine/Orc/OrcError.h"<br class=""> #include "llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h"<br class=""> #include "llvm/ExecutionEngine/SectionMemoryManager.h"<br class="">+#include "llvm/IR/GlobalVariable.h"<br class="">+#include "llvm/IR/IRBuilder.h"<br class=""> #include "llvm/IR/Mangler.h"<br class="">+#include "llvm/IR/Module.h"<br class="">+#include "llvm/Support/DynamicLibrary.h"<br class="">+<br class="">+#include <map><br class="">+<br class="">+using namespace llvm;<br class="">+using namespace llvm::orc;<br class="">+<br class="">+namespace {<br class="">+<br class="">+/// Add a reference to the __dso_handle global to the given module.<br class="">+/// Returns a reference to the __dso_handle IR decl.<br class="">+GlobalVariable *addDSOHandleDecl(Module &M) {<br class="">+ auto DSOHandleTy = StructType::create(M.getContext(), "lljit.dso_handle");<br class="">+ return new GlobalVariable(M, DSOHandleTy, true, GlobalValue::ExternalLinkage,<br class="">+ nullptr, "__dso_handle");<br class="">+}<br class="">+<br class="">+/// Adds helper function decls and wrapper functions that call the helper with<br class="">+/// some additional prefix arguments.<br class="">+///<br class="">+/// E.g. For wrapper "foo" with type i8(i8, i64), helper "bar", and prefix<br class="">+/// args i32 4 and i16 12345, this function will add:<br class="">+///<br class="">+/// declare i8 @bar(i32, i16, i8, i64)<br class="">+///<br class="">+/// define i8 @foo(i8, i64) {<br class="">+/// entry:<br class="">+/// %2 = call i8 @bar(i32 4, i16 12345, i8 %0, i64 %1)<br class="">+/// ret i8 %2<br class="">+/// }<br class="">+///<br class="">+Function *addHelperAndWrapper(Module &M, StringRef WrapperName,<br class="">+ FunctionType *WrapperFnType,<br class="">+ GlobalValue::VisibilityTypes WrapperVisibility,<br class="">+ StringRef HelperName,<br class="">+ ArrayRef<Value *> HelperPrefixArgs) {<br class="">+ std::vector<Type *> HelperArgTypes;<br class="">+ for (auto *Arg : HelperPrefixArgs)<br class="">+ HelperArgTypes.push_back(Arg->getType());<br class="">+ for (auto *T : WrapperFnType->params())<br class="">+ HelperArgTypes.push_back(T);<br class="">+ auto *HelperFnType =<br class="">+ FunctionType::get(WrapperFnType->getReturnType(), HelperArgTypes, false);<br class="">+ auto *HelperFn = Function::Create(HelperFnType, GlobalValue::ExternalLinkage,<br class="">+ HelperName, M);<br class="">+<br class="">+ auto *WrapperFn = Function::Create(<br class="">+ WrapperFnType, GlobalValue::ExternalLinkage, WrapperName, M);<br class="">+ WrapperFn->setVisibility(WrapperVisibility);<br class="">+<br class="">+ auto *EntryBlock = BasicBlock::Create(M.getContext(), "entry", WrapperFn);<br class="">+ IRBuilder<> IB(EntryBlock);<br class="">+<br class="">+ std::vector<Value *> HelperArgs;<br class="">+ for (auto *Arg : HelperPrefixArgs)<br class="">+ HelperArgs.push_back(Arg);<br class="">+ for (auto &Arg : WrapperFn->args())<br class="">+ HelperArgs.push_back(&Arg);<br class="">+ auto *HelperResult = IB.CreateCall(HelperFn, HelperArgs);<br class="">+ if (HelperFn->getReturnType()->isVoidTy())<br class="">+ IB.CreateRetVoid();<br class="">+ else<br class="">+ IB.CreateRet(HelperResult);<br class="">+<br class="">+ return WrapperFn;<br class="">+}<br class="">+<br class="">+class GenericLLVMIRPlatformSupport;<br class="">+<br class="">+/// orc::Platform component of Generic LLVM IR Platform support.<br class="">+/// Just forwards calls to the GenericLLVMIRPlatformSupport class below.<br class="">+class GenericLLVMIRPlatform : public Platform {<br class="">+public:<br class="">+ GenericLLVMIRPlatform(GenericLLVMIRPlatformSupport &S) : S(S) {}<br class="">+ Error setupJITDylib(JITDylib &JD) override;<br class="">+ Error notifyAdding(JITDylib &JD, const MaterializationUnit &MU) override;<br class="">+ Error notifyRemoving(JITDylib &JD, VModuleKey K) override {<br class="">+ // Noop -- Nothing to do (yet).<br class="">+ return Error::success();<br class="">+ }<br class="">+<br class="">+private:<br class="">+ GenericLLVMIRPlatformSupport &S;<br class="">+};<br class="">+<br class="">+/// This transform parses llvm.global_ctors to produce a single initialization<br class="">+/// function for the module, records the function, then deletes<br class="">+/// llvm.global_ctors.<br class="">+class GlobalCtorDtorScraper {<br class="">+public:<br class="">+ GlobalCtorDtorScraper(GenericLLVMIRPlatformSupport &PS) : PS(PS) {}<br class="">+ Expected<ThreadSafeModule> operator()(ThreadSafeModule TSM,<br class="">+ MaterializationResponsibility &R);<br class="">+<br class="">+private:<br class="">+ GenericLLVMIRPlatformSupport &PS;<br class="">+};<br class="">+<br class="">+/// Generic IR Platform Support<br class="">+///<br class="">+/// Scrapes llvm.global_ctors and llvm.global_dtors and replaces them with<br class="">+/// specially named 'init' and 'deinit'. Injects definitions / interposes for<br class="">+/// some runtime API, including __cxa_atexit, dlopen, and dlclose.<br class="">+class GenericLLVMIRPlatformSupport : public LLJIT::PlatformSupport {<br class="">+public:<br class="">+ // GenericLLVMIRPlatform &P) : P(P) {<br class="">+ GenericLLVMIRPlatformSupport(LLJIT &J) : J(J) {<br class="">+<br class="">+ getExecutionSession().setPlatform(<br class="">+ std::make_unique<GenericLLVMIRPlatform>(*this));<br class="">+<br class="">+ setInitTransform(J, GlobalCtorDtorScraper(*this));<br class="">+<br class="">+ MangleAndInterner Mangle(getExecutionSession(), J.getDataLayout());<br class="">+ SymbolMap StdInterposes;<br class="">+<br class="">+ StdInterposes[Mangle("__lljit.platform_support_instance")] =<br class="">+ JITEvaluatedSymbol(pointerToJITTargetAddress(this), JITSymbolFlags());<br class="">+ StdInterposes[Mangle("__lljit.cxa_atexit_helper")] = JITEvaluatedSymbol(<br class="">+ pointerToJITTargetAddress(registerAtExitHelper), JITSymbolFlags());<br class="">+ StdInterposes[Mangle("__lljit.run_atexits_helper")] = JITEvaluatedSymbol(<br class="">+ pointerToJITTargetAddress(runAtExitsHelper), JITSymbolFlags());<br class="">+<br class="">+ cantFail(<br class="">+ J.getMainJITDylib().define(absoluteSymbols(std::move(StdInterposes))));<br class="">+ cantFail(setupJITDylib(J.getMainJITDylib()));<br class="">+ cantFail(J.addIRModule(J.getMainJITDylib(), createPlatformRuntimeModule()));<br class="">+ }<br class="">+<br class="">+ ExecutionSession &getExecutionSession() { return J.getExecutionSession(); }<br class="">+<br class="">+ /// Adds a module that defines the __dso_handle global.<br class="">+ Error setupJITDylib(JITDylib &JD) {<br class="">+ auto Ctx = std::make_unique<LLVMContext>();<br class="">+ auto M = std::make_unique<Module>("__standard_lib", *Ctx);<br class="">+ M->setDataLayout(J.getDataLayout());<br class="">+<br class="">+ auto *Int64Ty = Type::getInt64Ty(*Ctx);<br class="">+ auto *DSOHandle = new GlobalVariable(<br class="">+ *M, Int64Ty, true, GlobalValue::ExternalLinkage,<br class="">+ ConstantInt::get(Int64Ty, reinterpret_cast<uintptr_t>(&JD)),<br class="">+ "__dso_handle");<br class="">+ DSOHandle->setVisibility(GlobalValue::HiddenVisibility);<br class="">+ DSOHandle->setInitializer(<br class="">+ ConstantInt::get(Int64Ty, pointerToJITTargetAddress(&JD)));<br class="">+ return J.addIRModule(JD, ThreadSafeModule(std::move(M), std::move(Ctx)));<br class="">+ }<br class="">+<br class="">+ Error notifyAdding(JITDylib &JD, const MaterializationUnit &MU) {<br class="">+ std::lock_guard<std::mutex> Lock(PlatformSupportMutex);<br class="">+ if (auto &InitSym = MU.getInitializerSymbol())<br class="">+ InitSymbols[&JD].add(InitSym);<br class="">+ return Error::success();<br class="">+ }<br class="">+<br class="">+ Error initialize(JITDylib &JD) override {<br class="">+ if (auto Initializers = getInitializers(JD)) {<br class="">+ for (auto InitFnAddr : *Initializers) {<br class="">+ auto *InitFn = jitTargetAddressToFunction<void (*)()>(InitFnAddr);<br class="">+ InitFn();<br class="">+ }<br class="">+ } else<br class="">+ return Initializers.takeError();<br class="">+ return Error::success();<br class="">+ }<br class="">+<br class="">+ Error deinitialize(JITDylib &JD) override {<br class="">+ if (auto Deinitializers = getDeinitializers(JD)) {<br class="">+ for (auto DeinitFnAddr : *Deinitializers) {<br class="">+ auto *DeinitFn = jitTargetAddressToFunction<void (*)()>(DeinitFnAddr);<br class="">+ DeinitFn();<br class="">+ }<br class="">+ } else<br class="">+ return Deinitializers.takeError();<br class="">+<br class="">+ return Error::success();<br class="">+ }<br class="">+<br class="">+ void registerInitFunc(JITDylib &JD, SymbolStringPtr InitName) {<br class="">+ std::lock_guard<std::mutex> Lock(PlatformSupportMutex);<br class="">+ InitFunctions[&JD].add(InitName);<br class="">+ }<br class="">+<br class="">+private:<br class="">+ Expected<std::vector<JITTargetAddress>> getInitializers(JITDylib &JD) {<br class="">+ if (auto Err = issueInitLookups(JD))<br class="">+ return std::move(Err);<br class="">+<br class="">+ DenseMap<JITDylib *, SymbolLookupSet> LookupSymbols;<br class="">+ std::vector<JITDylib *> DFSLinkOrder;<br class="">+<br class="">+ {<br class="">+ std::lock_guard<std::mutex> Lock(PlatformSupportMutex);<br class="">+ DFSLinkOrder = getDFSLinkOrder(JD);<br class="">+<br class="">+ for (auto *NextJD : DFSLinkOrder) {<br class="">+ auto IFItr = InitFunctions.find(NextJD);<br class="">+ if (IFItr != InitFunctions.end()) {<br class="">+ LookupSymbols[NextJD] = std::move(IFItr->second);<br class="">+ InitFunctions.erase(IFItr);<br class="">+ }<br class="">+ }<br class="">+ }<br class="">+<br class="">+ auto &ES = getExecutionSession();<br class="">+ auto LookupResult = Platform::lookupInitSymbols(ES, LookupSymbols);<br class="">+<br class="">+ if (!LookupResult)<br class="">+ return LookupResult.takeError();<br class="">+<br class="">+ std::vector<JITTargetAddress> Initializers;<br class="">+ while (!DFSLinkOrder.empty()) {<br class="">+ auto &NextJD = *DFSLinkOrder.back();<br class="">+ DFSLinkOrder.pop_back();<br class="">+ auto InitsItr = LookupResult->find(&NextJD);<br class="">+ if (InitsItr == LookupResult->end())<br class="">+ continue;<br class="">+ for (auto &KV : InitsItr->second)<br class="">+ Initializers.push_back(KV.second.getAddress());<br class="">+ }<br class="">+<br class="">+ return Initializers;<br class="">+ }<br class="">+<br class="">+ Expected<std::vector<JITTargetAddress>> getDeinitializers(JITDylib &JD) {<br class="">+ auto &ES = getExecutionSession();<br class="">+<br class="">+ MangleAndInterner Mangle(getExecutionSession(), J.getDataLayout());<br class="">+ auto LLJITRunAtExits = Mangle("__lljit_run_atexits");<br class="">+<br class="">+ DenseMap<JITDylib *, SymbolLookupSet> LookupSymbols;<br class="">+ std::vector<JITDylib *> DFSLinkOrder;<br class="">+<br class="">+ {<br class="">+ std::lock_guard<std::mutex> Lock(PlatformSupportMutex);<br class="">+ DFSLinkOrder = getDFSLinkOrder(JD);<br class="">+<br class="">+ for (auto *NextJD : DFSLinkOrder) {<br class="">+ auto &JDLookupSymbols = LookupSymbols[NextJD];<br class="">+ auto DIFItr = DeInitFunctions.find(NextJD);<br class="">+ if (DIFItr != DeInitFunctions.end()) {<br class="">+ LookupSymbols[NextJD] = std::move(DIFItr->second);<br class="">+ DeInitFunctions.erase(DIFItr);<br class="">+ }<br class="">+ JDLookupSymbols.add(LLJITRunAtExits,<br class="">+ SymbolLookupFlags::WeaklyReferencedSymbol);<br class="">+ }<br class="">+ }<br class="">+<br class="">+ auto LookupResult = Platform::lookupInitSymbols(ES, LookupSymbols);<br class="">+<br class="">+ if (!LookupResult)<br class="">+ return LookupResult.takeError();<br class="">+<br class="">+ std::vector<JITTargetAddress> DeInitializers;<br class="">+ for (auto *NextJD : DFSLinkOrder) {<br class="">+ auto DeInitsItr = LookupResult->find(NextJD);<br class="">+ assert(DeInitsItr != LookupResult->end() &&<br class="">+ "Every JD should have at least __lljit_run_atexits");<br class="">+<br class="">+ auto RunAtExitsItr = DeInitsItr->second.find(LLJITRunAtExits);<br class="">+ if (RunAtExitsItr != DeInitsItr->second.end())<br class="">+ DeInitializers.push_back(RunAtExitsItr->second.getAddress());<br class="">+<br class="">+ for (auto &KV : DeInitsItr->second)<br class="">+ if (KV.first != LLJITRunAtExits)<br class="">+ DeInitializers.push_back(KV.second.getAddress());<br class="">+ }<br class="">+<br class="">+ return DeInitializers;<br class="">+ }<br class="">+<br class="">+ // Returns a DFS traversal order of the JITDylibs reachable (via<br class="">+ // links-against edges) from JD, starting with JD itself.<br class="">+ static std::vector<JITDylib *> getDFSLinkOrder(JITDylib &JD) {<br class="">+ std::vector<JITDylib *> DFSLinkOrder;<br class="">+ std::vector<JITDylib *> WorkStack({&JD});<br class="">+ DenseSet<JITDylib *> Visited;<br class="">+<br class="">+ while (!WorkStack.empty()) {<br class="">+ auto &NextJD = *WorkStack.back();<br class="">+ WorkStack.pop_back();<br class="">+ if (Visited.count(&NextJD))<br class="">+ continue;<br class="">+ Visited.insert(&NextJD);<br class="">+ DFSLinkOrder.push_back(&NextJD);<br class="">+ NextJD.withSearchOrderDo([&](const JITDylibSearchOrder &SearchOrder) {<br class="">+ for (auto &KV : SearchOrder)<br class="">+ WorkStack.push_back(KV.first);<br class="">+ });<br class="">+ }<br class="">+<br class="">+ return DFSLinkOrder;<br class="">+ }<br class="">+<br class="">+ /// Issue lookups for all init symbols required to initialize JD (and any<br class="">+ /// JITDylibs that it depends on).<br class="">+ Error issueInitLookups(JITDylib &JD) {<br class="">+ DenseMap<JITDylib *, SymbolLookupSet> RequiredInitSymbols;<br class="">+<br class="">+ {<br class="">+ std::lock_guard<std::mutex> Lock(PlatformSupportMutex);<br class="">+<br class="">+ auto DFSLinkOrder = getDFSLinkOrder(JD);<br class="">+<br class="">+ for (auto *NextJD : DFSLinkOrder) {<br class="">+ auto ISItr = InitSymbols.find(NextJD);<br class="">+ if (ISItr != InitSymbols.end()) {<br class="">+ RequiredInitSymbols[NextJD] = std::move(ISItr->second);<br class="">+ InitSymbols.erase(ISItr);<br class="">+ }<br class="">+ }<br class="">+ }<br class="">+<br class="">+ return Platform::lookupInitSymbols(getExecutionSession(),<br class="">+ RequiredInitSymbols)<br class="">+ .takeError();<br class="">+ }<br class="">+<br class="">+ static void registerAtExitHelper(void *Self, void (*F)(void *), void *Ctx,<br class="">+ void *DSOHandle) {<br class="">+ static_cast<GenericLLVMIRPlatformSupport *>(Self)->AtExitMgr.registerAtExit(<br class="">+ F, Ctx, DSOHandle);<br class="">+ }<br class="">+<br class="">+ static void runAtExitsHelper(void *Self, void *DSOHandle) {<br class="">+ static_cast<GenericLLVMIRPlatformSupport *>(Self)->AtExitMgr.runAtExits(<br class="">+ DSOHandle);<br class="">+ }<br class="">+<br class="">+ // Constructs an LLVM IR module containing platform runtime globals,<br class="">+ // functions, and interposes.<br class="">+ ThreadSafeModule createPlatformRuntimeModule() {<br class="">+ auto Ctx = std::make_unique<LLVMContext>();<br class="">+ auto M = std::make_unique<Module>("__standard_lib", *Ctx);<br class="">+ M->setDataLayout(J.getDataLayout());<br class="">+<br class="">+ auto *GenericIRPlatformSupportTy =<br class="">+ StructType::create(*Ctx, "lljit.GenericLLJITIRPlatformSupport");<br class="">+<br class="">+ auto *PlatformInstanceDecl = new GlobalVariable(<br class="">+ *M, GenericIRPlatformSupportTy, true, GlobalValue::ExternalLinkage,<br class="">+ nullptr, "__lljit.platform_support_instance");<br class="">+<br class="">+ auto *DSOHandleDecl = addDSOHandleDecl(*M);<br class="">+<br class="">+ auto *Int8Ty = Type::getInt8Ty(*Ctx);<br class="">+ auto *IntTy = Type::getIntNTy(*Ctx, sizeof(int) * CHAR_BIT);<br class="">+ auto *VoidTy = Type::getVoidTy(*Ctx);<br class="">+ auto *BytePtrTy = PointerType::getUnqual(Int8Ty);<br class="">+ auto *AtExitCallbackTy = FunctionType::get(VoidTy, {BytePtrTy}, false);<br class="">+ auto *AtExitCallbackPtrTy = PointerType::getUnqual(AtExitCallbackTy);<br class="">+<br class="">+ addHelperAndWrapper(<br class="">+ *M, "__cxa_atexit",<br class="">+ FunctionType::get(IntTy, {AtExitCallbackPtrTy, BytePtrTy, BytePtrTy},<br class="">+ false),<br class="">+ GlobalValue::HiddenVisibility, "__lljit.cxa_atexit_helper",<br class="">+ {PlatformInstanceDecl});<br class="">+<br class="">+ addHelperAndWrapper(<br class="">+ *M, "__lljit_run_atexits", FunctionType::get(VoidTy, {}, false),<br class="">+ GlobalValue::HiddenVisibility, "__lljit.run_atexits_helper",<br class="">+ {PlatformInstanceDecl, DSOHandleDecl});<br class="">+<br class="">+ return ThreadSafeModule(std::move(M), std::move(Ctx));<br class="">+ }<br class="">+<br class="">+ std::mutex PlatformSupportMutex;<br class="">+ LLJIT &J;<br class="">+ DenseMap<JITDylib *, SymbolLookupSet> InitSymbols;<br class="">+ DenseMap<JITDylib *, SymbolLookupSet> InitFunctions;<br class="">+ DenseMap<JITDylib *, SymbolLookupSet> DeInitFunctions;<br class="">+ ItaniumCXAAtExitSupport AtExitMgr;<br class="">+};<br class="">+<br class="">+Error GenericLLVMIRPlatform::setupJITDylib(JITDylib &JD) {<br class="">+ return S.setupJITDylib(JD);<br class="">+}<br class="">+<br class="">+Error GenericLLVMIRPlatform::notifyAdding(JITDylib &JD,<br class="">+ const MaterializationUnit &MU) {<br class="">+ return S.notifyAdding(JD, MU);<br class="">+}<br class="">+<br class="">+Expected<ThreadSafeModule><br class="">+GlobalCtorDtorScraper::operator()(ThreadSafeModule TSM,<br class="">+ MaterializationResponsibility &R) {<br class="">+ auto Err = TSM.withModuleDo([&](Module &M) -> Error {<br class="">+ auto &Ctx = M.getContext();<br class="">+ auto *GlobalCtors = M.getNamedGlobal("llvm.global_ctors");<br class="">+<br class="">+ // If there's no llvm.global_ctors or it's just a decl then skip.<br class="">+ if (!GlobalCtors || GlobalCtors->isDeclaration())<br class="">+ return Error::success();<br class="">+<br class="">+ std::string InitFunctionName;<br class="">+ raw_string_ostream(InitFunctionName)<br class="">+ << "__orc_init." << M.getModuleIdentifier();<br class="">+<br class="">+ MangleAndInterner Mangle(PS.getExecutionSession(), M.getDataLayout());<br class="">+ auto InternedName = Mangle(InitFunctionName);<br class="">+ if (auto Err =<br class="">+ R.defineMaterializing({{InternedName, JITSymbolFlags::Callable}}))<br class="">+ return Err;<br class="">+<br class="">+ auto *InitFunc =<br class="">+ Function::Create(FunctionType::get(Type::getVoidTy(Ctx), {}, false),<br class="">+ GlobalValue::ExternalLinkage, InitFunctionName, &M);<br class="">+ InitFunc->setVisibility(GlobalValue::HiddenVisibility);<br class="">+ std::vector<std::pair<Function *, unsigned>> Inits;<br class="">+ for (auto E : getConstructors(M))<br class="">+ Inits.push_back(std::make_pair(E.Func, E.Priority));<br class="">+ llvm::sort(Inits, [](const std::pair<Function *, unsigned> &LHS,<br class="">+ const std::pair<Function *, unsigned> &RHS) {<br class="">+ return LHS.first < RHS.first;<br class="">+ });<br class="">+ auto *EntryBlock = BasicBlock::Create(Ctx, "entry", InitFunc);<br class="">+ IRBuilder<> IB(EntryBlock);<br class="">+ for (auto &KV : Inits)<br class="">+ IB.CreateCall(KV.first);<br class="">+ IB.CreateRetVoid();<br class="">+<br class="">+ PS.registerInitFunc(R.getTargetJITDylib(), InternedName);<br class="">+ GlobalCtors->eraseFromParent();<br class="">+ return Error::success();<br class="">+ });<br class="">+<br class="">+ if (Err)<br class="">+ return std::move(Err);<br class="">+<br class="">+ return TSM;<br class="">+}<br class="">+<br class="">+class MachOPlatformSupport : public LLJIT::PlatformSupport {<br class="">+public:<br class="">+ using DLOpenType = void *(*)(const char *Name, int Mode);<br class="">+ using DLCloseType = int (*)(void *Handle);<br class="">+ using DLSymType = void *(*)(void *Handle, const char *Name);<br class="">+ using DLErrorType = const char *(*)();<br class="">+<br class="">+ struct DlFcnValues {<br class="">+ Optional<void *> RTLDDefault;<br class="">+ DLOpenType dlopen = nullptr;<br class="">+ DLCloseType dlclose = nullptr;<br class="">+ DLSymType dlsym = nullptr;<br class="">+ DLErrorType dlerror = nullptr;<br class="">+ };<br class="">+<br class="">+ static Expected<std::unique_ptr<MachOPlatformSupport>><br class="">+ Create(LLJIT &J, JITDylib &PlatformJITDylib) {<br class="">+<br class="">+ // Make process symbols visible.<br class="">+ {<br class="">+ std::string ErrMsg;<br class="">+ auto Lib = sys::DynamicLibrary::getPermanentLibrary(nullptr, &ErrMsg);<br class="">+ if (!Lib.isValid())<br class="">+ return make_error<StringError>(std::move(ErrMsg),<br class="">+ inconvertibleErrorCode());<br class="">+ }<br class="">+<br class="">+ DlFcnValues DlFcn;<br class="">+<br class="">+ // Add support for RTLDDefault on known platforms.<br class="">+#ifdef __APPLE__<br class="">+ DlFcn.RTLDDefault = reinterpret_cast<void *>(-2);<br class="">+#endif // __APPLE__<br class="">+<br class="">+ if (auto Err = hookUpFunction(DlFcn.dlopen, "dlopen"))<br class="">+ return std::move(Err);<br class="">+ if (auto Err = hookUpFunction(DlFcn.dlclose, "dlclose"))<br class="">+ return std::move(Err);<br class="">+ if (auto Err = hookUpFunction(DlFcn.dlsym, "dlsym"))<br class="">+ return std::move(Err);<br class="">+ if (auto Err = hookUpFunction(DlFcn.dlerror, "dlerror"))<br class="">+ return std::move(Err);<br class="">+<br class="">+ std::unique_ptr<MachOPlatformSupport> MP(<br class="">+ new MachOPlatformSupport(J, PlatformJITDylib, DlFcn));<br class="">+ return std::move(MP);<br class="">+ }<br class="">+<br class="">+ Error initialize(JITDylib &JD) override {<br class="">+ if (auto InitSeq = MP.getInitializerSequence(JD)) {<br class="">+ for (auto &KV : *InitSeq) {<br class="">+ KV.second.registerObjCSelectors();<br class="">+ if (auto Err = KV.second.registerObjCClasses()) {<br class="">+ // FIXME: Roll back registrations on error?<br class="">+ return Err;<br class="">+ }<br class="">+ }<br class="">+ for (auto &KV : *InitSeq)<br class="">+ KV.second.runModInits();<br class="">+ } else<br class="">+ return InitSeq.takeError();<br class="">+ return Error::success();<br class="">+ }<br class="">+<br class="">+ Error deinitialize(JITDylib &JD) override {<br class="">+ auto &ES = J.getExecutionSession();<br class="">+ if (auto DeinitSeq = MP.getDeinitializerSequence(JD)) {<br class="">+ for (auto &KV : *DeinitSeq) {<br class="">+ auto DSOHandleName = ES.intern("___dso_handle");<br class="">+<br class="">+ // FIXME: Run DeInits here.<br class="">+ auto Result = ES.lookup(<br class="">+ {{KV.first, JITDylibLookupFlags::MatchAllSymbols}},<br class="">+ SymbolLookupSet(DSOHandleName,<br class="">+ SymbolLookupFlags::WeaklyReferencedSymbol));<br class="">+ if (!Result)<br class="">+ return Result.takeError();<br class="">+ if (Result->empty())<br class="">+ continue;<br class="">+ assert(Result->count(DSOHandleName) &&<br class="">+ "Result does not contain __dso_handle");<br class="">+ auto *DSOHandle = jitTargetAddressToPointer<void *>(<br class="">+ Result->begin()->second.getAddress());<br class="">+ AtExitMgr.runAtExits(DSOHandle);<br class="">+ }<br class="">+ } else<br class="">+ return DeinitSeq.takeError();<br class="">+ return Error::success();<br class="">+ }<br class="">+<br class="">+private:<br class="">+ template <typename FunctionPtrTy><br class="">+ static Error hookUpFunction(FunctionPtrTy &Fn, const char *Name) {<br class="">+ if (auto *FnAddr = sys::DynamicLibrary::SearchForAddressOfSymbol(Name)) {<br class="">+ Fn = reinterpret_cast<FunctionPtrTy>(Fn);<br class="">+ return Error::success();<br class="">+ }<br class="">+<br class="">+ return make_error<StringError>((Twine("Can not enable MachO JIT Platform: "<br class="">+ "missing function: ") +<br class="">+ Name)<br class="">+ .str(),<br class="">+ inconvertibleErrorCode());<br class="">+ }<br class="">+<br class="">+ MachOPlatformSupport(LLJIT &J, JITDylib &PlatformJITDylib, DlFcnValues DlFcn)<br class="">+ : J(J), MP(setupPlatform(J)), DlFcn(std::move(DlFcn)) {<br class="">+<br class="">+ MangleAndInterner Mangle(J.getExecutionSession(), J.getDataLayout());<br class="">+ SymbolMap HelperSymbols;<br class="">+<br class="">+ // platform and atexit helpers.<br class="">+ HelperSymbols[Mangle("__lljit.platform_support_instance")] =<br class="">+ JITEvaluatedSymbol(pointerToJITTargetAddress(this), JITSymbolFlags());<br class="">+ HelperSymbols[Mangle("__lljit.cxa_atexit_helper")] = JITEvaluatedSymbol(<br class="">+ pointerToJITTargetAddress(registerAtExitHelper), JITSymbolFlags());<br class="">+ HelperSymbols[Mangle("__lljit.run_atexits_helper")] = JITEvaluatedSymbol(<br class="">+ pointerToJITTargetAddress(runAtExitsHelper), JITSymbolFlags());<br class="">+<br class="">+ // dlfcn helpers.<br class="">+ HelperSymbols[Mangle("__lljit.dlopen_helper")] = JITEvaluatedSymbol(<br class="">+ pointerToJITTargetAddress(dlopenHelper), JITSymbolFlags());<br class="">+ HelperSymbols[Mangle("__lljit.dlclose_helper")] = JITEvaluatedSymbol(<br class="">+ pointerToJITTargetAddress(dlcloseHelper), JITSymbolFlags());<br class="">+ HelperSymbols[Mangle("__lljit.dlsym_helper")] = JITEvaluatedSymbol(<br class="">+ pointerToJITTargetAddress(dlsymHelper), JITSymbolFlags());<br class="">+ HelperSymbols[Mangle("__lljit.dlerror_helper")] = JITEvaluatedSymbol(<br class="">+ pointerToJITTargetAddress(dlerrorHelper), JITSymbolFlags());<br class="">+<br class="">+ cantFail(<br class="">+ PlatformJITDylib.define(absoluteSymbols(std::move(HelperSymbols))));<br class="">+ cantFail(MP.setupJITDylib(J.getMainJITDylib()));<br class="">+ cantFail(J.addIRModule(PlatformJITDylib, createPlatformRuntimeModule()));<br class="">+ }<br class="">+<br class="">+ static MachOPlatform &setupPlatform(LLJIT &J) {<br class="">+ auto Tmp = std::make_unique<MachOPlatform>(<br class="">+ J.getExecutionSession(),<br class="">+ static_cast<ObjectLinkingLayer &>(J.getObjLinkingLayer()),<br class="">+ createStandardSymbolsObject(J));<br class="">+ auto &MP = *Tmp;<br class="">+ J.getExecutionSession().setPlatform(std::move(Tmp));<br class="">+ return MP;<br class="">+ }<br class="">+<br class="">+ static std::unique_ptr<MemoryBuffer> createStandardSymbolsObject(LLJIT &J) {<br class="">+ LLVMContext Ctx;<br class="">+ Module M("__standard_symbols", Ctx);<br class="">+ M.setDataLayout(J.getDataLayout());<br class="">+<br class="">+ auto *Int64Ty = Type::getInt64Ty(Ctx);<br class="">+<br class="">+ auto *DSOHandle =<br class="">+ new GlobalVariable(M, Int64Ty, true, GlobalValue::ExternalLinkage,<br class="">+ ConstantInt::get(Int64Ty, 0), "__dso_handle");<br class="">+ DSOHandle->setVisibility(GlobalValue::HiddenVisibility);<br class="">+<br class="">+ return cantFail(J.getIRCompileLayer().getCompiler()(M));<br class="">+ }<br class="">+<br class="">+ ThreadSafeModule createPlatformRuntimeModule() {<br class="">+ auto Ctx = std::make_unique<LLVMContext>();<br class="">+ auto M = std::make_unique<Module>("__standard_lib", *Ctx);<br class="">+ M->setDataLayout(J.getDataLayout());<br class="">+<br class="">+ auto *MachOPlatformSupportTy =<br class="">+ StructType::create(*Ctx, "lljit.MachOPlatformSupport");<br class="">+<br class="">+ auto *PlatformInstanceDecl = new GlobalVariable(<br class="">+ *M, MachOPlatformSupportTy, true, GlobalValue::ExternalLinkage, nullptr,<br class="">+ "__lljit.platform_support_instance");<br class="">+<br class="">+ auto *Int8Ty = Type::getInt8Ty(*Ctx);<br class="">+ auto *IntTy = Type::getIntNTy(*Ctx, sizeof(int) * CHAR_BIT);<br class="">+ auto *VoidTy = Type::getVoidTy(*Ctx);<br class="">+ auto *BytePtrTy = PointerType::getUnqual(Int8Ty);<br class="">+ auto *AtExitCallbackTy = FunctionType::get(VoidTy, {BytePtrTy}, false);<br class="">+ auto *AtExitCallbackPtrTy = PointerType::getUnqual(AtExitCallbackTy);<br class="">+<br class="">+ addHelperAndWrapper(<br class="">+ *M, "__cxa_atexit",<br class="">+ FunctionType::get(IntTy, {AtExitCallbackPtrTy, BytePtrTy, BytePtrTy},<br class="">+ false),<br class="">+ GlobalValue::DefaultVisibility, "__lljit.cxa_atexit_helper",<br class="">+ {PlatformInstanceDecl});<br class="">+<br class="">+ addHelperAndWrapper(*M, "dlopen",<br class="">+ FunctionType::get(BytePtrTy, {BytePtrTy, IntTy}, false),<br class="">+ GlobalValue::DefaultVisibility, "__lljit.dlopen_helper",<br class="">+ {PlatformInstanceDecl});<br class="">+<br class="">+ addHelperAndWrapper(*M, "dlclose",<br class="">+ FunctionType::get(IntTy, {BytePtrTy}, false),<br class="">+ GlobalValue::DefaultVisibility,<br class="">+ "__lljit.dlclose_helper", {PlatformInstanceDecl});<br class="">+<br class="">+ addHelperAndWrapper(<br class="">+ *M, "dlsym",<br class="">+ FunctionType::get(BytePtrTy, {BytePtrTy, BytePtrTy}, false),<br class="">+ GlobalValue::DefaultVisibility, "__lljit.dlsym_helper",<br class="">+ {PlatformInstanceDecl});<br class="">+<br class="">+ addHelperAndWrapper(*M, "dlerror", FunctionType::get(BytePtrTy, {}, false),<br class="">+ GlobalValue::DefaultVisibility,<br class="">+ "__lljit.dlerror_helper", {PlatformInstanceDecl});<br class="">+<br class="">+ return ThreadSafeModule(std::move(M), std::move(Ctx));<br class="">+ }<br class="">+<br class="">+ static void registerAtExitHelper(void *Self, void (*F)(void *), void *Ctx,<br class="">+ void *DSOHandle) {<br class="">+ static_cast<MachOPlatformSupport *>(Self)->AtExitMgr.registerAtExit(<br class="">+ F, Ctx, DSOHandle);<br class="">+ }<br class="">+<br class="">+ static void runAtExitsHelper(void *Self, void *DSOHandle) {<br class="">+ static_cast<MachOPlatformSupport *>(Self)->AtExitMgr.runAtExits(DSOHandle);<br class="">+ }<br class="">+<br class="">+ void *jit_dlopen(const char *Path, int Mode) {<br class="">+ JITDylib *JDToOpen = nullptr;<br class="">+ // FIXME: Do the right thing with Mode flags.<br class="">+ {<br class="">+ std::lock_guard<std::mutex> Lock(PlatformSupportMutex);<br class="">+<br class="">+ // Clear any existing error messages.<br class="">+ dlErrorMsgs.erase(std::this_thread::get_id());<br class="">+<br class="">+ if (auto *JD = J.getExecutionSession().getJITDylibByName(Path)) {<br class="">+ auto I = JDRefCounts.find(JD);<br class="">+ if (I != JDRefCounts.end()) {<br class="">+ ++I->second;<br class="">+ return JD;<br class="">+ }<br class="">+<br class="">+ JDRefCounts[JD] = 1;<br class="">+ JDToOpen = JD;<br class="">+ }<br class="">+ }<br class="">+<br class="">+ if (JDToOpen) {<br class="">+ if (auto Err = initialize(*JDToOpen)) {<br class="">+ recordError(std::move(Err));<br class="">+ return 0;<br class="">+ }<br class="">+ }<br class="">+<br class="">+ // Fall through to dlopen if no JITDylib found for Path.<br class="">+ return DlFcn.dlopen(Path, Mode);<br class="">+ }<br class="">+<br class="">+ static void *dlopenHelper(void *Self, const char *Path, int Mode) {<br class="">+ return static_cast<MachOPlatformSupport *>(Self)->jit_dlopen(Path, Mode);<br class="">+ }<br class="">+<br class="">+ int jit_dlclose(void *Handle) {<br class="">+ JITDylib *JDToClose = nullptr;<br class="">+<br class="">+ {<br class="">+ std::lock_guard<std::mutex> Lock(PlatformSupportMutex);<br class="">+<br class="">+ // Clear any existing error messages.<br class="">+ dlErrorMsgs.erase(std::this_thread::get_id());<br class="">+<br class="">+ auto I = JDRefCounts.find(Handle);<br class="">+ if (I != JDRefCounts.end()) {<br class="">+ --I->second;<br class="">+ if (I->second == 0) {<br class="">+ JDRefCounts.erase(I);<br class="">+ JDToClose = static_cast<JITDylib *>(Handle);<br class="">+ } else<br class="">+ return 0;<br class="">+ }<br class="">+ }<br class="">+<br class="">+ if (JDToClose) {<br class="">+ if (auto Err = deinitialize(*JDToClose)) {<br class="">+ recordError(std::move(Err));<br class="">+ return -1;<br class="">+ }<br class="">+ return 0;<br class="">+ }<br class="">+<br class="">+ // Fall through to dlclose if no JITDylib found for Path.<br class="">+ return DlFcn.dlclose(Handle);<br class="">+ }<br class="">+<br class="">+ static int dlcloseHelper(void *Self, void *Handle) {<br class="">+ return static_cast<MachOPlatformSupport *>(Self)->jit_dlclose(Handle);<br class="">+ }<br class="">+<br class="">+ void *jit_dlsym(void *Handle, const char *Name) {<br class="">+ JITDylibSearchOrder JITSymSearchOrder;<br class="">+<br class="">+ // FIXME: RTLD_NEXT, RTLD_SELF not supported.<br class="">+ {<br class="">+ std::lock_guard<std::mutex> Lock(PlatformSupportMutex);<br class="">+<br class="">+ // Clear any existing error messages.<br class="">+ dlErrorMsgs.erase(std::this_thread::get_id());<br class="">+<br class="">+ if (JDRefCounts.count(Handle)) {<br class="">+ JITSymSearchOrder.push_back(<br class="">+ {static_cast<JITDylib *>(Handle),<br class="">+ JITDylibLookupFlags::MatchExportedSymbolsOnly});<br class="">+ } else if (Handle == DlFcn.RTLDDefault) {<br class="">+ for (auto &KV : JDRefCounts)<br class="">+ JITSymSearchOrder.push_back(<br class="">+ {static_cast<JITDylib *>(KV.first),<br class="">+ JITDylibLookupFlags::MatchExportedSymbolsOnly});<br class="">+ }<br class="">+ }<br class="">+<br class="">+ if (!JITSymSearchOrder.empty()) {<br class="">+ MangleAndInterner Mangle(J.getExecutionSession(), J.getDataLayout());<br class="">+ auto MangledName = Mangle(Name);<br class="">+ SymbolLookupSet Syms(MangledName,<br class="">+ SymbolLookupFlags::WeaklyReferencedSymbol);<br class="">+ if (auto Result = J.getExecutionSession().lookup(JITSymSearchOrder, Syms,<br class="">+ LookupKind::DLSym)) {<br class="">+ auto I = Result->find(MangledName);<br class="">+ if (I != Result->end())<br class="">+ return jitTargetAddressToPointer<void *>(I->second.getAddress());<br class="">+ } else {<br class="">+ recordError(Result.takeError());<br class="">+ return 0;<br class="">+ }<br class="">+ }<br class="">+<br class="">+ // Fall through to dlsym.<br class="">+ return DlFcn.dlsym(Handle, Name);<br class="">+ }<br class="">+<br class="">+ static void *dlsymHelper(void *Self, void *Handle, const char *Name) {<br class="">+ return static_cast<MachOPlatformSupport *>(Self)->jit_dlsym(Handle, Name);<br class="">+ }<br class="">+<br class="">+ const char *jit_dlerror() {<br class="">+ {<br class="">+ std::lock_guard<std::mutex> Lock(PlatformSupportMutex);<br class="">+ auto I = dlErrorMsgs.find(std::this_thread::get_id());<br class="">+ if (I != dlErrorMsgs.end())<br class="">+ return I->second->c_str();<br class="">+ }<br class="">+ return DlFcn.dlerror();<br class="">+ }<br class="">+<br class="">+ static const char *dlerrorHelper(void *Self) {<br class="">+ return static_cast<MachOPlatformSupport *>(Self)->jit_dlerror();<br class="">+ }<br class="">+<br class="">+ void recordError(Error Err) {<br class="">+ std::lock_guard<std::mutex> Lock(PlatformSupportMutex);<br class="">+ dlErrorMsgs[std::this_thread::get_id()] =<br class="">+ std::make_unique<std::string>(toString(std::move(Err)));<br class="">+ }<br class="">+<br class="">+ std::mutex PlatformSupportMutex;<br class="">+ LLJIT &J;<br class="">+ MachOPlatform &MP;<br class="">+ DlFcnValues DlFcn;<br class="">+ ItaniumCXAAtExitSupport AtExitMgr;<br class="">+ DenseMap<void *, unsigned> JDRefCounts;<br class="">+ std::map<std::thread::id, std::unique_ptr<std::string>> dlErrorMsgs;<br class="">+};<br class="">+<br class="">+} // end anonymous namespace<br class=""><br class=""> namespace llvm {<br class=""> namespace orc {<br class=""><br class="">+void LLJIT::PlatformSupport::setInitTransform(<br class="">+ LLJIT &J, IRTransformLayer::TransformFunction T) {<br class="">+ J.InitHelperTransformLayer->setTransform(std::move(T));<br class="">+}<br class="">+<br class="">+LLJIT::PlatformSupport::~PlatformSupport() {}<br class="">+<br class=""> Error LLJITBuilderState::prepareForConstruction() {<br class=""><br class=""> if (!JTMB) {<br class="">@@ -57,7 +869,7 @@ LLJIT::~LLJIT() {<br class=""> Error LLJIT::defineAbsolute(StringRef Name, JITEvaluatedSymbol Sym) {<br class=""> auto InternedName = ES->intern(Name);<br class=""> SymbolMap Symbols({{InternedName, Sym}});<br class="">- return Main.define(absoluteSymbols(std::move(Symbols)));<br class="">+ return Main->define(absoluteSymbols(std::move(Symbols)));<br class=""> }<br class=""><br class=""> Error LLJIT::addIRModule(JITDylib &JD, ThreadSafeModule TSM) {<br class="">@@ -67,7 +879,8 @@ Error LLJIT::addIRModule(JITDylib &JD, ThreadSafeModule TSM) {<br class=""> TSM.withModuleDo([&](Module &M) { return applyDataLayout(M); }))<br class=""> return Err;<br class=""><br class="">- return TransformLayer->add(JD, std::move(TSM), ES->allocateVModule());<br class="">+ return InitHelperTransformLayer->add(JD, std::move(TSM),<br class="">+ ES->allocateVModule());<br class=""> }<br class=""><br class=""> Error LLJIT::addObjectFile(JITDylib &JD, std::unique_ptr<MemoryBuffer> Obj) {<br class="">@@ -128,16 +941,23 @@ LLJIT::createCompileFunction(LLJITBuilderState &S,<br class=""> }<br class=""><br class=""> LLJIT::LLJIT(LLJITBuilderState &S, Error &Err)<br class="">- : ES(S.ES ? std::move(S.ES) : std::make_unique<ExecutionSession>()),<br class="">- Main(this->ES->createJITDylib("<main>")), DL(""),<br class="">- TT(S.JTMB->getTargetTriple()),<br class="">+ : ES(S.ES ? std::move(S.ES) : std::make_unique<ExecutionSession>()), Main(),<br class="">+ DL(""), TT(S.JTMB->getTargetTriple()),<br class=""> ObjLinkingLayer(createObjectLinkingLayer(S, *ES)),<br class="">- ObjTransformLayer(*this->ES, *ObjLinkingLayer), CtorRunner(Main),<br class="">- DtorRunner(Main) {<br class="">+ ObjTransformLayer(*this->ES, *ObjLinkingLayer) {<br class=""><br class=""> ErrorAsOutParameter _(&Err);<br class=""><br class="">- if (auto DLOrErr = S.JTMB->getDefaultDataLayoutForTarget())<br class="">+ if (auto MainOrErr = this->ES->createJITDylib("main"))<br class="">+ Main = &*MainOrErr;<br class="">+ else {<br class="">+ Err = MainOrErr.takeError();<br class="">+ return;<br class="">+ }<br class="">+<br class="">+ if (S.DL)<br class="">+ DL = std::move(*S.DL);<br class="">+ else if (auto DLOrErr = S.JTMB->getDefaultDataLayoutForTarget())<br class=""> DL = std::move(*DLOrErr);<br class=""> else {<br class=""> Err = DLOrErr.takeError();<br class="">@@ -153,10 +973,12 @@ LLJIT::LLJIT(LLJITBuilderState &S, Error &Err)<br class=""> CompileLayer = std::make_unique<IRCompileLayer>(<br class=""> *ES, ObjTransformLayer, std::move(*CompileFunction));<br class=""> TransformLayer = std::make_unique<IRTransformLayer>(*ES, *CompileLayer);<br class="">+ InitHelperTransformLayer =<br class="">+ std::make_unique<IRTransformLayer>(*ES, *TransformLayer);<br class=""> }<br class=""><br class=""> if (S.NumCompileThreads > 0) {<br class="">- TransformLayer->setCloneToNewContextOnEmit(true);<br class="">+ InitHelperTransformLayer->setCloneToNewContextOnEmit(true);<br class=""> CompileThreads =<br class=""> std::make_unique<ThreadPool>(hardware_concurrency(S.NumCompileThreads));<br class=""> ES->setDispatchMaterialization(<br class="">@@ -167,6 +989,11 @@ LLJIT::LLJIT(LLJITBuilderState &S, Error &Err)<br class=""> CompileThreads->async(std::move(Work));<br class=""> });<br class=""> }<br class="">+<br class="">+ if (S.SetUpPlatform)<br class="">+ Err = S.SetUpPlatform(*this);<br class="">+ else<br class="">+ setUpGenericLLVMIRPlatform(*this);<br class=""> }<br class=""><br class=""> std::string LLJIT::mangle(StringRef UnmangledName) {<br class="">@@ -184,15 +1011,24 @@ Error LLJIT::applyDataLayout(Module &M) {<br class=""><br class=""> if (M.getDataLayout() != DL)<br class=""> return make_error<StringError>(<br class="">- "Added modules have incompatible data layouts",<br class="">+ "Added modules have incompatible data layouts: " +<br class="">+ M.getDataLayout().getStringRepresentation() + " (module) vs " +<br class="">+ DL.getStringRepresentation() + " (jit)",<br class=""> inconvertibleErrorCode());<br class=""><br class=""> return Error::success();<br class=""> }<br class=""><br class="">-void LLJIT::recordCtorDtors(Module &M) {<br class="">- CtorRunner.add(getConstructors(M));<br class="">- DtorRunner.add(getDestructors(M));<br class="">+void setUpGenericLLVMIRPlatform(LLJIT &J) {<br class="">+ J.setPlatformSupport(std::make_unique<GenericLLVMIRPlatformSupport>(J));<br class="">+}<br class="">+<br class="">+Error setUpMachOPlatform(LLJIT &J) {<br class="">+ auto MP = MachOPlatformSupport::Create(J, J.getMainJITDylib());<br class="">+ if (!MP)<br class="">+ return MP.takeError();<br class="">+ J.setPlatformSupport(std::move(*MP));<br class="">+ return Error::success();<br class=""> }<br class=""><br class=""> Error LLLazyJITBuilderState::prepareForConstruction() {<br class="">@@ -205,13 +1041,8 @@ Error LLLazyJITBuilderState::prepareForConstruction() {<br class=""> Error LLLazyJIT::addLazyIRModule(JITDylib &JD, ThreadSafeModule TSM) {<br class=""> assert(TSM && "Can not add null module");<br class=""><br class="">- if (auto Err = TSM.withModuleDo([&](Module &M) -> Error {<br class="">- if (auto Err = applyDataLayout(M))<br class="">- return Err;<br class="">-<br class="">- recordCtorDtors(M);<br class="">- return Error::success();<br class="">- }))<br class="">+ if (auto Err = TSM.withModuleDo(<br class="">+ [&](Module &M) -> Error { return applyDataLayout(M); }))<br class=""> return Err;<br class=""><br class=""> return CODLayer->add(JD, std::move(TSM), ES->allocateVModule());<br class="">@@ -256,7 +1087,7 @@ LLLazyJIT::LLLazyJIT(LLLazyJITBuilderState &S, Error &Err) : LLJIT(S, Err) {<br class=""><br class=""> // Create the COD layer.<br class=""> CODLayer = std::make_unique<CompileOnDemandLayer>(<br class="">- *ES, *TransformLayer, *LCTMgr, std::move(ISMBuilder));<br class="">+ *ES, *InitHelperTransformLayer, *LCTMgr, std::move(ISMBuilder));<br class=""><br class=""> if (S.NumCompileThreads > 0)<br class=""> CODLayer->setCloneToNewContextOnEmit(true);<br class=""><br class="">diff --git a/llvm/lib/ExecutionEngine/Orc/Layer.cpp b/llvm/lib/ExecutionEngine/Orc/Layer.cpp<br class="">index ebc7801f11ff..63b04a0562bd 100644<br class="">--- a/llvm/lib/ExecutionEngine/Orc/Layer.cpp<br class="">+++ b/llvm/lib/ExecutionEngine/Orc/Layer.cpp<br class="">@@ -7,7 +7,10 @@<br class=""> //===----------------------------------------------------------------------===//<br class=""><br class=""> #include "llvm/ExecutionEngine/Orc/Layer.h"<br class="">+<br class="">+#include "llvm/ExecutionEngine/Orc/ExecutionUtils.h"<br class=""> #include "llvm/IR/Constants.h"<br class="">+#include "llvm/Object/MachO.h"<br class=""> #include "llvm/Object/ObjectFile.h"<br class=""> #include "llvm/Support/Debug.h"<br class=""><br class="">@@ -23,10 +26,11 @@ Error IRLayer::add(JITDylib &JD, ThreadSafeModule TSM, VModuleKey K) {<br class=""> *this, *getManglingOptions(), std::move(TSM), std::move(K)));<br class=""> }<br class=""><br class="">-IRMaterializationUnit::IRMaterializationUnit(ExecutionSession &ES,<br class="">- const ManglingOptions &MO,<br class="">- ThreadSafeModule TSM, VModuleKey K)<br class="">- : MaterializationUnit(SymbolFlagsMap(), std::move(K)), TSM(std::move(TSM)) {<br class="">+IRMaterializationUnit::IRMaterializationUnit(<br class="">+ ExecutionSession &ES, const IRSymbolMapper::ManglingOptions &MO,<br class="">+ ThreadSafeModule TSM, VModuleKey K)<br class="">+ : MaterializationUnit(SymbolFlagsMap(), nullptr, std::move(K)),<br class="">+ TSM(std::move(TSM)) {<br class=""><br class=""> assert(this->TSM && "Module must not be null");<br class=""><br class="">@@ -34,6 +38,7 @@ IRMaterializationUnit::IRMaterializationUnit(ExecutionSession &ES,<br class=""> this->TSM.withModuleDo([&](Module &M) {<br class=""> for (auto &G : M.global_values()) {<br class=""> // Skip globals that don't generate symbols.<br class="">+<br class=""> if (!G.hasName() || G.isDeclaration() || G.hasLocalLinkage() ||<br class=""> G.hasAvailableExternallyLinkage() || G.hasAppendingLinkage())<br class=""> continue;<br class="">@@ -72,13 +77,23 @@ IRMaterializationUnit::IRMaterializationUnit(ExecutionSession &ES,<br class=""> SymbolFlags[MangledName] = JITSymbolFlags::fromGlobalValue(G);<br class=""> SymbolToDefinition[MangledName] = &G;<br class=""> }<br class="">+<br class="">+ // If we need an init symbol for this module then create one.<br class="">+ if (!llvm::empty(getStaticInitGVs(M))) {<br class="">+ std::string InitSymbolName;<br class="">+ raw_string_ostream(InitSymbolName)<br class="">+ << "$." << M.getModuleIdentifier() << ".__inits";<br class="">+ InitSymbol = ES.intern(InitSymbolName);<br class="">+ SymbolFlags[InitSymbol] = JITSymbolFlags();<br class="">+ }<br class=""> });<br class=""> }<br class=""><br class=""> IRMaterializationUnit::IRMaterializationUnit(<br class=""> ThreadSafeModule TSM, VModuleKey K, SymbolFlagsMap SymbolFlags,<br class="">- SymbolNameToDefinitionMap SymbolToDefinition)<br class="">- : MaterializationUnit(std::move(SymbolFlags), std::move(K)),<br class="">+ SymbolStringPtr InitSymbol, SymbolNameToDefinitionMap SymbolToDefinition)<br class="">+ : MaterializationUnit(std::move(SymbolFlags), std::move(InitSymbol),<br class="">+ std::move(K)),<br class=""> TSM(std::move(TSM)), SymbolToDefinition(std::move(SymbolToDefinition)) {}<br class=""><br class=""> StringRef IRMaterializationUnit::getName() const {<br class="">@@ -105,7 +120,8 @@ void IRMaterializationUnit::discard(const JITDylib &JD,<br class=""> }<br class=""><br class=""> BasicIRLayerMaterializationUnit::BasicIRLayerMaterializationUnit(<br class="">- IRLayer &L, const ManglingOptions &MO, ThreadSafeModule TSM, VModuleKey K)<br class="">+ IRLayer &L, const IRSymbolMapper::ManglingOptions &MO, ThreadSafeModule TSM,<br class="">+ VModuleKey K)<br class=""> : IRMaterializationUnit(L.getExecutionSession(), MO, std::move(TSM),<br class=""> std::move(K)),<br class=""> L(L), K(std::move(K)) {}<br class="">@@ -150,22 +166,26 @@ Error ObjectLayer::add(JITDylib &JD, std::unique_ptr<MemoryBuffer> O,<br class=""> Expected<std::unique_ptr<BasicObjectLayerMaterializationUnit>><br class=""> BasicObjectLayerMaterializationUnit::Create(ObjectLayer &L, VModuleKey K,<br class=""> std::unique_ptr<MemoryBuffer> O) {<br class="">- auto SymbolFlags =<br class="">- getObjectSymbolFlags(L.getExecutionSession(), O->getMemBufferRef());<br class="">+ auto ObjSymInfo =<br class="">+ getObjectSymbolInfo(L.getExecutionSession(), O->getMemBufferRef());<br class="">+<br class="">+ if (!ObjSymInfo)<br class="">+ return ObjSymInfo.takeError();<br class=""><br class="">- if (!SymbolFlags)<br class="">- return SymbolFlags.takeError();<br class="">+ auto &SymbolFlags = ObjSymInfo->first;<br class="">+ auto &InitSymbol = ObjSymInfo->second;<br class=""><br class=""> return std::unique_ptr<BasicObjectLayerMaterializationUnit>(<br class="">- new BasicObjectLayerMaterializationUnit(L, K, std::move(O),<br class="">- std::move(*SymbolFlags)));<br class="">+ new BasicObjectLayerMaterializationUnit(<br class="">+ L, K, std::move(O), std::move(SymbolFlags), std::move(InitSymbol)));<br class=""> }<br class=""><br class=""> BasicObjectLayerMaterializationUnit::BasicObjectLayerMaterializationUnit(<br class=""> ObjectLayer &L, VModuleKey K, std::unique_ptr<MemoryBuffer> O,<br class="">- SymbolFlagsMap SymbolFlags)<br class="">- : MaterializationUnit(std::move(SymbolFlags), std::move(K)), L(L),<br class="">- O(std::move(O)) {}<br class="">+ SymbolFlagsMap SymbolFlags, SymbolStringPtr InitSymbol)<br class="">+ : MaterializationUnit(std::move(SymbolFlags), std::move(InitSymbol),<br class="">+ std::move(K)),<br class="">+ L(L), O(std::move(O)) {}<br class=""><br class=""> StringRef BasicObjectLayerMaterializationUnit::getName() const {<br class=""> if (O)<br class="">@@ -184,35 +204,5 @@ void BasicObjectLayerMaterializationUnit::discard(const JITDylib &JD,<br class=""> // filter to pass to the object layer along with the object itself.<br class=""> }<br class=""><br class="">-Expected<SymbolFlagsMap> getObjectSymbolFlags(ExecutionSession &ES,<br class="">- MemoryBufferRef ObjBuffer) {<br class="">- auto Obj = object::ObjectFile::createObjectFile(ObjBuffer);<br class="">-<br class="">- if (!Obj)<br class="">- return Obj.takeError();<br class="">-<br class="">- SymbolFlagsMap SymbolFlags;<br class="">- for (auto &Sym : (*Obj)->symbols()) {<br class="">- // Skip symbols not defined in this object file.<br class="">- if (Sym.getFlags() & object::BasicSymbolRef::SF_Undefined)<br class="">- continue;<br class="">-<br class="">- // Skip symbols that are not global.<br class="">- if (!(Sym.getFlags() & object::BasicSymbolRef::SF_Global))<br class="">- continue;<br class="">-<br class="">- auto Name = Sym.getName();<br class="">- if (!Name)<br class="">- return Name.takeError();<br class="">- auto InternedName = ES.intern(*Name);<br class="">- auto SymFlags = JITSymbolFlags::fromObjectSymbol(Sym);<br class="">- if (!SymFlags)<br class="">- return SymFlags.takeError();<br class="">- SymbolFlags[InternedName] = std::move(*SymFlags);<br class="">- }<br class="">-<br class="">- return SymbolFlags;<br class="">-}<br class="">-<br class=""> } // End namespace orc.<br class=""> } // End namespace llvm.<br class=""><br class="">diff --git a/llvm/lib/ExecutionEngine/Orc/LazyReexports.cpp b/llvm/lib/ExecutionEngine/Orc/LazyReexports.cpp<br class="">index 97f36ede713f..27d08e18270f 100644<br class="">--- a/llvm/lib/ExecutionEngine/Orc/LazyReexports.cpp<br class="">+++ b/llvm/lib/ExecutionEngine/Orc/LazyReexports.cpp<br class="">@@ -51,7 +51,7 @@ LazyCallThroughManager::callThroughToSymbol(JITTargetAddress TrampolineAddr) {<br class=""><br class=""> auto LookupResult = ES.lookup(<br class=""> makeJITDylibSearchOrder(SourceJD, JITDylibLookupFlags::MatchAllSymbols),<br class="">- SymbolName);<br class="">+ SymbolName, SymbolState::Ready);<br class=""><br class=""> if (!LookupResult) {<br class=""> ES.reportError(LookupResult.takeError());<br class="">@@ -123,10 +123,9 @@ LazyReexportsMaterializationUnit::LazyReexportsMaterializationUnit(<br class=""> LazyCallThroughManager &LCTManager, IndirectStubsManager &ISManager,<br class=""> JITDylib &SourceJD, SymbolAliasMap CallableAliases, ImplSymbolMap *SrcJDLoc,<br class=""> VModuleKey K)<br class="">- : MaterializationUnit(extractFlags(CallableAliases), std::move(K)),<br class="">+ : MaterializationUnit(extractFlags(CallableAliases), nullptr, std::move(K)),<br class=""> LCTManager(LCTManager), ISManager(ISManager), SourceJD(SourceJD),<br class="">- CallableAliases(std::move(CallableAliases)),<br class="">- AliaseeTable(SrcJDLoc) {}<br class="">+ CallableAliases(std::move(CallableAliases)), AliaseeTable(SrcJDLoc) {}<br class=""><br class=""> StringRef LazyReexportsMaterializationUnit::getName() const {<br class=""> return "<Lazy Reexports>";<br class=""><br class="">diff --git a/llvm/lib/ExecutionEngine/Orc/MachOPlatform.cpp b/llvm/lib/ExecutionEngine/Orc/MachOPlatform.cpp<br class="">new file mode 100644<br class="">index 000000000000..4618d53315ef<br class="">--- /dev/null<br class="">+++ b/llvm/lib/ExecutionEngine/Orc/MachOPlatform.cpp<br class="">@@ -0,0 +1,459 @@<br class="">+//===------ MachOPlatform.cpp - Utilities for executing MachO in Orc ------===//<br class="">+//<br class="">+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.<br class="">+// See <a href="https://llvm.org/LICENSE.txt" class="">https://llvm.org/LICENSE.txt</a> for license information.<br class="">+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception<br class="">+//<br class="">+//===----------------------------------------------------------------------===//<br class="">+<br class="">+#include "llvm/ExecutionEngine/Orc/MachOPlatform.h"<br class="">+<br class="">+#include "llvm/BinaryFormat/MachO.h"<br class="">+#include "llvm/Support/BinaryByteStream.h"<br class="">+<br class="">+namespace {<br class="">+<br class="">+struct objc_class;<br class="">+struct objc_image_info;<br class="">+struct objc_object;<br class="">+struct objc_selector;<br class="">+<br class="">+using Class = objc_class *;<br class="">+using id = objc_object *;<br class="">+using SEL = objc_selector *;<br class="">+<br class="">+using ObjCMsgSendTy = id (*)(id, SEL, ...);<br class="">+using ObjCReadClassPairTy = Class (*)(Class, const objc_image_info *);<br class="">+using SelRegisterNameTy = SEL (*)(const char *);<br class="">+<br class="">+enum class ObjCRegistrationAPI { Uninitialized, Unavailable, Initialized };<br class="">+<br class="">+ObjCRegistrationAPI ObjCRegistrationAPIState =<br class="">+ ObjCRegistrationAPI::Uninitialized;<br class="">+ObjCMsgSendTy objc_msgSend = nullptr;<br class="">+ObjCReadClassPairTy objc_readClassPair = nullptr;<br class="">+SelRegisterNameTy sel_registerName = nullptr;<br class="">+<br class="">+} // end anonymous namespace<br class="">+<br class="">+namespace llvm {<br class="">+namespace orc {<br class="">+<br class="">+template <typename FnTy><br class="">+static Error setUpObjCRegAPIFunc(FnTy &Target, sys::DynamicLibrary &LibObjC,<br class="">+ const char *Name) {<br class="">+ if (void *Addr = LibObjC.getAddressOfSymbol(Name))<br class="">+ Target = reinterpret_cast<FnTy>(Addr);<br class="">+ else<br class="">+ return make_error<StringError>(<br class="">+ (Twine("Could not find address for ") + Name).str(),<br class="">+ inconvertibleErrorCode());<br class="">+ return Error::success();<br class="">+}<br class="">+<br class="">+Error enableObjCRegistration(const char *PathToLibObjC) {<br class="">+ // If we've already tried to initialize then just bail out.<br class="">+ if (ObjCRegistrationAPIState != ObjCRegistrationAPI::Uninitialized)<br class="">+ return Error::success();<br class="">+<br class="">+ ObjCRegistrationAPIState = ObjCRegistrationAPI::Unavailable;<br class="">+<br class="">+ std::string ErrMsg;<br class="">+ auto LibObjC =<br class="">+ sys::DynamicLibrary::getPermanentLibrary(PathToLibObjC, &ErrMsg);<br class="">+<br class="">+ if (!LibObjC.isValid())<br class="">+ return make_error<StringError>(std::move(ErrMsg), inconvertibleErrorCode());<br class="">+<br class="">+ if (auto Err = setUpObjCRegAPIFunc(objc_msgSend, LibObjC, "objc_msgSend"))<br class="">+ return Err;<br class="">+ if (auto Err = setUpObjCRegAPIFunc(objc_readClassPair, LibObjC,<br class="">+ "objc_readClassPair"))<br class="">+ return Err;<br class="">+ if (auto Err =<br class="">+ setUpObjCRegAPIFunc(sel_registerName, LibObjC, "sel_registerName"))<br class="">+ return Err;<br class="">+<br class="">+ ObjCRegistrationAPIState = ObjCRegistrationAPI::Initialized;<br class="">+ return Error::success();<br class="">+}<br class="">+<br class="">+bool objcRegistrationEnabled() {<br class="">+ return ObjCRegistrationAPIState == ObjCRegistrationAPI::Initialized;<br class="">+}<br class="">+<br class="">+void MachOJITDylibInitializers::runModInits() const {<br class="">+ for (const auto &ModInit : ModInitSections) {<br class="">+ for (uint64_t I = 0; I != ModInit.NumPtrs; ++I) {<br class="">+ auto *InitializerAddr = jitTargetAddressToPointer<uintptr_t *>(<br class="">+ ModInit.Address + (I * sizeof(uintptr_t)));<br class="">+ auto *Initializer =<br class="">+ jitTargetAddressToFunction<void (*)()>(*InitializerAddr);<br class="">+ Initializer();<br class="">+ }<br class="">+ }<br class="">+}<br class="">+<br class="">+void MachOJITDylibInitializers::registerObjCSelectors() const {<br class="">+ assert(objcRegistrationEnabled() && "ObjC registration not enabled.");<br class="">+<br class="">+ for (const auto &ObjCSelRefs : ObjCSelRefsSections) {<br class="">+ for (uint64_t I = 0; I != ObjCSelRefs.NumPtrs; ++I) {<br class="">+ auto SelEntryAddr = ObjCSelRefs.Address + (I * sizeof(uintptr_t));<br class="">+ const auto *SelName =<br class="">+ *jitTargetAddressToPointer<const char **>(SelEntryAddr);<br class="">+ auto Sel = sel_registerName(SelName);<br class="">+ *jitTargetAddressToPointer<SEL *>(SelEntryAddr) = Sel;<br class="">+ }<br class="">+ }<br class="">+}<br class="">+<br class="">+Error MachOJITDylibInitializers::registerObjCClasses() const {<br class="">+ assert(objcRegistrationEnabled() && "ObjC registration not enabled.");<br class="">+<br class="">+ struct ObjCClassCompiled {<br class="">+ void *Metaclass;<br class="">+ void *Parent;<br class="">+ void *Cache1;<br class="">+ void *Cache2;<br class="">+ void *Data;<br class="">+ };<br class="">+<br class="">+ auto *ImageInfo =<br class="">+ jitTargetAddressToPointer<const objc_image_info *>(ObjCImageInfoAddr);<br class="">+ auto ClassSelector = sel_registerName("class");<br class="">+<br class="">+ for (const auto &ObjCClassList : ObjCClassListSections) {<br class="">+ for (uint64_t I = 0; I != ObjCClassList.NumPtrs; ++I) {<br class="">+ auto ClassPtrAddr = ObjCClassList.Address + (I * sizeof(uintptr_t));<br class="">+ auto Cls = *jitTargetAddressToPointer<Class *>(ClassPtrAddr);<br class="">+ auto *ClassCompiled =<br class="">+ *jitTargetAddressToPointer<ObjCClassCompiled **>(ClassPtrAddr);<br class="">+ objc_msgSend(reinterpret_cast<id>(ClassCompiled->Parent), ClassSelector);<br class="">+ auto Registered = objc_readClassPair(Cls, ImageInfo);<br class="">+<br class="">+ // FIXME: Improve diagnostic by reporting the failed class's name.<br class="">+ if (Registered != Cls)<br class="">+ return make_error<StringError>("Unable to register Objective-C class",<br class="">+ inconvertibleErrorCode());<br class="">+ }<br class="">+ }<br class="">+ return Error::success();<br class="">+}<br class="">+<br class="">+void MachOJITDylibInitializers::dump() const {<br class="">+ for (auto &Extent : ModInitSections)<br class="">+ dbgs() << formatv("{0:x16}", Extent.Address) << " -- "<br class="">+ << formatv("{0:x16}", Extent.Address + 8 * Extent.NumPtrs) << "\n";<br class="">+}<br class="">+<br class="">+MachOPlatform::MachOPlatform(<br class="">+ ExecutionSession &ES, ObjectLinkingLayer &ObjLinkingLayer,<br class="">+ std::unique_ptr<MemoryBuffer> StandardSymbolsObject)<br class="">+ : ES(ES), ObjLinkingLayer(ObjLinkingLayer),<br class="">+ StandardSymbolsObject(std::move(StandardSymbolsObject)) {<br class="">+ ObjLinkingLayer.addPlugin(std::make_unique<InitScraperPlugin>(*this));<br class="">+}<br class="">+<br class="">+Error MachOPlatform::setupJITDylib(JITDylib &JD) {<br class="">+ auto ObjBuffer = MemoryBuffer::getMemBuffer(<br class="">+ StandardSymbolsObject->getMemBufferRef(), false);<br class="">+ return ObjLinkingLayer.add(JD, std::move(ObjBuffer));<br class="">+}<br class="">+<br class="">+Error MachOPlatform::notifyAdding(JITDylib &JD, const MaterializationUnit &MU) {<br class="">+ const auto &InitSym = MU.getInitializerSymbol();<br class="">+ if (!InitSym)<br class="">+ return Error::success();<br class="">+<br class="">+ std::lock_guard<std::mutex> Lock(PlatformMutex);<br class="">+ RegisteredInitSymbols[&JD].add(InitSym);<br class="">+ return Error::success();<br class="">+}<br class="">+<br class="">+Error MachOPlatform::notifyRemoving(JITDylib &JD, VModuleKey K) {<br class="">+ llvm_unreachable("Not supported yet");<br class="">+}<br class="">+<br class="">+Expected<MachOPlatform::InitializerSequence><br class="">+MachOPlatform::getInitializerSequence(JITDylib &JD) {<br class="">+<br class="">+ std::vector<JITDylib *> DFSLinkOrder;<br class="">+<br class="">+ while (true) {<br class="">+ // Lock the platform while we search for any initializer symbols to<br class="">+ // look up.<br class="">+ DenseMap<JITDylib *, SymbolLookupSet> NewInitSymbols;<br class="">+ {<br class="">+ std::lock_guard<std::mutex> Lock(PlatformMutex);<br class="">+ DFSLinkOrder = getDFSLinkOrder(JD);<br class="">+<br class="">+ for (auto *InitJD : DFSLinkOrder) {<br class="">+ auto RISItr = RegisteredInitSymbols.find(InitJD);<br class="">+ if (RISItr != RegisteredInitSymbols.end()) {<br class="">+ NewInitSymbols[InitJD] = std::move(RISItr->second);<br class="">+ RegisteredInitSymbols.erase(RISItr);<br class="">+ }<br class="">+ }<br class="">+ }<br class="">+<br class="">+ if (NewInitSymbols.empty())<br class="">+ break;<br class="">+<br class="">+ // Outside the lock, issue the lookup.<br class="">+ if (auto R = lookupInitSymbols(JD.getExecutionSession(), NewInitSymbols))<br class="">+ ; // Nothing to do in the success case.<br class="">+ else<br class="">+ return R.takeError();<br class="">+ }<br class="">+<br class="">+ // Lock again to collect the initializers.<br class="">+ InitializerSequence FullInitSeq;<br class="">+ {<br class="">+ std::lock_guard<std::mutex> Lock(PlatformMutex);<br class="">+ for (auto *InitJD : reverse(DFSLinkOrder)) {<br class="">+ auto ISItr = InitSeqs.find(InitJD);<br class="">+ if (ISItr != InitSeqs.end()) {<br class="">+ FullInitSeq.emplace_back(InitJD, std::move(ISItr->second));<br class="">+ InitSeqs.erase(ISItr);<br class="">+ }<br class="">+ }<br class="">+ }<br class="">+<br class="">+ return FullInitSeq;<br class="">+}<br class="">+<br class="">+Expected<MachOPlatform::DeinitializerSequence><br class="">+MachOPlatform::getDeinitializerSequence(JITDylib &JD) {<br class="">+ std::vector<JITDylib *> DFSLinkOrder = getDFSLinkOrder(JD);<br class="">+<br class="">+ DeinitializerSequence FullDeinitSeq;<br class="">+ {<br class="">+ std::lock_guard<std::mutex> Lock(PlatformMutex);<br class="">+ for (auto *DeinitJD : DFSLinkOrder) {<br class="">+ FullDeinitSeq.emplace_back(DeinitJD, MachOJITDylibDeinitializers());<br class="">+ }<br class="">+ }<br class="">+<br class="">+ return FullDeinitSeq;<br class="">+}<br class="">+<br class="">+std::vector<JITDylib *> MachOPlatform::getDFSLinkOrder(JITDylib &JD) {<br class="">+ std::vector<JITDylib *> Result, WorkStack({&JD});<br class="">+ DenseSet<JITDylib *> Visited;<br class="">+<br class="">+ while (!WorkStack.empty()) {<br class="">+ auto *NextJD = WorkStack.back();<br class="">+ WorkStack.pop_back();<br class="">+ if (Visited.count(NextJD))<br class="">+ continue;<br class="">+ Visited.insert(NextJD);<br class="">+ Result.push_back(NextJD);<br class="">+ NextJD->withSearchOrderDo([&](const JITDylibSearchOrder &SO) {<br class="">+ for (auto &KV : SO)<br class="">+ WorkStack.push_back(KV.first);<br class="">+ });<br class="">+ }<br class="">+<br class="">+ return Result;<br class="">+}<br class="">+<br class="">+void MachOPlatform::registerInitInfo(<br class="">+ JITDylib &JD, JITTargetAddress ObjCImageInfoAddr,<br class="">+ MachOJITDylibInitializers::SectionExtent ModInits,<br class="">+ MachOJITDylibInitializers::SectionExtent ObjCSelRefs,<br class="">+ MachOJITDylibInitializers::SectionExtent ObjCClassList) {<br class="">+ std::lock_guard<std::mutex> Lock(PlatformMutex);<br class="">+<br class="">+ auto &InitSeq = InitSeqs[&JD];<br class="">+<br class="">+ InitSeq.setObjCImageInfoAddr(ObjCImageInfoAddr);<br class="">+<br class="">+ if (ModInits.Address)<br class="">+ InitSeq.addModInitsSection(std::move(ModInits));<br class="">+<br class="">+ if (ObjCSelRefs.Address)<br class="">+ InitSeq.addObjCSelRefsSection(std::move(ObjCSelRefs));<br class="">+<br class="">+ if (ObjCClassList.Address)<br class="">+ InitSeq.addObjCClassListSection(std::move(ObjCClassList));<br class="">+}<br class="">+<br class="">+static Expected<MachOJITDylibInitializers::SectionExtent><br class="">+getSectionExtent(jitlink::LinkGraph &G, StringRef SectionName) {<br class="">+ auto *Sec = G.findSectionByName(SectionName);<br class="">+ if (!Sec)<br class="">+ return MachOJITDylibInitializers::SectionExtent();<br class="">+ jitlink::SectionRange R(*Sec);<br class="">+ if (R.getSize() % G.getPointerSize() != 0)<br class="">+ return make_error<StringError>(SectionName + " section size is not a "<br class="">+ "multiple of the pointer size",<br class="">+ inconvertibleErrorCode());<br class="">+ return MachOJITDylibInitializers::SectionExtent(<br class="">+ R.getStart(), R.getSize() / G.getPointerSize());<br class="">+}<br class="">+<br class="">+void MachOPlatform::InitScraperPlugin::modifyPassConfig(<br class="">+ MaterializationResponsibility &MR, const Triple &TT,<br class="">+ jitlink::PassConfiguration &Config) {<br class="">+<br class="">+ Config.PrePrunePasses.push_back([this, &MR](jitlink::LinkGraph &G) -> Error {<br class="">+ JITLinkSymbolVector InitSectionSymbols;<br class="">+ preserveInitSectionIfPresent(InitSectionSymbols, G, "__mod_init_func");<br class="">+ preserveInitSectionIfPresent(InitSectionSymbols, G, "__objc_selrefs");<br class="">+ preserveInitSectionIfPresent(InitSectionSymbols, G, "__objc_classlist");<br class="">+<br class="">+ if (!InitSymbolDeps.empty()) {<br class="">+ std::lock_guard<std::mutex> Lock(InitScraperMutex);<br class="">+ InitSymbolDeps[&MR] = std::move(InitSectionSymbols);<br class="">+ }<br class="">+<br class="">+ if (auto Err = processObjCImageInfo(G, MR))<br class="">+ return Err;<br class="">+<br class="">+ return Error::success();<br class="">+ });<br class="">+<br class="">+ Config.PostFixupPasses.push_back([this, &JD = MR.getTargetJITDylib()](<br class="">+ jitlink::LinkGraph &G) -> Error {<br class="">+ MachOJITDylibInitializers::SectionExtent ModInits, ObjCSelRefs,<br class="">+ ObjCClassList;<br class="">+<br class="">+ JITTargetAddress ObjCImageInfoAddr = 0;<br class="">+ if (auto *ObjCImageInfoSec = G.findSectionByName("__objc_image_info")) {<br class="">+ if (auto Addr = jitlink::SectionRange(*ObjCImageInfoSec).getStart()) {<br class="">+ ObjCImageInfoAddr = Addr;<br class="">+ dbgs() << "Recorded __objc_imageinfo @ " << formatv("{0:x16}", Addr);<br class="">+ }<br class="">+ }<br class="">+<br class="">+ // Record __mod_init_func.<br class="">+ if (auto ModInitsOrErr = getSectionExtent(G, "__mod_init_func"))<br class="">+ ModInits = std::move(*ModInitsOrErr);<br class="">+ else<br class="">+ return ModInitsOrErr.takeError();<br class="">+<br class="">+ // Record __objc_selrefs.<br class="">+ if (auto ObjCSelRefsOrErr = getSectionExtent(G, "__objc_selrefs"))<br class="">+ ObjCSelRefs = std::move(*ObjCSelRefsOrErr);<br class="">+ else<br class="">+ return ObjCSelRefsOrErr.takeError();<br class="">+<br class="">+ // Record __objc_classlist.<br class="">+ if (auto ObjCClassListOrErr = getSectionExtent(G, "__objc_classlist"))<br class="">+ ObjCClassList = std::move(*ObjCClassListOrErr);<br class="">+ else<br class="">+ return ObjCClassListOrErr.takeError();<br class="">+<br class="">+ MP.registerInitInfo(JD, ObjCImageInfoAddr, std::move(ModInits),<br class="">+ std::move(ObjCSelRefs), std::move(ObjCClassList));<br class="">+<br class="">+ return Error::success();<br class="">+ });<br class="">+}<br class="">+<br class="">+ObjectLinkingLayer::Plugin::LocalDependenciesMap<br class="">+MachOPlatform::InitScraperPlugin::getSyntheticSymbolLocalDependencies(<br class="">+ MaterializationResponsibility &MR) {<br class="">+ std::lock_guard<std::mutex> Lock(InitScraperMutex);<br class="">+ auto I = InitSymbolDeps.find(&MR);<br class="">+ if (I != InitSymbolDeps.end()) {<br class="">+ LocalDependenciesMap Result;<br class="">+ Result[MR.getInitializerSymbol()] = std::move(I->second);<br class="">+ InitSymbolDeps.erase(&MR);<br class="">+ return Result;<br class="">+ }<br class="">+ return LocalDependenciesMap();<br class="">+}<br class="">+<br class="">+void MachOPlatform::InitScraperPlugin::preserveInitSectionIfPresent(<br class="">+ JITLinkSymbolVector &Symbols, jitlink::LinkGraph &G,<br class="">+ StringRef SectionName) {<br class="">+ if (auto *Sec = G.findSectionByName(SectionName)) {<br class="">+ auto SecBlocks = Sec->blocks();<br class="">+ if (!llvm::empty(SecBlocks))<br class="">+ Symbols.push_back(<br class="">+ &G.addAnonymousSymbol(**SecBlocks.begin(), 0, 0, false, true));<br class="">+ }<br class="">+}<br class="">+<br class="">+Error MachOPlatform::InitScraperPlugin::processObjCImageInfo(<br class="">+ jitlink::LinkGraph &G, MaterializationResponsibility &MR) {<br class="">+<br class="">+ // If there's an ObjC imagine info then either<br class="">+ // (1) It's the first __objc_imageinfo we've seen in this JITDylib. In<br class="">+ // this case we name and record it.<br class="">+ // OR<br class="">+ // (2) We already have a recorded __objc_imageinfo for this JITDylib,<br class="">+ // in which case we just verify it.<br class="">+ auto *ObjCImageInfo = G.findSectionByName("__objc_imageinfo");<br class="">+ if (!ObjCImageInfo)<br class="">+ return Error::success();<br class="">+<br class="">+ auto ObjCImageInfoBlocks = ObjCImageInfo->blocks();<br class="">+<br class="">+ // Check that the section is not empty if present.<br class="">+ if (llvm::empty(ObjCImageInfoBlocks))<br class="">+ return make_error<StringError>("Empty __objc_imageinfo section in " +<br class="">+ G.getName(),<br class="">+ inconvertibleErrorCode());<br class="">+<br class="">+ // Check that there's only one block in the section.<br class="">+ if (std::next(ObjCImageInfoBlocks.begin()) != ObjCImageInfoBlocks.end())<br class="">+ return make_error<StringError>("Multiple blocks in __objc_imageinfo "<br class="">+ "section in " +<br class="">+ G.getName(),<br class="">+ inconvertibleErrorCode());<br class="">+<br class="">+ // Check that the __objc_imageinfo section is unreferenced.<br class="">+ // FIXME: We could optimize this check if Symbols had a ref-count.<br class="">+ for (auto &Sec : G.sections()) {<br class="">+ if (&Sec != ObjCImageInfo)<br class="">+ for (auto *B : Sec.blocks())<br class="">+ for (auto &E : B->edges())<br class="">+ if (E.getTarget().isDefined() &&<br class="">+ &E.getTarget().getBlock().getSection() == ObjCImageInfo)<br class="">+ return make_error<StringError>("__objc_imageinfo is referenced "<br class="">+ "within file " +<br class="">+ G.getName(),<br class="">+ inconvertibleErrorCode());<br class="">+ }<br class="">+<br class="">+ auto &ObjCImageInfoBlock = **ObjCImageInfoBlocks.begin();<br class="">+ auto *ObjCImageInfoData = ObjCImageInfoBlock.getContent().data();<br class="">+ auto Version = support::endian::read32(ObjCImageInfoData, G.getEndianness());<br class="">+ auto Flags =<br class="">+ support::endian::read32(ObjCImageInfoData + 4, G.getEndianness());<br class="">+<br class="">+ // Lock the mutex while we verify / update the ObjCImageInfos map.<br class="">+ std::lock_guard<std::mutex> Lock(InitScraperMutex);<br class="">+<br class="">+ auto ObjCImageInfoItr = ObjCImageInfos.find(&MR.getTargetJITDylib());<br class="">+ if (ObjCImageInfoItr != ObjCImageInfos.end()) {<br class="">+ // We've already registered an __objc_imageinfo section. Verify the<br class="">+ // content of this new section matches, then delete it.<br class="">+ if (ObjCImageInfoItr->second.first != Version)<br class="">+ return make_error<StringError>(<br class="">+ "ObjC version in " + G.getName() +<br class="">+ " does not match first registered version",<br class="">+ inconvertibleErrorCode());<br class="">+ if (ObjCImageInfoItr->second.second != Flags)<br class="">+ return make_error<StringError>("ObjC flags in " + G.getName() +<br class="">+ " do not match first registered flags",<br class="">+ inconvertibleErrorCode());<br class="">+<br class="">+ // __objc_imageinfo is valid. Delete the block.<br class="">+ for (auto *S : ObjCImageInfo->symbols())<br class="">+ G.removeDefinedSymbol(*S);<br class="">+ G.removeBlock(ObjCImageInfoBlock);<br class="">+ } else {<br class="">+ // We haven't registered an __objc_imageinfo section yet. Register and<br class="">+ // move on. The section should already be marked no-dead-strip.<br class="">+ ObjCImageInfos[&MR.getTargetJITDylib()] = std::make_pair(Version, Flags);<br class="">+ }<br class="">+<br class="">+ return Error::success();<br class="">+}<br class="">+<br class="">+} // End namespace orc.<br class="">+} // End namespace llvm.<br class=""><br class="">diff --git a/llvm/lib/ExecutionEngine/Orc/Mangling.cpp b/llvm/lib/ExecutionEngine/Orc/Mangling.cpp<br class="">new file mode 100644<br class="">index 000000000000..7ec7869d00b5<br class="">--- /dev/null<br class="">+++ b/llvm/lib/ExecutionEngine/Orc/Mangling.cpp<br class="">@@ -0,0 +1,131 @@<br class="">+//===----------- Mangling.cpp -- Name Mangling Utilities for ORC ----------===//<br class="">+//<br class="">+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.<br class="">+// See <a href="https://llvm.org/LICENSE.txt" class="">https://llvm.org/LICENSE.txt</a> for license information.<br class="">+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception<br class="">+//<br class="">+//===----------------------------------------------------------------------===//<br class="">+<br class="">+#include "llvm/ExecutionEngine/Orc/Mangling.h"<br class="">+#include "llvm/IR/Constants.h"<br class="">+#include "llvm/IR/Mangler.h"<br class="">+#include "llvm/Object/MachO.h"<br class="">+#include "llvm/Object/ObjectFile.h"<br class="">+#include "llvm/Support/Debug.h"<br class="">+<br class="">+#define DEBUG_TYPE "orc"<br class="">+<br class="">+namespace llvm {<br class="">+namespace orc {<br class="">+<br class="">+MangleAndInterner::MangleAndInterner(ExecutionSession &ES, const DataLayout &DL)<br class="">+ : ES(ES), DL(DL) {}<br class="">+<br class="">+SymbolStringPtr MangleAndInterner::operator()(StringRef Name) {<br class="">+ std::string MangledName;<br class="">+ {<br class="">+ raw_string_ostream MangledNameStream(MangledName);<br class="">+ Mangler::getNameWithPrefix(MangledNameStream, Name, DL);<br class="">+ }<br class="">+ return ES.intern(MangledName);<br class="">+}<br class="">+<br class="">+void IRSymbolMapper::add(ExecutionSession &ES, const ManglingOptions &MO,<br class="">+ ArrayRef<GlobalValue *> GVs,<br class="">+ SymbolFlagsMap &SymbolFlags,<br class="">+ SymbolNameToDefinitionMap *SymbolToDefinition) {<br class="">+ if (GVs.empty())<br class="">+ return;<br class="">+<br class="">+ MangleAndInterner Mangle(ES, GVs[0]->getParent()->getDataLayout());<br class="">+ for (auto *G : GVs) {<br class="">+ assert(G && "GVs cannot contain null elements");<br class="">+ if (!G->hasName() || G->isDeclaration() || G->hasLocalLinkage() ||<br class="">+ G->hasAvailableExternallyLinkage() || G->hasAppendingLinkage())<br class="">+ continue;<br class="">+<br class="">+ if (G->isThreadLocal() && MO.EmulatedTLS) {<br class="">+ auto *GV = cast<GlobalVariable>(G);<br class="">+<br class="">+ auto Flags = JITSymbolFlags::fromGlobalValue(*GV);<br class="">+<br class="">+ auto EmuTLSV = Mangle(("__emutls_v." + GV->getName()).str());<br class="">+ SymbolFlags[EmuTLSV] = Flags;<br class="">+ if (SymbolToDefinition)<br class="">+ (*SymbolToDefinition)[EmuTLSV] = GV;<br class="">+<br class="">+ // If this GV has a non-zero initializer we'll need to emit an<br class="">+ // __emutls.t symbol too.<br class="">+ if (GV->hasInitializer()) {<br class="">+ const auto *InitVal = GV->getInitializer();<br class="">+<br class="">+ // Skip zero-initializers.<br class="">+ if (isa<ConstantAggregateZero>(InitVal))<br class="">+ continue;<br class="">+ const auto *InitIntValue = dyn_cast<ConstantInt>(InitVal);<br class="">+ if (InitIntValue && InitIntValue->isZero())<br class="">+ continue;<br class="">+<br class="">+ auto EmuTLST = Mangle(("__emutls_t." + GV->getName()).str());<br class="">+ SymbolFlags[EmuTLST] = Flags;<br class="">+ if (SymbolToDefinition)<br class="">+ (*SymbolToDefinition)[EmuTLST] = GV;<br class="">+ }<br class="">+ continue;<br class="">+ }<br class="">+<br class="">+ // Otherwise we just need a normal linker mangling.<br class="">+ auto MangledName = Mangle(G->getName());<br class="">+ SymbolFlags[MangledName] = JITSymbolFlags::fromGlobalValue(*G);<br class="">+ if (SymbolToDefinition)<br class="">+ (*SymbolToDefinition)[MangledName] = G;<br class="">+ }<br class="">+}<br class="">+<br class="">+Expected<std::pair<SymbolFlagsMap, SymbolStringPtr>><br class="">+getObjectSymbolInfo(ExecutionSession &ES, MemoryBufferRef ObjBuffer) {<br class="">+ auto Obj = object::ObjectFile::createObjectFile(ObjBuffer);<br class="">+<br class="">+ if (!Obj)<br class="">+ return Obj.takeError();<br class="">+<br class="">+ SymbolFlagsMap SymbolFlags;<br class="">+ for (auto &Sym : (*Obj)->symbols()) {<br class="">+ // Skip symbols not defined in this object file.<br class="">+ if (Sym.getFlags() & object::BasicSymbolRef::SF_Undefined)<br class="">+ continue;<br class="">+<br class="">+ // Skip symbols that are not global.<br class="">+ if (!(Sym.getFlags() & object::BasicSymbolRef::SF_Global))<br class="">+ continue;<br class="">+<br class="">+ auto Name = Sym.getName();<br class="">+ if (!Name)<br class="">+ return Name.takeError();<br class="">+ auto InternedName = ES.intern(*Name);<br class="">+ auto SymFlags = JITSymbolFlags::fromObjectSymbol(Sym);<br class="">+ if (!SymFlags)<br class="">+ return SymFlags.takeError();<br class="">+ SymbolFlags[InternedName] = std::move(*SymFlags);<br class="">+ }<br class="">+<br class="">+ SymbolStringPtr InitSymbol;<br class="">+<br class="">+ if (auto *MachOObj = dyn_cast<object::MachOObjectFile>(Obj->get())) {<br class="">+ for (auto &Sec : MachOObj->sections()) {<br class="">+ auto SecType = MachOObj->getSectionType(Sec);<br class="">+ if ((SecType & MachO::SECTION_TYPE) == MachO::S_MOD_INIT_FUNC_POINTERS) {<br class="">+ std::string InitSymString;<br class="">+ raw_string_ostream(InitSymString)<br class="">+ << "$." << ObjBuffer.getBufferIdentifier() << ".__inits";<br class="">+ InitSymbol = ES.intern(InitSymString);<br class="">+ break;<br class="">+ }<br class="">+ }<br class="">+ }<br class="">+<br class="">+ return std::make_pair(std::move(SymbolFlags), std::move(InitSymbol));<br class="">+}<br class="">+<br class="">+} // End namespace orc.<br class="">+} // End namespace llvm.<br class=""><br class="">diff --git a/llvm/lib/ExecutionEngine/Orc/ObjectLinkingLayer.cpp b/llvm/lib/ExecutionEngine/Orc/ObjectLinkingLayer.cpp<br class="">index 6575b0a6ccb2..db88ec8517bf 100644<br class="">--- a/llvm/lib/ExecutionEngine/Orc/ObjectLinkingLayer.cpp<br class="">+++ b/llvm/lib/ExecutionEngine/Orc/ObjectLinkingLayer.cpp<br class="">@@ -144,6 +144,10 @@ class ObjectLinkingLayerJITLinkContext final : public JITLinkContext {<br class=""> if (!ExtraSymbolsToClaim.empty())<br class=""> if (auto Err = MR.defineMaterializing(ExtraSymbolsToClaim))<br class=""> return notifyFailed(std::move(Err));<br class="">+<br class="">+ if (const auto &InitSym = MR.getInitializerSymbol())<br class="">+ InternedResult[InitSym] = JITEvaluatedSymbol();<br class="">+<br class=""> if (auto Err = MR.notifyResolved(InternedResult)) {<br class=""> Layer.getExecutionSession().reportError(std::move(Err));<br class=""> MR.failMaterialization();<br class="">@@ -184,8 +188,12 @@ class ObjectLinkingLayerJITLinkContext final : public JITLinkContext {<br class=""> }<br class=""><br class=""> private:<br class="">- using JITLinkSymbolSet = DenseSet<const Symbol *>;<br class="">- using LocalToNamedDependenciesMap = DenseMap<const Symbol *, JITLinkSymbolSet>;<br class="">+ struct LocalSymbolNamedDependencies {<br class="">+ SymbolNameSet Internal, External;<br class="">+ };<br class="">+<br class="">+ using LocalSymbolNamedDependenciesMap =<br class="">+ DenseMap<const Symbol *, LocalSymbolNamedDependencies>;<br class=""><br class=""> Error externalizeWeakAndCommonSymbols(LinkGraph &G) {<br class=""> auto &ES = Layer.getExecutionSession();<br class="">@@ -216,6 +224,7 @@ class ObjectLinkingLayerJITLinkContext final : public JITLinkContext {<br class=""> auto &ES = MR.getTargetJITDylib().getExecutionSession();<br class=""> auto LocalDeps = computeLocalDeps(G);<br class=""><br class="">+ // Compute dependencies for symbols defined in the JITLink graph.<br class=""> for (auto *Sym : G.defined_symbols()) {<br class=""><br class=""> // Skip local symbols: we do not track dependencies for these.<br class="">@@ -239,15 +248,12 @@ class ObjectLinkingLayerJITLinkContext final : public JITLinkContext {<br class=""> assert(TargetSym.isDefined() &&<br class=""> "local symbols must be defined");<br class=""> auto I = LocalDeps.find(&TargetSym);<br class="">- if (I != LocalDeps.end())<br class="">- for (auto &S : I->second) {<br class="">- assert(S->hasName() &&<br class="">- "LocalDeps should only contain named values");<br class="">- if (S->isExternal())<br class="">- ExternalSymDeps.insert(ES.intern(S->getName()));<br class="">- else if (S != Sym)<br class="">- InternalSymDeps.insert(ES.intern(S->getName()));<br class="">- }<br class="">+ if (I != LocalDeps.end()) {<br class="">+ for (auto &S : I->second.External)<br class="">+ ExternalSymDeps.insert(S);<br class="">+ for (auto &S : I->second.Internal)<br class="">+ InternalSymDeps.insert(S);<br class="">+ }<br class=""> }<br class=""> }<br class=""><br class="">@@ -261,11 +267,33 @@ class ObjectLinkingLayerJITLinkContext final : public JITLinkContext {<br class=""> InternalNamedSymbolDeps[SymName] = std::move(InternalSymDeps);<br class=""> }<br class=""><br class="">+ for (auto &P : Layer.Plugins) {<br class="">+ auto SyntheticLocalDeps = P->getSyntheticSymbolLocalDependencies(MR);<br class="">+ if (SyntheticLocalDeps.empty())<br class="">+ continue;<br class="">+<br class="">+ for (auto &KV : SyntheticLocalDeps) {<br class="">+ auto &Name = KV.first;<br class="">+ auto &LocalDepsForName = KV.second;<br class="">+ for (auto *Local : LocalDepsForName) {<br class="">+ assert(Local->getScope() == Scope::Local &&<br class="">+ "Dependence on non-local symbol");<br class="">+ auto LocalNamedDepsItr = LocalDeps.find(Local);<br class="">+ if (LocalNamedDepsItr == LocalDeps.end())<br class="">+ continue;<br class="">+ for (auto &S : LocalNamedDepsItr->second.Internal)<br class="">+ InternalNamedSymbolDeps[Name].insert(S);<br class="">+ for (auto &S : LocalNamedDepsItr->second.External)<br class="">+ ExternalNamedSymbolDeps[Name].insert(S);<br class="">+ }<br class="">+ }<br class="">+ }<br class="">+<br class=""> return Error::success();<br class=""> }<br class=""><br class="">- LocalToNamedDependenciesMap computeLocalDeps(LinkGraph &G) {<br class="">- LocalToNamedDependenciesMap DepMap;<br class="">+ LocalSymbolNamedDependenciesMap computeLocalDeps(LinkGraph &G) {<br class="">+ DenseMap<jitlink::Symbol *, DenseSet<jitlink::Symbol *>> DepMap;<br class=""><br class=""> // For all local symbols:<br class=""> // (1) Add their named dependencies.<br class="">@@ -319,7 +347,26 @@ class ObjectLinkingLayerJITLinkContext final : public JITLinkContext {<br class=""> }<br class=""> } while (Changed);<br class=""><br class="">- return DepMap;<br class="">+ // Intern the results to produce a mapping of jitlink::Symbol* to internal<br class="">+ // and external symbol names.<br class="">+ auto &ES = Layer.getExecutionSession();<br class="">+ LocalSymbolNamedDependenciesMap Result;<br class="">+ for (auto &KV : DepMap) {<br class="">+ auto *Local = KV.first;<br class="">+ assert(Local->getScope() == Scope::Local &&<br class="">+ "DepMap keys should all be local symbols");<br class="">+ auto &LocalNamedDeps = Result[Local];<br class="">+ for (auto *Named : KV.second) {<br class="">+ assert(Named->getScope() != Scope::Local &&<br class="">+ "DepMap values should all be non-local symbol sets");<br class="">+ if (Named->isExternal())<br class="">+ LocalNamedDeps.External.insert(ES.intern(Named->getName()));<br class="">+ else<br class="">+ LocalNamedDeps.Internal.insert(ES.intern(Named->getName()));<br class="">+ }<br class="">+ }<br class="">+<br class="">+ return Result;<br class=""> }<br class=""><br class=""> void registerDependencies(const SymbolDependenceMap &QueryDeps) {<br class=""><br class="">diff --git a/llvm/lib/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.cpp b/llvm/lib/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.cpp<br class="">index ff8289a264c8..0eaf13c6e3c8 100644<br class="">--- a/llvm/lib/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.cpp<br class="">+++ b/llvm/lib/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.cpp<br class="">@@ -88,7 +88,8 @@ RTDyldObjectLinkingLayer::~RTDyldObjectLinkingLayer() {<br class=""> void RTDyldObjectLinkingLayer::emit(MaterializationResponsibility R,<br class=""> std::unique_ptr<MemoryBuffer> O) {<br class=""> assert(O && "Object must not be null");<br class="">-<br class="">+ dbgs() << "Emitting via RTDyldObjectLinkingLayer:\n"<br class="">+ << R.getSymbols() << "\n";<br class=""> // This method launches an asynchronous link step that will fulfill our<br class=""> // materialization responsibility. We need to switch R to be heap<br class=""> // allocated before that happens so it can live as long as the asynchronous<br class="">@@ -229,6 +230,9 @@ Error RTDyldObjectLinkingLayer::onObjLoad(<br class=""> Symbols.erase(KV.first);<br class=""> }<br class=""><br class="">+ if (const auto &InitSym = R.getInitializerSymbol())<br class="">+ Symbols[InitSym] = JITEvaluatedSymbol();<br class="">+<br class=""> if (auto Err = R.notifyResolved(Symbols)) {<br class=""> R.failMaterialization();<br class=""> return Err;<br class=""><br class="">diff --git a/llvm/test/ExecutionEngine/OrcLazy/objc-minimal.ll b/llvm/test/ExecutionEngine/OrcLazy/objc-minimal.ll<br class="">new file mode 100644<br class="">index 000000000000..8a940573dcb9<br class="">--- /dev/null<br class="">+++ b/llvm/test/ExecutionEngine/OrcLazy/objc-minimal.ll<br class="">@@ -0,0 +1,73 @@<br class="">+; REQUIRES: system-darwin<br class="">+; RUN: lli -jit-kind=orc-lazy %s<br class="">+;<br class="">+; Sanity check MachO Platform support: Call a no-op method (returning int 0) on<br class="">+; an ObjC object. This test checks that we correctly auto-identify this as a<br class="">+; MachO target, configure MachOPlatform support, and correctly register the<br class="">+; class metadata and method selector with the Objective-C runtime.<br class="">+<br class="">+source_filename = "objc-minimal.mm"<br class="">+target datalayout = "e-m:o-i64:64-f80:128-n8:16:32:64-S128"<br class="">+target triple = "x86_64-apple-macosx10.14.0"<br class="">+<br class="">+%0 = type opaque<br class="">+%struct._objc_cache = type opaque<br class="">+%struct._class_t = type { %struct._class_t*, %struct._class_t*, %struct._objc_cache*, i8* (i8*, i8*)**, %struct._class_ro_t* }<br class="">+%struct._class_ro_t = type { i32, i32, i32, i8*, i8*, %struct.__method_list_t*, %struct._objc_protocol_list*, %struct._ivar_list_t*, i8*, %struct._prop_list_t* }<br class="">+%struct.__method_list_t = type { i32, i32, [0 x %struct._objc_method] }<br class="">+%struct._objc_method = type { i8*, i8*, i8* }<br class="">+%struct._objc_protocol_list = type { i64, [0 x %struct._protocol_t*] }<br class="">+%struct._protocol_t = type { i8*, i8*, %struct._objc_protocol_list*, %struct.__method_list_t*, %struct.__method_list_t*, %struct.__method_list_t*, %struct.__method_list_t*, %struct._prop_list_t*, i32, i32, i8**, i8*, %struct._prop_list_t* }<br class="">+%struct._ivar_list_t = type { i32, i32, [0 x %struct._ivar_t] }<br class="">+%struct._ivar_t = type { i64*, i8*, i8*, i32, i32 }<br class="">+%struct._prop_list_t = type { i32, i32, [0 x %struct._prop_t] }<br class="">+%struct._prop_t = type { i8*, i8* }<br class="">+<br class="">+@_objc_empty_cache = external global %struct._objc_cache<br class="">+@"OBJC_METACLASS_$_NSObject" = external global %struct._class_t<br class="">+@OBJC_CLASS_NAME_ = private unnamed_addr constant [4 x i8] c"Foo\00", section "__TEXT,__objc_classname,cstring_literals", align 1<br class="">+@"\01l_OBJC_METACLASS_RO_$_Foo" = private global %struct._class_ro_t { i32 1, i32 40, i32 40, i8* null, i8* getelementptr inbounds ([4 x i8], [4 x i8]* @OBJC_CLASS_NAME_, i32 0, i32 0), %struct.__method_list_t* null, %struct._objc_protocol_list* null, %struct._ivar_list_t* null, i8* null, %struct._prop_list_t* null }, section "__DATA, __objc_const", align 8<br class="">+@"OBJC_METACLASS_$_Foo" = global %struct._class_t { %struct._class_t* @"OBJC_METACLASS_$_NSObject", %struct._class_t* @"OBJC_METACLASS_$_NSObject", %struct._objc_cache* @_objc_empty_cache, i8* (i8*, i8*)** null, %struct._class_ro_t* @"\01l_OBJC_METACLASS_RO_$_Foo" }, section "__DATA, __objc_data", align 8<br class="">+@"OBJC_CLASS_$_NSObject" = external global %struct._class_t<br class="">+@OBJC_METH_VAR_NAME_ = private unnamed_addr constant [4 x i8] c"foo\00", section "__TEXT,__objc_methname,cstring_literals", align 1<br class="">+@OBJC_METH_VAR_TYPE_ = private unnamed_addr constant [8 x i8] c"i16@0:8\00", section "__TEXT,__objc_methtype,cstring_literals", align 1<br class="">+@"\01l_OBJC_$_INSTANCE_METHODS_Foo" = private global { i32, i32, [1 x %struct._objc_method] } { i32 24, i32 1, [1 x %struct._objc_method] [%struct._objc_method { i8* getelementptr inbounds ([4 x i8], [4 x i8]* @OBJC_METH_VAR_NAME_, i32 0, i32 0), i8* getelementptr inbounds ([8 x i8], [8 x i8]* @OBJC_METH_VAR_TYPE_, i32 0, i32 0), i8* bitcast (i32 (%0*, i8*)* @"\01-[Foo foo]" to i8*) }] }, section "__DATA, __objc_const", align 8<br class="">+@"\01l_OBJC_CLASS_RO_$_Foo" = private global %struct._class_ro_t { i32 0, i32 8, i32 8, i8* null, i8* getelementptr inbounds ([4 x i8], [4 x i8]* @OBJC_CLASS_NAME_, i32 0, i32 0), %struct.__method_list_t* bitcast ({ i32, i32, [1 x %struct._objc_method] }* @"\01l_OBJC_$_INSTANCE_METHODS_Foo" to %struct.__method_list_t*), %struct._objc_protocol_list* null, %struct._ivar_list_t* null, i8* null, %struct._prop_list_t* null }, section "__DATA, __objc_const", align 8<br class="">+@"OBJC_CLASS_$_Foo" = global %struct._class_t { %struct._class_t* @"OBJC_METACLASS_$_Foo", %struct._class_t* @"OBJC_CLASS_$_NSObject", %struct._objc_cache* @_objc_empty_cache, i8* (i8*, i8*)** null, %struct._class_ro_t* @"\01l_OBJC_CLASS_RO_$_Foo" }, section "__DATA, __objc_data", align 8<br class="">+@"OBJC_CLASSLIST_REFERENCES_$_" = private global %struct._class_t* @"OBJC_CLASS_$_Foo", section "__DATA,__objc_classrefs,regular,no_dead_strip", align 8<br class="">+@OBJC_METH_VAR_NAME_.1 = private unnamed_addr constant [5 x i8] c"init\00", section "__TEXT,__objc_methname,cstring_literals", align 1<br class="">+@OBJC_SELECTOR_REFERENCES_ = private externally_initialized global i8* getelementptr inbounds ([5 x i8], [5 x i8]* @OBJC_METH_VAR_NAME_.1, i64 0, i64 0), section "__DATA,__objc_selrefs,literal_pointers,no_dead_strip", align 8<br class="">+@OBJC_SELECTOR_REFERENCES_.2 = private externally_initialized global i8* getelementptr inbounds ([4 x i8], [4 x i8]* @OBJC_METH_VAR_NAME_, i64 0, i64 0), section "__DATA,__objc_selrefs,literal_pointers,no_dead_strip", align 8<br class="">+@"OBJC_LABEL_CLASS_$" = private global [1 x i8*] [i8* bitcast (%struct._class_t* @"OBJC_CLASS_$_Foo" to i8*)], section "__DATA,__objc_classlist,regular,no_dead_strip", align 8<br class="">+@llvm.compiler.used = appending global [9 x i8*] [i8* bitcast ({ i32, i32, [1 x %struct._objc_method] }* @"\01l_OBJC_$_INSTANCE_METHODS_Foo" to i8*), i8* bitcast (%struct._class_t** @"OBJC_CLASSLIST_REFERENCES_$_" to i8*), i8* getelementptr inbounds ([4 x i8], [4 x i8]* @OBJC_CLASS_NAME_, i32 0, i32 0), i8* bitcast ([1 x i8*]* @"OBJC_LABEL_CLASS_$" to i8*), i8* getelementptr inbounds ([4 x i8], [4 x i8]* @OBJC_METH_VAR_NAME_, i32 0, i32 0), i8* getelementptr inbounds ([5 x i8], [5 x i8]* @OBJC_METH_VAR_NAME_.1, i32 0, i32 0), i8* getelementptr inbounds ([8 x i8], [8 x i8]* @OBJC_METH_VAR_TYPE_, i32 0, i32 0), i8* bitcast (i8** @OBJC_SELECTOR_REFERENCES_ to i8*), i8* bitcast (i8** @OBJC_SELECTOR_REFERENCES_.2 to i8*)], section "llvm.metadata"<br class="">+<br class="">+; Function Attrs: noinline norecurse nounwind readnone ssp uwtable<br class="">+define internal i32 @"\01-[Foo foo]"(%0* nocapture readnone, i8* nocapture readnone) {<br class="">+ ret i32 0<br class="">+}<br class="">+<br class="">+; Function Attrs: noinline norecurse ssp uwtable<br class="">+define i32 @main(i32, i8** nocapture readnone) local_unnamed_addr {<br class="">+ %3 = load i8*, i8** bitcast (%struct._class_t** @"OBJC_CLASSLIST_REFERENCES_$_" to i8**), align 8<br class="">+ %4 = tail call i8* @objc_alloc(i8* %3)<br class="">+ %5 = load i8*, i8** @OBJC_SELECTOR_REFERENCES_, align 8<br class="">+ %6 = tail call i8* bitcast (i8* (i8*, i8*, ...)* @objc_msgSend to i8* (i8*, i8*)*)(i8* %4, i8* %5)<br class="">+ %7 = load i8*, i8** @OBJC_SELECTOR_REFERENCES_.2, align 8<br class="">+ %8 = tail call i32 bitcast (i8* (i8*, i8*, ...)* @objc_msgSend to i32 (i8*, i8*)*)(i8* %6, i8* %7)<br class="">+ ret i32 %8<br class="">+}<br class="">+<br class="">+declare i8* @objc_alloc(i8*) local_unnamed_addr<br class="">+<br class="">+; Function Attrs: nonlazybind<br class="">+declare i8* @objc_msgSend(i8*, i8*, ...) local_unnamed_addr<br class="">+<br class="">+!llvm.module.flags = !{!0, !1, !2, !3, !4, !5, !6 }<br class="">+<br class="">+!0 = !{i32 2, !"SDK Version", [2 x i32] [i32 10, i32 15]}<br class="">+!1 = !{i32 1, !"Objective-C Version", i32 2}<br class="">+!2 = !{i32 1, !"Objective-C Image Info Version", i32 0}<br class="">+!3 = !{i32 1, !"Objective-C Image Info Section", !"__DATA,__objc_imageinfo,regular,no_dead_strip"}<br class="">+!4 = !{i32 4, !"Objective-C Garbage Collection", i32 0}<br class="">+!5 = !{i32 1, !"Objective-C Class Properties", i32 64}<br class="">+!6 = !{i32 1, !"wchar_size", i32 4}<br class=""><br class="">diff --git a/llvm/tools/lli/lli.cpp b/llvm/tools/lli/lli.cpp<br class="">index e7251b12f7fc..55b9557bddfe 100644<br class="">--- a/llvm/tools/lli/lli.cpp<br class="">+++ b/llvm/tools/lli/lli.cpp<br class="">@@ -24,9 +24,11 @@<br class=""> #include "llvm/ExecutionEngine/JITEventListener.h"<br class=""> #include "llvm/ExecutionEngine/MCJIT.h"<br class=""> #include "llvm/ExecutionEngine/ObjectCache.h"<br class="">+#include "llvm/ExecutionEngine/Orc/DebugUtils.h"<br class=""> #include "llvm/ExecutionEngine/Orc/ExecutionUtils.h"<br class=""> #include "llvm/ExecutionEngine/Orc/JITTargetMachineBuilder.h"<br class=""> #include "llvm/ExecutionEngine/Orc/LLJIT.h"<br class="">+#include "llvm/ExecutionEngine/Orc/MachOPlatform.h"<br class=""> #include "llvm/ExecutionEngine/Orc/OrcRemoteTargetClient.h"<br class=""> #include "llvm/ExecutionEngine/OrcMCJITReplacement.h"<br class=""> #include "llvm/ExecutionEngine/SectionMemoryManager.h"<br class="">@@ -206,6 +208,19 @@ namespace {<br class=""> cl::desc("Do not resolve lli process symbols in JIT'd code"),<br class=""> cl::init(false));<br class=""><br class="">+ enum class LLJITPlatform { DetectHost, GenericIR, MachO };<br class="">+<br class="">+ cl::opt<LLJITPlatform><br class="">+ Platform("lljit-platform", cl::desc("Platform to use with LLJIT"),<br class="">+ cl::init(LLJITPlatform::DetectHost),<br class="">+ cl::values(clEnumValN(LLJITPlatform::DetectHost, "DetectHost",<br class="">+ "Select based on JIT target triple"),<br class="">+ clEnumValN(LLJITPlatform::GenericIR, "GenericIR",<br class="">+ "Use LLJITGenericIRPlatform"),<br class="">+ clEnumValN(LLJITPlatform::MachO, "MachO",<br class="">+ "Use LLJITMachOPlatform")),<br class="">+ cl::Hidden);<br class="">+<br class=""> enum class DumpKind {<br class=""> NoDump,<br class=""> DumpFuncsToStdOut,<br class="">@@ -772,16 +787,20 @@ int runOrcLazyJIT(const char *ProgName) {<br class=""> if (!MainModule)<br class=""> reportError(Err, ProgName);<br class=""><br class="">- const auto &TT = MainModule->getTargetTriple();<br class="">+ Triple TT(MainModule->getTargetTriple());<br class=""> orc::LLLazyJITBuilder Builder;<br class=""><br class=""> Builder.setJITTargetMachineBuilder(<br class="">- TT.empty() ? ExitOnErr(orc::JITTargetMachineBuilder::detectHost())<br class="">- : orc::JITTargetMachineBuilder(Triple(TT)));<br class="">+ MainModule->getTargetTriple().empty()<br class="">+ ? ExitOnErr(orc::JITTargetMachineBuilder::detectHost())<br class="">+ : orc::JITTargetMachineBuilder(TT));<br class=""><br class=""> if (!MArch.empty())<br class=""> Builder.getJITTargetMachineBuilder()->getTargetTriple().setArchName(MArch);<br class=""><br class="">+ if (!MainModule->getDataLayout().isDefault())<br class="">+ Builder.setDataLayout(MainModule->getDataLayout());<br class="">+<br class=""> Builder.getJITTargetMachineBuilder()<br class=""> ->setCPU(getCPUStr())<br class=""> .addFeatures(getFeatureList())<br class="">@@ -796,6 +815,29 @@ int runOrcLazyJIT(const char *ProgName) {<br class=""> pointerToJITTargetAddress(exitOnLazyCallThroughFailure));<br class=""> Builder.setNumCompileThreads(LazyJITCompileThreads);<br class=""><br class="">+ // Set up LLJIT platform.<br class="">+ {<br class="">+ LLJITPlatform P = Platform;<br class="">+ if (P == LLJITPlatform::DetectHost) {<br class="">+ if (TT.isOSBinFormatMachO())<br class="">+ P = LLJITPlatform::MachO;<br class="">+ else<br class="">+ P = LLJITPlatform::GenericIR;<br class="">+ }<br class="">+<br class="">+ switch (P) {<br class="">+ case LLJITPlatform::GenericIR:<br class="">+ // Nothing to do: LLJITBuilder will use this by default.<br class="">+ break;<br class="">+ case LLJITPlatform::MachO:<br class="">+ Builder.setPlatformSetUp(orc::setUpMachOPlatform);<br class="">+ ExitOnErr(orc::enableObjCRegistration("libobjc.dylib"));<br class="">+ break;<br class="">+ default:<br class="">+ llvm_unreachable("Unrecognized platform value");<br class="">+ }<br class="">+ }<br class="">+<br class=""> auto J = ExitOnErr(Builder.create());<br class=""><br class=""> if (PerModuleLazy)<br class="">@@ -828,9 +870,6 @@ int runOrcLazyJIT(const char *ProgName) {<br class=""> return Name != MainName;<br class=""> })));<br class=""><br class="">- orc::LocalCXXRuntimeOverrides CXXRuntimeOverrides;<br class="">- ExitOnErr(CXXRuntimeOverrides.enable(J->getMainJITDylib(), Mangle));<br class="">-<br class=""> // Add the main module.<br class=""> ExitOnErr(<br class=""> J->addLazyIRModule(orc::ThreadSafeModule(std::move(MainModule), TSCtx)));<br class="">@@ -845,8 +884,11 @@ int runOrcLazyJIT(const char *ProgName) {<br class=""> for (auto JDItr = JITDylibs.begin(), JDEnd = JITDylibs.end();<br class=""> JDItr != JDEnd; ++JDItr) {<br class=""> orc::JITDylib *JD = J->getJITDylibByName(*JDItr);<br class="">- if (!JD)<br class="">- JD = &J->createJITDylib(*JDItr);<br class="">+ if (!JD) {<br class="">+ JD = &ExitOnErr(J->createJITDylib(*JDItr));<br class="">+ J->getMainJITDylib().addToSearchOrder(*JD);<br class="">+ JD->addToSearchOrder(J->getMainJITDylib());<br class="">+ }<br class=""> IdxToDylib[JITDylibs.getPosition(JDItr - JITDylibs.begin())] = JD;<br class=""> }<br class=""><br class="">@@ -882,7 +924,7 @@ int runOrcLazyJIT(const char *ProgName) {<br class=""> }<br class=""><br class=""> // Run any static constructors.<br class="">- ExitOnErr(J->runConstructors());<br class="">+ ExitOnErr(J->initialize(J->getMainJITDylib()));<br class=""><br class=""> // Run any -thread-entry points.<br class=""> std::vector<std::thread> AltEntryThreads;<br class="">@@ -907,8 +949,7 @@ int runOrcLazyJIT(const char *ProgName) {<br class=""> AltEntryThread.join();<br class=""><br class=""> // Run destructors.<br class="">- ExitOnErr(J->runDestructors());<br class="">- CXXRuntimeOverrides.runDestructors();<br class="">+ ExitOnErr(J->deinitialize(J->getMainJITDylib()));<br class=""><br class=""> return Result;<br class=""> }<br class=""><br class="">diff --git a/llvm/tools/llvm-jitlink/llvm-jitlink.cpp b/llvm/tools/llvm-jitlink/llvm-jitlink.cpp<br class="">index 26bcf46d60a6..7765d53f402d 100644<br class="">--- a/llvm/tools/llvm-jitlink/llvm-jitlink.cpp<br class="">+++ b/llvm/tools/llvm-jitlink/llvm-jitlink.cpp<br class="">@@ -396,9 +396,18 @@ static std::unique_ptr<jitlink::JITLinkMemoryManager> createMemoryManager() {<br class=""> return std::make_unique<jitlink::InProcessMemoryManager>();<br class=""> }<br class=""><br class="">-Session::Session(Triple TT)<br class="">- : MainJD(ES.createJITDylib("<main>")), ObjLayer(ES, createMemoryManager()),<br class="">- TT(std::move(TT)) {<br class="">+Expected<std::unique_ptr<Session>> Session::Create(Triple TT) {<br class="">+ Error Err = Error::success();<br class="">+ std::unique_ptr<Session> S(new Session(std::move(TT), Err));<br class="">+ if (Err)<br class="">+ return std::move(Err);<br class="">+ return std::move(S);<br class="">+}<br class="">+<br class="">+// FIXME: Move to createJITDylib if/when we start using Platform support in<br class="">+// llvm-jitlink.<br class="">+Session::Session(Triple TT, Error &Err)<br class="">+ : ObjLayer(ES, createMemoryManager()), TT(std::move(TT)) {<br class=""><br class=""> /// Local ObjectLinkingLayer::Plugin class to forward modifyPassConfig to the<br class=""> /// Session.<br class="">@@ -414,6 +423,15 @@ Session::Session(Triple TT)<br class=""> Session &S;<br class=""> };<br class=""><br class="">+ ErrorAsOutParameter _(&Err);<br class="">+<br class="">+ if (auto MainJDOrErr = ES.createJITDylib("main"))<br class="">+ MainJD = &*MainJDOrErr;<br class="">+ else {<br class="">+ Err = MainJDOrErr.takeError();<br class="">+ return;<br class="">+ }<br class="">+<br class=""> if (!NoExec && !TT.isOSWindows())<br class=""> ObjLayer.addPlugin(std::make_unique<EHFrameRegistrationPlugin>(<br class=""> InProcessEHFrameRegistrar::getInstance()));<br class="">@@ -561,7 +579,7 @@ Error loadProcessSymbols(Session &S) {<br class=""> auto FilterMainEntryPoint = [InternedEntryPointName](SymbolStringPtr Name) {<br class=""> return Name != InternedEntryPointName;<br class=""> };<br class="">- S.MainJD.addGenerator(<br class="">+ S.MainJD->addGenerator(<br class=""> ExitOnErr(orc::DynamicLibrarySearchGenerator::GetForCurrentProcess(<br class=""> GlobalPrefix, FilterMainEntryPoint)));<br class=""><br class="">@@ -590,20 +608,22 @@ Error loadObjects(Session &S) {<br class=""> LLVM_DEBUG(dbgs() << "Creating JITDylibs...\n");<br class=""> {<br class=""> // Create a "main" JITLinkDylib.<br class="">- IdxToJLD[0] = &S.MainJD;<br class="">- S.JDSearchOrder.push_back(&S.MainJD);<br class="">- LLVM_DEBUG(dbgs() << " 0: " << S.MainJD.getName() << "\n");<br class="">+ IdxToJLD[0] = S.MainJD;<br class="">+ S.JDSearchOrder.push_back(S.MainJD);<br class="">+ LLVM_DEBUG(dbgs() << " 0: " << S.MainJD->getName() << "\n");<br class=""><br class=""> // Add any extra JITLinkDylibs from the command line.<br class=""> std::string JDNamePrefix("lib");<br class=""> for (auto JLDItr = JITLinkDylibs.begin(), JLDEnd = JITLinkDylibs.end();<br class=""> JLDItr != JLDEnd; ++JLDItr) {<br class="">- auto &JD = S.ES.createJITDylib(JDNamePrefix + *JLDItr);<br class="">+ auto JD = S.ES.createJITDylib(JDNamePrefix + *JLDItr);<br class="">+ if (!JD)<br class="">+ return JD.takeError();<br class=""> unsigned JDIdx =<br class=""> JITLinkDylibs.getPosition(JLDItr - JITLinkDylibs.begin());<br class="">- IdxToJLD[JDIdx] = &JD;<br class="">- S.JDSearchOrder.push_back(&JD);<br class="">- LLVM_DEBUG(dbgs() << " " << JDIdx << ": " << JD.getName() << "\n");<br class="">+ IdxToJLD[JDIdx] = &*JD;<br class="">+ S.JDSearchOrder.push_back(&*JD);<br class="">+ LLVM_DEBUG(dbgs() << " " << JDIdx << ": " << JD->getName() << "\n");<br class=""> }<br class=""><br class=""> // Set every dylib to link against every other, in command line order.<br class="">@@ -790,32 +810,32 @@ int main(int argc, char *argv[]) {<br class=""> std::unique_ptr<JITLinkTimers> Timers =<br class=""> ShowTimes ? std::make_unique<JITLinkTimers>() : nullptr;<br class=""><br class="">- Session S(getFirstFileTriple());<br class="">+ auto S = ExitOnErr(Session::Create(getFirstFileTriple()));<br class=""><br class="">- ExitOnErr(sanitizeArguments(S));<br class="">+ ExitOnErr(sanitizeArguments(*S));<br class=""><br class=""> if (!NoProcessSymbols)<br class="">- ExitOnErr(loadProcessSymbols(S));<br class="">+ ExitOnErr(loadProcessSymbols(*S));<br class=""> ExitOnErr(loadDylibs());<br class=""><br class=""><br class=""> {<br class=""> TimeRegion TR(Timers ? &Timers->LoadObjectsTimer : nullptr);<br class="">- ExitOnErr(loadObjects(S));<br class="">+ ExitOnErr(loadObjects(*S));<br class=""> }<br class=""><br class=""> JITEvaluatedSymbol EntryPoint = 0;<br class=""> {<br class=""> TimeRegion TR(Timers ? &Timers->LinkTimer : nullptr);<br class="">- EntryPoint = ExitOnErr(getMainEntryPoint(S));<br class="">+ EntryPoint = ExitOnErr(getMainEntryPoint(*S));<br class=""> }<br class=""><br class=""> if (ShowAddrs)<br class="">- S.dumpSessionInfo(outs());<br class="">+ S->dumpSessionInfo(outs());<br class=""><br class="">- ExitOnErr(runChecks(S));<br class="">+ ExitOnErr(runChecks(*S));<br class=""><br class="">- dumpSessionStats(S);<br class="">+ dumpSessionStats(*S);<br class=""><br class=""> if (NoExec)<br class=""> return 0;<br class=""><br class="">diff --git a/llvm/tools/llvm-jitlink/llvm-jitlink.h b/llvm/tools/llvm-jitlink/llvm-jitlink.h<br class="">index 0f92d7605012..c888baec9adf 100644<br class="">--- a/llvm/tools/llvm-jitlink/llvm-jitlink.h<br class="">+++ b/llvm/tools/llvm-jitlink/llvm-jitlink.h<br class="">@@ -26,12 +26,13 @@ namespace llvm {<br class=""><br class=""> struct Session {<br class=""> orc::ExecutionSession ES;<br class="">- orc::JITDylib &MainJD;<br class="">+ orc::JITDylib *MainJD;<br class=""> orc::ObjectLinkingLayer ObjLayer;<br class=""> std::vector<orc::JITDylib *> JDSearchOrder;<br class=""> Triple TT;<br class=""><br class=""> Session(Triple TT);<br class="">+ static Expected<std::unique_ptr<Session>> Create(Triple TT);<br class=""> void dumpSessionInfo(raw_ostream &OS);<br class=""> void modifyPassConfig(const Triple &FTT,<br class=""> jitlink::PassConfiguration &PassConfig);<br class="">@@ -63,6 +64,9 @@ struct Session {<br class=""> FileInfoMap FileInfos;<br class=""> uint64_t SizeBeforePruning = 0;<br class=""> uint64_t SizeAfterFixups = 0;<br class="">+<br class="">+private:<br class="">+ Session(Triple TT, Error &Err);<br class=""> };<br class=""><br class=""> Error registerMachOStubsAndGOT(Session &S, jitlink::LinkGraph &G);<br class=""><br class="">diff --git a/llvm/unittests/ExecutionEngine/Orc/CoreAPIsTest.cpp b/llvm/unittests/ExecutionEngine/Orc/CoreAPIsTest.cpp<br class="">index 68a6d2ed2ca2..171ff37f0069 100644<br class="">--- a/llvm/unittests/ExecutionEngine/Orc/CoreAPIsTest.cpp<br class="">+++ b/llvm/unittests/ExecutionEngine/Orc/CoreAPIsTest.cpp<br class="">@@ -113,6 +113,7 @@ TEST_F(CoreAPIsStandardTest, RemoveSymbolsTest) {<br class=""> cantFail(R.notifyResolved({{Bar, BarSym}}));<br class=""> cantFail(R.notifyEmitted());<br class=""> },<br class="">+ nullptr,<br class=""> [&](const JITDylib &JD, const SymbolStringPtr &Name) {<br class=""> EXPECT_EQ(Name, Bar) << "Expected \"Bar\" to be discarded";<br class=""> if (Name == Bar)<br class="">@@ -126,6 +127,7 @@ TEST_F(CoreAPIsStandardTest, RemoveSymbolsTest) {<br class=""> cantFail(JD.define(std::make_unique<SimpleMaterializationUnit>(<br class=""> SymbolFlagsMap({{Baz, BazSym.getFlags()}}),<br class=""> [&](MaterializationResponsibility R) { BazR.emplace(std::move(R)); },<br class="">+ nullptr,<br class=""> [](const JITDylib &JD, const SymbolStringPtr &Name) {<br class=""> ADD_FAILURE() << "\"Baz\" discarded unexpectedly";<br class=""> })));<br class="">@@ -176,7 +178,7 @@ TEST_F(CoreAPIsStandardTest, RemoveSymbolsTest) {<br class=""> TEST_F(CoreAPIsStandardTest, ChainedJITDylibLookup) {<br class=""> cantFail(JD.define(absoluteSymbols({{Foo, FooSym}})));<br class=""><br class="">- auto &JD2 = ES.createJITDylib("JD2");<br class="">+ auto &JD2 = ES.createBareJITDylib("JD2");<br class=""><br class=""> bool OnCompletionRun = false;<br class=""><br class="">@@ -198,7 +200,7 @@ TEST_F(CoreAPIsStandardTest, LookupWithHiddenSymbols) {<br class=""><br class=""> cantFail(JD.define(absoluteSymbols({{Foo, FooSym}, {Bar, BarHiddenSym}})));<br class=""><br class="">- auto &JD2 = ES.createJITDylib("JD2");<br class="">+ auto &JD2 = ES.createBareJITDylib("JD2");<br class=""> cantFail(JD2.define(absoluteSymbols({{Bar, QuxSym}})));<br class=""><br class=""> /// Try a blocking lookup.<br class="">@@ -307,7 +309,7 @@ TEST_F(CoreAPIsStandardTest, TestBasicReExports) {<br class=""> // JITDylib works.<br class=""> cantFail(JD.define(absoluteSymbols({{Foo, FooSym}})));<br class=""><br class="">- auto &JD2 = ES.createJITDylib("JD2");<br class="">+ auto &JD2 = ES.createBareJITDylib("JD2");<br class=""><br class=""> cantFail(JD2.define(reexports(JD, {{Bar, {Foo, BarSym.getFlags()}}})));<br class=""><br class="">@@ -332,7 +334,7 @@ TEST_F(CoreAPIsStandardTest, TestThatReExportsDontUnnecessarilyMaterialize) {<br class=""><br class=""> cantFail(JD.define(BarMU));<br class=""><br class="">- auto &JD2 = ES.createJITDylib("JD2");<br class="">+ auto &JD2 = ES.createBareJITDylib("JD2");<br class=""><br class=""> cantFail(JD2.define(reexports(<br class=""> JD, {{Baz, {Foo, BazSym.getFlags()}}, {Qux, {Bar, QuxSym.getFlags()}}})));<br class="">@@ -347,7 +349,7 @@ TEST_F(CoreAPIsStandardTest, TestThatReExportsDontUnnecessarilyMaterialize) {<br class=""> TEST_F(CoreAPIsStandardTest, TestReexportsGenerator) {<br class=""> // Test that a re-exports generator can dynamically generate reexports.<br class=""><br class="">- auto &JD2 = ES.createJITDylib("JD2");<br class="">+ auto &JD2 = ES.createBareJITDylib("JD2");<br class=""> cantFail(JD2.define(absoluteSymbols({{Foo, FooSym}, {Bar, BarSym}})));<br class=""><br class=""> auto Filter = [this](SymbolStringPtr Name) { return Name != Bar; };<br class="">@@ -838,6 +840,7 @@ TEST_F(CoreAPIsStandardTest, DropMaterializerWhenEmpty) {<br class=""> [](MaterializationResponsibility R) {<br class=""> llvm_unreachable("Unexpected call to materialize");<br class=""> },<br class="">+ nullptr,<br class=""> [&](const JITDylib &JD, SymbolStringPtr Name) {<br class=""> EXPECT_TRUE(Name == Foo || Name == Bar)<br class=""> << "Discard of unexpected symbol?";<br class="">@@ -872,6 +875,7 @@ TEST_F(CoreAPIsStandardTest, AddAndMaterializeLazySymbol) {<br class=""> cantFail(R.notifyEmitted());<br class=""> FooMaterialized = true;<br class=""> },<br class="">+ nullptr,<br class=""> [&](const JITDylib &JD, SymbolStringPtr Name) {<br class=""> EXPECT_EQ(Name, Bar) << "Expected Name to be Bar";<br class=""> BarDiscarded = true;<br class="">@@ -920,6 +924,7 @@ TEST_F(CoreAPIsStandardTest, TestBasicWeakSymbolMaterialization) {<br class=""> ADD_FAILURE() << "Attempt to materialize Bar from the wrong unit";<br class=""> R.failMaterialization();<br class=""> },<br class="">+ nullptr,<br class=""> [&](const JITDylib &JD, SymbolStringPtr Name) {<br class=""> EXPECT_EQ(Name, Bar) << "Expected \"Bar\" to be discarded";<br class=""> DuplicateBarDiscarded = true;<br class=""><br class="">diff --git a/llvm/unittests/ExecutionEngine/Orc/OrcTestCommon.h b/llvm/unittests/ExecutionEngine/Orc/OrcTestCommon.h<br class="">index d82012b5a853..b25851d8f796 100644<br class="">--- a/llvm/unittests/ExecutionEngine/Orc/OrcTestCommon.h<br class="">+++ b/llvm/unittests/ExecutionEngine/Orc/OrcTestCommon.h<br class="">@@ -47,7 +47,7 @@ class CoreAPIsBasedStandardTest : public testing::Test {<br class=""> protected:<br class=""> std::shared_ptr<SymbolStringPool> SSP = std::make_shared<SymbolStringPool>();<br class=""> ExecutionSession ES{SSP};<br class="">- JITDylib &JD = ES.createJITDylib("JD");<br class="">+ JITDylib &JD = ES.createBareJITDylib("JD");<br class=""> SymbolStringPtr Foo = ES.intern("foo");<br class=""> SymbolStringPtr Bar = ES.intern("bar");<br class=""> SymbolStringPtr Baz = ES.intern("baz");<br class="">@@ -93,9 +93,11 @@ class SimpleMaterializationUnit : public orc::MaterializationUnit {<br class=""><br class=""> SimpleMaterializationUnit(<br class=""> orc::SymbolFlagsMap SymbolFlags, MaterializeFunction Materialize,<br class="">+ orc::SymbolStringPtr InitSym = nullptr,<br class=""> DiscardFunction Discard = DiscardFunction(),<br class=""> DestructorFunction Destructor = DestructorFunction())<br class="">- : MaterializationUnit(std::move(SymbolFlags), orc::VModuleKey()),<br class="">+ : MaterializationUnit(std::move(SymbolFlags), std::move(InitSym),<br class="">+ orc::VModuleKey()),<br class=""> Materialize(std::move(Materialize)), Discard(std::move(Discard)),<br class=""> Destructor(std::move(Destructor)) {}<br class=""><br class=""><br class="">diff --git a/llvm/unittests/ExecutionEngine/Orc/RTDyldObjectLinkingLayerTest.cpp b/llvm/unittests/ExecutionEngine/Orc/RTDyldObjectLinkingLayerTest.cpp<br class="">index 0c66841e9af0..9a50571b1969 100644<br class="">--- a/llvm/unittests/ExecutionEngine/Orc/RTDyldObjectLinkingLayerTest.cpp<br class="">+++ b/llvm/unittests/ExecutionEngine/Orc/RTDyldObjectLinkingLayerTest.cpp<br class="">@@ -50,7 +50,7 @@ static bool testSetProcessAllSections(std::unique_ptr<MemoryBuffer> Obj,<br class=""> bool DebugSectionSeen = false;<br class=""><br class=""> ExecutionSession ES;<br class="">- auto &JD = ES.createJITDylib("main");<br class="">+ auto &JD = ES.createBareJITDylib("main");<br class=""> auto Foo = ES.intern("foo");<br class=""><br class=""> RTDyldObjectLinkingLayer ObjLayer(ES, [&DebugSectionSeen]() {<br class="">@@ -151,7 +151,7 @@ TEST(RTDyldObjectLinkingLayerTest, TestOverrideObjectFlags) {<br class=""><br class=""> // Create a simple stack and set the override flags option.<br class=""> ExecutionSession ES;<br class="">- auto &JD = ES.createJITDylib("main");<br class="">+ auto &JD = ES.createBareJITDylib("main");<br class=""> auto Foo = ES.intern("foo");<br class=""> RTDyldObjectLinkingLayer ObjLayer(<br class=""> ES, []() { return std::make_unique<SectionMemoryManager>(); });<br class="">@@ -218,7 +218,7 @@ TEST(RTDyldObjectLinkingLayerTest, TestAutoClaimResponsibilityForSymbols) {<br class=""><br class=""> // Create a simple stack and set the override flags option.<br class=""> ExecutionSession ES;<br class="">- auto &JD = ES.createJITDylib("main");<br class="">+ auto &JD = ES.createBareJITDylib("main");<br class=""> auto Foo = ES.intern("foo");<br class=""> RTDyldObjectLinkingLayer ObjLayer(<br class=""> ES, []() { return std::make_unique<SectionMemoryManager>(); });<br class=""><br class=""><br class=""><br class="">_______________________________________________<br class="">llvm-commits mailing list<br class=""><a href="mailto:llvm-commits@lists.llvm.org" class="">llvm-commits@lists.llvm.org</a><br class="">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits<br class=""></div></div></blockquote></div><br class=""></div></body></html>