[llvm] 85fb997 - [ORC] Add generic initializer/deinitializer support.
Lang Hames via llvm-commits
llvm-commits at lists.llvm.org
Mon Feb 24 13:44:45 PST 2020
Hi Eric,
Thanks for letting me know! I’ll take a look and see if I can figure this out.
Regards,
Lang.
Sent from my iPad
> On Feb 24, 2020, at 10:29 AM, Erik Pilkington <erik.pilkington at gmail.com> wrote:
>
> Hi Lang,
>
> It looks like a test you added in this commit is failing on macOS: http://lab.llvm.org:8080/green/job/clang-stage2-cmake-RgSan/6977
>
> Can you please take a look?
>
> Thanks!
> Erik
>
>
> ******************** TEST 'LLVM :: ExecutionEngine/OrcLazy/objc-minimal.ll' FAILED ********************
> Script:
> --
> : 'RUN: at line 2'; /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/lli -jit-kind=orc-lazy /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm-project/llvm/test/ExecutionEngine/OrcLazy/objc-minimal.ll
> --
> Exit Code: 1
>
> Command Output (stderr):
> --
>
> =================================================================
> ==96779==ERROR: LeakSanitizer: detected memory leaks
>
> Direct leak of 80 byte(s) in 1 object(s) allocated from:
> #0 0x1127fbef2 in wrap_calloc+0xa2 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x45ef2)
> #1 0x7fff5addac5d in allocateBuckets(unsigned int)+0x1d (libobjc.A.dylib:x86_64h+0x9c5d)
> #2 0x7fff5addabec in cache_t::reallocate(unsigned int, unsigned int)+0x2a (libobjc.A.dylib:x86_64h+0x9bec)
> #3 0x7fff5adda3a8 in cache_fill+0xc2 (libobjc.A.dylib:x86_64h+0x93a8)
> #4 0x7fff5add9b94 in lookUpImpOrForward+0x269 (libobjc.A.dylib:x86_64h+0x8b94)
> #5 0x7fff5add9493 in _objc_msgSend_uncached+0x43 (libobjc.A.dylib:x86_64h+0x8493)
> #6 0x1166f702a (<unknown module>)
> #7 0x10a89cd22 in runOrcLazyJIT(char const*) lli.cpp:991
> #8 0x10a890b79 in main lli.cpp:409
> #9 0x7fff5beb5084 in start+0x0 (libdyld.dylib:x86_64+0x17084)
>
> Direct leak of 80 byte(s) in 1 object(s) allocated from:
> #0 0x1127fbef2 in wrap_calloc+0xa2 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x45ef2)
> #1 0x7fff5addac5d in allocateBuckets(unsigned int)+0x1d (libobjc.A.dylib:x86_64h+0x9c5d)
> #2 0x7fff5addabec in cache_t::reallocate(unsigned int, unsigned int)+0x2a (libobjc.A.dylib:x86_64h+0x9bec)
> #3 0x7fff5adda3a8 in cache_fill+0xc2 (libobjc.A.dylib:x86_64h+0x93a8)
> #4 0x7fff5add9b94 in lookUpImpOrForward+0x269 (libobjc.A.dylib:x86_64h+0x8b94)
> #5 0x7fff5add9493 in _objc_msgSend_uncached+0x43 (libobjc.A.dylib:x86_64h+0x8493)
> #6 0x1166f701b (<unknown module>)
> #7 0x10a89cd22 in runOrcLazyJIT(char const*) lli.cpp:991
> #8 0x10a890b79 in main lli.cpp:409
> #9 0x7fff5beb5084 in start+0x0 (libdyld.dylib:x86_64+0x17084)
>
> Direct leak of 64 byte(s) in 1 object(s) allocated from:
> #0 0x1127fbef2 in wrap_calloc+0xa2 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x45ef2)
> #1 0x7fff5add6510 in realizeClass(objc_class*)+0x4c (libobjc.A.dylib:x86_64h+0x5510)
> #2 0x7fff5add65e2 in realizeClass(objc_class*)+0x11e (libobjc.A.dylib:x86_64h+0x55e2)
> #3 0x7fff5adee3f5 in objc_readClassPair+0x76 (libobjc.A.dylib:x86_64h+0x1d3f5)
> #4 0x10be4620c in llvm::orc::MachOJITDylibInitializers::registerObjCClasses() const MachOPlatform.cpp:133
> #5 0x10be246a7 in (anonymous namespace)::MachOPlatformSupport::initialize(llvm::orc::JITDylib&) LLJIT.cpp:525
> #6 0x10a8a73f7 in llvm::orc::LLJIT::initialize(llvm::orc::JITDylib&) LLJIT.h:141
> #7 0x10a89c514 in runOrcLazyJIT(char const*) lli.cpp:975
> #8 0x10a890b79 in main lli.cpp:409
> #9 0x7fff5beb5084 in start+0x0 (libdyld.dylib:x86_64+0x17084)
>
> Direct leak of 64 byte(s) in 1 object(s) allocated from:
> #0 0x1127fbef2 in wrap_calloc+0xa2 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x45ef2)
> #1 0x7fff5add6510 in realizeClass(objc_class*)+0x4c (libobjc.A.dylib:x86_64h+0x5510)
> #2 0x7fff5adee3f5 in objc_readClassPair+0x76 (libobjc.A.dylib:x86_64h+0x1d3f5)
> #3 0x10be4620c in llvm::orc::MachOJITDylibInitializers::registerObjCClasses() const MachOPlatform.cpp:133
> #4 0x10be246a7 in (anonymous namespace)::MachOPlatformSupport::initialize(llvm::orc::JITDylib&) LLJIT.cpp:525
> #5 0x10a8a73f7 in llvm::orc::LLJIT::initialize(llvm::orc::JITDylib&) LLJIT.h:141
> #6 0x10a89c514 in runOrcLazyJIT(char const*) lli.cpp:975
> #7 0x10a890b79 in main lli.cpp:409
> #8 0x7fff5beb5084 in start+0x0 (libdyld.dylib:x86_64+0x17084)
>
> Direct leak of 16 byte(s) in 1 object(s) allocated from:
> #0 0x1127fbef2 in wrap_calloc+0xa2 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x45ef2)
> #1 0x7fff5add883d in class_createInstance+0x52 (libobjc.A.dylib:x86_64h+0x783d)
> #2 0x7fff5addc595 in _objc_rootAlloc+0x2c (libobjc.A.dylib:x86_64h+0xb595)
> #3 0x1166f701b (<unknown module>)
> #4 0x10a89cd22 in runOrcLazyJIT(char const*) lli.cpp:991
> #5 0x10a890b79 in main lli.cpp:409
> #6 0x7fff5beb5084 in start+0x0 (libdyld.dylib:x86_64+0x17084)
>
> SUMMARY: AddressSanitizer: 304 byte(s) leaked in 5 allocation(s)
>> On Feb 19, 2020, at 2:03 PM, Lang Hames via llvm-commits <llvm-commits at lists.llvm.org> wrote:
>>
>>
>> Author: Lang Hames
>> Date: 2020-02-19T13:59:32-08:00
>> New Revision: 85fb997659b55101b58e3545733e581bf4ed9cf2
>>
>> URL: https://github.com/llvm/llvm-project/commit/85fb997659b55101b58e3545733e581bf4ed9cf2
>> DIFF: https://github.com/llvm/llvm-project/commit/85fb997659b55101b58e3545733e581bf4ed9cf2.diff
>>
>> LOG: [ORC] Add generic initializer/deinitializer support.
>>
>> Initializers and deinitializers are used to implement C++ static constructors
>> and destructors, runtime registration for some languages (e.g. with the
>> Objective-C runtime for Objective-C/C++ code) and other tasks that would
>> typically be performed when a shared-object/dylib is loaded or unloaded by a
>> statically compiled program.
>>
>> MCJIT and ORC have historically provided limited support for discovering and
>> running initializers/deinitializers by scanning the llvm.global_ctors and
>> llvm.global_dtors variables and recording the functions to be run. This approach
>> suffers from several drawbacks: (1) It only works for IR inputs, not for object
>> files (including cached JIT'd objects). (2) It only works for initializers
>> described by llvm.global_ctors and llvm.global_dtors, however not all
>> initializers are described in this way (Objective-C, for example, describes
>> initializers via specially named metadata sections). (3) To make the
>> initializer/deinitializer functions described by llvm.global_ctors and
>> llvm.global_dtors searchable they must be promoted to extern linkage, polluting
>> the JIT symbol table (extra care must be taken to ensure this promotion does
>> not result in symbol name clashes).
>>
>> This patch introduces several interdependent changes to ORCv2 to support the
>> construction of new initialization schemes, and includes an implementation of a
>> backwards-compatible llvm.global_ctor/llvm.global_dtor scanning scheme, and a
>> MachO specific scheme that handles Objective-C runtime registration (if the
>> Objective-C runtime is available) enabling execution of LLVM IR compiled from
>> Objective-C and Swift.
>>
>> The major changes included in this patch are:
>>
>> (1) The MaterializationUnit and MaterializationResponsibility classes are
>> extended to describe an optional "initializer" symbol for the module (see the
>> getInitializerSymbol method on each class). The presence or absence of this
>> symbol indicates whether the module contains any initializers or
>> deinitializers. The initializer symbol otherwise behaves like any other:
>> searching for it triggers materialization.
>>
>> (2) A new Platform interface is introduced in llvm/ExecutionEngine/Orc/Core.h
>> which provides the following callback interface:
>>
>> - Error setupJITDylib(JITDylib &JD): Can be used to install standard symbols
>> in JITDylibs upon creation. E.g. __dso_handle.
>>
>> - Error notifyAdding(JITDylib &JD, const MaterializationUnit &MU): Generally
>> used to record initializer symbols.
>>
>> - Error notifyRemoving(JITDylib &JD, VModuleKey K): Used to notify a platform
>> that a module is being removed.
>>
>> Platform implementations can use these callbacks to track outstanding
>> initializers and implement a platform-specific approach for executing them. For
>> example, the MachOPlatform installs a plugin in the JIT linker to scan for both
>> __mod_inits sections (for C++ static constructors) and ObjC metadata sections.
>> If discovered, these are processed in the usual platform order: Objective-C
>> registration is carried out first, then static initializers are executed,
>> ensuring that calls to Objective-C from static initializers will be safe.
>>
>> This patch updates LLJIT to use the new scheme for initialization. Two
>> LLJIT::PlatformSupport classes are implemented: A GenericIR platform and a MachO
>> platform. The GenericIR platform implements a modified version of the previous
>> llvm.global-ctor scraping scheme to provide support for Windows and
>> Linux. LLJIT's MachO platform uses the MachOPlatform class to provide MachO
>> specific initialization as described above.
>>
>> Reviewers: sgraenitz, dblaikie
>>
>> Subscribers: mgorny, hiraditya, mgrang, ributzka, llvm-commits
>>
>> Tags: #llvm
>>
>> Differential Revision: https://reviews.llvm.org/D74300
>>
>> Added:
>> llvm/include/llvm/ExecutionEngine/Orc/MachOPlatform.h
>> llvm/include/llvm/ExecutionEngine/Orc/Mangling.h
>> llvm/lib/ExecutionEngine/Orc/MachOPlatform.cpp
>> llvm/lib/ExecutionEngine/Orc/Mangling.cpp
>> llvm/test/ExecutionEngine/OrcLazy/objc-minimal.ll
>>
>> Modified:
>> llvm/examples/Kaleidoscope/BuildingAJIT/Chapter1/KaleidoscopeJIT.h
>> llvm/examples/Kaleidoscope/BuildingAJIT/Chapter2/KaleidoscopeJIT.h
>> llvm/include/llvm/ExecutionEngine/JITLink/JITLink.h
>> llvm/include/llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h
>> llvm/include/llvm/ExecutionEngine/Orc/CompileUtils.h
>> llvm/include/llvm/ExecutionEngine/Orc/Core.h
>> llvm/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h
>> llvm/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h
>> llvm/include/llvm/ExecutionEngine/Orc/IRTransformLayer.h
>> llvm/include/llvm/ExecutionEngine/Orc/IndirectionUtils.h
>> llvm/include/llvm/ExecutionEngine/Orc/LLJIT.h
>> llvm/include/llvm/ExecutionEngine/Orc/Layer.h
>> llvm/include/llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h
>> llvm/include/llvm/ExecutionEngine/Orc/SymbolStringPool.h
>> llvm/lib/ExecutionEngine/Orc/CMakeLists.txt
>> llvm/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp
>> llvm/lib/ExecutionEngine/Orc/CompileUtils.cpp
>> llvm/lib/ExecutionEngine/Orc/Core.cpp
>> llvm/lib/ExecutionEngine/Orc/ExecutionUtils.cpp
>> llvm/lib/ExecutionEngine/Orc/IndirectionUtils.cpp
>> llvm/lib/ExecutionEngine/Orc/LLJIT.cpp
>> llvm/lib/ExecutionEngine/Orc/Layer.cpp
>> llvm/lib/ExecutionEngine/Orc/LazyReexports.cpp
>> llvm/lib/ExecutionEngine/Orc/ObjectLinkingLayer.cpp
>> llvm/lib/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.cpp
>> llvm/tools/lli/lli.cpp
>> llvm/tools/llvm-jitlink/llvm-jitlink.cpp
>> llvm/tools/llvm-jitlink/llvm-jitlink.h
>> llvm/unittests/ExecutionEngine/Orc/CoreAPIsTest.cpp
>> llvm/unittests/ExecutionEngine/Orc/OrcTestCommon.h
>> llvm/unittests/ExecutionEngine/Orc/RTDyldObjectLinkingLayerTest.cpp
>>
>> Removed:
>>
>>
>>
>> ################################################################################
>> diff --git a/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter1/KaleidoscopeJIT.h b/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter1/KaleidoscopeJIT.h
>> index b7404baf1ff0..adb6be436660 100644
>> --- a/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter1/KaleidoscopeJIT.h
>> +++ b/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter1/KaleidoscopeJIT.h
>> @@ -49,7 +49,7 @@ class KaleidoscopeJIT {
>> std::make_unique<ConcurrentIRCompiler>(std::move(JTMB))),
>> DL(std::move(DL)), Mangle(ES, this->DL),
>> Ctx(std::make_unique<LLVMContext>()),
>> - MainJD(ES.createJITDylib("<main>")) {
>> + MainJD(ES.createBareJITDylib("<main>")) {
>> MainJD.addGenerator(
>> cantFail(DynamicLibrarySearchGenerator::GetForCurrentProcess(
>> DL.getGlobalPrefix())));
>>
>> diff --git a/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter2/KaleidoscopeJIT.h b/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter2/KaleidoscopeJIT.h
>> index efb19349e3e1..28649ffe4910 100644
>> --- a/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter2/KaleidoscopeJIT.h
>> +++ b/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter2/KaleidoscopeJIT.h
>> @@ -55,7 +55,7 @@ class KaleidoscopeJIT {
>> std::make_unique<ConcurrentIRCompiler>(std::move(JTMB))),
>> OptimizeLayer(ES, CompileLayer, optimizeModule), DL(std::move(DL)),
>> Mangle(ES, this->DL), Ctx(std::make_unique<LLVMContext>()),
>> - MainJD(ES.createJITDylib("<main>")) {
>> + MainJD(ES.createBareJITDylib("<main>")) {
>> MainJD.addGenerator(
>> cantFail(DynamicLibrarySearchGenerator::GetForCurrentProcess(
>> DL.getGlobalPrefix())));
>>
>> diff --git a/llvm/include/llvm/ExecutionEngine/JITLink/JITLink.h b/llvm/include/llvm/ExecutionEngine/JITLink/JITLink.h
>> index 1a8f5cacbcd4..d673a89386da 100644
>> --- a/llvm/include/llvm/ExecutionEngine/JITLink/JITLink.h
>> +++ b/llvm/include/llvm/ExecutionEngine/JITLink/JITLink.h
>> @@ -488,6 +488,8 @@ class Symbol {
>>
>> /// Set the visibility for this Symbol.
>> void setScope(Scope S) {
>> + assert((!Name.empty() || S == Scope::Local) &&
>> + "Can not set anonymous symbol to non-local scope");
>> assert((S == Scope::Default || Base->isDefined() || Base->isAbsolute()) &&
>> "Invalid visibility for symbol type");
>> this->S = static_cast<uint8_t>(S);
>>
>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h b/llvm/include/llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h
>> index 65295f97e2c6..ef251ac302df 100644
>> --- a/llvm/include/llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h
>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h
>> @@ -94,6 +94,7 @@ class CompileOnDemandLayer : public IRLayer {
>>
>> /// Sets the ImplSymbolMap
>> void setImplMap(ImplSymbolMap *Imp);
>> +
>> /// Emits the given module. This should not be called by clients: it will be
>> /// called by the JIT when a definition added via the add method is requested.
>> void emit(MaterializationResponsibility R, ThreadSafeModule TSM) override;
>>
>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/CompileUtils.h b/llvm/include/llvm/ExecutionEngine/Orc/CompileUtils.h
>> index 218afda1b546..8376d163d57a 100644
>> --- a/llvm/include/llvm/ExecutionEngine/Orc/CompileUtils.h
>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/CompileUtils.h
>> @@ -30,7 +30,7 @@ namespace orc {
>>
>> class JITTargetMachineBuilder;
>>
>> -IRMaterializationUnit::ManglingOptions
>> +IRSymbolMapper::ManglingOptions
>> irManglingOptionsFromTargetOptions(const TargetOptions &Opts);
>>
>> /// Simple compile functor: Takes a single IR module and returns an ObjectFile.
>> @@ -52,7 +52,7 @@ class SimpleCompiler : public IRCompileLayer::IRCompiler {
>> Expected<CompileResult> operator()(Module &M) override;
>>
>> private:
>> - IRMaterializationUnit::ManglingOptions
>> + IRSymbolMapper::ManglingOptions
>> manglingOptionsForTargetMachine(const TargetMachine &TM);
>>
>> CompileResult tryToLoadFromObjectCache(const Module &M);
>>
>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/Core.h b/llvm/include/llvm/ExecutionEngine/Orc/Core.h
>> index 1a55801b9d87..94600f2286b8 100644
>> --- a/llvm/include/llvm/ExecutionEngine/Orc/Core.h
>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/Core.h
>> @@ -14,11 +14,11 @@
>> #define LLVM_EXECUTIONENGINE_ORC_CORE_H
>>
>> #include "llvm/ADT/BitmaskEnum.h"
>> +#include "llvm/ADT/DenseSet.h"
>> #include "llvm/ADT/FunctionExtras.h"
>> #include "llvm/ExecutionEngine/JITSymbol.h"
>> #include "llvm/ExecutionEngine/Orc/SymbolStringPool.h"
>> #include "llvm/ExecutionEngine/OrcV1Deprecation.h"
>> -#include "llvm/IR/Module.h"
>> #include "llvm/Support/Debug.h"
>>
>> #include <memory>
>> @@ -456,6 +456,11 @@ class MaterializationResponsibility {
>> /// before using.
>> const SymbolFlagsMap &getSymbols() const { return SymbolFlags; }
>>
>> + /// Returns the initialization pseudo-symbol, if any. This symbol will also
>> + /// be present in the SymbolFlagsMap for this MaterializationResponsibility
>> + /// object.
>> + const SymbolStringPtr &getInitializerSymbol() const { return InitSymbol; }
>> +
>> /// Returns the names of any symbols covered by this
>> /// MaterializationResponsibility object that have queries pending. This
>> /// information can be used to return responsibility for unrequested symbols
>> @@ -532,10 +537,15 @@ class MaterializationResponsibility {
>> /// Create a MaterializationResponsibility for the given JITDylib and
>> /// initial symbols.
>> MaterializationResponsibility(JITDylib &JD, SymbolFlagsMap SymbolFlags,
>> - VModuleKey K);
>> + SymbolStringPtr InitSymbol, VModuleKey K)
>> + : JD(JD), SymbolFlags(std::move(SymbolFlags)),
>> + InitSymbol(std::move(InitSymbol)), K(std::move(K)) {
>> + assert(!this->SymbolFlags.empty() && "Materializing nothing?");
>> + }
>>
>> JITDylib &JD;
>> SymbolFlagsMap SymbolFlags;
>> + SymbolStringPtr InitSymbol;
>> VModuleKey K;
>> };
>>
>> @@ -549,8 +559,13 @@ class MaterializationResponsibility {
>> /// stronger definition is added or already present.
>> class MaterializationUnit {
>> public:
>> - MaterializationUnit(SymbolFlagsMap InitalSymbolFlags, VModuleKey K)
>> - : SymbolFlags(std::move(InitalSymbolFlags)), K(std::move(K)) {}
>> + MaterializationUnit(SymbolFlagsMap InitalSymbolFlags,
>> + SymbolStringPtr InitSymbol, VModuleKey K)
>> + : SymbolFlags(std::move(InitalSymbolFlags)),
>> + InitSymbol(std::move(InitSymbol)), K(std::move(K)) {
>> + assert((!this->InitSymbol || this->SymbolFlags.count(this->InitSymbol)) &&
>> + "If set, InitSymbol should appear in InitialSymbolFlags map");
>> + }
>>
>> virtual ~MaterializationUnit() {}
>>
>> @@ -561,12 +576,15 @@ class MaterializationUnit {
>> /// Return the set of symbols that this source provides.
>> const SymbolFlagsMap &getSymbols() const { return SymbolFlags; }
>>
>> + /// Returns the initialization symbol for this MaterializationUnit (if any).
>> + const SymbolStringPtr &getInitializerSymbol() const { return InitSymbol; }
>> +
>> /// Called by materialization dispatchers (see
>> /// ExecutionSession::DispatchMaterializationFunction) to trigger
>> /// materialization of this MaterializationUnit.
>> void doMaterialize(JITDylib &JD) {
>> - materialize(MaterializationResponsibility(JD, std::move(SymbolFlags),
>> - std::move(K)));
>> + materialize(MaterializationResponsibility(
>> + JD, std::move(SymbolFlags), std::move(InitSymbol), std::move(K)));
>> }
>>
>> /// Called by JITDylibs to notify MaterializationUnits that the given symbol
>> @@ -578,6 +596,7 @@ class MaterializationUnit {
>>
>> protected:
>> SymbolFlagsMap SymbolFlags;
>> + SymbolStringPtr InitSymbol;
>> VModuleKey K;
>>
>> private:
>> @@ -774,6 +793,7 @@ class AsynchronousSymbolQuery {
>> class JITDylib {
>> friend class AsynchronousSymbolQuery;
>> friend class ExecutionSession;
>> + friend class Platform;
>> friend class MaterializationResponsibility;
>> public:
>> /// Definition generators can be attached to JITDylibs to generate new
>> @@ -1054,6 +1074,35 @@ class JITDylib {
>> JITDylibSearchOrder SearchOrder;
>> };
>>
>> +/// Platforms set up standard symbols and mediate interactions between dynamic
>> +/// initializers (e.g. C++ static constructors) and ExecutionSession state.
>> +/// Note that Platforms do not automatically run initializers: clients are still
>> +/// responsible for doing this.
>> +class Platform {
>> +public:
>> + virtual ~Platform();
>> +
>> + /// This method will be called outside the session lock each time a JITDylib
>> + /// is created (unless it is created with EmptyJITDylib set) to allow the
>> + /// Platform to install any JITDylib specific standard symbols (e.g
>> + /// __dso_handle).
>> + virtual Error setupJITDylib(JITDylib &JD) = 0;
>> +
>> + /// This method will be called under the ExecutionSession lock each time a
>> + /// MaterializationUnit is added to a JITDylib.
>> + virtual Error notifyAdding(JITDylib &JD, const MaterializationUnit &MU) = 0;
>> +
>> + /// This method will be called under the ExecutionSession lock when a
>> + /// VModuleKey is removed.
>> + virtual Error notifyRemoving(JITDylib &JD, VModuleKey K) = 0;
>> +
>> + /// A utility function for looking up initializer symbols. Performs a blocking
>> + /// lookup for the given symbols in each of the given JITDylibs.
>> + static Expected<DenseMap<JITDylib *, SymbolMap>>
>> + lookupInitSymbols(ExecutionSession &ES,
>> + const DenseMap<JITDylib *, SymbolLookupSet> &InitSyms);
>> +};
>> +
>> /// An ExecutionSession represents a running JIT program.
>> class ExecutionSession {
>> // FIXME: Remove this when we remove the old ORC layers.
>> @@ -1078,6 +1127,13 @@ class ExecutionSession {
>> /// Returns a shared_ptr to the SymbolStringPool for this ExecutionSession.
>> std::shared_ptr<SymbolStringPool> getSymbolStringPool() const { return SSP; }
>>
>> + /// Set the Platform for this ExecutionSession.
>> + void setPlatform(std::unique_ptr<Platform> P) { this->P = std::move(P); }
>> +
>> + /// Get the Platform for this session.
>> + /// Will return null if no Platform has been set for this ExecutionSession.
>> + Platform *getPlatform() { return P.get(); }
>> +
>> /// Run the given lambda with the session mutex locked.
>> template <typename Func> decltype(auto) runSessionLocked(Func &&F) {
>> std::lock_guard<std::recursive_mutex> Lock(SessionMutex);
>> @@ -1088,12 +1144,26 @@ class ExecutionSession {
>> /// Ownership of JITDylib remains within Execution Session
>> JITDylib *getJITDylibByName(StringRef Name);
>>
>> + /// Add a new bare JITDylib to this ExecutionSession.
>> + ///
>> + /// The JITDylib Name is required to be unique. Clients should verify that
>> + /// names are not being re-used (E.g. by calling getJITDylibByName) if names
>> + /// are based on user input.
>> + ///
>> + /// This call does not install any library code or symbols into the newly
>> + /// created JITDylib. The client is responsible for all configuration.
>> + JITDylib &createBareJITDylib(std::string Name);
>> +
>> /// Add a new JITDylib to this ExecutionSession.
>> ///
>> /// The JITDylib Name is required to be unique. Clients should verify that
>> /// names are not being re-used (e.g. by calling getJITDylibByName) if names
>> /// are based on user input.
>> - JITDylib &createJITDylib(std::string Name);
>> + ///
>> + /// If a Platform is attached then Platform::setupJITDylib will be called to
>> + /// install standard platform symbols (e.g. standard library interposes).
>> + /// If no Platform is attached this call is equivalent to createBareJITDylib.
>> + Expected<JITDylib &> createJITDylib(std::string Name);
>>
>> /// Allocate a module key for a new module to add to the JIT.
>> VModuleKey allocateVModule() {
>> @@ -1177,20 +1247,23 @@ class ExecutionSession {
>> /// Convenience version of blocking lookup.
>> /// Searches each of the JITDylibs in the search order in turn for the given
>> /// symbol.
>> - Expected<JITEvaluatedSymbol> lookup(const JITDylibSearchOrder &SearchOrder,
>> - SymbolStringPtr Symbol);
>> + Expected<JITEvaluatedSymbol>
>> + lookup(const JITDylibSearchOrder &SearchOrder, SymbolStringPtr Symbol,
>> + SymbolState RequiredState = SymbolState::Ready);
>>
>> /// Convenience version of blocking lookup.
>> /// Searches each of the JITDylibs in the search order in turn for the given
>> /// symbol. The search will not find non-exported symbols.
>> - Expected<JITEvaluatedSymbol> lookup(ArrayRef<JITDylib *> SearchOrder,
>> - SymbolStringPtr Symbol);
>> + Expected<JITEvaluatedSymbol>
>> + lookup(ArrayRef<JITDylib *> SearchOrder, SymbolStringPtr Symbol,
>> + SymbolState RequiredState = SymbolState::Ready);
>>
>> /// Convenience version of blocking lookup.
>> /// Searches each of the JITDylibs in the search order in turn for the given
>> /// symbol. The search will not find non-exported symbols.
>> - Expected<JITEvaluatedSymbol> lookup(ArrayRef<JITDylib *> SearchOrder,
>> - StringRef Symbol);
>> + Expected<JITEvaluatedSymbol>
>> + lookup(ArrayRef<JITDylib *> SearchOrder, StringRef Symbol,
>> + SymbolState RequiredState = SymbolState::Ready);
>>
>> /// Materialize the given unit.
>> void dispatchMaterialization(JITDylib &JD,
>> @@ -1221,6 +1294,7 @@ class ExecutionSession {
>>
>> mutable std::recursive_mutex SessionMutex;
>> std::shared_ptr<SymbolStringPool> SSP;
>> + std::unique_ptr<Platform> P;
>> VModuleKey LastKey = 0;
>> ErrorReporter ReportError = logErrorsToStdErr;
>> DispatchMaterializationFunction DispatchMaterialization =
>> @@ -1256,6 +1330,11 @@ Error JITDylib::define(std::unique_ptr<MaterializationUnitType> &&MU) {
>> if (auto Err = defineImpl(*MU))
>> return Err;
>>
>> + if (auto *P = ES.getPlatform()) {
>> + if (auto Err = P->notifyAdding(*this, *MU))
>> + return Err;
>> + }
>> +
>> /// defineImpl succeeded.
>> auto UMI = std::make_shared<UnmaterializedInfo>(std::move(MU));
>> for (auto &KV : UMI->MU->getSymbols())
>> @@ -1273,6 +1352,11 @@ Error JITDylib::define(std::unique_ptr<MaterializationUnitType> &MU) {
>> if (auto Err = defineImpl(*MU))
>> return Err;
>>
>> + if (auto *P = ES.getPlatform()) {
>> + if (auto Err = P->notifyAdding(*this, *MU))
>> + return Err;
>> + }
>> +
>> /// defineImpl succeeded.
>> auto UMI = std::make_shared<UnmaterializedInfo>(std::move(MU));
>> for (auto &KV : UMI->MU->getSymbols())
>> @@ -1305,18 +1389,6 @@ class ReexportsGenerator : public JITDylib::DefinitionGenerator {
>> SymbolPredicate Allow;
>> };
>>
>> -/// Mangles symbol names then uniques them in the context of an
>> -/// ExecutionSession.
>> -class MangleAndInterner {
>> -public:
>> - MangleAndInterner(ExecutionSession &ES, const DataLayout &DL);
>> - SymbolStringPtr operator()(StringRef Name);
>> -
>> -private:
>> - ExecutionSession &ES;
>> - const DataLayout &DL;
>> -};
>> -
>> } // End namespace orc
>> } // End namespace llvm
>>
>>
>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h b/llvm/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h
>> index f7255c5af845..49ab0d628b0b 100644
>> --- a/llvm/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h
>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h
>> @@ -17,6 +17,7 @@
>> #include "llvm/ADT/iterator_range.h"
>> #include "llvm/ExecutionEngine/JITSymbol.h"
>> #include "llvm/ExecutionEngine/Orc/Core.h"
>> +#include "llvm/ExecutionEngine/Orc/Mangling.h"
>> #include "llvm/ExecutionEngine/Orc/OrcError.h"
>> #include "llvm/ExecutionEngine/RuntimeDyld.h"
>> #include "llvm/Object/Archive.h"
>> @@ -104,6 +105,53 @@ iterator_range<CtorDtorIterator> getConstructors(const Module &M);
>> /// array.
>> iterator_range<CtorDtorIterator> getDestructors(const Module &M);
>>
>> +/// This iterator provides a convenient way to iterate over GlobalValues that
>> +/// have initialization effects.
>> +class StaticInitGVIterator {
>> +public:
>> + StaticInitGVIterator() = default;
>> +
>> + StaticInitGVIterator(Module &M)
>> + : I(M.global_values().begin()), E(M.global_values().end()),
>> + ObjFmt(Triple(M.getTargetTriple()).getObjectFormat()) {
>> + if (I != E) {
>> + if (!isStaticInitGlobal(*I))
>> + moveToNextStaticInitGlobal();
>> + } else
>> + I = E = Module::global_value_iterator();
>> + }
>> +
>> + bool operator==(const StaticInitGVIterator &O) const { return I == O.I; }
>> + bool operator!=(const StaticInitGVIterator &O) const { return I != O.I; }
>> +
>> + StaticInitGVIterator &operator++() {
>> + assert(I != E && "Increment past end of range");
>> + moveToNextStaticInitGlobal();
>> + return *this;
>> + }
>> +
>> + GlobalValue &operator*() { return *I; }
>> +
>> +private:
>> + bool isStaticInitGlobal(GlobalValue &GV);
>> + void moveToNextStaticInitGlobal() {
>> + ++I;
>> + while (I != E && !isStaticInitGlobal(*I))
>> + ++I;
>> + if (I == E)
>> + I = E = Module::global_value_iterator();
>> + }
>> +
>> + Module::global_value_iterator I, E;
>> + Triple::ObjectFormatType ObjFmt;
>> +};
>> +
>> +/// Create an iterator range over the GlobalValues that contribute to static
>> +/// initialization.
>> +inline iterator_range<StaticInitGVIterator> getStaticInitGVs(Module &M) {
>> + return make_range(StaticInitGVIterator(M), StaticInitGVIterator());
>> +}
>> +
>> /// Convenience class for recording constructor/destructor names for
>> /// later execution.
>> template <typename JITLayerT>
>> @@ -246,6 +294,22 @@ class LocalCXXRuntimeOverrides : public LocalCXXRuntimeOverridesBase {
>> Error enable(JITDylib &JD, MangleAndInterner &Mangler);
>> };
>>
>> +/// An interface for Itanium __cxa_atexit interposer implementations.
>> +class ItaniumCXAAtExitSupport {
>> +public:
>> + struct AtExitRecord {
>> + void (*F)(void *);
>> + void *Ctx;
>> + };
>> +
>> + void registerAtExit(void (*F)(void *), void *Ctx, void *DSOHandle);
>> + void runAtExits(void *DSOHandle);
>> +
>> +private:
>> + std::mutex AtExitsMutex;
>> + DenseMap<void *, std::vector<AtExitRecord>> AtExitRecords;
>> +};
>> +
>> /// A utility class to expose symbols found via dlsym to the JIT.
>> ///
>> /// If an instance of this class is attached to a JITDylib as a fallback
>>
>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h b/llvm/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h
>> index bb8270fe80a3..eb74d283f043 100644
>> --- a/llvm/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h
>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h
>> @@ -31,18 +31,18 @@ class IRCompileLayer : public IRLayer {
>> public:
>> class IRCompiler {
>> public:
>> - IRCompiler(IRMaterializationUnit::ManglingOptions MO) : MO(std::move(MO)) {}
>> + IRCompiler(IRSymbolMapper::ManglingOptions MO) : MO(std::move(MO)) {}
>> virtual ~IRCompiler();
>> - const IRMaterializationUnit::ManglingOptions &getManglingOptions() const {
>> + const IRSymbolMapper::ManglingOptions &getManglingOptions() const {
>> return MO;
>> }
>> virtual Expected<std::unique_ptr<MemoryBuffer>> operator()(Module &M) = 0;
>>
>> protected:
>> - IRMaterializationUnit::ManglingOptions &manglingOptions() { return MO; }
>> + IRSymbolMapper::ManglingOptions &manglingOptions() { return MO; }
>>
>> private:
>> - IRMaterializationUnit::ManglingOptions MO;
>> + IRSymbolMapper::ManglingOptions MO;
>> };
>>
>> using NotifyCompiledFunction =
>> @@ -61,7 +61,7 @@ class IRCompileLayer : public IRLayer {
>> mutable std::mutex IRLayerMutex;
>> ObjectLayer &BaseLayer;
>> std::unique_ptr<IRCompiler> Compile;
>> - const IRMaterializationUnit::ManglingOptions *ManglingOpts;
>> + const IRSymbolMapper::ManglingOptions *ManglingOpts;
>> NotifyCompiledFunction NotifyCompiled = NotifyCompiledFunction();
>> };
>>
>>
>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/IRTransformLayer.h b/llvm/include/llvm/ExecutionEngine/Orc/IRTransformLayer.h
>> index b71e5b339711..296d74ae6b86 100644
>> --- a/llvm/include/llvm/ExecutionEngine/Orc/IRTransformLayer.h
>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/IRTransformLayer.h
>> @@ -28,7 +28,7 @@ namespace orc {
>> class IRTransformLayer : public IRLayer {
>> public:
>> using TransformFunction = std::function<Expected<ThreadSafeModule>(
>> - ThreadSafeModule, const MaterializationResponsibility &R)>;
>> + ThreadSafeModule, MaterializationResponsibility &R)>;
>>
>> IRTransformLayer(ExecutionSession &ES, IRLayer &BaseLayer,
>> TransformFunction Transform = identityTransform);
>> @@ -39,9 +39,8 @@ class IRTransformLayer : public IRLayer {
>>
>> void emit(MaterializationResponsibility R, ThreadSafeModule TSM) override;
>>
>> - static ThreadSafeModule
>> - identityTransform(ThreadSafeModule TSM,
>> - const MaterializationResponsibility &R) {
>> + static ThreadSafeModule identityTransform(ThreadSafeModule TSM,
>> + MaterializationResponsibility &R) {
>> return TSM;
>> }
>>
>>
>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/IndirectionUtils.h b/llvm/include/llvm/ExecutionEngine/Orc/IndirectionUtils.h
>> index a9ab3a630a64..880c8ff44c5f 100644
>> --- a/llvm/include/llvm/ExecutionEngine/Orc/IndirectionUtils.h
>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/IndirectionUtils.h
>> @@ -201,7 +201,7 @@ class JITCompileCallbackManager {
>> ExecutionSession &ES,
>> JITTargetAddress ErrorHandlerAddress)
>> : TP(std::move(TP)), ES(ES),
>> - CallbacksJD(ES.createJITDylib("<Callbacks>")),
>> + CallbacksJD(ES.createBareJITDylib("<Callbacks>")),
>> ErrorHandlerAddress(ErrorHandlerAddress) {}
>>
>> void setTrampolinePool(std::unique_ptr<TrampolinePool> TP) {
>>
>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/LLJIT.h b/llvm/include/llvm/ExecutionEngine/Orc/LLJIT.h
>> index 3374a29f04df..c0b5012f801f 100644
>> --- a/llvm/include/llvm/ExecutionEngine/Orc/LLJIT.h
>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/LLJIT.h
>> @@ -35,7 +35,23 @@ class LLLazyJITBuilderState;
>> class LLJIT {
>> template <typename, typename, typename> friend class LLJITBuilderSetters;
>>
>> + friend void setUpGenericLLVMIRPlatform(LLJIT &J);
>> +
>> public:
>> + /// Initializer support for LLJIT.
>> + class PlatformSupport {
>> + public:
>> + virtual ~PlatformSupport();
>> +
>> + virtual Error initialize(JITDylib &JD) = 0;
>> +
>> + virtual Error deinitialize(JITDylib &JD) = 0;
>> +
>> + protected:
>> + static void setInitTransform(LLJIT &J,
>> + IRTransformLayer::TransformFunction T);
>> + };
>> +
>> static Expected<std::unique_ptr<LLJIT>> Create(LLJITBuilderState &S);
>>
>> /// Destruct this instance. If a multi-threaded instance, waits for all
>> @@ -52,7 +68,7 @@ class LLJIT {
>> const DataLayout &getDataLayout() const { return DL; }
>>
>> /// Returns a reference to the JITDylib representing the JIT'd main program.
>> - JITDylib &getMainJITDylib() { return Main; }
>> + JITDylib &getMainJITDylib() { return *Main; }
>>
>> /// Returns the JITDylib with the given name, or nullptr if no JITDylib with
>> /// that name exists.
>> @@ -66,7 +82,7 @@ class LLJIT {
>> /// input or elsewhere in the environment then the client should check
>> /// (e.g. by calling getJITDylibByName) that the given name is not already in
>> /// use.
>> - JITDylib &createJITDylib(std::string Name) {
>> + Expected<JITDylib &> createJITDylib(std::string Name) {
>> return ES->createJITDylib(std::move(Name));
>> }
>>
>> @@ -78,7 +94,7 @@ class LLJIT {
>>
>> /// Adds an IR module to the Main JITDylib.
>> Error addIRModule(ThreadSafeModule TSM) {
>> - return addIRModule(Main, std::move(TSM));
>> + return addIRModule(*Main, std::move(TSM));
>> }
>>
>> /// Adds an object file to the given JITDylib.
>> @@ -86,7 +102,7 @@ class LLJIT {
>>
>> /// Adds an object file to the given JITDylib.
>> Error addObjectFile(std::unique_ptr<MemoryBuffer> Obj) {
>> - return addObjectFile(Main, std::move(Obj));
>> + return addObjectFile(*Main, std::move(Obj));
>> }
>>
>> /// Look up a symbol in JITDylib JD by the symbol's linker-mangled name (to
>> @@ -98,7 +114,7 @@ class LLJIT {
>> /// (to look up symbols based on their IR name use the lookup function
>> /// instead).
>> Expected<JITEvaluatedSymbol> lookupLinkerMangled(StringRef Name) {
>> - return lookupLinkerMangled(Main, Name);
>> + return lookupLinkerMangled(*Main, Name);
>> }
>>
>> /// Look up a symbol in JITDylib JD based on its IR symbol name.
>> @@ -108,14 +124,28 @@ class LLJIT {
>>
>> /// Look up a symbol in the main JITDylib based on its IR symbol name.
>> Expected<JITEvaluatedSymbol> lookup(StringRef UnmangledName) {
>> - return lookup(Main, UnmangledName);
>> + return lookup(*Main, UnmangledName);
>> }
>>
>> - /// Runs all not-yet-run static constructors.
>> - Error runConstructors() { return CtorRunner.run(); }
>> + /// Set the PlatformSupport instance.
>> + void setPlatformSupport(std::unique_ptr<PlatformSupport> PS) {
>> + this->PS = std::move(PS);
>> + }
>>
>> - /// Runs all not-yet-run static destructors.
>> - Error runDestructors() { return DtorRunner.run(); }
>> + /// Get the PlatformSupport instance.
>> + PlatformSupport *getPlatformSupport() { return PS.get(); }
>> +
>> + /// Run the initializers for the given JITDylib.
>> + Error initialize(JITDylib &JD) {
>> + assert(PS && "PlatformSupport must be set to run initializers.");
>> + return PS->initialize(JD);
>> + }
>> +
>> + /// Run the deinitializers for the given JITDylib.
>> + Error deinitialize(JITDylib &JD) {
>> + assert(PS && "PlatformSupport must be set to run initializers.");
>> + return PS->deinitialize(JD);
>> + }
>>
>> /// Returns a reference to the ObjLinkingLayer
>> ObjectLayer &getObjLinkingLayer() { return *ObjLinkingLayer; }
>> @@ -126,6 +156,9 @@ class LLJIT {
>> /// Returns a reference to the IR transform layer.
>> IRTransformLayer &getIRTransformLayer() { return *TransformLayer; }
>>
>> + /// Returns a reference to the IR compile layer.
>> + IRCompileLayer &getIRCompileLayer() { return *CompileLayer; }
>> +
>> protected:
>> static std::unique_ptr<ObjectLayer>
>> createObjectLinkingLayer(LLJITBuilderState &S, ExecutionSession &ES);
>> @@ -143,7 +176,9 @@ class LLJIT {
>> void recordCtorDtors(Module &M);
>>
>> std::unique_ptr<ExecutionSession> ES;
>> - JITDylib &Main;
>> + std::unique_ptr<PlatformSupport> PS;
>> +
>> + JITDylib *Main = nullptr;
>>
>> DataLayout DL;
>> Triple TT;
>> @@ -153,8 +188,7 @@ class LLJIT {
>> ObjectTransformLayer ObjTransformLayer;
>> std::unique_ptr<IRCompileLayer> CompileLayer;
>> std::unique_ptr<IRTransformLayer> TransformLayer;
>> -
>> - CtorDtorRunner CtorRunner, DtorRunner;
>> + std::unique_ptr<IRTransformLayer> InitHelperTransformLayer;
>> };
>>
>> /// An extended version of LLJIT that supports lazy function-at-a-time
>> @@ -175,7 +209,7 @@ class LLLazyJIT : public LLJIT {
>>
>> /// Add a module to be lazily compiled to the main JITDylib.
>> Error addLazyIRModule(ThreadSafeModule M) {
>> - return addLazyIRModule(Main, std::move(M));
>> + return addLazyIRModule(*Main, std::move(M));
>> }
>>
>> private:
>> @@ -196,10 +230,14 @@ class LLJITBuilderState {
>> std::function<Expected<std::unique_ptr<IRCompileLayer::IRCompiler>>(
>> JITTargetMachineBuilder JTMB)>;
>>
>> + using PlatformSetupFunction = std::function<Error(LLJIT &J)>;
>> +
>> std::unique_ptr<ExecutionSession> ES;
>> Optional<JITTargetMachineBuilder> JTMB;
>> + Optional<DataLayout> DL;
>> ObjectLinkingLayerCreator CreateObjectLinkingLayer;
>> CompileFunctionCreator CreateCompileFunction;
>> + PlatformSetupFunction SetUpPlatform;
>> unsigned NumCompileThreads = 0;
>>
>> /// Called prior to JIT class construcion to fix up defaults.
>> @@ -224,6 +262,13 @@ class LLJITBuilderSetters {
>> return impl().JTMB;
>> }
>>
>> + /// Set a DataLayout for this instance. If no data layout is specified then
>> + /// the target's default data layout will be used.
>> + SetterImpl &setDataLayout(Optional<DataLayout> DL) {
>> + impl().DL = std::move(DL);
>> + return impl();
>> + }
>> +
>> /// Set an ObjectLinkingLayer creation function.
>> ///
>> /// If this method is not called, a default creation function will be used
>> @@ -246,6 +291,16 @@ class LLJITBuilderSetters {
>> return impl();
>> }
>>
>> + /// Set up an PlatformSetupFunction.
>> + ///
>> + /// If this method is not called then setUpGenericLLVMIRPlatform
>> + /// will be used to configure the JIT's platform support.
>> + SetterImpl &
>> + setPlatformSetUp(LLJITBuilderState::PlatformSetupFunction SetUpPlatform) {
>> + impl().SetUpPlatform = std::move(SetUpPlatform);
>> + return impl();
>> + }
>> +
>> /// Set the number of compile threads to use.
>> ///
>> /// If set to zero, compilation will be performed on the execution thread when
>> @@ -334,6 +389,26 @@ class LLLazyJITBuilder
>> public LLLazyJITBuilderSetters<LLLazyJIT, LLLazyJITBuilder,
>> LLLazyJITBuilderState> {};
>>
>> +/// Configure the LLJIT instance to scrape modules for llvm.global_ctors and
>> +/// llvm.global_dtors variables and (if present) build initialization and
>> +/// deinitialization functions. Platform specific initialization configurations
>> +/// should be preferred where available.
>> +void setUpGenericLLVMIRPlatform(LLJIT &J);
>> +
>> +/// Configure the LLJIT instance to use MachOPlatform support.
>> +///
>> +/// Warning: MachOPlatform *requires* that LLJIT be configured to use
>> +/// ObjectLinkingLayer (default on platforms supported by JITLink). If
>> +/// MachOPlatform is used with RTDyldObjectLinkingLayer it will result in
>> +/// undefined behavior).
>> +///
>> +/// MachOPlatform installs an ObjectLinkingLayer plugin to scrape initializers
>> +/// from the __mod_inits section. It also provides interposes for the dlfcn
>> +/// functions (dlopen, dlclose, dlsym, dlerror) that work for JITDylibs as
>> +/// well as regular libraries (JITDylibs will be preferenced, so make sure
>> +/// your JITDylib names do not shadow any real library paths).
>> +Error setUpMachOPlatform(LLJIT &J);
>> +
>> } // End namespace orc
>> } // End namespace llvm
>>
>>
>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/Layer.h b/llvm/include/llvm/ExecutionEngine/Orc/Layer.h
>> index 95e32b2431a0..e843d0f56245 100644
>> --- a/llvm/include/llvm/ExecutionEngine/Orc/Layer.h
>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/Layer.h
>> @@ -14,6 +14,7 @@
>> #define LLVM_EXECUTIONENGINE_ORC_LAYER_H
>>
>> #include "llvm/ExecutionEngine/Orc/Core.h"
>> +#include "llvm/ExecutionEngine/Orc/Mangling.h"
>> #include "llvm/ExecutionEngine/Orc/ThreadSafeModule.h"
>> #include "llvm/IR/Module.h"
>> #include "llvm/Support/MemoryBuffer.h"
>> @@ -27,15 +28,12 @@ namespace orc {
>> /// their linkage is changed to available-externally.
>> class IRMaterializationUnit : public MaterializationUnit {
>> public:
>> - struct ManglingOptions {
>> - bool EmulatedTLS = false;
>> - };
>> -
>> using SymbolNameToDefinitionMap = std::map<SymbolStringPtr, GlobalValue *>;
>>
>> /// Create an IRMaterializationLayer. Scans the module to build the
>> /// SymbolFlags and SymbolToDefinition maps.
>> - IRMaterializationUnit(ExecutionSession &ES, const ManglingOptions &MO,
>> + IRMaterializationUnit(ExecutionSession &ES,
>> + const IRSymbolMapper::ManglingOptions &MO,
>> ThreadSafeModule TSM, VModuleKey K);
>>
>> /// Create an IRMaterializationLayer from a module, and pre-existing
>> @@ -44,12 +42,13 @@ class IRMaterializationUnit : public MaterializationUnit {
>> /// This constructor is useful for delegating work from one
>> /// IRMaterializationUnit to another.
>> IRMaterializationUnit(ThreadSafeModule TSM, VModuleKey K,
>> - SymbolFlagsMap SymbolFlags,
>> + SymbolFlagsMap SymbolFlags, SymbolStringPtr InitSymbol,
>> SymbolNameToDefinitionMap SymbolToDefinition);
>>
>> /// Return the ModuleIdentifier as the name for this MaterializationUnit.
>> StringRef getName() const override;
>>
>> + /// Return a reference to the contained ThreadSafeModule.
>> const ThreadSafeModule &getModule() const { return TSM; }
>>
>> protected:
>> @@ -57,14 +56,16 @@ class IRMaterializationUnit : public MaterializationUnit {
>> SymbolNameToDefinitionMap SymbolToDefinition;
>>
>> private:
>> + static SymbolStringPtr getInitSymbol(ExecutionSession &ES,
>> + const ThreadSafeModule &TSM);
>> +
>> void discard(const JITDylib &JD, const SymbolStringPtr &Name) override;
>> };
>>
>> /// Interface for layers that accept LLVM IR.
>> class IRLayer {
>> public:
>> - IRLayer(ExecutionSession &ES,
>> - const IRMaterializationUnit::ManglingOptions *&MO)
>> + IRLayer(ExecutionSession &ES, const IRSymbolMapper::ManglingOptions *&MO)
>> : ES(ES), MO(MO) {}
>>
>> virtual ~IRLayer();
>> @@ -73,7 +74,7 @@ class IRLayer {
>> ExecutionSession &getExecutionSession() { return ES; }
>>
>> /// Get the mangling options for this layer.
>> - const IRMaterializationUnit::ManglingOptions *&getManglingOptions() const {
>> + const IRSymbolMapper::ManglingOptions *&getManglingOptions() const {
>> return MO;
>> }
>>
>> @@ -104,14 +105,15 @@ class IRLayer {
>> private:
>> bool CloneToNewContextOnEmit = false;
>> ExecutionSession &ES;
>> - const IRMaterializationUnit::ManglingOptions *&MO;
>> + const IRSymbolMapper::ManglingOptions *&MO;
>> };
>>
>> /// MaterializationUnit that materializes modules by calling the 'emit' method
>> /// on the given IRLayer.
>> class BasicIRLayerMaterializationUnit : public IRMaterializationUnit {
>> public:
>> - BasicIRLayerMaterializationUnit(IRLayer &L, const ManglingOptions &MO,
>> + BasicIRLayerMaterializationUnit(IRLayer &L,
>> + const IRSymbolMapper::ManglingOptions &MO,
>> ThreadSafeModule TSM, VModuleKey K);
>>
>> private:
>> @@ -153,7 +155,8 @@ class BasicObjectLayerMaterializationUnit : public MaterializationUnit {
>>
>> BasicObjectLayerMaterializationUnit(ObjectLayer &L, VModuleKey K,
>> std::unique_ptr<MemoryBuffer> O,
>> - SymbolFlagsMap SymbolFlags);
>> + SymbolFlagsMap SymbolFlags,
>> + SymbolStringPtr InitSymbol);
>>
>> /// Return the buffer's identifier as the name for this MaterializationUnit.
>> StringRef getName() const override;
>> @@ -167,12 +170,6 @@ class BasicObjectLayerMaterializationUnit : public MaterializationUnit {
>> std::unique_ptr<MemoryBuffer> O;
>> };
>>
>> -/// Returns a SymbolFlagsMap for the object file represented by the given
>> -/// buffer, or an error if the buffer does not contain a valid object file.
>> -// FIXME: Maybe move to Core.h?
>> -Expected<SymbolFlagsMap> getObjectSymbolFlags(ExecutionSession &ES,
>> - MemoryBufferRef ObjBuffer);
>> -
>> } // End namespace orc
>> } // End namespace llvm
>>
>>
>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/MachOPlatform.h b/llvm/include/llvm/ExecutionEngine/Orc/MachOPlatform.h
>> new file mode 100644
>> index 000000000000..92c1a04921b8
>> --- /dev/null
>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/MachOPlatform.h
>> @@ -0,0 +1,147 @@
>> +//===-- MachOPlatform.h - Utilities for executing MachO in Orc --*- C++ -*-===//
>> +//
>> +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
>> +// See https://llvm.org/LICENSE.txt for license information.
>> +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
>> +//
>> +//===----------------------------------------------------------------------===//
>> +//
>> +// Utilities for executing JIT'd MachO in Orc.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +
>> +#ifndef LLVM_EXECUTIONENGINE_ORC_MACHOPLATFORM_H
>> +#define LLVM_EXECUTIONENGINE_ORC_MACHOPLATFORM_H
>> +
>> +#include "llvm/ADT/StringRef.h"
>> +#include "llvm/ExecutionEngine/Orc/Core.h"
>> +#include "llvm/ExecutionEngine/Orc/ExecutionUtils.h"
>> +#include "llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h"
>> +
>> +#include <future>
>> +#include <thread>
>> +#include <vector>
>> +
>> +namespace llvm {
>> +namespace orc {
>> +
>> +/// Enable registration of JIT'd ObjC classes and selectors.
>> +Error enableObjCRegistration(const char *PathToLibObjC);
>> +bool objCRegistrationEnabled();
>> +
>> +class MachOJITDylibInitializers {
>> +public:
>> + struct SectionExtent {
>> + SectionExtent() = default;
>> + SectionExtent(JITTargetAddress Address, uint64_t NumPtrs)
>> + : Address(Address), NumPtrs(NumPtrs) {}
>> + JITTargetAddress Address = 0;
>> + uint64_t NumPtrs = 0;
>> + };
>> +
>> + void setObjCImageInfoAddr(JITTargetAddress ObjCImageInfoAddr) {
>> + this->ObjCImageInfoAddr = ObjCImageInfoAddr;
>> + }
>> +
>> + void addModInitsSection(SectionExtent ModInit) {
>> + ModInitSections.push_back(std::move(ModInit));
>> + }
>> +
>> + void addObjCSelRefsSection(SectionExtent ObjCSelRefs) {
>> + ObjCSelRefsSections.push_back(std::move(ObjCSelRefs));
>> + }
>> +
>> + void addObjCClassListSection(SectionExtent ObjCClassList) {
>> + ObjCClassListSections.push_back(std::move(ObjCClassList));
>> + }
>> +
>> + void runModInits() const;
>> + void registerObjCSelectors() const;
>> + Error registerObjCClasses() const;
>> +
>> + void dump() const;
>> +
>> +private:
>> + using RawPointerSectionList = std::vector<SectionExtent>;
>> +
>> + JITTargetAddress ObjCImageInfoAddr;
>> + RawPointerSectionList ModInitSections;
>> + RawPointerSectionList ObjCSelRefsSections;
>> + RawPointerSectionList ObjCClassListSections;
>> +};
>> +
>> +class MachOJITDylibDeinitializers {};
>> +
>> +/// Mediates between MachO initialization and ExecutionSession state.
>> +class MachOPlatform : public Platform {
>> +public:
>> + using InitializerSequence =
>> + std::vector<std::pair<JITDylib *, MachOJITDylibInitializers>>;
>> +
>> + using DeinitializerSequence =
>> + std::vector<std::pair<JITDylib *, MachOJITDylibDeinitializers>>;
>> +
>> + MachOPlatform(ExecutionSession &ES, ObjectLinkingLayer &ObjLinkingLayer,
>> + std::unique_ptr<MemoryBuffer> StandardSymbolsObject);
>> +
>> + ExecutionSession &getExecutionSession() const { return ES; }
>> +
>> + Error setupJITDylib(JITDylib &JD) override;
>> + Error notifyAdding(JITDylib &JD, const MaterializationUnit &MU) override;
>> + Error notifyRemoving(JITDylib &JD, VModuleKey K) override;
>> +
>> + Expected<InitializerSequence> getInitializerSequence(JITDylib &JD);
>> +
>> + Expected<DeinitializerSequence> getDeinitializerSequence(JITDylib &JD);
>> +
>> +private:
>> + // This ObjectLinkingLayer plugin scans JITLink graphs for __mod_init_func,
>> + // __objc_classlist and __sel_ref sections and records their extents so that
>> + // they can be run in the target process.
>> + class InitScraperPlugin : public ObjectLinkingLayer::Plugin {
>> + public:
>> + InitScraperPlugin(MachOPlatform &MP) : MP(MP) {}
>> +
>> + void modifyPassConfig(MaterializationResponsibility &MR, const Triple &TT,
>> + jitlink::PassConfiguration &Config) override;
>> +
>> + LocalDependenciesMap getSyntheticSymbolLocalDependencies(
>> + MaterializationResponsibility &MR) override;
>> +
>> + private:
>> + using InitSymbolDepMap =
>> + DenseMap<MaterializationResponsibility *, JITLinkSymbolVector>;
>> +
>> + void preserveInitSectionIfPresent(JITLinkSymbolVector &Syms,
>> + jitlink::LinkGraph &G,
>> + StringRef SectionName);
>> +
>> + Error processObjCImageInfo(jitlink::LinkGraph &G,
>> + MaterializationResponsibility &MR);
>> +
>> + std::mutex InitScraperMutex;
>> + MachOPlatform &MP;
>> + DenseMap<JITDylib *, std::pair<uint32_t, uint32_t>> ObjCImageInfos;
>> + InitSymbolDepMap InitSymbolDeps;
>> + };
>> +
>> + static std::vector<JITDylib *> getDFSLinkOrder(JITDylib &JD);
>> +
>> + void registerInitInfo(JITDylib &JD, JITTargetAddress ObjCImageInfoAddr,
>> + MachOJITDylibInitializers::SectionExtent ModInits,
>> + MachOJITDylibInitializers::SectionExtent ObjCSelRefs,
>> + MachOJITDylibInitializers::SectionExtent ObjCClassList);
>> +
>> + std::mutex PlatformMutex;
>> + ExecutionSession &ES;
>> + ObjectLinkingLayer &ObjLinkingLayer;
>> + std::unique_ptr<MemoryBuffer> StandardSymbolsObject;
>> +
>> + DenseMap<JITDylib *, SymbolLookupSet> RegisteredInitSymbols;
>> + DenseMap<JITDylib *, MachOJITDylibInitializers> InitSeqs;
>> +};
>> +
>> +} // end namespace orc
>> +} // end namespace llvm
>> +
>> +#endif // LLVM_EXECUTIONENGINE_ORC_MACHOPLATFORM_H
>>
>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/Mangling.h b/llvm/include/llvm/ExecutionEngine/Orc/Mangling.h
>> new file mode 100644
>> index 000000000000..e0f770a601fb
>> --- /dev/null
>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/Mangling.h
>> @@ -0,0 +1,66 @@
>> +//===------ Mangling.h -- Name Mangling Utilities for ORC -------*- C++ -*-===//
>> +//
>> +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
>> +// See https://llvm.org/LICENSE.txt for license information.
>> +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
>> +//
>> +//===----------------------------------------------------------------------===//
>> +//
>> +// Name mangling utilities for ORC.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +
>> +#ifndef LLVM_EXECUTIONENGINE_ORC_MANGLING_H
>> +#define LLVM_EXECUTIONENGINE_ORC_MANGLING_H
>> +
>> +#include "llvm/ExecutionEngine/Orc/Core.h"
>> +#include "llvm/ExecutionEngine/Orc/ThreadSafeModule.h"
>> +#include "llvm/IR/Module.h"
>> +#include "llvm/Support/MemoryBuffer.h"
>> +
>> +namespace llvm {
>> +namespace orc {
>> +
>> +/// Mangles symbol names then uniques them in the context of an
>> +/// ExecutionSession.
>> +class MangleAndInterner {
>> +public:
>> + MangleAndInterner(ExecutionSession &ES, const DataLayout &DL);
>> + SymbolStringPtr operator()(StringRef Name);
>> +
>> +private:
>> + ExecutionSession &ES;
>> + const DataLayout &DL;
>> +};
>> +
>> +/// Maps IR global values to their linker symbol names / flags.
>> +///
>> +/// This utility can be used when adding new IR globals in the JIT.
>> +class IRSymbolMapper {
>> +public:
>> + struct ManglingOptions {
>> + bool EmulatedTLS = false;
>> + };
>> +
>> + using SymbolNameToDefinitionMap = std::map<SymbolStringPtr, GlobalValue *>;
>> +
>> + /// Add mangled symbols for the given GlobalValues to SymbolFlags.
>> + /// If a SymbolToDefinitionMap pointer is supplied then it will be populated
>> + /// with Name-to-GlobalValue* mappings. Note that this mapping is not
>> + /// necessarily one-to-one: thread-local GlobalValues, for example, may
>> + /// produce more than one symbol, in which case the map will contain duplicate
>> + /// values.
>> + static void add(ExecutionSession &ES, const ManglingOptions &MO,
>> + ArrayRef<GlobalValue *> GVs, SymbolFlagsMap &SymbolFlags,
>> + SymbolNameToDefinitionMap *SymbolToDefinition = nullptr);
>> +};
>> +
>> +/// Returns a SymbolFlagsMap for the object file represented by the given
>> +/// buffer, or an error if the buffer does not contain a valid object file.
>> +Expected<std::pair<SymbolFlagsMap, SymbolStringPtr>>
>> +getObjectSymbolInfo(ExecutionSession &ES, MemoryBufferRef ObjBuffer);
>> +
>> +} // End namespace orc
>> +} // End namespace llvm
>> +
>> +#endif // LLVM_EXECUTIONENGINE_ORC_MANGLING_H
>>
>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h b/llvm/include/llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h
>> index 50d25f18891e..1b4c03bd2cfd 100644
>> --- a/llvm/include/llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h
>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h
>> @@ -35,6 +35,7 @@ namespace llvm {
>>
>> namespace jitlink {
>> class EHFrameRegistrar;
>> +class Symbol;
>> } // namespace jitlink
>>
>> namespace object {
>> @@ -59,10 +60,14 @@ class ObjectLinkingLayer : public ObjectLayer {
>> /// configured.
>> class Plugin {
>> public:
>> + using JITLinkSymbolVector = std::vector<const jitlink::Symbol *>;
>> + using LocalDependenciesMap = DenseMap<SymbolStringPtr, JITLinkSymbolVector>;
>> +
>> virtual ~Plugin();
>> virtual void modifyPassConfig(MaterializationResponsibility &MR,
>> const Triple &TT,
>> jitlink::PassConfiguration &Config) {}
>> +
>> virtual void notifyLoaded(MaterializationResponsibility &MR) {}
>> virtual Error notifyEmitted(MaterializationResponsibility &MR) {
>> return Error::success();
>> @@ -71,6 +76,15 @@ class ObjectLinkingLayer : public ObjectLayer {
>> return Error::success();
>> }
>> virtual Error notifyRemovingAllModules() { return Error::success(); }
>> +
>> + /// Return any dependencies that synthetic symbols (e.g. init symbols)
>> + /// have on locally scoped jitlink::Symbols. This is used by the
>> + /// ObjectLinkingLayer to update the dependencies for the synthetic
>> + /// symbols.
>> + virtual LocalDependenciesMap
>> + getSyntheticSymbolLocalDependencies(MaterializationResponsibility &MR) {
>> + return LocalDependenciesMap();
>> + }
>> };
>>
>> using ReturnObjectBufferFunction =
>>
>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/SymbolStringPool.h b/llvm/include/llvm/ExecutionEngine/Orc/SymbolStringPool.h
>> index c354f6c3559c..c8462dc3d2c8 100644
>> --- a/llvm/include/llvm/ExecutionEngine/Orc/SymbolStringPool.h
>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/SymbolStringPool.h
>> @@ -53,6 +53,7 @@ class SymbolStringPtr {
>>
>> public:
>> SymbolStringPtr() = default;
>> + SymbolStringPtr(nullptr_t) {}
>> SymbolStringPtr(const SymbolStringPtr &Other)
>> : S(Other.S) {
>> if (isRealPoolEntry(S))
>> @@ -85,6 +86,8 @@ class SymbolStringPtr {
>> --S->getValue();
>> }
>>
>> + explicit operator bool() const { return S; }
>> +
>> StringRef operator*() const { return S->first(); }
>>
>> friend bool operator==(const SymbolStringPtr &LHS,
>>
>> diff --git a/llvm/lib/ExecutionEngine/Orc/CMakeLists.txt b/llvm/lib/ExecutionEngine/Orc/CMakeLists.txt
>> index e3a7a67c8a0e..7eb2742d7b53 100644
>> --- a/llvm/lib/ExecutionEngine/Orc/CMakeLists.txt
>> +++ b/llvm/lib/ExecutionEngine/Orc/CMakeLists.txt
>> @@ -12,6 +12,8 @@ add_llvm_component_library(LLVMOrcJIT
>> Legacy.cpp
>> Layer.cpp
>> LLJIT.cpp
>> + MachOPlatform.cpp
>> + Mangling.cpp
>> NullResolver.cpp
>> ObjectLinkingLayer.cpp
>> ObjectTransformLayer.cpp
>>
>> diff --git a/llvm/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp b/llvm/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp
>> index 29d18b6e4d7b..f22ae01a2080 100644
>> --- a/llvm/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp
>> +++ b/llvm/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp
>> @@ -7,6 +7,7 @@
>> //===----------------------------------------------------------------------===//
>>
>> #include "llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h"
>> +#include "llvm/ExecutionEngine/Orc/ExecutionUtils.h"
>> #include "llvm/IR/Mangler.h"
>> #include "llvm/IR/Module.h"
>>
>> @@ -68,18 +69,18 @@ namespace orc {
>> class PartitioningIRMaterializationUnit : public IRMaterializationUnit {
>> public:
>> PartitioningIRMaterializationUnit(ExecutionSession &ES,
>> - const ManglingOptions &MO,
>> + const IRSymbolMapper::ManglingOptions &MO,
>> ThreadSafeModule TSM, VModuleKey K,
>> CompileOnDemandLayer &Parent)
>> : IRMaterializationUnit(ES, MO, std::move(TSM), std::move(K)),
>> Parent(Parent) {}
>>
>> PartitioningIRMaterializationUnit(
>> - ThreadSafeModule TSM, SymbolFlagsMap SymbolFlags,
>> - SymbolNameToDefinitionMap SymbolToDefinition,
>> + ThreadSafeModule TSM, VModuleKey K, SymbolFlagsMap SymbolFlags,
>> + SymbolStringPtr InitSymbol, SymbolNameToDefinitionMap SymbolToDefinition,
>> CompileOnDemandLayer &Parent)
>> : IRMaterializationUnit(std::move(TSM), std::move(K),
>> - std::move(SymbolFlags),
>> + std::move(SymbolFlags), std::move(InitSymbol),
>> std::move(SymbolToDefinition)),
>> Parent(Parent) {}
>>
>> @@ -172,21 +173,23 @@ CompileOnDemandLayer::getPerDylibResources(JITDylib &TargetD) {
>> auto I = DylibResources.find(&TargetD);
>> if (I == DylibResources.end()) {
>> auto &ImplD =
>> - getExecutionSession().createJITDylib(TargetD.getName() + ".impl");
>> + getExecutionSession().createBareJITDylib(TargetD.getName() + ".impl");
>> + JITDylibSearchOrder NewSearchOrder;
>> TargetD.withSearchOrderDo(
>> [&](const JITDylibSearchOrder &TargetSearchOrder) {
>> - auto NewSearchOrder = TargetSearchOrder;
>> - assert(
>> - !NewSearchOrder.empty() &&
>> - NewSearchOrder.front().first == &TargetD &&
>> - NewSearchOrder.front().second ==
>> - JITDylibLookupFlags::MatchAllSymbols &&
>> - "TargetD must be at the front of its own search order and match "
>> - "non-exported symbol");
>> - NewSearchOrder.insert(std::next(NewSearchOrder.begin()),
>> - {&ImplD, JITDylibLookupFlags::MatchAllSymbols});
>> - ImplD.setSearchOrder(std::move(NewSearchOrder), false);
>> + NewSearchOrder = TargetSearchOrder;
>> });
>> +
>> + assert(
>> + !NewSearchOrder.empty() && NewSearchOrder.front().first == &TargetD &&
>> + NewSearchOrder.front().second == JITDylibLookupFlags::MatchAllSymbols &&
>> + "TargetD must be at the front of its own search order and match "
>> + "non-exported symbol");
>> + NewSearchOrder.insert(std::next(NewSearchOrder.begin()),
>> + {&ImplD, JITDylibLookupFlags::MatchAllSymbols});
>> + ImplD.setSearchOrder(NewSearchOrder, false);
>> + TargetD.setSearchOrder(std::move(NewSearchOrder), false);
>> +
>> PerDylibResources PDR(ImplD, BuildIndirectStubsManager());
>> I = DylibResources.insert(std::make_pair(&TargetD, std::move(PDR))).first;
>> }
>> @@ -251,8 +254,15 @@ void CompileOnDemandLayer::emitPartition(
>> auto &ES = getExecutionSession();
>> GlobalValueSet RequestedGVs;
>> for (auto &Name : R.getRequestedSymbols()) {
>> - assert(Defs.count(Name) && "No definition for symbol");
>> - RequestedGVs.insert(Defs[Name]);
>> + if (Name == R.getInitializerSymbol())
>> + TSM.withModuleDo([&](Module &M) {
>> + for (auto &GV : getStaticInitGVs(M))
>> + RequestedGVs.insert(&GV);
>> + });
>> + else {
>> + assert(Defs.count(Name) && "No definition for symbol");
>> + RequestedGVs.insert(Defs[Name]);
>> + }
>> }
>>
>> /// Perform partitioning with the context lock held, since the partition
>> @@ -272,7 +282,8 @@ void CompileOnDemandLayer::emitPartition(
>> // If the partition is empty, return the whole module to the symbol table.
>> if (GVsToExtract->empty()) {
>> R.replace(std::make_unique<PartitioningIRMaterializationUnit>(
>> - std::move(TSM), R.getSymbols(), std::move(Defs), *this));
>> + std::move(TSM), R.getVModuleKey(), R.getSymbols(),
>> + R.getInitializerSymbol(), std::move(Defs), *this));
>> return;
>> }
>>
>>
>> diff --git a/llvm/lib/ExecutionEngine/Orc/CompileUtils.cpp b/llvm/lib/ExecutionEngine/Orc/CompileUtils.cpp
>> index 160e5ba50311..f8efed15edea 100644
>> --- a/llvm/lib/ExecutionEngine/Orc/CompileUtils.cpp
>> +++ b/llvm/lib/ExecutionEngine/Orc/CompileUtils.cpp
>> @@ -24,9 +24,9 @@
>> namespace llvm {
>> namespace orc {
>>
>> -IRMaterializationUnit::ManglingOptions
>> +IRSymbolMapper::ManglingOptions
>> irManglingOptionsFromTargetOptions(const TargetOptions &Opts) {
>> - IRMaterializationUnit::ManglingOptions MO;
>> + IRSymbolMapper::ManglingOptions MO;
>>
>> MO.EmulatedTLS = Opts.EmulatedTLS;
>>
>>
>> diff --git a/llvm/lib/ExecutionEngine/Orc/Core.cpp b/llvm/lib/ExecutionEngine/Orc/Core.cpp
>> index a7af34a21926..6a0df10f7c3f 100644
>> --- a/llvm/lib/ExecutionEngine/Orc/Core.cpp
>> +++ b/llvm/lib/ExecutionEngine/Orc/Core.cpp
>> @@ -11,7 +11,6 @@
>> #include "llvm/ADT/STLExtras.h"
>> #include "llvm/Config/llvm-config.h"
>> #include "llvm/ExecutionEngine/Orc/OrcError.h"
>> -#include "llvm/IR/Mangler.h"
>> #include "llvm/Support/CommandLine.h"
>> #include "llvm/Support/Debug.h"
>> #include "llvm/Support/Format.h"
>> @@ -421,12 +420,6 @@ void AsynchronousSymbolQuery::detach() {
>> QueryRegistrations.clear();
>> }
>>
>> -MaterializationResponsibility::MaterializationResponsibility(
>> - JITDylib &JD, SymbolFlagsMap SymbolFlags, VModuleKey K)
>> - : JD(JD), SymbolFlags(std::move(SymbolFlags)), K(std::move(K)) {
>> - assert(!this->SymbolFlags.empty() && "Materializing nothing?");
>> -}
>> -
>> MaterializationResponsibility::~MaterializationResponsibility() {
>> assert(SymbolFlags.empty() &&
>> "All symbols should have been explicitly materialized or failed");
>> @@ -501,9 +494,13 @@ void MaterializationResponsibility::failMaterialization() {
>>
>> void MaterializationResponsibility::replace(
>> std::unique_ptr<MaterializationUnit> MU) {
>> +
>> for (auto &KV : MU->getSymbols())
>> SymbolFlags.erase(KV.first);
>>
>> + if (MU->getInitializerSymbol() == InitSymbol)
>> + InitSymbol = nullptr;
>> +
>> LLVM_DEBUG(JD.getExecutionSession().runSessionLocked([&]() {
>> dbgs() << "In " << JD.getName() << " replacing symbols with " << *MU
>> << "\n";
>> @@ -519,6 +516,7 @@ MaterializationResponsibility::delegate(const SymbolNameSet &Symbols,
>> if (NewKey == VModuleKey())
>> NewKey = K;
>>
>> + SymbolStringPtr DelegatedInitSymbol;
>> SymbolFlagsMap DelegatedFlags;
>>
>> for (auto &Name : Symbols) {
>> @@ -528,10 +526,14 @@ MaterializationResponsibility::delegate(const SymbolNameSet &Symbols,
>> "instance");
>>
>> DelegatedFlags[Name] = std::move(I->second);
>> + if (Name == InitSymbol)
>> + std::swap(InitSymbol, DelegatedInitSymbol);
>> +
>> SymbolFlags.erase(I);
>> }
>>
>> return MaterializationResponsibility(JD, std::move(DelegatedFlags),
>> + std::move(DelegatedInitSymbol),
>> std::move(NewKey));
>> }
>>
>> @@ -550,7 +552,7 @@ void MaterializationResponsibility::addDependenciesForAll(
>>
>> AbsoluteSymbolsMaterializationUnit::AbsoluteSymbolsMaterializationUnit(
>> SymbolMap Symbols, VModuleKey K)
>> - : MaterializationUnit(extractFlags(Symbols), std::move(K)),
>> + : MaterializationUnit(extractFlags(Symbols), nullptr, std::move(K)),
>> Symbols(std::move(Symbols)) {}
>>
>> StringRef AbsoluteSymbolsMaterializationUnit::getName() const {
>> @@ -581,7 +583,7 @@ AbsoluteSymbolsMaterializationUnit::extractFlags(const SymbolMap &Symbols) {
>> ReExportsMaterializationUnit::ReExportsMaterializationUnit(
>> JITDylib *SourceJD, JITDylibLookupFlags SourceJDLookupFlags,
>> SymbolAliasMap Aliases, VModuleKey K)
>> - : MaterializationUnit(extractFlags(Aliases), std::move(K)),
>> + : MaterializationUnit(extractFlags(Aliases), nullptr, std::move(K)),
>> SourceJD(SourceJD), SourceJDLookupFlags(SourceJDLookupFlags),
>> Aliases(std::move(Aliases)) {}
>>
>> @@ -972,7 +974,7 @@ void JITDylib::addDependencies(const SymbolStringPtr &Name,
>> // Assert that this symbol exists and has not reached the ready state
>> // already.
>> assert(OtherSymI != OtherJITDylib.Symbols.end() &&
>> - (OtherSymI->second.getState() != SymbolState::Ready &&
>> + (OtherSymI->second.getState() < SymbolState::Ready &&
>> "Dependency on emitted/ready symbol"));
>> #endif
>>
>> @@ -1101,6 +1103,7 @@ Error JITDylib::resolve(const SymbolMap &Resolved) {
>> Error JITDylib::emit(const SymbolFlagsMap &Emitted) {
>> AsynchronousSymbolQuerySet CompletedQueries;
>> SymbolNameSet SymbolsInErrorState;
>> + DenseMap<JITDylib *, SymbolNameVector> ReadySymbols;
>>
>> ES.runSessionLocked([&, this]() {
>> std::vector<SymbolTable::iterator> Worklist;
>> @@ -1145,6 +1148,7 @@ Error JITDylib::emit(const SymbolFlagsMap &Emitted) {
>> // dependencies) then notify any pending queries.
>> for (auto &KV : MI.Dependants) {
>> auto &DependantJD = *KV.first;
>> + auto &DependantJDReadySymbols = ReadySymbols[&DependantJD];
>> for (auto &DependantName : KV.second) {
>> auto DependantMII =
>> DependantJD.MaterializingInfos.find(DependantName);
>> @@ -1184,6 +1188,7 @@ Error JITDylib::emit(const SymbolFlagsMap &Emitted) {
>> // Since this dependant is now ready, we erase its MaterializingInfo
>> // and update its materializing state.
>> DependantSymEntry.setState(SymbolState::Ready);
>> + DependantJDReadySymbols.push_back(DependantName);
>>
>> for (auto &Q : DependantMI.takeQueriesMeeting(SymbolState::Ready)) {
>> Q->notifySymbolMetRequiredState(
>> @@ -1192,22 +1197,21 @@ Error JITDylib::emit(const SymbolFlagsMap &Emitted) {
>> CompletedQueries.insert(Q);
>> Q->removeQueryDependence(DependantJD, DependantName);
>> }
>> -
>> - DependantJD.MaterializingInfos.erase(DependantMII);
>> }
>> }
>> }
>>
>> + auto &ThisJDReadySymbols = ReadySymbols[this];
>> MI.Dependants.clear();
>> if (MI.UnemittedDependencies.empty()) {
>> SymI->second.setState(SymbolState::Ready);
>> + ThisJDReadySymbols.push_back(Name);
>> for (auto &Q : MI.takeQueriesMeeting(SymbolState::Ready)) {
>> Q->notifySymbolMetRequiredState(Name, SymI->second.getSymbol());
>> if (Q->isComplete())
>> CompletedQueries.insert(Q);
>> Q->removeQueryDependence(*this, Name);
>> }
>> - MaterializingInfos.erase(MII);
>> }
>> }
>> });
>> @@ -1882,6 +1886,57 @@ void JITDylib::transferEmittedNodeDependencies(
>> }
>> }
>>
>> +Platform::~Platform() {}
>> +
>> +Expected<DenseMap<JITDylib *, SymbolMap>> Platform::lookupInitSymbols(
>> + ExecutionSession &ES,
>> + const DenseMap<JITDylib *, SymbolLookupSet> &InitSyms) {
>> +
>> + DenseMap<JITDylib *, SymbolMap> CompoundResult;
>> + Error CompoundErr = Error::success();
>> + std::mutex LookupMutex;
>> + std::condition_variable CV;
>> + uint64_t Count = InitSyms.size();
>> +
>> + LLVM_DEBUG({
>> + dbgs() << "Issuing init-symbol lookup:\n";
>> + for (auto &KV : InitSyms)
>> + dbgs() << " " << KV.first->getName() << ": " << KV.second << "\n";
>> + });
>> +
>> + for (auto &KV : InitSyms) {
>> + auto *JD = KV.first;
>> + auto Names = std::move(KV.second);
>> + ES.lookup(
>> + LookupKind::Static,
>> + JITDylibSearchOrder({{JD, JITDylibLookupFlags::MatchAllSymbols}}),
>> + std::move(Names), SymbolState::Ready,
>> + [&, JD](Expected<SymbolMap> Result) {
>> + {
>> + std::lock_guard<std::mutex> Lock(LookupMutex);
>> + --Count;
>> + if (Result) {
>> + assert(!CompoundResult.count(JD) &&
>> + "Duplicate JITDylib in lookup?");
>> + CompoundResult[JD] = std::move(*Result);
>> + } else
>> + CompoundErr =
>> + joinErrors(std::move(CompoundErr), Result.takeError());
>> + }
>> + CV.notify_one();
>> + },
>> + NoDependenciesToRegister);
>> + }
>> +
>> + std::unique_lock<std::mutex> Lock(LookupMutex);
>> + CV.wait(Lock, [&] { return Count == 0 || CompoundErr; });
>> +
>> + if (CompoundErr)
>> + return std::move(CompoundErr);
>> +
>> + return std::move(CompoundResult);
>> +}
>> +
>> ExecutionSession::ExecutionSession(std::shared_ptr<SymbolStringPool> SSP)
>> : SSP(SSP ? std::move(SSP) : std::make_shared<SymbolStringPool>()) {
>> }
>> @@ -1895,7 +1950,7 @@ JITDylib *ExecutionSession::getJITDylibByName(StringRef Name) {
>> });
>> }
>>
>> -JITDylib &ExecutionSession::createJITDylib(std::string Name) {
>> +JITDylib &ExecutionSession::createBareJITDylib(std::string Name) {
>> assert(!getJITDylibByName(Name) && "JITDylib with that name already exists");
>> return runSessionLocked([&, this]() -> JITDylib & {
>> JDs.push_back(
>> @@ -1904,6 +1959,14 @@ JITDylib &ExecutionSession::createJITDylib(std::string Name) {
>> });
>> }
>>
>> +Expected<JITDylib &> ExecutionSession::createJITDylib(std::string Name) {
>> + auto &JD = createBareJITDylib(Name);
>> + if (P)
>> + if (auto Err = P->setupJITDylib(JD))
>> + return std::move(Err);
>> + return JD;
>> +}
>> +
>> void ExecutionSession::legacyFailQuery(AsynchronousSymbolQuery &Q, Error Err) {
>> assert(!!Err && "Error should be in failure state");
>>
>> @@ -2144,11 +2207,11 @@ ExecutionSession::lookup(const JITDylibSearchOrder &SearchOrder,
>>
>> Expected<JITEvaluatedSymbol>
>> ExecutionSession::lookup(const JITDylibSearchOrder &SearchOrder,
>> - SymbolStringPtr Name) {
>> + SymbolStringPtr Name, SymbolState RequiredState) {
>> SymbolLookupSet Names({Name});
>>
>> if (auto ResultMap = lookup(SearchOrder, std::move(Names), LookupKind::Static,
>> - SymbolState::Ready, NoDependenciesToRegister)) {
>> + RequiredState, NoDependenciesToRegister)) {
>> assert(ResultMap->size() == 1 && "Unexpected number of results");
>> assert(ResultMap->count(Name) && "Missing result for symbol");
>> return std::move(ResultMap->begin()->second);
>> @@ -2157,14 +2220,15 @@ ExecutionSession::lookup(const JITDylibSearchOrder &SearchOrder,
>> }
>>
>> Expected<JITEvaluatedSymbol>
>> -ExecutionSession::lookup(ArrayRef<JITDylib *> SearchOrder,
>> - SymbolStringPtr Name) {
>> - return lookup(makeJITDylibSearchOrder(SearchOrder), Name);
>> +ExecutionSession::lookup(ArrayRef<JITDylib *> SearchOrder, SymbolStringPtr Name,
>> + SymbolState RequiredState) {
>> + return lookup(makeJITDylibSearchOrder(SearchOrder), Name, RequiredState);
>> }
>>
>> Expected<JITEvaluatedSymbol>
>> -ExecutionSession::lookup(ArrayRef<JITDylib *> SearchOrder, StringRef Name) {
>> - return lookup(SearchOrder, intern(Name));
>> +ExecutionSession::lookup(ArrayRef<JITDylib *> SearchOrder, StringRef Name,
>> + SymbolState RequiredState) {
>> + return lookup(SearchOrder, intern(Name), RequiredState);
>> }
>>
>> void ExecutionSession::dump(raw_ostream &OS) {
>> @@ -2195,17 +2259,5 @@ void ExecutionSession::runOutstandingMUs() {
>> }
>> }
>>
>> -MangleAndInterner::MangleAndInterner(ExecutionSession &ES, const DataLayout &DL)
>> - : ES(ES), DL(DL) {}
>> -
>> -SymbolStringPtr MangleAndInterner::operator()(StringRef Name) {
>> - std::string MangledName;
>> - {
>> - raw_string_ostream MangledNameStream(MangledName);
>> - Mangler::getNameWithPrefix(MangledNameStream, Name, DL);
>> - }
>> - return ES.intern(MangledName);
>> -}
>> -
>> } // End namespace orc.
>> } // End namespace llvm.
>>
>> diff --git a/llvm/lib/ExecutionEngine/Orc/ExecutionUtils.cpp b/llvm/lib/ExecutionEngine/Orc/ExecutionUtils.cpp
>> index 3d97fe9eeab1..a98445a2295e 100644
>> --- a/llvm/lib/ExecutionEngine/Orc/ExecutionUtils.cpp
>> +++ b/llvm/lib/ExecutionEngine/Orc/ExecutionUtils.cpp
>> @@ -113,6 +113,26 @@ iterator_range<CtorDtorIterator> getDestructors(const Module &M) {
>> CtorDtorIterator(DtorsList, true));
>> }
>>
>> +bool StaticInitGVIterator::isStaticInitGlobal(GlobalValue &GV) {
>> + if (GV.isDeclaration())
>> + return false;
>> +
>> + if (GV.hasName() && (GV.getName() == "llvm.global_ctors" ||
>> + GV.getName() == "llvm.global_dtors"))
>> + return true;
>> +
>> + if (ObjFmt == Triple::MachO) {
>> + // FIXME: These section checks are too strict: We should match first and
>> + // second word split by comma.
>> + if (GV.hasSection() &&
>> + (GV.getSection().startswith("__DATA,__objc_classlist") ||
>> + GV.getSection().startswith("__DATA,__objc_selrefs")))
>> + return true;
>> + }
>> +
>> + return false;
>> +}
>> +
>> void CtorDtorRunner::add(iterator_range<CtorDtorIterator> CtorDtors) {
>> if (CtorDtors.empty())
>> return;
>> @@ -198,6 +218,30 @@ Error LocalCXXRuntimeOverrides::enable(JITDylib &JD,
>> return JD.define(absoluteSymbols(std::move(RuntimeInterposes)));
>> }
>>
>> +void ItaniumCXAAtExitSupport::registerAtExit(void (*F)(void *), void *Ctx,
>> + void *DSOHandle) {
>> + std::lock_guard<std::mutex> Lock(AtExitsMutex);
>> + AtExitRecords[DSOHandle].push_back({F, Ctx});
>> +}
>> +
>> +void ItaniumCXAAtExitSupport::runAtExits(void *DSOHandle) {
>> + std::vector<AtExitRecord> AtExitsToRun;
>> +
>> + {
>> + std::lock_guard<std::mutex> Lock(AtExitsMutex);
>> + auto I = AtExitRecords.find(DSOHandle);
>> + if (I != AtExitRecords.end()) {
>> + AtExitsToRun = std::move(I->second);
>> + AtExitRecords.erase(I);
>> + }
>> + }
>> +
>> + while (!AtExitsToRun.empty()) {
>> + AtExitsToRun.back().F(AtExitsToRun.back().Ctx);
>> + AtExitsToRun.pop_back();
>> + }
>> +}
>> +
>> DynamicLibrarySearchGenerator::DynamicLibrarySearchGenerator(
>> sys::DynamicLibrary Dylib, char GlobalPrefix, SymbolPredicate Allow)
>> : Dylib(std::move(Dylib)), Allow(std::move(Allow)),
>>
>> diff --git a/llvm/lib/ExecutionEngine/Orc/IndirectionUtils.cpp b/llvm/lib/ExecutionEngine/Orc/IndirectionUtils.cpp
>> index 1ac9a58aeaef..1e991d8f58f2 100644
>> --- a/llvm/lib/ExecutionEngine/Orc/IndirectionUtils.cpp
>> +++ b/llvm/lib/ExecutionEngine/Orc/IndirectionUtils.cpp
>> @@ -28,7 +28,7 @@ class CompileCallbackMaterializationUnit : public orc::MaterializationUnit {
>> CompileCallbackMaterializationUnit(SymbolStringPtr Name,
>> CompileFunction Compile, VModuleKey K)
>> : MaterializationUnit(SymbolFlagsMap({{Name, JITSymbolFlags::Exported}}),
>> - std::move(K)),
>> + nullptr, std::move(K)),
>> Name(std::move(Name)), Compile(std::move(Compile)) {}
>>
>> StringRef getName() const override { return "<Compile Callbacks>"; }
>>
>> diff --git a/llvm/lib/ExecutionEngine/Orc/LLJIT.cpp b/llvm/lib/ExecutionEngine/Orc/LLJIT.cpp
>> index 4218ca4e481f..5b2f86823355 100644
>> --- a/llvm/lib/ExecutionEngine/Orc/LLJIT.cpp
>> +++ b/llvm/lib/ExecutionEngine/Orc/LLJIT.cpp
>> @@ -8,15 +8,827 @@
>>
>> #include "llvm/ExecutionEngine/Orc/LLJIT.h"
>> #include "llvm/ExecutionEngine/JITLink/JITLinkMemoryManager.h"
>> +#include "llvm/ExecutionEngine/Orc/MachOPlatform.h"
>> #include "llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h"
>> #include "llvm/ExecutionEngine/Orc/OrcError.h"
>> #include "llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h"
>> #include "llvm/ExecutionEngine/SectionMemoryManager.h"
>> +#include "llvm/IR/GlobalVariable.h"
>> +#include "llvm/IR/IRBuilder.h"
>> #include "llvm/IR/Mangler.h"
>> +#include "llvm/IR/Module.h"
>> +#include "llvm/Support/DynamicLibrary.h"
>> +
>> +#include <map>
>> +
>> +using namespace llvm;
>> +using namespace llvm::orc;
>> +
>> +namespace {
>> +
>> +/// Add a reference to the __dso_handle global to the given module.
>> +/// Returns a reference to the __dso_handle IR decl.
>> +GlobalVariable *addDSOHandleDecl(Module &M) {
>> + auto DSOHandleTy = StructType::create(M.getContext(), "lljit.dso_handle");
>> + return new GlobalVariable(M, DSOHandleTy, true, GlobalValue::ExternalLinkage,
>> + nullptr, "__dso_handle");
>> +}
>> +
>> +/// Adds helper function decls and wrapper functions that call the helper with
>> +/// some additional prefix arguments.
>> +///
>> +/// E.g. For wrapper "foo" with type i8(i8, i64), helper "bar", and prefix
>> +/// args i32 4 and i16 12345, this function will add:
>> +///
>> +/// declare i8 @bar(i32, i16, i8, i64)
>> +///
>> +/// define i8 @foo(i8, i64) {
>> +/// entry:
>> +/// %2 = call i8 @bar(i32 4, i16 12345, i8 %0, i64 %1)
>> +/// ret i8 %2
>> +/// }
>> +///
>> +Function *addHelperAndWrapper(Module &M, StringRef WrapperName,
>> + FunctionType *WrapperFnType,
>> + GlobalValue::VisibilityTypes WrapperVisibility,
>> + StringRef HelperName,
>> + ArrayRef<Value *> HelperPrefixArgs) {
>> + std::vector<Type *> HelperArgTypes;
>> + for (auto *Arg : HelperPrefixArgs)
>> + HelperArgTypes.push_back(Arg->getType());
>> + for (auto *T : WrapperFnType->params())
>> + HelperArgTypes.push_back(T);
>> + auto *HelperFnType =
>> + FunctionType::get(WrapperFnType->getReturnType(), HelperArgTypes, false);
>> + auto *HelperFn = Function::Create(HelperFnType, GlobalValue::ExternalLinkage,
>> + HelperName, M);
>> +
>> + auto *WrapperFn = Function::Create(
>> + WrapperFnType, GlobalValue::ExternalLinkage, WrapperName, M);
>> + WrapperFn->setVisibility(WrapperVisibility);
>> +
>> + auto *EntryBlock = BasicBlock::Create(M.getContext(), "entry", WrapperFn);
>> + IRBuilder<> IB(EntryBlock);
>> +
>> + std::vector<Value *> HelperArgs;
>> + for (auto *Arg : HelperPrefixArgs)
>> + HelperArgs.push_back(Arg);
>> + for (auto &Arg : WrapperFn->args())
>> + HelperArgs.push_back(&Arg);
>> + auto *HelperResult = IB.CreateCall(HelperFn, HelperArgs);
>> + if (HelperFn->getReturnType()->isVoidTy())
>> + IB.CreateRetVoid();
>> + else
>> + IB.CreateRet(HelperResult);
>> +
>> + return WrapperFn;
>> +}
>> +
>> +class GenericLLVMIRPlatformSupport;
>> +
>> +/// orc::Platform component of Generic LLVM IR Platform support.
>> +/// Just forwards calls to the GenericLLVMIRPlatformSupport class below.
>> +class GenericLLVMIRPlatform : public Platform {
>> +public:
>> + GenericLLVMIRPlatform(GenericLLVMIRPlatformSupport &S) : S(S) {}
>> + Error setupJITDylib(JITDylib &JD) override;
>> + Error notifyAdding(JITDylib &JD, const MaterializationUnit &MU) override;
>> + Error notifyRemoving(JITDylib &JD, VModuleKey K) override {
>> + // Noop -- Nothing to do (yet).
>> + return Error::success();
>> + }
>> +
>> +private:
>> + GenericLLVMIRPlatformSupport &S;
>> +};
>> +
>> +/// This transform parses llvm.global_ctors to produce a single initialization
>> +/// function for the module, records the function, then deletes
>> +/// llvm.global_ctors.
>> +class GlobalCtorDtorScraper {
>> +public:
>> + GlobalCtorDtorScraper(GenericLLVMIRPlatformSupport &PS) : PS(PS) {}
>> + Expected<ThreadSafeModule> operator()(ThreadSafeModule TSM,
>> + MaterializationResponsibility &R);
>> +
>> +private:
>> + GenericLLVMIRPlatformSupport &PS;
>> +};
>> +
>> +/// Generic IR Platform Support
>> +///
>> +/// Scrapes llvm.global_ctors and llvm.global_dtors and replaces them with
>> +/// specially named 'init' and 'deinit'. Injects definitions / interposes for
>> +/// some runtime API, including __cxa_atexit, dlopen, and dlclose.
>> +class GenericLLVMIRPlatformSupport : public LLJIT::PlatformSupport {
>> +public:
>> + // GenericLLVMIRPlatform &P) : P(P) {
>> + GenericLLVMIRPlatformSupport(LLJIT &J) : J(J) {
>> +
>> + getExecutionSession().setPlatform(
>> + std::make_unique<GenericLLVMIRPlatform>(*this));
>> +
>> + setInitTransform(J, GlobalCtorDtorScraper(*this));
>> +
>> + MangleAndInterner Mangle(getExecutionSession(), J.getDataLayout());
>> + SymbolMap StdInterposes;
>> +
>> + StdInterposes[Mangle("__lljit.platform_support_instance")] =
>> + JITEvaluatedSymbol(pointerToJITTargetAddress(this), JITSymbolFlags());
>> + StdInterposes[Mangle("__lljit.cxa_atexit_helper")] = JITEvaluatedSymbol(
>> + pointerToJITTargetAddress(registerAtExitHelper), JITSymbolFlags());
>> + StdInterposes[Mangle("__lljit.run_atexits_helper")] = JITEvaluatedSymbol(
>> + pointerToJITTargetAddress(runAtExitsHelper), JITSymbolFlags());
>> +
>> + cantFail(
>> + J.getMainJITDylib().define(absoluteSymbols(std::move(StdInterposes))));
>> + cantFail(setupJITDylib(J.getMainJITDylib()));
>> + cantFail(J.addIRModule(J.getMainJITDylib(), createPlatformRuntimeModule()));
>> + }
>> +
>> + ExecutionSession &getExecutionSession() { return J.getExecutionSession(); }
>> +
>> + /// Adds a module that defines the __dso_handle global.
>> + Error setupJITDylib(JITDylib &JD) {
>> + auto Ctx = std::make_unique<LLVMContext>();
>> + auto M = std::make_unique<Module>("__standard_lib", *Ctx);
>> + M->setDataLayout(J.getDataLayout());
>> +
>> + auto *Int64Ty = Type::getInt64Ty(*Ctx);
>> + auto *DSOHandle = new GlobalVariable(
>> + *M, Int64Ty, true, GlobalValue::ExternalLinkage,
>> + ConstantInt::get(Int64Ty, reinterpret_cast<uintptr_t>(&JD)),
>> + "__dso_handle");
>> + DSOHandle->setVisibility(GlobalValue::HiddenVisibility);
>> + DSOHandle->setInitializer(
>> + ConstantInt::get(Int64Ty, pointerToJITTargetAddress(&JD)));
>> + return J.addIRModule(JD, ThreadSafeModule(std::move(M), std::move(Ctx)));
>> + }
>> +
>> + Error notifyAdding(JITDylib &JD, const MaterializationUnit &MU) {
>> + std::lock_guard<std::mutex> Lock(PlatformSupportMutex);
>> + if (auto &InitSym = MU.getInitializerSymbol())
>> + InitSymbols[&JD].add(InitSym);
>> + return Error::success();
>> + }
>> +
>> + Error initialize(JITDylib &JD) override {
>> + if (auto Initializers = getInitializers(JD)) {
>> + for (auto InitFnAddr : *Initializers) {
>> + auto *InitFn = jitTargetAddressToFunction<void (*)()>(InitFnAddr);
>> + InitFn();
>> + }
>> + } else
>> + return Initializers.takeError();
>> + return Error::success();
>> + }
>> +
>> + Error deinitialize(JITDylib &JD) override {
>> + if (auto Deinitializers = getDeinitializers(JD)) {
>> + for (auto DeinitFnAddr : *Deinitializers) {
>> + auto *DeinitFn = jitTargetAddressToFunction<void (*)()>(DeinitFnAddr);
>> + DeinitFn();
>> + }
>> + } else
>> + return Deinitializers.takeError();
>> +
>> + return Error::success();
>> + }
>> +
>> + void registerInitFunc(JITDylib &JD, SymbolStringPtr InitName) {
>> + std::lock_guard<std::mutex> Lock(PlatformSupportMutex);
>> + InitFunctions[&JD].add(InitName);
>> + }
>> +
>> +private:
>> + Expected<std::vector<JITTargetAddress>> getInitializers(JITDylib &JD) {
>> + if (auto Err = issueInitLookups(JD))
>> + return std::move(Err);
>> +
>> + DenseMap<JITDylib *, SymbolLookupSet> LookupSymbols;
>> + std::vector<JITDylib *> DFSLinkOrder;
>> +
>> + {
>> + std::lock_guard<std::mutex> Lock(PlatformSupportMutex);
>> + DFSLinkOrder = getDFSLinkOrder(JD);
>> +
>> + for (auto *NextJD : DFSLinkOrder) {
>> + auto IFItr = InitFunctions.find(NextJD);
>> + if (IFItr != InitFunctions.end()) {
>> + LookupSymbols[NextJD] = std::move(IFItr->second);
>> + InitFunctions.erase(IFItr);
>> + }
>> + }
>> + }
>> +
>> + auto &ES = getExecutionSession();
>> + auto LookupResult = Platform::lookupInitSymbols(ES, LookupSymbols);
>> +
>> + if (!LookupResult)
>> + return LookupResult.takeError();
>> +
>> + std::vector<JITTargetAddress> Initializers;
>> + while (!DFSLinkOrder.empty()) {
>> + auto &NextJD = *DFSLinkOrder.back();
>> + DFSLinkOrder.pop_back();
>> + auto InitsItr = LookupResult->find(&NextJD);
>> + if (InitsItr == LookupResult->end())
>> + continue;
>> + for (auto &KV : InitsItr->second)
>> + Initializers.push_back(KV.second.getAddress());
>> + }
>> +
>> + return Initializers;
>> + }
>> +
>> + Expected<std::vector<JITTargetAddress>> getDeinitializers(JITDylib &JD) {
>> + auto &ES = getExecutionSession();
>> +
>> + MangleAndInterner Mangle(getExecutionSession(), J.getDataLayout());
>> + auto LLJITRunAtExits = Mangle("__lljit_run_atexits");
>> +
>> + DenseMap<JITDylib *, SymbolLookupSet> LookupSymbols;
>> + std::vector<JITDylib *> DFSLinkOrder;
>> +
>> + {
>> + std::lock_guard<std::mutex> Lock(PlatformSupportMutex);
>> + DFSLinkOrder = getDFSLinkOrder(JD);
>> +
>> + for (auto *NextJD : DFSLinkOrder) {
>> + auto &JDLookupSymbols = LookupSymbols[NextJD];
>> + auto DIFItr = DeInitFunctions.find(NextJD);
>> + if (DIFItr != DeInitFunctions.end()) {
>> + LookupSymbols[NextJD] = std::move(DIFItr->second);
>> + DeInitFunctions.erase(DIFItr);
>> + }
>> + JDLookupSymbols.add(LLJITRunAtExits,
>> + SymbolLookupFlags::WeaklyReferencedSymbol);
>> + }
>> + }
>> +
>> + auto LookupResult = Platform::lookupInitSymbols(ES, LookupSymbols);
>> +
>> + if (!LookupResult)
>> + return LookupResult.takeError();
>> +
>> + std::vector<JITTargetAddress> DeInitializers;
>> + for (auto *NextJD : DFSLinkOrder) {
>> + auto DeInitsItr = LookupResult->find(NextJD);
>> + assert(DeInitsItr != LookupResult->end() &&
>> + "Every JD should have at least __lljit_run_atexits");
>> +
>> + auto RunAtExitsItr = DeInitsItr->second.find(LLJITRunAtExits);
>> + if (RunAtExitsItr != DeInitsItr->second.end())
>> + DeInitializers.push_back(RunAtExitsItr->second.getAddress());
>> +
>> + for (auto &KV : DeInitsItr->second)
>> + if (KV.first != LLJITRunAtExits)
>> + DeInitializers.push_back(KV.second.getAddress());
>> + }
>> +
>> + return DeInitializers;
>> + }
>> +
>> + // Returns a DFS traversal order of the JITDylibs reachable (via
>> + // links-against edges) from JD, starting with JD itself.
>> + static std::vector<JITDylib *> getDFSLinkOrder(JITDylib &JD) {
>> + std::vector<JITDylib *> DFSLinkOrder;
>> + std::vector<JITDylib *> WorkStack({&JD});
>> + DenseSet<JITDylib *> Visited;
>> +
>> + while (!WorkStack.empty()) {
>> + auto &NextJD = *WorkStack.back();
>> + WorkStack.pop_back();
>> + if (Visited.count(&NextJD))
>> + continue;
>> + Visited.insert(&NextJD);
>> + DFSLinkOrder.push_back(&NextJD);
>> + NextJD.withSearchOrderDo([&](const JITDylibSearchOrder &SearchOrder) {
>> + for (auto &KV : SearchOrder)
>> + WorkStack.push_back(KV.first);
>> + });
>> + }
>> +
>> + return DFSLinkOrder;
>> + }
>> +
>> + /// Issue lookups for all init symbols required to initialize JD (and any
>> + /// JITDylibs that it depends on).
>> + Error issueInitLookups(JITDylib &JD) {
>> + DenseMap<JITDylib *, SymbolLookupSet> RequiredInitSymbols;
>> +
>> + {
>> + std::lock_guard<std::mutex> Lock(PlatformSupportMutex);
>> +
>> + auto DFSLinkOrder = getDFSLinkOrder(JD);
>> +
>> + for (auto *NextJD : DFSLinkOrder) {
>> + auto ISItr = InitSymbols.find(NextJD);
>> + if (ISItr != InitSymbols.end()) {
>> + RequiredInitSymbols[NextJD] = std::move(ISItr->second);
>> + InitSymbols.erase(ISItr);
>> + }
>> + }
>> + }
>> +
>> + return Platform::lookupInitSymbols(getExecutionSession(),
>> + RequiredInitSymbols)
>> + .takeError();
>> + }
>> +
>> + static void registerAtExitHelper(void *Self, void (*F)(void *), void *Ctx,
>> + void *DSOHandle) {
>> + static_cast<GenericLLVMIRPlatformSupport *>(Self)->AtExitMgr.registerAtExit(
>> + F, Ctx, DSOHandle);
>> + }
>> +
>> + static void runAtExitsHelper(void *Self, void *DSOHandle) {
>> + static_cast<GenericLLVMIRPlatformSupport *>(Self)->AtExitMgr.runAtExits(
>> + DSOHandle);
>> + }
>> +
>> + // Constructs an LLVM IR module containing platform runtime globals,
>> + // functions, and interposes.
>> + ThreadSafeModule createPlatformRuntimeModule() {
>> + auto Ctx = std::make_unique<LLVMContext>();
>> + auto M = std::make_unique<Module>("__standard_lib", *Ctx);
>> + M->setDataLayout(J.getDataLayout());
>> +
>> + auto *GenericIRPlatformSupportTy =
>> + StructType::create(*Ctx, "lljit.GenericLLJITIRPlatformSupport");
>> +
>> + auto *PlatformInstanceDecl = new GlobalVariable(
>> + *M, GenericIRPlatformSupportTy, true, GlobalValue::ExternalLinkage,
>> + nullptr, "__lljit.platform_support_instance");
>> +
>> + auto *DSOHandleDecl = addDSOHandleDecl(*M);
>> +
>> + auto *Int8Ty = Type::getInt8Ty(*Ctx);
>> + auto *IntTy = Type::getIntNTy(*Ctx, sizeof(int) * CHAR_BIT);
>> + auto *VoidTy = Type::getVoidTy(*Ctx);
>> + auto *BytePtrTy = PointerType::getUnqual(Int8Ty);
>> + auto *AtExitCallbackTy = FunctionType::get(VoidTy, {BytePtrTy}, false);
>> + auto *AtExitCallbackPtrTy = PointerType::getUnqual(AtExitCallbackTy);
>> +
>> + addHelperAndWrapper(
>> + *M, "__cxa_atexit",
>> + FunctionType::get(IntTy, {AtExitCallbackPtrTy, BytePtrTy, BytePtrTy},
>> + false),
>> + GlobalValue::HiddenVisibility, "__lljit.cxa_atexit_helper",
>> + {PlatformInstanceDecl});
>> +
>> + addHelperAndWrapper(
>> + *M, "__lljit_run_atexits", FunctionType::get(VoidTy, {}, false),
>> + GlobalValue::HiddenVisibility, "__lljit.run_atexits_helper",
>> + {PlatformInstanceDecl, DSOHandleDecl});
>> +
>> + return ThreadSafeModule(std::move(M), std::move(Ctx));
>> + }
>> +
>> + std::mutex PlatformSupportMutex;
>> + LLJIT &J;
>> + DenseMap<JITDylib *, SymbolLookupSet> InitSymbols;
>> + DenseMap<JITDylib *, SymbolLookupSet> InitFunctions;
>> + DenseMap<JITDylib *, SymbolLookupSet> DeInitFunctions;
>> + ItaniumCXAAtExitSupport AtExitMgr;
>> +};
>> +
>> +Error GenericLLVMIRPlatform::setupJITDylib(JITDylib &JD) {
>> + return S.setupJITDylib(JD);
>> +}
>> +
>> +Error GenericLLVMIRPlatform::notifyAdding(JITDylib &JD,
>> + const MaterializationUnit &MU) {
>> + return S.notifyAdding(JD, MU);
>> +}
>> +
>> +Expected<ThreadSafeModule>
>> +GlobalCtorDtorScraper::operator()(ThreadSafeModule TSM,
>> + MaterializationResponsibility &R) {
>> + auto Err = TSM.withModuleDo([&](Module &M) -> Error {
>> + auto &Ctx = M.getContext();
>> + auto *GlobalCtors = M.getNamedGlobal("llvm.global_ctors");
>> +
>> + // If there's no llvm.global_ctors or it's just a decl then skip.
>> + if (!GlobalCtors || GlobalCtors->isDeclaration())
>> + return Error::success();
>> +
>> + std::string InitFunctionName;
>> + raw_string_ostream(InitFunctionName)
>> + << "__orc_init." << M.getModuleIdentifier();
>> +
>> + MangleAndInterner Mangle(PS.getExecutionSession(), M.getDataLayout());
>> + auto InternedName = Mangle(InitFunctionName);
>> + if (auto Err =
>> + R.defineMaterializing({{InternedName, JITSymbolFlags::Callable}}))
>> + return Err;
>> +
>> + auto *InitFunc =
>> + Function::Create(FunctionType::get(Type::getVoidTy(Ctx), {}, false),
>> + GlobalValue::ExternalLinkage, InitFunctionName, &M);
>> + InitFunc->setVisibility(GlobalValue::HiddenVisibility);
>> + std::vector<std::pair<Function *, unsigned>> Inits;
>> + for (auto E : getConstructors(M))
>> + Inits.push_back(std::make_pair(E.Func, E.Priority));
>> + llvm::sort(Inits, [](const std::pair<Function *, unsigned> &LHS,
>> + const std::pair<Function *, unsigned> &RHS) {
>> + return LHS.first < RHS.first;
>> + });
>> + auto *EntryBlock = BasicBlock::Create(Ctx, "entry", InitFunc);
>> + IRBuilder<> IB(EntryBlock);
>> + for (auto &KV : Inits)
>> + IB.CreateCall(KV.first);
>> + IB.CreateRetVoid();
>> +
>> + PS.registerInitFunc(R.getTargetJITDylib(), InternedName);
>> + GlobalCtors->eraseFromParent();
>> + return Error::success();
>> + });
>> +
>> + if (Err)
>> + return std::move(Err);
>> +
>> + return TSM;
>> +}
>> +
>> +class MachOPlatformSupport : public LLJIT::PlatformSupport {
>> +public:
>> + using DLOpenType = void *(*)(const char *Name, int Mode);
>> + using DLCloseType = int (*)(void *Handle);
>> + using DLSymType = void *(*)(void *Handle, const char *Name);
>> + using DLErrorType = const char *(*)();
>> +
>> + struct DlFcnValues {
>> + Optional<void *> RTLDDefault;
>> + DLOpenType dlopen = nullptr;
>> + DLCloseType dlclose = nullptr;
>> + DLSymType dlsym = nullptr;
>> + DLErrorType dlerror = nullptr;
>> + };
>> +
>> + static Expected<std::unique_ptr<MachOPlatformSupport>>
>> + Create(LLJIT &J, JITDylib &PlatformJITDylib) {
>> +
>> + // Make process symbols visible.
>> + {
>> + std::string ErrMsg;
>> + auto Lib = sys::DynamicLibrary::getPermanentLibrary(nullptr, &ErrMsg);
>> + if (!Lib.isValid())
>> + return make_error<StringError>(std::move(ErrMsg),
>> + inconvertibleErrorCode());
>> + }
>> +
>> + DlFcnValues DlFcn;
>> +
>> + // Add support for RTLDDefault on known platforms.
>> +#ifdef __APPLE__
>> + DlFcn.RTLDDefault = reinterpret_cast<void *>(-2);
>> +#endif // __APPLE__
>> +
>> + if (auto Err = hookUpFunction(DlFcn.dlopen, "dlopen"))
>> + return std::move(Err);
>> + if (auto Err = hookUpFunction(DlFcn.dlclose, "dlclose"))
>> + return std::move(Err);
>> + if (auto Err = hookUpFunction(DlFcn.dlsym, "dlsym"))
>> + return std::move(Err);
>> + if (auto Err = hookUpFunction(DlFcn.dlerror, "dlerror"))
>> + return std::move(Err);
>> +
>> + std::unique_ptr<MachOPlatformSupport> MP(
>> + new MachOPlatformSupport(J, PlatformJITDylib, DlFcn));
>> + return std::move(MP);
>> + }
>> +
>> + Error initialize(JITDylib &JD) override {
>> + if (auto InitSeq = MP.getInitializerSequence(JD)) {
>> + for (auto &KV : *InitSeq) {
>> + KV.second.registerObjCSelectors();
>> + if (auto Err = KV.second.registerObjCClasses()) {
>> + // FIXME: Roll back registrations on error?
>> + return Err;
>> + }
>> + }
>> + for (auto &KV : *InitSeq)
>> + KV.second.runModInits();
>> + } else
>> + return InitSeq.takeError();
>> + return Error::success();
>> + }
>> +
>> + Error deinitialize(JITDylib &JD) override {
>> + auto &ES = J.getExecutionSession();
>> + if (auto DeinitSeq = MP.getDeinitializerSequence(JD)) {
>> + for (auto &KV : *DeinitSeq) {
>> + auto DSOHandleName = ES.intern("___dso_handle");
>> +
>> + // FIXME: Run DeInits here.
>> + auto Result = ES.lookup(
>> + {{KV.first, JITDylibLookupFlags::MatchAllSymbols}},
>> + SymbolLookupSet(DSOHandleName,
>> + SymbolLookupFlags::WeaklyReferencedSymbol));
>> + if (!Result)
>> + return Result.takeError();
>> + if (Result->empty())
>> + continue;
>> + assert(Result->count(DSOHandleName) &&
>> + "Result does not contain __dso_handle");
>> + auto *DSOHandle = jitTargetAddressToPointer<void *>(
>> + Result->begin()->second.getAddress());
>> + AtExitMgr.runAtExits(DSOHandle);
>> + }
>> + } else
>> + return DeinitSeq.takeError();
>> + return Error::success();
>> + }
>> +
>> +private:
>> + template <typename FunctionPtrTy>
>> + static Error hookUpFunction(FunctionPtrTy &Fn, const char *Name) {
>> + if (auto *FnAddr = sys::DynamicLibrary::SearchForAddressOfSymbol(Name)) {
>> + Fn = reinterpret_cast<FunctionPtrTy>(Fn);
>> + return Error::success();
>> + }
>> +
>> + return make_error<StringError>((Twine("Can not enable MachO JIT Platform: "
>> + "missing function: ") +
>> + Name)
>> + .str(),
>> + inconvertibleErrorCode());
>> + }
>> +
>> + MachOPlatformSupport(LLJIT &J, JITDylib &PlatformJITDylib, DlFcnValues DlFcn)
>> + : J(J), MP(setupPlatform(J)), DlFcn(std::move(DlFcn)) {
>> +
>> + MangleAndInterner Mangle(J.getExecutionSession(), J.getDataLayout());
>> + SymbolMap HelperSymbols;
>> +
>> + // platform and atexit helpers.
>> + HelperSymbols[Mangle("__lljit.platform_support_instance")] =
>> + JITEvaluatedSymbol(pointerToJITTargetAddress(this), JITSymbolFlags());
>> + HelperSymbols[Mangle("__lljit.cxa_atexit_helper")] = JITEvaluatedSymbol(
>> + pointerToJITTargetAddress(registerAtExitHelper), JITSymbolFlags());
>> + HelperSymbols[Mangle("__lljit.run_atexits_helper")] = JITEvaluatedSymbol(
>> + pointerToJITTargetAddress(runAtExitsHelper), JITSymbolFlags());
>> +
>> + // dlfcn helpers.
>> + HelperSymbols[Mangle("__lljit.dlopen_helper")] = JITEvaluatedSymbol(
>> + pointerToJITTargetAddress(dlopenHelper), JITSymbolFlags());
>> + HelperSymbols[Mangle("__lljit.dlclose_helper")] = JITEvaluatedSymbol(
>> + pointerToJITTargetAddress(dlcloseHelper), JITSymbolFlags());
>> + HelperSymbols[Mangle("__lljit.dlsym_helper")] = JITEvaluatedSymbol(
>> + pointerToJITTargetAddress(dlsymHelper), JITSymbolFlags());
>> + HelperSymbols[Mangle("__lljit.dlerror_helper")] = JITEvaluatedSymbol(
>> + pointerToJITTargetAddress(dlerrorHelper), JITSymbolFlags());
>> +
>> + cantFail(
>> + PlatformJITDylib.define(absoluteSymbols(std::move(HelperSymbols))));
>> + cantFail(MP.setupJITDylib(J.getMainJITDylib()));
>> + cantFail(J.addIRModule(PlatformJITDylib, createPlatformRuntimeModule()));
>> + }
>> +
>> + static MachOPlatform &setupPlatform(LLJIT &J) {
>> + auto Tmp = std::make_unique<MachOPlatform>(
>> + J.getExecutionSession(),
>> + static_cast<ObjectLinkingLayer &>(J.getObjLinkingLayer()),
>> + createStandardSymbolsObject(J));
>> + auto &MP = *Tmp;
>> + J.getExecutionSession().setPlatform(std::move(Tmp));
>> + return MP;
>> + }
>> +
>> + static std::unique_ptr<MemoryBuffer> createStandardSymbolsObject(LLJIT &J) {
>> + LLVMContext Ctx;
>> + Module M("__standard_symbols", Ctx);
>> + M.setDataLayout(J.getDataLayout());
>> +
>> + auto *Int64Ty = Type::getInt64Ty(Ctx);
>> +
>> + auto *DSOHandle =
>> + new GlobalVariable(M, Int64Ty, true, GlobalValue::ExternalLinkage,
>> + ConstantInt::get(Int64Ty, 0), "__dso_handle");
>> + DSOHandle->setVisibility(GlobalValue::HiddenVisibility);
>> +
>> + return cantFail(J.getIRCompileLayer().getCompiler()(M));
>> + }
>> +
>> + ThreadSafeModule createPlatformRuntimeModule() {
>> + auto Ctx = std::make_unique<LLVMContext>();
>> + auto M = std::make_unique<Module>("__standard_lib", *Ctx);
>> + M->setDataLayout(J.getDataLayout());
>> +
>> + auto *MachOPlatformSupportTy =
>> + StructType::create(*Ctx, "lljit.MachOPlatformSupport");
>> +
>> + auto *PlatformInstanceDecl = new GlobalVariable(
>> + *M, MachOPlatformSupportTy, true, GlobalValue::ExternalLinkage, nullptr,
>> + "__lljit.platform_support_instance");
>> +
>> + auto *Int8Ty = Type::getInt8Ty(*Ctx);
>> + auto *IntTy = Type::getIntNTy(*Ctx, sizeof(int) * CHAR_BIT);
>> + auto *VoidTy = Type::getVoidTy(*Ctx);
>> + auto *BytePtrTy = PointerType::getUnqual(Int8Ty);
>> + auto *AtExitCallbackTy = FunctionType::get(VoidTy, {BytePtrTy}, false);
>> + auto *AtExitCallbackPtrTy = PointerType::getUnqual(AtExitCallbackTy);
>> +
>> + addHelperAndWrapper(
>> + *M, "__cxa_atexit",
>> + FunctionType::get(IntTy, {AtExitCallbackPtrTy, BytePtrTy, BytePtrTy},
>> + false),
>> + GlobalValue::DefaultVisibility, "__lljit.cxa_atexit_helper",
>> + {PlatformInstanceDecl});
>> +
>> + addHelperAndWrapper(*M, "dlopen",
>> + FunctionType::get(BytePtrTy, {BytePtrTy, IntTy}, false),
>> + GlobalValue::DefaultVisibility, "__lljit.dlopen_helper",
>> + {PlatformInstanceDecl});
>> +
>> + addHelperAndWrapper(*M, "dlclose",
>> + FunctionType::get(IntTy, {BytePtrTy}, false),
>> + GlobalValue::DefaultVisibility,
>> + "__lljit.dlclose_helper", {PlatformInstanceDecl});
>> +
>> + addHelperAndWrapper(
>> + *M, "dlsym",
>> + FunctionType::get(BytePtrTy, {BytePtrTy, BytePtrTy}, false),
>> + GlobalValue::DefaultVisibility, "__lljit.dlsym_helper",
>> + {PlatformInstanceDecl});
>> +
>> + addHelperAndWrapper(*M, "dlerror", FunctionType::get(BytePtrTy, {}, false),
>> + GlobalValue::DefaultVisibility,
>> + "__lljit.dlerror_helper", {PlatformInstanceDecl});
>> +
>> + return ThreadSafeModule(std::move(M), std::move(Ctx));
>> + }
>> +
>> + static void registerAtExitHelper(void *Self, void (*F)(void *), void *Ctx,
>> + void *DSOHandle) {
>> + static_cast<MachOPlatformSupport *>(Self)->AtExitMgr.registerAtExit(
>> + F, Ctx, DSOHandle);
>> + }
>> +
>> + static void runAtExitsHelper(void *Self, void *DSOHandle) {
>> + static_cast<MachOPlatformSupport *>(Self)->AtExitMgr.runAtExits(DSOHandle);
>> + }
>> +
>> + void *jit_dlopen(const char *Path, int Mode) {
>> + JITDylib *JDToOpen = nullptr;
>> + // FIXME: Do the right thing with Mode flags.
>> + {
>> + std::lock_guard<std::mutex> Lock(PlatformSupportMutex);
>> +
>> + // Clear any existing error messages.
>> + dlErrorMsgs.erase(std::this_thread::get_id());
>> +
>> + if (auto *JD = J.getExecutionSession().getJITDylibByName(Path)) {
>> + auto I = JDRefCounts.find(JD);
>> + if (I != JDRefCounts.end()) {
>> + ++I->second;
>> + return JD;
>> + }
>> +
>> + JDRefCounts[JD] = 1;
>> + JDToOpen = JD;
>> + }
>> + }
>> +
>> + if (JDToOpen) {
>> + if (auto Err = initialize(*JDToOpen)) {
>> + recordError(std::move(Err));
>> + return 0;
>> + }
>> + }
>> +
>> + // Fall through to dlopen if no JITDylib found for Path.
>> + return DlFcn.dlopen(Path, Mode);
>> + }
>> +
>> + static void *dlopenHelper(void *Self, const char *Path, int Mode) {
>> + return static_cast<MachOPlatformSupport *>(Self)->jit_dlopen(Path, Mode);
>> + }
>> +
>> + int jit_dlclose(void *Handle) {
>> + JITDylib *JDToClose = nullptr;
>> +
>> + {
>> + std::lock_guard<std::mutex> Lock(PlatformSupportMutex);
>> +
>> + // Clear any existing error messages.
>> + dlErrorMsgs.erase(std::this_thread::get_id());
>> +
>> + auto I = JDRefCounts.find(Handle);
>> + if (I != JDRefCounts.end()) {
>> + --I->second;
>> + if (I->second == 0) {
>> + JDRefCounts.erase(I);
>> + JDToClose = static_cast<JITDylib *>(Handle);
>> + } else
>> + return 0;
>> + }
>> + }
>> +
>> + if (JDToClose) {
>> + if (auto Err = deinitialize(*JDToClose)) {
>> + recordError(std::move(Err));
>> + return -1;
>> + }
>> + return 0;
>> + }
>> +
>> + // Fall through to dlclose if no JITDylib found for Path.
>> + return DlFcn.dlclose(Handle);
>> + }
>> +
>> + static int dlcloseHelper(void *Self, void *Handle) {
>> + return static_cast<MachOPlatformSupport *>(Self)->jit_dlclose(Handle);
>> + }
>> +
>> + void *jit_dlsym(void *Handle, const char *Name) {
>> + JITDylibSearchOrder JITSymSearchOrder;
>> +
>> + // FIXME: RTLD_NEXT, RTLD_SELF not supported.
>> + {
>> + std::lock_guard<std::mutex> Lock(PlatformSupportMutex);
>> +
>> + // Clear any existing error messages.
>> + dlErrorMsgs.erase(std::this_thread::get_id());
>> +
>> + if (JDRefCounts.count(Handle)) {
>> + JITSymSearchOrder.push_back(
>> + {static_cast<JITDylib *>(Handle),
>> + JITDylibLookupFlags::MatchExportedSymbolsOnly});
>> + } else if (Handle == DlFcn.RTLDDefault) {
>> + for (auto &KV : JDRefCounts)
>> + JITSymSearchOrder.push_back(
>> + {static_cast<JITDylib *>(KV.first),
>> + JITDylibLookupFlags::MatchExportedSymbolsOnly});
>> + }
>> + }
>> +
>> + if (!JITSymSearchOrder.empty()) {
>> + MangleAndInterner Mangle(J.getExecutionSession(), J.getDataLayout());
>> + auto MangledName = Mangle(Name);
>> + SymbolLookupSet Syms(MangledName,
>> + SymbolLookupFlags::WeaklyReferencedSymbol);
>> + if (auto Result = J.getExecutionSession().lookup(JITSymSearchOrder, Syms,
>> + LookupKind::DLSym)) {
>> + auto I = Result->find(MangledName);
>> + if (I != Result->end())
>> + return jitTargetAddressToPointer<void *>(I->second.getAddress());
>> + } else {
>> + recordError(Result.takeError());
>> + return 0;
>> + }
>> + }
>> +
>> + // Fall through to dlsym.
>> + return DlFcn.dlsym(Handle, Name);
>> + }
>> +
>> + static void *dlsymHelper(void *Self, void *Handle, const char *Name) {
>> + return static_cast<MachOPlatformSupport *>(Self)->jit_dlsym(Handle, Name);
>> + }
>> +
>> + const char *jit_dlerror() {
>> + {
>> + std::lock_guard<std::mutex> Lock(PlatformSupportMutex);
>> + auto I = dlErrorMsgs.find(std::this_thread::get_id());
>> + if (I != dlErrorMsgs.end())
>> + return I->second->c_str();
>> + }
>> + return DlFcn.dlerror();
>> + }
>> +
>> + static const char *dlerrorHelper(void *Self) {
>> + return static_cast<MachOPlatformSupport *>(Self)->jit_dlerror();
>> + }
>> +
>> + void recordError(Error Err) {
>> + std::lock_guard<std::mutex> Lock(PlatformSupportMutex);
>> + dlErrorMsgs[std::this_thread::get_id()] =
>> + std::make_unique<std::string>(toString(std::move(Err)));
>> + }
>> +
>> + std::mutex PlatformSupportMutex;
>> + LLJIT &J;
>> + MachOPlatform &MP;
>> + DlFcnValues DlFcn;
>> + ItaniumCXAAtExitSupport AtExitMgr;
>> + DenseMap<void *, unsigned> JDRefCounts;
>> + std::map<std::thread::id, std::unique_ptr<std::string>> dlErrorMsgs;
>> +};
>> +
>> +} // end anonymous namespace
>>
>> namespace llvm {
>> namespace orc {
>>
>> +void LLJIT::PlatformSupport::setInitTransform(
>> + LLJIT &J, IRTransformLayer::TransformFunction T) {
>> + J.InitHelperTransformLayer->setTransform(std::move(T));
>> +}
>> +
>> +LLJIT::PlatformSupport::~PlatformSupport() {}
>> +
>> Error LLJITBuilderState::prepareForConstruction() {
>>
>> if (!JTMB) {
>> @@ -57,7 +869,7 @@ LLJIT::~LLJIT() {
>> Error LLJIT::defineAbsolute(StringRef Name, JITEvaluatedSymbol Sym) {
>> auto InternedName = ES->intern(Name);
>> SymbolMap Symbols({{InternedName, Sym}});
>> - return Main.define(absoluteSymbols(std::move(Symbols)));
>> + return Main->define(absoluteSymbols(std::move(Symbols)));
>> }
>>
>> Error LLJIT::addIRModule(JITDylib &JD, ThreadSafeModule TSM) {
>> @@ -67,7 +879,8 @@ Error LLJIT::addIRModule(JITDylib &JD, ThreadSafeModule TSM) {
>> TSM.withModuleDo([&](Module &M) { return applyDataLayout(M); }))
>> return Err;
>>
>> - return TransformLayer->add(JD, std::move(TSM), ES->allocateVModule());
>> + return InitHelperTransformLayer->add(JD, std::move(TSM),
>> + ES->allocateVModule());
>> }
>>
>> Error LLJIT::addObjectFile(JITDylib &JD, std::unique_ptr<MemoryBuffer> Obj) {
>> @@ -128,16 +941,23 @@ LLJIT::createCompileFunction(LLJITBuilderState &S,
>> }
>>
>> LLJIT::LLJIT(LLJITBuilderState &S, Error &Err)
>> - : ES(S.ES ? std::move(S.ES) : std::make_unique<ExecutionSession>()),
>> - Main(this->ES->createJITDylib("<main>")), DL(""),
>> - TT(S.JTMB->getTargetTriple()),
>> + : ES(S.ES ? std::move(S.ES) : std::make_unique<ExecutionSession>()), Main(),
>> + DL(""), TT(S.JTMB->getTargetTriple()),
>> ObjLinkingLayer(createObjectLinkingLayer(S, *ES)),
>> - ObjTransformLayer(*this->ES, *ObjLinkingLayer), CtorRunner(Main),
>> - DtorRunner(Main) {
>> + ObjTransformLayer(*this->ES, *ObjLinkingLayer) {
>>
>> ErrorAsOutParameter _(&Err);
>>
>> - if (auto DLOrErr = S.JTMB->getDefaultDataLayoutForTarget())
>> + if (auto MainOrErr = this->ES->createJITDylib("main"))
>> + Main = &*MainOrErr;
>> + else {
>> + Err = MainOrErr.takeError();
>> + return;
>> + }
>> +
>> + if (S.DL)
>> + DL = std::move(*S.DL);
>> + else if (auto DLOrErr = S.JTMB->getDefaultDataLayoutForTarget())
>> DL = std::move(*DLOrErr);
>> else {
>> Err = DLOrErr.takeError();
>> @@ -153,10 +973,12 @@ LLJIT::LLJIT(LLJITBuilderState &S, Error &Err)
>> CompileLayer = std::make_unique<IRCompileLayer>(
>> *ES, ObjTransformLayer, std::move(*CompileFunction));
>> TransformLayer = std::make_unique<IRTransformLayer>(*ES, *CompileLayer);
>> + InitHelperTransformLayer =
>> + std::make_unique<IRTransformLayer>(*ES, *TransformLayer);
>> }
>>
>> if (S.NumCompileThreads > 0) {
>> - TransformLayer->setCloneToNewContextOnEmit(true);
>> + InitHelperTransformLayer->setCloneToNewContextOnEmit(true);
>> CompileThreads =
>> std::make_unique<ThreadPool>(hardware_concurrency(S.NumCompileThreads));
>> ES->setDispatchMaterialization(
>> @@ -167,6 +989,11 @@ LLJIT::LLJIT(LLJITBuilderState &S, Error &Err)
>> CompileThreads->async(std::move(Work));
>> });
>> }
>> +
>> + if (S.SetUpPlatform)
>> + Err = S.SetUpPlatform(*this);
>> + else
>> + setUpGenericLLVMIRPlatform(*this);
>> }
>>
>> std::string LLJIT::mangle(StringRef UnmangledName) {
>> @@ -184,15 +1011,24 @@ Error LLJIT::applyDataLayout(Module &M) {
>>
>> if (M.getDataLayout() != DL)
>> return make_error<StringError>(
>> - "Added modules have incompatible data layouts",
>> + "Added modules have incompatible data layouts: " +
>> + M.getDataLayout().getStringRepresentation() + " (module) vs " +
>> + DL.getStringRepresentation() + " (jit)",
>> inconvertibleErrorCode());
>>
>> return Error::success();
>> }
>>
>> -void LLJIT::recordCtorDtors(Module &M) {
>> - CtorRunner.add(getConstructors(M));
>> - DtorRunner.add(getDestructors(M));
>> +void setUpGenericLLVMIRPlatform(LLJIT &J) {
>> + J.setPlatformSupport(std::make_unique<GenericLLVMIRPlatformSupport>(J));
>> +}
>> +
>> +Error setUpMachOPlatform(LLJIT &J) {
>> + auto MP = MachOPlatformSupport::Create(J, J.getMainJITDylib());
>> + if (!MP)
>> + return MP.takeError();
>> + J.setPlatformSupport(std::move(*MP));
>> + return Error::success();
>> }
>>
>> Error LLLazyJITBuilderState::prepareForConstruction() {
>> @@ -205,13 +1041,8 @@ Error LLLazyJITBuilderState::prepareForConstruction() {
>> Error LLLazyJIT::addLazyIRModule(JITDylib &JD, ThreadSafeModule TSM) {
>> assert(TSM && "Can not add null module");
>>
>> - if (auto Err = TSM.withModuleDo([&](Module &M) -> Error {
>> - if (auto Err = applyDataLayout(M))
>> - return Err;
>> -
>> - recordCtorDtors(M);
>> - return Error::success();
>> - }))
>> + if (auto Err = TSM.withModuleDo(
>> + [&](Module &M) -> Error { return applyDataLayout(M); }))
>> return Err;
>>
>> return CODLayer->add(JD, std::move(TSM), ES->allocateVModule());
>> @@ -256,7 +1087,7 @@ LLLazyJIT::LLLazyJIT(LLLazyJITBuilderState &S, Error &Err) : LLJIT(S, Err) {
>>
>> // Create the COD layer.
>> CODLayer = std::make_unique<CompileOnDemandLayer>(
>> - *ES, *TransformLayer, *LCTMgr, std::move(ISMBuilder));
>> + *ES, *InitHelperTransformLayer, *LCTMgr, std::move(ISMBuilder));
>>
>> if (S.NumCompileThreads > 0)
>> CODLayer->setCloneToNewContextOnEmit(true);
>>
>> diff --git a/llvm/lib/ExecutionEngine/Orc/Layer.cpp b/llvm/lib/ExecutionEngine/Orc/Layer.cpp
>> index ebc7801f11ff..63b04a0562bd 100644
>> --- a/llvm/lib/ExecutionEngine/Orc/Layer.cpp
>> +++ b/llvm/lib/ExecutionEngine/Orc/Layer.cpp
>> @@ -7,7 +7,10 @@
>> //===----------------------------------------------------------------------===//
>>
>> #include "llvm/ExecutionEngine/Orc/Layer.h"
>> +
>> +#include "llvm/ExecutionEngine/Orc/ExecutionUtils.h"
>> #include "llvm/IR/Constants.h"
>> +#include "llvm/Object/MachO.h"
>> #include "llvm/Object/ObjectFile.h"
>> #include "llvm/Support/Debug.h"
>>
>> @@ -23,10 +26,11 @@ Error IRLayer::add(JITDylib &JD, ThreadSafeModule TSM, VModuleKey K) {
>> *this, *getManglingOptions(), std::move(TSM), std::move(K)));
>> }
>>
>> -IRMaterializationUnit::IRMaterializationUnit(ExecutionSession &ES,
>> - const ManglingOptions &MO,
>> - ThreadSafeModule TSM, VModuleKey K)
>> - : MaterializationUnit(SymbolFlagsMap(), std::move(K)), TSM(std::move(TSM)) {
>> +IRMaterializationUnit::IRMaterializationUnit(
>> + ExecutionSession &ES, const IRSymbolMapper::ManglingOptions &MO,
>> + ThreadSafeModule TSM, VModuleKey K)
>> + : MaterializationUnit(SymbolFlagsMap(), nullptr, std::move(K)),
>> + TSM(std::move(TSM)) {
>>
>> assert(this->TSM && "Module must not be null");
>>
>> @@ -34,6 +38,7 @@ IRMaterializationUnit::IRMaterializationUnit(ExecutionSession &ES,
>> this->TSM.withModuleDo([&](Module &M) {
>> for (auto &G : M.global_values()) {
>> // Skip globals that don't generate symbols.
>> +
>> if (!G.hasName() || G.isDeclaration() || G.hasLocalLinkage() ||
>> G.hasAvailableExternallyLinkage() || G.hasAppendingLinkage())
>> continue;
>> @@ -72,13 +77,23 @@ IRMaterializationUnit::IRMaterializationUnit(ExecutionSession &ES,
>> SymbolFlags[MangledName] = JITSymbolFlags::fromGlobalValue(G);
>> SymbolToDefinition[MangledName] = &G;
>> }
>> +
>> + // If we need an init symbol for this module then create one.
>> + if (!llvm::empty(getStaticInitGVs(M))) {
>> + std::string InitSymbolName;
>> + raw_string_ostream(InitSymbolName)
>> + << "$." << M.getModuleIdentifier() << ".__inits";
>> + InitSymbol = ES.intern(InitSymbolName);
>> + SymbolFlags[InitSymbol] = JITSymbolFlags();
>> + }
>> });
>> }
>>
>> IRMaterializationUnit::IRMaterializationUnit(
>> ThreadSafeModule TSM, VModuleKey K, SymbolFlagsMap SymbolFlags,
>> - SymbolNameToDefinitionMap SymbolToDefinition)
>> - : MaterializationUnit(std::move(SymbolFlags), std::move(K)),
>> + SymbolStringPtr InitSymbol, SymbolNameToDefinitionMap SymbolToDefinition)
>> + : MaterializationUnit(std::move(SymbolFlags), std::move(InitSymbol),
>> + std::move(K)),
>> TSM(std::move(TSM)), SymbolToDefinition(std::move(SymbolToDefinition)) {}
>>
>> StringRef IRMaterializationUnit::getName() const {
>> @@ -105,7 +120,8 @@ void IRMaterializationUnit::discard(const JITDylib &JD,
>> }
>>
>> BasicIRLayerMaterializationUnit::BasicIRLayerMaterializationUnit(
>> - IRLayer &L, const ManglingOptions &MO, ThreadSafeModule TSM, VModuleKey K)
>> + IRLayer &L, const IRSymbolMapper::ManglingOptions &MO, ThreadSafeModule TSM,
>> + VModuleKey K)
>> : IRMaterializationUnit(L.getExecutionSession(), MO, std::move(TSM),
>> std::move(K)),
>> L(L), K(std::move(K)) {}
>> @@ -150,22 +166,26 @@ Error ObjectLayer::add(JITDylib &JD, std::unique_ptr<MemoryBuffer> O,
>> Expected<std::unique_ptr<BasicObjectLayerMaterializationUnit>>
>> BasicObjectLayerMaterializationUnit::Create(ObjectLayer &L, VModuleKey K,
>> std::unique_ptr<MemoryBuffer> O) {
>> - auto SymbolFlags =
>> - getObjectSymbolFlags(L.getExecutionSession(), O->getMemBufferRef());
>> + auto ObjSymInfo =
>> + getObjectSymbolInfo(L.getExecutionSession(), O->getMemBufferRef());
>> +
>> + if (!ObjSymInfo)
>> + return ObjSymInfo.takeError();
>>
>> - if (!SymbolFlags)
>> - return SymbolFlags.takeError();
>> + auto &SymbolFlags = ObjSymInfo->first;
>> + auto &InitSymbol = ObjSymInfo->second;
>>
>> return std::unique_ptr<BasicObjectLayerMaterializationUnit>(
>> - new BasicObjectLayerMaterializationUnit(L, K, std::move(O),
>> - std::move(*SymbolFlags)));
>> + new BasicObjectLayerMaterializationUnit(
>> + L, K, std::move(O), std::move(SymbolFlags), std::move(InitSymbol)));
>> }
>>
>> BasicObjectLayerMaterializationUnit::BasicObjectLayerMaterializationUnit(
>> ObjectLayer &L, VModuleKey K, std::unique_ptr<MemoryBuffer> O,
>> - SymbolFlagsMap SymbolFlags)
>> - : MaterializationUnit(std::move(SymbolFlags), std::move(K)), L(L),
>> - O(std::move(O)) {}
>> + SymbolFlagsMap SymbolFlags, SymbolStringPtr InitSymbol)
>> + : MaterializationUnit(std::move(SymbolFlags), std::move(InitSymbol),
>> + std::move(K)),
>> + L(L), O(std::move(O)) {}
>>
>> StringRef BasicObjectLayerMaterializationUnit::getName() const {
>> if (O)
>> @@ -184,35 +204,5 @@ void BasicObjectLayerMaterializationUnit::discard(const JITDylib &JD,
>> // filter to pass to the object layer along with the object itself.
>> }
>>
>> -Expected<SymbolFlagsMap> getObjectSymbolFlags(ExecutionSession &ES,
>> - MemoryBufferRef ObjBuffer) {
>> - auto Obj = object::ObjectFile::createObjectFile(ObjBuffer);
>> -
>> - if (!Obj)
>> - return Obj.takeError();
>> -
>> - SymbolFlagsMap SymbolFlags;
>> - for (auto &Sym : (*Obj)->symbols()) {
>> - // Skip symbols not defined in this object file.
>> - if (Sym.getFlags() & object::BasicSymbolRef::SF_Undefined)
>> - continue;
>> -
>> - // Skip symbols that are not global.
>> - if (!(Sym.getFlags() & object::BasicSymbolRef::SF_Global))
>> - continue;
>> -
>> - auto Name = Sym.getName();
>> - if (!Name)
>> - return Name.takeError();
>> - auto InternedName = ES.intern(*Name);
>> - auto SymFlags = JITSymbolFlags::fromObjectSymbol(Sym);
>> - if (!SymFlags)
>> - return SymFlags.takeError();
>> - SymbolFlags[InternedName] = std::move(*SymFlags);
>> - }
>> -
>> - return SymbolFlags;
>> -}
>> -
>> } // End namespace orc.
>> } // End namespace llvm.
>>
>> diff --git a/llvm/lib/ExecutionEngine/Orc/LazyReexports.cpp b/llvm/lib/ExecutionEngine/Orc/LazyReexports.cpp
>> index 97f36ede713f..27d08e18270f 100644
>> --- a/llvm/lib/ExecutionEngine/Orc/LazyReexports.cpp
>> +++ b/llvm/lib/ExecutionEngine/Orc/LazyReexports.cpp
>> @@ -51,7 +51,7 @@ LazyCallThroughManager::callThroughToSymbol(JITTargetAddress TrampolineAddr) {
>>
>> auto LookupResult = ES.lookup(
>> makeJITDylibSearchOrder(SourceJD, JITDylibLookupFlags::MatchAllSymbols),
>> - SymbolName);
>> + SymbolName, SymbolState::Ready);
>>
>> if (!LookupResult) {
>> ES.reportError(LookupResult.takeError());
>> @@ -123,10 +123,9 @@ LazyReexportsMaterializationUnit::LazyReexportsMaterializationUnit(
>> LazyCallThroughManager &LCTManager, IndirectStubsManager &ISManager,
>> JITDylib &SourceJD, SymbolAliasMap CallableAliases, ImplSymbolMap *SrcJDLoc,
>> VModuleKey K)
>> - : MaterializationUnit(extractFlags(CallableAliases), std::move(K)),
>> + : MaterializationUnit(extractFlags(CallableAliases), nullptr, std::move(K)),
>> LCTManager(LCTManager), ISManager(ISManager), SourceJD(SourceJD),
>> - CallableAliases(std::move(CallableAliases)),
>> - AliaseeTable(SrcJDLoc) {}
>> + CallableAliases(std::move(CallableAliases)), AliaseeTable(SrcJDLoc) {}
>>
>> StringRef LazyReexportsMaterializationUnit::getName() const {
>> return "<Lazy Reexports>";
>>
>> diff --git a/llvm/lib/ExecutionEngine/Orc/MachOPlatform.cpp b/llvm/lib/ExecutionEngine/Orc/MachOPlatform.cpp
>> new file mode 100644
>> index 000000000000..4618d53315ef
>> --- /dev/null
>> +++ b/llvm/lib/ExecutionEngine/Orc/MachOPlatform.cpp
>> @@ -0,0 +1,459 @@
>> +//===------ MachOPlatform.cpp - Utilities for executing MachO in Orc ------===//
>> +//
>> +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
>> +// See https://llvm.org/LICENSE.txt for license information.
>> +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
>> +//
>> +//===----------------------------------------------------------------------===//
>> +
>> +#include "llvm/ExecutionEngine/Orc/MachOPlatform.h"
>> +
>> +#include "llvm/BinaryFormat/MachO.h"
>> +#include "llvm/Support/BinaryByteStream.h"
>> +
>> +namespace {
>> +
>> +struct objc_class;
>> +struct objc_image_info;
>> +struct objc_object;
>> +struct objc_selector;
>> +
>> +using Class = objc_class *;
>> +using id = objc_object *;
>> +using SEL = objc_selector *;
>> +
>> +using ObjCMsgSendTy = id (*)(id, SEL, ...);
>> +using ObjCReadClassPairTy = Class (*)(Class, const objc_image_info *);
>> +using SelRegisterNameTy = SEL (*)(const char *);
>> +
>> +enum class ObjCRegistrationAPI { Uninitialized, Unavailable, Initialized };
>> +
>> +ObjCRegistrationAPI ObjCRegistrationAPIState =
>> + ObjCRegistrationAPI::Uninitialized;
>> +ObjCMsgSendTy objc_msgSend = nullptr;
>> +ObjCReadClassPairTy objc_readClassPair = nullptr;
>> +SelRegisterNameTy sel_registerName = nullptr;
>> +
>> +} // end anonymous namespace
>> +
>> +namespace llvm {
>> +namespace orc {
>> +
>> +template <typename FnTy>
>> +static Error setUpObjCRegAPIFunc(FnTy &Target, sys::DynamicLibrary &LibObjC,
>> + const char *Name) {
>> + if (void *Addr = LibObjC.getAddressOfSymbol(Name))
>> + Target = reinterpret_cast<FnTy>(Addr);
>> + else
>> + return make_error<StringError>(
>> + (Twine("Could not find address for ") + Name).str(),
>> + inconvertibleErrorCode());
>> + return Error::success();
>> +}
>> +
>> +Error enableObjCRegistration(const char *PathToLibObjC) {
>> + // If we've already tried to initialize then just bail out.
>> + if (ObjCRegistrationAPIState != ObjCRegistrationAPI::Uninitialized)
>> + return Error::success();
>> +
>> + ObjCRegistrationAPIState = ObjCRegistrationAPI::Unavailable;
>> +
>> + std::string ErrMsg;
>> + auto LibObjC =
>> + sys::DynamicLibrary::getPermanentLibrary(PathToLibObjC, &ErrMsg);
>> +
>> + if (!LibObjC.isValid())
>> + return make_error<StringError>(std::move(ErrMsg), inconvertibleErrorCode());
>> +
>> + if (auto Err = setUpObjCRegAPIFunc(objc_msgSend, LibObjC, "objc_msgSend"))
>> + return Err;
>> + if (auto Err = setUpObjCRegAPIFunc(objc_readClassPair, LibObjC,
>> + "objc_readClassPair"))
>> + return Err;
>> + if (auto Err =
>> + setUpObjCRegAPIFunc(sel_registerName, LibObjC, "sel_registerName"))
>> + return Err;
>> +
>> + ObjCRegistrationAPIState = ObjCRegistrationAPI::Initialized;
>> + return Error::success();
>> +}
>> +
>> +bool objcRegistrationEnabled() {
>> + return ObjCRegistrationAPIState == ObjCRegistrationAPI::Initialized;
>> +}
>> +
>> +void MachOJITDylibInitializers::runModInits() const {
>> + for (const auto &ModInit : ModInitSections) {
>> + for (uint64_t I = 0; I != ModInit.NumPtrs; ++I) {
>> + auto *InitializerAddr = jitTargetAddressToPointer<uintptr_t *>(
>> + ModInit.Address + (I * sizeof(uintptr_t)));
>> + auto *Initializer =
>> + jitTargetAddressToFunction<void (*)()>(*InitializerAddr);
>> + Initializer();
>> + }
>> + }
>> +}
>> +
>> +void MachOJITDylibInitializers::registerObjCSelectors() const {
>> + assert(objcRegistrationEnabled() && "ObjC registration not enabled.");
>> +
>> + for (const auto &ObjCSelRefs : ObjCSelRefsSections) {
>> + for (uint64_t I = 0; I != ObjCSelRefs.NumPtrs; ++I) {
>> + auto SelEntryAddr = ObjCSelRefs.Address + (I * sizeof(uintptr_t));
>> + const auto *SelName =
>> + *jitTargetAddressToPointer<const char **>(SelEntryAddr);
>> + auto Sel = sel_registerName(SelName);
>> + *jitTargetAddressToPointer<SEL *>(SelEntryAddr) = Sel;
>> + }
>> + }
>> +}
>> +
>> +Error MachOJITDylibInitializers::registerObjCClasses() const {
>> + assert(objcRegistrationEnabled() && "ObjC registration not enabled.");
>> +
>> + struct ObjCClassCompiled {
>> + void *Metaclass;
>> + void *Parent;
>> + void *Cache1;
>> + void *Cache2;
>> + void *Data;
>> + };
>> +
>> + auto *ImageInfo =
>> + jitTargetAddressToPointer<const objc_image_info *>(ObjCImageInfoAddr);
>> + auto ClassSelector = sel_registerName("class");
>> +
>> + for (const auto &ObjCClassList : ObjCClassListSections) {
>> + for (uint64_t I = 0; I != ObjCClassList.NumPtrs; ++I) {
>> + auto ClassPtrAddr = ObjCClassList.Address + (I * sizeof(uintptr_t));
>> + auto Cls = *jitTargetAddressToPointer<Class *>(ClassPtrAddr);
>> + auto *ClassCompiled =
>> + *jitTargetAddressToPointer<ObjCClassCompiled **>(ClassPtrAddr);
>> + objc_msgSend(reinterpret_cast<id>(ClassCompiled->Parent), ClassSelector);
>> + auto Registered = objc_readClassPair(Cls, ImageInfo);
>> +
>> + // FIXME: Improve diagnostic by reporting the failed class's name.
>> + if (Registered != Cls)
>> + return make_error<StringError>("Unable to register Objective-C class",
>> + inconvertibleErrorCode());
>> + }
>> + }
>> + return Error::success();
>> +}
>> +
>> +void MachOJITDylibInitializers::dump() const {
>> + for (auto &Extent : ModInitSections)
>> + dbgs() << formatv("{0:x16}", Extent.Address) << " -- "
>> + << formatv("{0:x16}", Extent.Address + 8 * Extent.NumPtrs) << "\n";
>> +}
>> +
>> +MachOPlatform::MachOPlatform(
>> + ExecutionSession &ES, ObjectLinkingLayer &ObjLinkingLayer,
>> + std::unique_ptr<MemoryBuffer> StandardSymbolsObject)
>> + : ES(ES), ObjLinkingLayer(ObjLinkingLayer),
>> + StandardSymbolsObject(std::move(StandardSymbolsObject)) {
>> + ObjLinkingLayer.addPlugin(std::make_unique<InitScraperPlugin>(*this));
>> +}
>> +
>> +Error MachOPlatform::setupJITDylib(JITDylib &JD) {
>> + auto ObjBuffer = MemoryBuffer::getMemBuffer(
>> + StandardSymbolsObject->getMemBufferRef(), false);
>> + return ObjLinkingLayer.add(JD, std::move(ObjBuffer));
>> +}
>> +
>> +Error MachOPlatform::notifyAdding(JITDylib &JD, const MaterializationUnit &MU) {
>> + const auto &InitSym = MU.getInitializerSymbol();
>> + if (!InitSym)
>> + return Error::success();
>> +
>> + std::lock_guard<std::mutex> Lock(PlatformMutex);
>> + RegisteredInitSymbols[&JD].add(InitSym);
>> + return Error::success();
>> +}
>> +
>> +Error MachOPlatform::notifyRemoving(JITDylib &JD, VModuleKey K) {
>> + llvm_unreachable("Not supported yet");
>> +}
>> +
>> +Expected<MachOPlatform::InitializerSequence>
>> +MachOPlatform::getInitializerSequence(JITDylib &JD) {
>> +
>> + std::vector<JITDylib *> DFSLinkOrder;
>> +
>> + while (true) {
>> + // Lock the platform while we search for any initializer symbols to
>> + // look up.
>> + DenseMap<JITDylib *, SymbolLookupSet> NewInitSymbols;
>> + {
>> + std::lock_guard<std::mutex> Lock(PlatformMutex);
>> + DFSLinkOrder = getDFSLinkOrder(JD);
>> +
>> + for (auto *InitJD : DFSLinkOrder) {
>> + auto RISItr = RegisteredInitSymbols.find(InitJD);
>> + if (RISItr != RegisteredInitSymbols.end()) {
>> + NewInitSymbols[InitJD] = std::move(RISItr->second);
>> + RegisteredInitSymbols.erase(RISItr);
>> + }
>> + }
>> + }
>> +
>> + if (NewInitSymbols.empty())
>> + break;
>> +
>> + // Outside the lock, issue the lookup.
>> + if (auto R = lookupInitSymbols(JD.getExecutionSession(), NewInitSymbols))
>> + ; // Nothing to do in the success case.
>> + else
>> + return R.takeError();
>> + }
>> +
>> + // Lock again to collect the initializers.
>> + InitializerSequence FullInitSeq;
>> + {
>> + std::lock_guard<std::mutex> Lock(PlatformMutex);
>> + for (auto *InitJD : reverse(DFSLinkOrder)) {
>> + auto ISItr = InitSeqs.find(InitJD);
>> + if (ISItr != InitSeqs.end()) {
>> + FullInitSeq.emplace_back(InitJD, std::move(ISItr->second));
>> + InitSeqs.erase(ISItr);
>> + }
>> + }
>> + }
>> +
>> + return FullInitSeq;
>> +}
>> +
>> +Expected<MachOPlatform::DeinitializerSequence>
>> +MachOPlatform::getDeinitializerSequence(JITDylib &JD) {
>> + std::vector<JITDylib *> DFSLinkOrder = getDFSLinkOrder(JD);
>> +
>> + DeinitializerSequence FullDeinitSeq;
>> + {
>> + std::lock_guard<std::mutex> Lock(PlatformMutex);
>> + for (auto *DeinitJD : DFSLinkOrder) {
>> + FullDeinitSeq.emplace_back(DeinitJD, MachOJITDylibDeinitializers());
>> + }
>> + }
>> +
>> + return FullDeinitSeq;
>> +}
>> +
>> +std::vector<JITDylib *> MachOPlatform::getDFSLinkOrder(JITDylib &JD) {
>> + std::vector<JITDylib *> Result, WorkStack({&JD});
>> + DenseSet<JITDylib *> Visited;
>> +
>> + while (!WorkStack.empty()) {
>> + auto *NextJD = WorkStack.back();
>> + WorkStack.pop_back();
>> + if (Visited.count(NextJD))
>> + continue;
>> + Visited.insert(NextJD);
>> + Result.push_back(NextJD);
>> + NextJD->withSearchOrderDo([&](const JITDylibSearchOrder &SO) {
>> + for (auto &KV : SO)
>> + WorkStack.push_back(KV.first);
>> + });
>> + }
>> +
>> + return Result;
>> +}
>> +
>> +void MachOPlatform::registerInitInfo(
>> + JITDylib &JD, JITTargetAddress ObjCImageInfoAddr,
>> + MachOJITDylibInitializers::SectionExtent ModInits,
>> + MachOJITDylibInitializers::SectionExtent ObjCSelRefs,
>> + MachOJITDylibInitializers::SectionExtent ObjCClassList) {
>> + std::lock_guard<std::mutex> Lock(PlatformMutex);
>> +
>> + auto &InitSeq = InitSeqs[&JD];
>> +
>> + InitSeq.setObjCImageInfoAddr(ObjCImageInfoAddr);
>> +
>> + if (ModInits.Address)
>> + InitSeq.addModInitsSection(std::move(ModInits));
>> +
>> + if (ObjCSelRefs.Address)
>> + InitSeq.addObjCSelRefsSection(std::move(ObjCSelRefs));
>> +
>> + if (ObjCClassList.Address)
>> + InitSeq.addObjCClassListSection(std::move(ObjCClassList));
>> +}
>> +
>> +static Expected<MachOJITDylibInitializers::SectionExtent>
>> +getSectionExtent(jitlink::LinkGraph &G, StringRef SectionName) {
>> + auto *Sec = G.findSectionByName(SectionName);
>> + if (!Sec)
>> + return MachOJITDylibInitializers::SectionExtent();
>> + jitlink::SectionRange R(*Sec);
>> + if (R.getSize() % G.getPointerSize() != 0)
>> + return make_error<StringError>(SectionName + " section size is not a "
>> + "multiple of the pointer size",
>> + inconvertibleErrorCode());
>> + return MachOJITDylibInitializers::SectionExtent(
>> + R.getStart(), R.getSize() / G.getPointerSize());
>> +}
>> +
>> +void MachOPlatform::InitScraperPlugin::modifyPassConfig(
>> + MaterializationResponsibility &MR, const Triple &TT,
>> + jitlink::PassConfiguration &Config) {
>> +
>> + Config.PrePrunePasses.push_back([this, &MR](jitlink::LinkGraph &G) -> Error {
>> + JITLinkSymbolVector InitSectionSymbols;
>> + preserveInitSectionIfPresent(InitSectionSymbols, G, "__mod_init_func");
>> + preserveInitSectionIfPresent(InitSectionSymbols, G, "__objc_selrefs");
>> + preserveInitSectionIfPresent(InitSectionSymbols, G, "__objc_classlist");
>> +
>> + if (!InitSymbolDeps.empty()) {
>> + std::lock_guard<std::mutex> Lock(InitScraperMutex);
>> + InitSymbolDeps[&MR] = std::move(InitSectionSymbols);
>> + }
>> +
>> + if (auto Err = processObjCImageInfo(G, MR))
>> + return Err;
>> +
>> + return Error::success();
>> + });
>> +
>> + Config.PostFixupPasses.push_back([this, &JD = MR.getTargetJITDylib()](
>> + jitlink::LinkGraph &G) -> Error {
>> + MachOJITDylibInitializers::SectionExtent ModInits, ObjCSelRefs,
>> + ObjCClassList;
>> +
>> + JITTargetAddress ObjCImageInfoAddr = 0;
>> + if (auto *ObjCImageInfoSec = G.findSectionByName("__objc_image_info")) {
>> + if (auto Addr = jitlink::SectionRange(*ObjCImageInfoSec).getStart()) {
>> + ObjCImageInfoAddr = Addr;
>> + dbgs() << "Recorded __objc_imageinfo @ " << formatv("{0:x16}", Addr);
>> + }
>> + }
>> +
>> + // Record __mod_init_func.
>> + if (auto ModInitsOrErr = getSectionExtent(G, "__mod_init_func"))
>> + ModInits = std::move(*ModInitsOrErr);
>> + else
>> + return ModInitsOrErr.takeError();
>> +
>> + // Record __objc_selrefs.
>> + if (auto ObjCSelRefsOrErr = getSectionExtent(G, "__objc_selrefs"))
>> + ObjCSelRefs = std::move(*ObjCSelRefsOrErr);
>> + else
>> + return ObjCSelRefsOrErr.takeError();
>> +
>> + // Record __objc_classlist.
>> + if (auto ObjCClassListOrErr = getSectionExtent(G, "__objc_classlist"))
>> + ObjCClassList = std::move(*ObjCClassListOrErr);
>> + else
>> + return ObjCClassListOrErr.takeError();
>> +
>> + MP.registerInitInfo(JD, ObjCImageInfoAddr, std::move(ModInits),
>> + std::move(ObjCSelRefs), std::move(ObjCClassList));
>> +
>> + return Error::success();
>> + });
>> +}
>> +
>> +ObjectLinkingLayer::Plugin::LocalDependenciesMap
>> +MachOPlatform::InitScraperPlugin::getSyntheticSymbolLocalDependencies(
>> + MaterializationResponsibility &MR) {
>> + std::lock_guard<std::mutex> Lock(InitScraperMutex);
>> + auto I = InitSymbolDeps.find(&MR);
>> + if (I != InitSymbolDeps.end()) {
>> + LocalDependenciesMap Result;
>> + Result[MR.getInitializerSymbol()] = std::move(I->second);
>> + InitSymbolDeps.erase(&MR);
>> + return Result;
>> + }
>> + return LocalDependenciesMap();
>> +}
>> +
>> +void MachOPlatform::InitScraperPlugin::preserveInitSectionIfPresent(
>> + JITLinkSymbolVector &Symbols, jitlink::LinkGraph &G,
>> + StringRef SectionName) {
>> + if (auto *Sec = G.findSectionByName(SectionName)) {
>> + auto SecBlocks = Sec->blocks();
>> + if (!llvm::empty(SecBlocks))
>> + Symbols.push_back(
>> + &G.addAnonymousSymbol(**SecBlocks.begin(), 0, 0, false, true));
>> + }
>> +}
>> +
>> +Error MachOPlatform::InitScraperPlugin::processObjCImageInfo(
>> + jitlink::LinkGraph &G, MaterializationResponsibility &MR) {
>> +
>> + // If there's an ObjC imagine info then either
>> + // (1) It's the first __objc_imageinfo we've seen in this JITDylib. In
>> + // this case we name and record it.
>> + // OR
>> + // (2) We already have a recorded __objc_imageinfo for this JITDylib,
>> + // in which case we just verify it.
>> + auto *ObjCImageInfo = G.findSectionByName("__objc_imageinfo");
>> + if (!ObjCImageInfo)
>> + return Error::success();
>> +
>> + auto ObjCImageInfoBlocks = ObjCImageInfo->blocks();
>> +
>> + // Check that the section is not empty if present.
>> + if (llvm::empty(ObjCImageInfoBlocks))
>> + return make_error<StringError>("Empty __objc_imageinfo section in " +
>> + G.getName(),
>> + inconvertibleErrorCode());
>> +
>> + // Check that there's only one block in the section.
>> + if (std::next(ObjCImageInfoBlocks.begin()) != ObjCImageInfoBlocks.end())
>> + return make_error<StringError>("Multiple blocks in __objc_imageinfo "
>> + "section in " +
>> + G.getName(),
>> + inconvertibleErrorCode());
>> +
>> + // Check that the __objc_imageinfo section is unreferenced.
>> + // FIXME: We could optimize this check if Symbols had a ref-count.
>> + for (auto &Sec : G.sections()) {
>> + if (&Sec != ObjCImageInfo)
>> + for (auto *B : Sec.blocks())
>> + for (auto &E : B->edges())
>> + if (E.getTarget().isDefined() &&
>> + &E.getTarget().getBlock().getSection() == ObjCImageInfo)
>> + return make_error<StringError>("__objc_imageinfo is referenced "
>> + "within file " +
>> + G.getName(),
>> + inconvertibleErrorCode());
>> + }
>> +
>> + auto &ObjCImageInfoBlock = **ObjCImageInfoBlocks.begin();
>> + auto *ObjCImageInfoData = ObjCImageInfoBlock.getContent().data();
>> + auto Version = support::endian::read32(ObjCImageInfoData, G.getEndianness());
>> + auto Flags =
>> + support::endian::read32(ObjCImageInfoData + 4, G.getEndianness());
>> +
>> + // Lock the mutex while we verify / update the ObjCImageInfos map.
>> + std::lock_guard<std::mutex> Lock(InitScraperMutex);
>> +
>> + auto ObjCImageInfoItr = ObjCImageInfos.find(&MR.getTargetJITDylib());
>> + if (ObjCImageInfoItr != ObjCImageInfos.end()) {
>> + // We've already registered an __objc_imageinfo section. Verify the
>> + // content of this new section matches, then delete it.
>> + if (ObjCImageInfoItr->second.first != Version)
>> + return make_error<StringError>(
>> + "ObjC version in " + G.getName() +
>> + " does not match first registered version",
>> + inconvertibleErrorCode());
>> + if (ObjCImageInfoItr->second.second != Flags)
>> + return make_error<StringError>("ObjC flags in " + G.getName() +
>> + " do not match first registered flags",
>> + inconvertibleErrorCode());
>> +
>> + // __objc_imageinfo is valid. Delete the block.
>> + for (auto *S : ObjCImageInfo->symbols())
>> + G.removeDefinedSymbol(*S);
>> + G.removeBlock(ObjCImageInfoBlock);
>> + } else {
>> + // We haven't registered an __objc_imageinfo section yet. Register and
>> + // move on. The section should already be marked no-dead-strip.
>> + ObjCImageInfos[&MR.getTargetJITDylib()] = std::make_pair(Version, Flags);
>> + }
>> +
>> + return Error::success();
>> +}
>> +
>> +} // End namespace orc.
>> +} // End namespace llvm.
>>
>> diff --git a/llvm/lib/ExecutionEngine/Orc/Mangling.cpp b/llvm/lib/ExecutionEngine/Orc/Mangling.cpp
>> new file mode 100644
>> index 000000000000..7ec7869d00b5
>> --- /dev/null
>> +++ b/llvm/lib/ExecutionEngine/Orc/Mangling.cpp
>> @@ -0,0 +1,131 @@
>> +//===----------- Mangling.cpp -- Name Mangling Utilities for ORC ----------===//
>> +//
>> +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
>> +// See https://llvm.org/LICENSE.txt for license information.
>> +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
>> +//
>> +//===----------------------------------------------------------------------===//
>> +
>> +#include "llvm/ExecutionEngine/Orc/Mangling.h"
>> +#include "llvm/IR/Constants.h"
>> +#include "llvm/IR/Mangler.h"
>> +#include "llvm/Object/MachO.h"
>> +#include "llvm/Object/ObjectFile.h"
>> +#include "llvm/Support/Debug.h"
>> +
>> +#define DEBUG_TYPE "orc"
>> +
>> +namespace llvm {
>> +namespace orc {
>> +
>> +MangleAndInterner::MangleAndInterner(ExecutionSession &ES, const DataLayout &DL)
>> + : ES(ES), DL(DL) {}
>> +
>> +SymbolStringPtr MangleAndInterner::operator()(StringRef Name) {
>> + std::string MangledName;
>> + {
>> + raw_string_ostream MangledNameStream(MangledName);
>> + Mangler::getNameWithPrefix(MangledNameStream, Name, DL);
>> + }
>> + return ES.intern(MangledName);
>> +}
>> +
>> +void IRSymbolMapper::add(ExecutionSession &ES, const ManglingOptions &MO,
>> + ArrayRef<GlobalValue *> GVs,
>> + SymbolFlagsMap &SymbolFlags,
>> + SymbolNameToDefinitionMap *SymbolToDefinition) {
>> + if (GVs.empty())
>> + return;
>> +
>> + MangleAndInterner Mangle(ES, GVs[0]->getParent()->getDataLayout());
>> + for (auto *G : GVs) {
>> + assert(G && "GVs cannot contain null elements");
>> + if (!G->hasName() || G->isDeclaration() || G->hasLocalLinkage() ||
>> + G->hasAvailableExternallyLinkage() || G->hasAppendingLinkage())
>> + continue;
>> +
>> + if (G->isThreadLocal() && MO.EmulatedTLS) {
>> + auto *GV = cast<GlobalVariable>(G);
>> +
>> + auto Flags = JITSymbolFlags::fromGlobalValue(*GV);
>> +
>> + auto EmuTLSV = Mangle(("__emutls_v." + GV->getName()).str());
>> + SymbolFlags[EmuTLSV] = Flags;
>> + if (SymbolToDefinition)
>> + (*SymbolToDefinition)[EmuTLSV] = GV;
>> +
>> + // If this GV has a non-zero initializer we'll need to emit an
>> + // __emutls.t symbol too.
>> + if (GV->hasInitializer()) {
>> + const auto *InitVal = GV->getInitializer();
>> +
>> + // Skip zero-initializers.
>> + if (isa<ConstantAggregateZero>(InitVal))
>> + continue;
>> + const auto *InitIntValue = dyn_cast<ConstantInt>(InitVal);
>> + if (InitIntValue && InitIntValue->isZero())
>> + continue;
>> +
>> + auto EmuTLST = Mangle(("__emutls_t." + GV->getName()).str());
>> + SymbolFlags[EmuTLST] = Flags;
>> + if (SymbolToDefinition)
>> + (*SymbolToDefinition)[EmuTLST] = GV;
>> + }
>> + continue;
>> + }
>> +
>> + // Otherwise we just need a normal linker mangling.
>> + auto MangledName = Mangle(G->getName());
>> + SymbolFlags[MangledName] = JITSymbolFlags::fromGlobalValue(*G);
>> + if (SymbolToDefinition)
>> + (*SymbolToDefinition)[MangledName] = G;
>> + }
>> +}
>> +
>> +Expected<std::pair<SymbolFlagsMap, SymbolStringPtr>>
>> +getObjectSymbolInfo(ExecutionSession &ES, MemoryBufferRef ObjBuffer) {
>> + auto Obj = object::ObjectFile::createObjectFile(ObjBuffer);
>> +
>> + if (!Obj)
>> + return Obj.takeError();
>> +
>> + SymbolFlagsMap SymbolFlags;
>> + for (auto &Sym : (*Obj)->symbols()) {
>> + // Skip symbols not defined in this object file.
>> + if (Sym.getFlags() & object::BasicSymbolRef::SF_Undefined)
>> + continue;
>> +
>> + // Skip symbols that are not global.
>> + if (!(Sym.getFlags() & object::BasicSymbolRef::SF_Global))
>> + continue;
>> +
>> + auto Name = Sym.getName();
>> + if (!Name)
>> + return Name.takeError();
>> + auto InternedName = ES.intern(*Name);
>> + auto SymFlags = JITSymbolFlags::fromObjectSymbol(Sym);
>> + if (!SymFlags)
>> + return SymFlags.takeError();
>> + SymbolFlags[InternedName] = std::move(*SymFlags);
>> + }
>> +
>> + SymbolStringPtr InitSymbol;
>> +
>> + if (auto *MachOObj = dyn_cast<object::MachOObjectFile>(Obj->get())) {
>> + for (auto &Sec : MachOObj->sections()) {
>> + auto SecType = MachOObj->getSectionType(Sec);
>> + if ((SecType & MachO::SECTION_TYPE) == MachO::S_MOD_INIT_FUNC_POINTERS) {
>> + std::string InitSymString;
>> + raw_string_ostream(InitSymString)
>> + << "$." << ObjBuffer.getBufferIdentifier() << ".__inits";
>> + InitSymbol = ES.intern(InitSymString);
>> + break;
>> + }
>> + }
>> + }
>> +
>> + return std::make_pair(std::move(SymbolFlags), std::move(InitSymbol));
>> +}
>> +
>> +} // End namespace orc.
>> +} // End namespace llvm.
>>
>> diff --git a/llvm/lib/ExecutionEngine/Orc/ObjectLinkingLayer.cpp b/llvm/lib/ExecutionEngine/Orc/ObjectLinkingLayer.cpp
>> index 6575b0a6ccb2..db88ec8517bf 100644
>> --- a/llvm/lib/ExecutionEngine/Orc/ObjectLinkingLayer.cpp
>> +++ b/llvm/lib/ExecutionEngine/Orc/ObjectLinkingLayer.cpp
>> @@ -144,6 +144,10 @@ class ObjectLinkingLayerJITLinkContext final : public JITLinkContext {
>> if (!ExtraSymbolsToClaim.empty())
>> if (auto Err = MR.defineMaterializing(ExtraSymbolsToClaim))
>> return notifyFailed(std::move(Err));
>> +
>> + if (const auto &InitSym = MR.getInitializerSymbol())
>> + InternedResult[InitSym] = JITEvaluatedSymbol();
>> +
>> if (auto Err = MR.notifyResolved(InternedResult)) {
>> Layer.getExecutionSession().reportError(std::move(Err));
>> MR.failMaterialization();
>> @@ -184,8 +188,12 @@ class ObjectLinkingLayerJITLinkContext final : public JITLinkContext {
>> }
>>
>> private:
>> - using JITLinkSymbolSet = DenseSet<const Symbol *>;
>> - using LocalToNamedDependenciesMap = DenseMap<const Symbol *, JITLinkSymbolSet>;
>> + struct LocalSymbolNamedDependencies {
>> + SymbolNameSet Internal, External;
>> + };
>> +
>> + using LocalSymbolNamedDependenciesMap =
>> + DenseMap<const Symbol *, LocalSymbolNamedDependencies>;
>>
>> Error externalizeWeakAndCommonSymbols(LinkGraph &G) {
>> auto &ES = Layer.getExecutionSession();
>> @@ -216,6 +224,7 @@ class ObjectLinkingLayerJITLinkContext final : public JITLinkContext {
>> auto &ES = MR.getTargetJITDylib().getExecutionSession();
>> auto LocalDeps = computeLocalDeps(G);
>>
>> + // Compute dependencies for symbols defined in the JITLink graph.
>> for (auto *Sym : G.defined_symbols()) {
>>
>> // Skip local symbols: we do not track dependencies for these.
>> @@ -239,15 +248,12 @@ class ObjectLinkingLayerJITLinkContext final : public JITLinkContext {
>> assert(TargetSym.isDefined() &&
>> "local symbols must be defined");
>> auto I = LocalDeps.find(&TargetSym);
>> - if (I != LocalDeps.end())
>> - for (auto &S : I->second) {
>> - assert(S->hasName() &&
>> - "LocalDeps should only contain named values");
>> - if (S->isExternal())
>> - ExternalSymDeps.insert(ES.intern(S->getName()));
>> - else if (S != Sym)
>> - InternalSymDeps.insert(ES.intern(S->getName()));
>> - }
>> + if (I != LocalDeps.end()) {
>> + for (auto &S : I->second.External)
>> + ExternalSymDeps.insert(S);
>> + for (auto &S : I->second.Internal)
>> + InternalSymDeps.insert(S);
>> + }
>> }
>> }
>>
>> @@ -261,11 +267,33 @@ class ObjectLinkingLayerJITLinkContext final : public JITLinkContext {
>> InternalNamedSymbolDeps[SymName] = std::move(InternalSymDeps);
>> }
>>
>> + for (auto &P : Layer.Plugins) {
>> + auto SyntheticLocalDeps = P->getSyntheticSymbolLocalDependencies(MR);
>> + if (SyntheticLocalDeps.empty())
>> + continue;
>> +
>> + for (auto &KV : SyntheticLocalDeps) {
>> + auto &Name = KV.first;
>> + auto &LocalDepsForName = KV.second;
>> + for (auto *Local : LocalDepsForName) {
>> + assert(Local->getScope() == Scope::Local &&
>> + "Dependence on non-local symbol");
>> + auto LocalNamedDepsItr = LocalDeps.find(Local);
>> + if (LocalNamedDepsItr == LocalDeps.end())
>> + continue;
>> + for (auto &S : LocalNamedDepsItr->second.Internal)
>> + InternalNamedSymbolDeps[Name].insert(S);
>> + for (auto &S : LocalNamedDepsItr->second.External)
>> + ExternalNamedSymbolDeps[Name].insert(S);
>> + }
>> + }
>> + }
>> +
>> return Error::success();
>> }
>>
>> - LocalToNamedDependenciesMap computeLocalDeps(LinkGraph &G) {
>> - LocalToNamedDependenciesMap DepMap;
>> + LocalSymbolNamedDependenciesMap computeLocalDeps(LinkGraph &G) {
>> + DenseMap<jitlink::Symbol *, DenseSet<jitlink::Symbol *>> DepMap;
>>
>> // For all local symbols:
>> // (1) Add their named dependencies.
>> @@ -319,7 +347,26 @@ class ObjectLinkingLayerJITLinkContext final : public JITLinkContext {
>> }
>> } while (Changed);
>>
>> - return DepMap;
>> + // Intern the results to produce a mapping of jitlink::Symbol* to internal
>> + // and external symbol names.
>> + auto &ES = Layer.getExecutionSession();
>> + LocalSymbolNamedDependenciesMap Result;
>> + for (auto &KV : DepMap) {
>> + auto *Local = KV.first;
>> + assert(Local->getScope() == Scope::Local &&
>> + "DepMap keys should all be local symbols");
>> + auto &LocalNamedDeps = Result[Local];
>> + for (auto *Named : KV.second) {
>> + assert(Named->getScope() != Scope::Local &&
>> + "DepMap values should all be non-local symbol sets");
>> + if (Named->isExternal())
>> + LocalNamedDeps.External.insert(ES.intern(Named->getName()));
>> + else
>> + LocalNamedDeps.Internal.insert(ES.intern(Named->getName()));
>> + }
>> + }
>> +
>> + return Result;
>> }
>>
>> void registerDependencies(const SymbolDependenceMap &QueryDeps) {
>>
>> diff --git a/llvm/lib/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.cpp b/llvm/lib/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.cpp
>> index ff8289a264c8..0eaf13c6e3c8 100644
>> --- a/llvm/lib/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.cpp
>> +++ b/llvm/lib/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.cpp
>> @@ -88,7 +88,8 @@ RTDyldObjectLinkingLayer::~RTDyldObjectLinkingLayer() {
>> void RTDyldObjectLinkingLayer::emit(MaterializationResponsibility R,
>> std::unique_ptr<MemoryBuffer> O) {
>> assert(O && "Object must not be null");
>> -
>> + dbgs() << "Emitting via RTDyldObjectLinkingLayer:\n"
>> + << R.getSymbols() << "\n";
>> // This method launches an asynchronous link step that will fulfill our
>> // materialization responsibility. We need to switch R to be heap
>> // allocated before that happens so it can live as long as the asynchronous
>> @@ -229,6 +230,9 @@ Error RTDyldObjectLinkingLayer::onObjLoad(
>> Symbols.erase(KV.first);
>> }
>>
>> + if (const auto &InitSym = R.getInitializerSymbol())
>> + Symbols[InitSym] = JITEvaluatedSymbol();
>> +
>> if (auto Err = R.notifyResolved(Symbols)) {
>> R.failMaterialization();
>> return Err;
>>
>> diff --git a/llvm/test/ExecutionEngine/OrcLazy/objc-minimal.ll b/llvm/test/ExecutionEngine/OrcLazy/objc-minimal.ll
>> new file mode 100644
>> index 000000000000..8a940573dcb9
>> --- /dev/null
>> +++ b/llvm/test/ExecutionEngine/OrcLazy/objc-minimal.ll
>> @@ -0,0 +1,73 @@
>> +; REQUIRES: system-darwin
>> +; RUN: lli -jit-kind=orc-lazy %s
>> +;
>> +; Sanity check MachO Platform support: Call a no-op method (returning int 0) on
>> +; an ObjC object. This test checks that we correctly auto-identify this as a
>> +; MachO target, configure MachOPlatform support, and correctly register the
>> +; class metadata and method selector with the Objective-C runtime.
>> +
>> +source_filename = "objc-minimal.mm"
>> +target datalayout = "e-m:o-i64:64-f80:128-n8:16:32:64-S128"
>> +target triple = "x86_64-apple-macosx10.14.0"
>> +
>> +%0 = type opaque
>> +%struct._objc_cache = type opaque
>> +%struct._class_t = type { %struct._class_t*, %struct._class_t*, %struct._objc_cache*, i8* (i8*, i8*)**, %struct._class_ro_t* }
>> +%struct._class_ro_t = type { i32, i32, i32, i8*, i8*, %struct.__method_list_t*, %struct._objc_protocol_list*, %struct._ivar_list_t*, i8*, %struct._prop_list_t* }
>> +%struct.__method_list_t = type { i32, i32, [0 x %struct._objc_method] }
>> +%struct._objc_method = type { i8*, i8*, i8* }
>> +%struct._objc_protocol_list = type { i64, [0 x %struct._protocol_t*] }
>> +%struct._protocol_t = type { i8*, i8*, %struct._objc_protocol_list*, %struct.__method_list_t*, %struct.__method_list_t*, %struct.__method_list_t*, %struct.__method_list_t*, %struct._prop_list_t*, i32, i32, i8**, i8*, %struct._prop_list_t* }
>> +%struct._ivar_list_t = type { i32, i32, [0 x %struct._ivar_t] }
>> +%struct._ivar_t = type { i64*, i8*, i8*, i32, i32 }
>> +%struct._prop_list_t = type { i32, i32, [0 x %struct._prop_t] }
>> +%struct._prop_t = type { i8*, i8* }
>> +
>> + at _objc_empty_cache = external global %struct._objc_cache
>> +@"OBJC_METACLASS_$_NSObject" = external global %struct._class_t
>> + at OBJC_CLASS_NAME_ = private unnamed_addr constant [4 x i8] c"Foo\00", section "__TEXT,__objc_classname,cstring_literals", align 1
>> +@"\01l_OBJC_METACLASS_RO_$_Foo" = private global %struct._class_ro_t { i32 1, i32 40, i32 40, i8* null, i8* getelementptr inbounds ([4 x i8], [4 x i8]* @OBJC_CLASS_NAME_, i32 0, i32 0), %struct.__method_list_t* null, %struct._objc_protocol_list* null, %struct._ivar_list_t* null, i8* null, %struct._prop_list_t* null }, section "__DATA, __objc_const", align 8
>> +@"OBJC_METACLASS_$_Foo" = global %struct._class_t { %struct._class_t* @"OBJC_METACLASS_$_NSObject", %struct._class_t* @"OBJC_METACLASS_$_NSObject", %struct._objc_cache* @_objc_empty_cache, i8* (i8*, i8*)** null, %struct._class_ro_t* @"\01l_OBJC_METACLASS_RO_$_Foo" }, section "__DATA, __objc_data", align 8
>> +@"OBJC_CLASS_$_NSObject" = external global %struct._class_t
>> + at OBJC_METH_VAR_NAME_ = private unnamed_addr constant [4 x i8] c"foo\00", section "__TEXT,__objc_methname,cstring_literals", align 1
>> + at OBJC_METH_VAR_TYPE_ = private unnamed_addr constant [8 x i8] c"i16 at 0:8\00", section "__TEXT,__objc_methtype,cstring_literals", align 1
>> +@"\01l_OBJC_$_INSTANCE_METHODS_Foo" = private global { i32, i32, [1 x %struct._objc_method] } { i32 24, i32 1, [1 x %struct._objc_method] [%struct._objc_method { i8* getelementptr inbounds ([4 x i8], [4 x i8]* @OBJC_METH_VAR_NAME_, i32 0, i32 0), i8* getelementptr inbounds ([8 x i8], [8 x i8]* @OBJC_METH_VAR_TYPE_, i32 0, i32 0), i8* bitcast (i32 (%0*, i8*)* @"\01-[Foo foo]" to i8*) }] }, section "__DATA, __objc_const", align 8
>> +@"\01l_OBJC_CLASS_RO_$_Foo" = private global %struct._class_ro_t { i32 0, i32 8, i32 8, i8* null, i8* getelementptr inbounds ([4 x i8], [4 x i8]* @OBJC_CLASS_NAME_, i32 0, i32 0), %struct.__method_list_t* bitcast ({ i32, i32, [1 x %struct._objc_method] }* @"\01l_OBJC_$_INSTANCE_METHODS_Foo" to %struct.__method_list_t*), %struct._objc_protocol_list* null, %struct._ivar_list_t* null, i8* null, %struct._prop_list_t* null }, section "__DATA, __objc_const", align 8
>> +@"OBJC_CLASS_$_Foo" = global %struct._class_t { %struct._class_t* @"OBJC_METACLASS_$_Foo", %struct._class_t* @"OBJC_CLASS_$_NSObject", %struct._objc_cache* @_objc_empty_cache, i8* (i8*, i8*)** null, %struct._class_ro_t* @"\01l_OBJC_CLASS_RO_$_Foo" }, section "__DATA, __objc_data", align 8
>> +@"OBJC_CLASSLIST_REFERENCES_$_" = private global %struct._class_t* @"OBJC_CLASS_$_Foo", section "__DATA,__objc_classrefs,regular,no_dead_strip", align 8
>> + at OBJC_METH_VAR_NAME_.1 = private unnamed_addr constant [5 x i8] c"init\00", section "__TEXT,__objc_methname,cstring_literals", align 1
>> + at OBJC_SELECTOR_REFERENCES_ = private externally_initialized global i8* getelementptr inbounds ([5 x i8], [5 x i8]* @OBJC_METH_VAR_NAME_.1, i64 0, i64 0), section "__DATA,__objc_selrefs,literal_pointers,no_dead_strip", align 8
>> + at OBJC_SELECTOR_REFERENCES_.2 = private externally_initialized global i8* getelementptr inbounds ([4 x i8], [4 x i8]* @OBJC_METH_VAR_NAME_, i64 0, i64 0), section "__DATA,__objc_selrefs,literal_pointers,no_dead_strip", align 8
>> +@"OBJC_LABEL_CLASS_$" = private global [1 x i8*] [i8* bitcast (%struct._class_t* @"OBJC_CLASS_$_Foo" to i8*)], section "__DATA,__objc_classlist,regular,no_dead_strip", align 8
>> + at llvm.compiler.used = appending global [9 x i8*] [i8* bitcast ({ i32, i32, [1 x %struct._objc_method] }* @"\01l_OBJC_$_INSTANCE_METHODS_Foo" to i8*), i8* bitcast (%struct._class_t** @"OBJC_CLASSLIST_REFERENCES_$_" to i8*), i8* getelementptr inbounds ([4 x i8], [4 x i8]* @OBJC_CLASS_NAME_, i32 0, i32 0), i8* bitcast ([1 x i8*]* @"OBJC_LABEL_CLASS_$" to i8*), i8* getelementptr inbounds ([4 x i8], [4 x i8]* @OBJC_METH_VAR_NAME_, i32 0, i32 0), i8* getelementptr inbounds ([5 x i8], [5 x i8]* @OBJC_METH_VAR_NAME_.1, i32 0, i32 0), i8* getelementptr inbounds ([8 x i8], [8 x i8]* @OBJC_METH_VAR_TYPE_, i32 0, i32 0), i8* bitcast (i8** @OBJC_SELECTOR_REFERENCES_ to i8*), i8* bitcast (i8** @OBJC_SELECTOR_REFERENCES_.2 to i8*)], section "llvm.metadata"
>> +
>> +; Function Attrs: noinline norecurse nounwind readnone ssp uwtable
>> +define internal i32 @"\01-[Foo foo]"(%0* nocapture readnone, i8* nocapture readnone) {
>> + ret i32 0
>> +}
>> +
>> +; Function Attrs: noinline norecurse ssp uwtable
>> +define i32 @main(i32, i8** nocapture readnone) local_unnamed_addr {
>> + %3 = load i8*, i8** bitcast (%struct._class_t** @"OBJC_CLASSLIST_REFERENCES_$_" to i8**), align 8
>> + %4 = tail call i8* @objc_alloc(i8* %3)
>> + %5 = load i8*, i8** @OBJC_SELECTOR_REFERENCES_, align 8
>> + %6 = tail call i8* bitcast (i8* (i8*, i8*, ...)* @objc_msgSend to i8* (i8*, i8*)*)(i8* %4, i8* %5)
>> + %7 = load i8*, i8** @OBJC_SELECTOR_REFERENCES_.2, align 8
>> + %8 = tail call i32 bitcast (i8* (i8*, i8*, ...)* @objc_msgSend to i32 (i8*, i8*)*)(i8* %6, i8* %7)
>> + ret i32 %8
>> +}
>> +
>> +declare i8* @objc_alloc(i8*) local_unnamed_addr
>> +
>> +; Function Attrs: nonlazybind
>> +declare i8* @objc_msgSend(i8*, i8*, ...) local_unnamed_addr
>> +
>> +!llvm.module.flags = !{!0, !1, !2, !3, !4, !5, !6 }
>> +
>> +!0 = !{i32 2, !"SDK Version", [2 x i32] [i32 10, i32 15]}
>> +!1 = !{i32 1, !"Objective-C Version", i32 2}
>> +!2 = !{i32 1, !"Objective-C Image Info Version", i32 0}
>> +!3 = !{i32 1, !"Objective-C Image Info Section", !"__DATA,__objc_imageinfo,regular,no_dead_strip"}
>> +!4 = !{i32 4, !"Objective-C Garbage Collection", i32 0}
>> +!5 = !{i32 1, !"Objective-C Class Properties", i32 64}
>> +!6 = !{i32 1, !"wchar_size", i32 4}
>>
>> diff --git a/llvm/tools/lli/lli.cpp b/llvm/tools/lli/lli.cpp
>> index e7251b12f7fc..55b9557bddfe 100644
>> --- a/llvm/tools/lli/lli.cpp
>> +++ b/llvm/tools/lli/lli.cpp
>> @@ -24,9 +24,11 @@
>> #include "llvm/ExecutionEngine/JITEventListener.h"
>> #include "llvm/ExecutionEngine/MCJIT.h"
>> #include "llvm/ExecutionEngine/ObjectCache.h"
>> +#include "llvm/ExecutionEngine/Orc/DebugUtils.h"
>> #include "llvm/ExecutionEngine/Orc/ExecutionUtils.h"
>> #include "llvm/ExecutionEngine/Orc/JITTargetMachineBuilder.h"
>> #include "llvm/ExecutionEngine/Orc/LLJIT.h"
>> +#include "llvm/ExecutionEngine/Orc/MachOPlatform.h"
>> #include "llvm/ExecutionEngine/Orc/OrcRemoteTargetClient.h"
>> #include "llvm/ExecutionEngine/OrcMCJITReplacement.h"
>> #include "llvm/ExecutionEngine/SectionMemoryManager.h"
>> @@ -206,6 +208,19 @@ namespace {
>> cl::desc("Do not resolve lli process symbols in JIT'd code"),
>> cl::init(false));
>>
>> + enum class LLJITPlatform { DetectHost, GenericIR, MachO };
>> +
>> + cl::opt<LLJITPlatform>
>> + Platform("lljit-platform", cl::desc("Platform to use with LLJIT"),
>> + cl::init(LLJITPlatform::DetectHost),
>> + cl::values(clEnumValN(LLJITPlatform::DetectHost, "DetectHost",
>> + "Select based on JIT target triple"),
>> + clEnumValN(LLJITPlatform::GenericIR, "GenericIR",
>> + "Use LLJITGenericIRPlatform"),
>> + clEnumValN(LLJITPlatform::MachO, "MachO",
>> + "Use LLJITMachOPlatform")),
>> + cl::Hidden);
>> +
>> enum class DumpKind {
>> NoDump,
>> DumpFuncsToStdOut,
>> @@ -772,16 +787,20 @@ int runOrcLazyJIT(const char *ProgName) {
>> if (!MainModule)
>> reportError(Err, ProgName);
>>
>> - const auto &TT = MainModule->getTargetTriple();
>> + Triple TT(MainModule->getTargetTriple());
>> orc::LLLazyJITBuilder Builder;
>>
>> Builder.setJITTargetMachineBuilder(
>> - TT.empty() ? ExitOnErr(orc::JITTargetMachineBuilder::detectHost())
>> - : orc::JITTargetMachineBuilder(Triple(TT)));
>> + MainModule->getTargetTriple().empty()
>> + ? ExitOnErr(orc::JITTargetMachineBuilder::detectHost())
>> + : orc::JITTargetMachineBuilder(TT));
>>
>> if (!MArch.empty())
>> Builder.getJITTargetMachineBuilder()->getTargetTriple().setArchName(MArch);
>>
>> + if (!MainModule->getDataLayout().isDefault())
>> + Builder.setDataLayout(MainModule->getDataLayout());
>> +
>> Builder.getJITTargetMachineBuilder()
>> ->setCPU(getCPUStr())
>> .addFeatures(getFeatureList())
>> @@ -796,6 +815,29 @@ int runOrcLazyJIT(const char *ProgName) {
>> pointerToJITTargetAddress(exitOnLazyCallThroughFailure));
>> Builder.setNumCompileThreads(LazyJITCompileThreads);
>>
>> + // Set up LLJIT platform.
>> + {
>> + LLJITPlatform P = Platform;
>> + if (P == LLJITPlatform::DetectHost) {
>> + if (TT.isOSBinFormatMachO())
>> + P = LLJITPlatform::MachO;
>> + else
>> + P = LLJITPlatform::GenericIR;
>> + }
>> +
>> + switch (P) {
>> + case LLJITPlatform::GenericIR:
>> + // Nothing to do: LLJITBuilder will use this by default.
>> + break;
>> + case LLJITPlatform::MachO:
>> + Builder.setPlatformSetUp(orc::setUpMachOPlatform);
>> + ExitOnErr(orc::enableObjCRegistration("libobjc.dylib"));
>> + break;
>> + default:
>> + llvm_unreachable("Unrecognized platform value");
>> + }
>> + }
>> +
>> auto J = ExitOnErr(Builder.create());
>>
>> if (PerModuleLazy)
>> @@ -828,9 +870,6 @@ int runOrcLazyJIT(const char *ProgName) {
>> return Name != MainName;
>> })));
>>
>> - orc::LocalCXXRuntimeOverrides CXXRuntimeOverrides;
>> - ExitOnErr(CXXRuntimeOverrides.enable(J->getMainJITDylib(), Mangle));
>> -
>> // Add the main module.
>> ExitOnErr(
>> J->addLazyIRModule(orc::ThreadSafeModule(std::move(MainModule), TSCtx)));
>> @@ -845,8 +884,11 @@ int runOrcLazyJIT(const char *ProgName) {
>> for (auto JDItr = JITDylibs.begin(), JDEnd = JITDylibs.end();
>> JDItr != JDEnd; ++JDItr) {
>> orc::JITDylib *JD = J->getJITDylibByName(*JDItr);
>> - if (!JD)
>> - JD = &J->createJITDylib(*JDItr);
>> + if (!JD) {
>> + JD = &ExitOnErr(J->createJITDylib(*JDItr));
>> + J->getMainJITDylib().addToSearchOrder(*JD);
>> + JD->addToSearchOrder(J->getMainJITDylib());
>> + }
>> IdxToDylib[JITDylibs.getPosition(JDItr - JITDylibs.begin())] = JD;
>> }
>>
>> @@ -882,7 +924,7 @@ int runOrcLazyJIT(const char *ProgName) {
>> }
>>
>> // Run any static constructors.
>> - ExitOnErr(J->runConstructors());
>> + ExitOnErr(J->initialize(J->getMainJITDylib()));
>>
>> // Run any -thread-entry points.
>> std::vector<std::thread> AltEntryThreads;
>> @@ -907,8 +949,7 @@ int runOrcLazyJIT(const char *ProgName) {
>> AltEntryThread.join();
>>
>> // Run destructors.
>> - ExitOnErr(J->runDestructors());
>> - CXXRuntimeOverrides.runDestructors();
>> + ExitOnErr(J->deinitialize(J->getMainJITDylib()));
>>
>> return Result;
>> }
>>
>> diff --git a/llvm/tools/llvm-jitlink/llvm-jitlink.cpp b/llvm/tools/llvm-jitlink/llvm-jitlink.cpp
>> index 26bcf46d60a6..7765d53f402d 100644
>> --- a/llvm/tools/llvm-jitlink/llvm-jitlink.cpp
>> +++ b/llvm/tools/llvm-jitlink/llvm-jitlink.cpp
>> @@ -396,9 +396,18 @@ static std::unique_ptr<jitlink::JITLinkMemoryManager> createMemoryManager() {
>> return std::make_unique<jitlink::InProcessMemoryManager>();
>> }
>>
>> -Session::Session(Triple TT)
>> - : MainJD(ES.createJITDylib("<main>")), ObjLayer(ES, createMemoryManager()),
>> - TT(std::move(TT)) {
>> +Expected<std::unique_ptr<Session>> Session::Create(Triple TT) {
>> + Error Err = Error::success();
>> + std::unique_ptr<Session> S(new Session(std::move(TT), Err));
>> + if (Err)
>> + return std::move(Err);
>> + return std::move(S);
>> +}
>> +
>> +// FIXME: Move to createJITDylib if/when we start using Platform support in
>> +// llvm-jitlink.
>> +Session::Session(Triple TT, Error &Err)
>> + : ObjLayer(ES, createMemoryManager()), TT(std::move(TT)) {
>>
>> /// Local ObjectLinkingLayer::Plugin class to forward modifyPassConfig to the
>> /// Session.
>> @@ -414,6 +423,15 @@ Session::Session(Triple TT)
>> Session &S;
>> };
>>
>> + ErrorAsOutParameter _(&Err);
>> +
>> + if (auto MainJDOrErr = ES.createJITDylib("main"))
>> + MainJD = &*MainJDOrErr;
>> + else {
>> + Err = MainJDOrErr.takeError();
>> + return;
>> + }
>> +
>> if (!NoExec && !TT.isOSWindows())
>> ObjLayer.addPlugin(std::make_unique<EHFrameRegistrationPlugin>(
>> InProcessEHFrameRegistrar::getInstance()));
>> @@ -561,7 +579,7 @@ Error loadProcessSymbols(Session &S) {
>> auto FilterMainEntryPoint = [InternedEntryPointName](SymbolStringPtr Name) {
>> return Name != InternedEntryPointName;
>> };
>> - S.MainJD.addGenerator(
>> + S.MainJD->addGenerator(
>> ExitOnErr(orc::DynamicLibrarySearchGenerator::GetForCurrentProcess(
>> GlobalPrefix, FilterMainEntryPoint)));
>>
>> @@ -590,20 +608,22 @@ Error loadObjects(Session &S) {
>> LLVM_DEBUG(dbgs() << "Creating JITDylibs...\n");
>> {
>> // Create a "main" JITLinkDylib.
>> - IdxToJLD[0] = &S.MainJD;
>> - S.JDSearchOrder.push_back(&S.MainJD);
>> - LLVM_DEBUG(dbgs() << " 0: " << S.MainJD.getName() << "\n");
>> + IdxToJLD[0] = S.MainJD;
>> + S.JDSearchOrder.push_back(S.MainJD);
>> + LLVM_DEBUG(dbgs() << " 0: " << S.MainJD->getName() << "\n");
>>
>> // Add any extra JITLinkDylibs from the command line.
>> std::string JDNamePrefix("lib");
>> for (auto JLDItr = JITLinkDylibs.begin(), JLDEnd = JITLinkDylibs.end();
>> JLDItr != JLDEnd; ++JLDItr) {
>> - auto &JD = S.ES.createJITDylib(JDNamePrefix + *JLDItr);
>> + auto JD = S.ES.createJITDylib(JDNamePrefix + *JLDItr);
>> + if (!JD)
>> + return JD.takeError();
>> unsigned JDIdx =
>> JITLinkDylibs.getPosition(JLDItr - JITLinkDylibs.begin());
>> - IdxToJLD[JDIdx] = &JD;
>> - S.JDSearchOrder.push_back(&JD);
>> - LLVM_DEBUG(dbgs() << " " << JDIdx << ": " << JD.getName() << "\n");
>> + IdxToJLD[JDIdx] = &*JD;
>> + S.JDSearchOrder.push_back(&*JD);
>> + LLVM_DEBUG(dbgs() << " " << JDIdx << ": " << JD->getName() << "\n");
>> }
>>
>> // Set every dylib to link against every other, in command line order.
>> @@ -790,32 +810,32 @@ int main(int argc, char *argv[]) {
>> std::unique_ptr<JITLinkTimers> Timers =
>> ShowTimes ? std::make_unique<JITLinkTimers>() : nullptr;
>>
>> - Session S(getFirstFileTriple());
>> + auto S = ExitOnErr(Session::Create(getFirstFileTriple()));
>>
>> - ExitOnErr(sanitizeArguments(S));
>> + ExitOnErr(sanitizeArguments(*S));
>>
>> if (!NoProcessSymbols)
>> - ExitOnErr(loadProcessSymbols(S));
>> + ExitOnErr(loadProcessSymbols(*S));
>> ExitOnErr(loadDylibs());
>>
>>
>> {
>> TimeRegion TR(Timers ? &Timers->LoadObjectsTimer : nullptr);
>> - ExitOnErr(loadObjects(S));
>> + ExitOnErr(loadObjects(*S));
>> }
>>
>> JITEvaluatedSymbol EntryPoint = 0;
>> {
>> TimeRegion TR(Timers ? &Timers->LinkTimer : nullptr);
>> - EntryPoint = ExitOnErr(getMainEntryPoint(S));
>> + EntryPoint = ExitOnErr(getMainEntryPoint(*S));
>> }
>>
>> if (ShowAddrs)
>> - S.dumpSessionInfo(outs());
>> + S->dumpSessionInfo(outs());
>>
>> - ExitOnErr(runChecks(S));
>> + ExitOnErr(runChecks(*S));
>>
>> - dumpSessionStats(S);
>> + dumpSessionStats(*S);
>>
>> if (NoExec)
>> return 0;
>>
>> diff --git a/llvm/tools/llvm-jitlink/llvm-jitlink.h b/llvm/tools/llvm-jitlink/llvm-jitlink.h
>> index 0f92d7605012..c888baec9adf 100644
>> --- a/llvm/tools/llvm-jitlink/llvm-jitlink.h
>> +++ b/llvm/tools/llvm-jitlink/llvm-jitlink.h
>> @@ -26,12 +26,13 @@ namespace llvm {
>>
>> struct Session {
>> orc::ExecutionSession ES;
>> - orc::JITDylib &MainJD;
>> + orc::JITDylib *MainJD;
>> orc::ObjectLinkingLayer ObjLayer;
>> std::vector<orc::JITDylib *> JDSearchOrder;
>> Triple TT;
>>
>> Session(Triple TT);
>> + static Expected<std::unique_ptr<Session>> Create(Triple TT);
>> void dumpSessionInfo(raw_ostream &OS);
>> void modifyPassConfig(const Triple &FTT,
>> jitlink::PassConfiguration &PassConfig);
>> @@ -63,6 +64,9 @@ struct Session {
>> FileInfoMap FileInfos;
>> uint64_t SizeBeforePruning = 0;
>> uint64_t SizeAfterFixups = 0;
>> +
>> +private:
>> + Session(Triple TT, Error &Err);
>> };
>>
>> Error registerMachOStubsAndGOT(Session &S, jitlink::LinkGraph &G);
>>
>> diff --git a/llvm/unittests/ExecutionEngine/Orc/CoreAPIsTest.cpp b/llvm/unittests/ExecutionEngine/Orc/CoreAPIsTest.cpp
>> index 68a6d2ed2ca2..171ff37f0069 100644
>> --- a/llvm/unittests/ExecutionEngine/Orc/CoreAPIsTest.cpp
>> +++ b/llvm/unittests/ExecutionEngine/Orc/CoreAPIsTest.cpp
>> @@ -113,6 +113,7 @@ TEST_F(CoreAPIsStandardTest, RemoveSymbolsTest) {
>> cantFail(R.notifyResolved({{Bar, BarSym}}));
>> cantFail(R.notifyEmitted());
>> },
>> + nullptr,
>> [&](const JITDylib &JD, const SymbolStringPtr &Name) {
>> EXPECT_EQ(Name, Bar) << "Expected \"Bar\" to be discarded";
>> if (Name == Bar)
>> @@ -126,6 +127,7 @@ TEST_F(CoreAPIsStandardTest, RemoveSymbolsTest) {
>> cantFail(JD.define(std::make_unique<SimpleMaterializationUnit>(
>> SymbolFlagsMap({{Baz, BazSym.getFlags()}}),
>> [&](MaterializationResponsibility R) { BazR.emplace(std::move(R)); },
>> + nullptr,
>> [](const JITDylib &JD, const SymbolStringPtr &Name) {
>> ADD_FAILURE() << "\"Baz\" discarded unexpectedly";
>> })));
>> @@ -176,7 +178,7 @@ TEST_F(CoreAPIsStandardTest, RemoveSymbolsTest) {
>> TEST_F(CoreAPIsStandardTest, ChainedJITDylibLookup) {
>> cantFail(JD.define(absoluteSymbols({{Foo, FooSym}})));
>>
>> - auto &JD2 = ES.createJITDylib("JD2");
>> + auto &JD2 = ES.createBareJITDylib("JD2");
>>
>> bool OnCompletionRun = false;
>>
>> @@ -198,7 +200,7 @@ TEST_F(CoreAPIsStandardTest, LookupWithHiddenSymbols) {
>>
>> cantFail(JD.define(absoluteSymbols({{Foo, FooSym}, {Bar, BarHiddenSym}})));
>>
>> - auto &JD2 = ES.createJITDylib("JD2");
>> + auto &JD2 = ES.createBareJITDylib("JD2");
>> cantFail(JD2.define(absoluteSymbols({{Bar, QuxSym}})));
>>
>> /// Try a blocking lookup.
>> @@ -307,7 +309,7 @@ TEST_F(CoreAPIsStandardTest, TestBasicReExports) {
>> // JITDylib works.
>> cantFail(JD.define(absoluteSymbols({{Foo, FooSym}})));
>>
>> - auto &JD2 = ES.createJITDylib("JD2");
>> + auto &JD2 = ES.createBareJITDylib("JD2");
>>
>> cantFail(JD2.define(reexports(JD, {{Bar, {Foo, BarSym.getFlags()}}})));
>>
>> @@ -332,7 +334,7 @@ TEST_F(CoreAPIsStandardTest, TestThatReExportsDontUnnecessarilyMaterialize) {
>>
>> cantFail(JD.define(BarMU));
>>
>> - auto &JD2 = ES.createJITDylib("JD2");
>> + auto &JD2 = ES.createBareJITDylib("JD2");
>>
>> cantFail(JD2.define(reexports(
>> JD, {{Baz, {Foo, BazSym.getFlags()}}, {Qux, {Bar, QuxSym.getFlags()}}})));
>> @@ -347,7 +349,7 @@ TEST_F(CoreAPIsStandardTest, TestThatReExportsDontUnnecessarilyMaterialize) {
>> TEST_F(CoreAPIsStandardTest, TestReexportsGenerator) {
>> // Test that a re-exports generator can dynamically generate reexports.
>>
>> - auto &JD2 = ES.createJITDylib("JD2");
>> + auto &JD2 = ES.createBareJITDylib("JD2");
>> cantFail(JD2.define(absoluteSymbols({{Foo, FooSym}, {Bar, BarSym}})));
>>
>> auto Filter = [this](SymbolStringPtr Name) { return Name != Bar; };
>> @@ -838,6 +840,7 @@ TEST_F(CoreAPIsStandardTest, DropMaterializerWhenEmpty) {
>> [](MaterializationResponsibility R) {
>> llvm_unreachable("Unexpected call to materialize");
>> },
>> + nullptr,
>> [&](const JITDylib &JD, SymbolStringPtr Name) {
>> EXPECT_TRUE(Name == Foo || Name == Bar)
>> << "Discard of unexpected symbol?";
>> @@ -872,6 +875,7 @@ TEST_F(CoreAPIsStandardTest, AddAndMaterializeLazySymbol) {
>> cantFail(R.notifyEmitted());
>> FooMaterialized = true;
>> },
>> + nullptr,
>> [&](const JITDylib &JD, SymbolStringPtr Name) {
>> EXPECT_EQ(Name, Bar) << "Expected Name to be Bar";
>> BarDiscarded = true;
>> @@ -920,6 +924,7 @@ TEST_F(CoreAPIsStandardTest, TestBasicWeakSymbolMaterialization) {
>> ADD_FAILURE() << "Attempt to materialize Bar from the wrong unit";
>> R.failMaterialization();
>> },
>> + nullptr,
>> [&](const JITDylib &JD, SymbolStringPtr Name) {
>> EXPECT_EQ(Name, Bar) << "Expected \"Bar\" to be discarded";
>> DuplicateBarDiscarded = true;
>>
>> diff --git a/llvm/unittests/ExecutionEngine/Orc/OrcTestCommon.h b/llvm/unittests/ExecutionEngine/Orc/OrcTestCommon.h
>> index d82012b5a853..b25851d8f796 100644
>> --- a/llvm/unittests/ExecutionEngine/Orc/OrcTestCommon.h
>> +++ b/llvm/unittests/ExecutionEngine/Orc/OrcTestCommon.h
>> @@ -47,7 +47,7 @@ class CoreAPIsBasedStandardTest : public testing::Test {
>> protected:
>> std::shared_ptr<SymbolStringPool> SSP = std::make_shared<SymbolStringPool>();
>> ExecutionSession ES{SSP};
>> - JITDylib &JD = ES.createJITDylib("JD");
>> + JITDylib &JD = ES.createBareJITDylib("JD");
>> SymbolStringPtr Foo = ES.intern("foo");
>> SymbolStringPtr Bar = ES.intern("bar");
>> SymbolStringPtr Baz = ES.intern("baz");
>> @@ -93,9 +93,11 @@ class SimpleMaterializationUnit : public orc::MaterializationUnit {
>>
>> SimpleMaterializationUnit(
>> orc::SymbolFlagsMap SymbolFlags, MaterializeFunction Materialize,
>> + orc::SymbolStringPtr InitSym = nullptr,
>> DiscardFunction Discard = DiscardFunction(),
>> DestructorFunction Destructor = DestructorFunction())
>> - : MaterializationUnit(std::move(SymbolFlags), orc::VModuleKey()),
>> + : MaterializationUnit(std::move(SymbolFlags), std::move(InitSym),
>> + orc::VModuleKey()),
>> Materialize(std::move(Materialize)), Discard(std::move(Discard)),
>> Destructor(std::move(Destructor)) {}
>>
>>
>> diff --git a/llvm/unittests/ExecutionEngine/Orc/RTDyldObjectLinkingLayerTest.cpp b/llvm/unittests/ExecutionEngine/Orc/RTDyldObjectLinkingLayerTest.cpp
>> index 0c66841e9af0..9a50571b1969 100644
>> --- a/llvm/unittests/ExecutionEngine/Orc/RTDyldObjectLinkingLayerTest.cpp
>> +++ b/llvm/unittests/ExecutionEngine/Orc/RTDyldObjectLinkingLayerTest.cpp
>> @@ -50,7 +50,7 @@ static bool testSetProcessAllSections(std::unique_ptr<MemoryBuffer> Obj,
>> bool DebugSectionSeen = false;
>>
>> ExecutionSession ES;
>> - auto &JD = ES.createJITDylib("main");
>> + auto &JD = ES.createBareJITDylib("main");
>> auto Foo = ES.intern("foo");
>>
>> RTDyldObjectLinkingLayer ObjLayer(ES, [&DebugSectionSeen]() {
>> @@ -151,7 +151,7 @@ TEST(RTDyldObjectLinkingLayerTest, TestOverrideObjectFlags) {
>>
>> // Create a simple stack and set the override flags option.
>> ExecutionSession ES;
>> - auto &JD = ES.createJITDylib("main");
>> + auto &JD = ES.createBareJITDylib("main");
>> auto Foo = ES.intern("foo");
>> RTDyldObjectLinkingLayer ObjLayer(
>> ES, []() { return std::make_unique<SectionMemoryManager>(); });
>> @@ -218,7 +218,7 @@ TEST(RTDyldObjectLinkingLayerTest, TestAutoClaimResponsibilityForSymbols) {
>>
>> // Create a simple stack and set the override flags option.
>> ExecutionSession ES;
>> - auto &JD = ES.createJITDylib("main");
>> + auto &JD = ES.createBareJITDylib("main");
>> auto Foo = ES.intern("foo");
>> RTDyldObjectLinkingLayer ObjLayer(
>> ES, []() { return std::make_unique<SectionMemoryManager>(); });
>>
>>
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20200224/6cdece27/attachment-0001.html>
More information about the llvm-commits
mailing list