[llvm] 85fb997 - [ORC] Add generic initializer/deinitializer support.
Erik Pilkington via llvm-commits
llvm-commits at lists.llvm.org
Wed Feb 26 14:57:14 PST 2020
I can’t seem to easily reproduce it either, I committed your suggested workaround in 30f4362040f, I’ll keep an eye on the bot to verify that it fixes the issue.
Thanks,
Erik
> On Feb 25, 2020, at 9:15 AM, Lang Hames <lhames at gmail.com> wrote:
>
> Hi Eric,
>
> I think this leak is expected (i.e. that cache might not be cleared until process termination), but I will need to talk to the ObjC runtime folks to confirm.
>
> Any chance you could test a workaround for me in the mean time? I think if we replace the current run line,
>
> RUN: lli -jit-kind=orc-lazy %s
>
> with
>
> RUN: ASAN_OPTIONS=detect_leaks=0 lli -jit-kind=orc-lazy %s
>
> That should allow us to at least keep running asan on the test. If that doesn’t work we can just disable the test entirely when asan is enabled by adding
>
> ; UNSUPPORTED: asan
>
> at the top.
>
> I’d test these myself but my asan build doesn’t seem to have leak checking enabled, so I wasn’t able to reproduce this failure locally.
>
> Regards,
> Lang.
>
>> On Feb 24, 2020, at 1:44 PM, Lang Hames <lhames at gmail.com> wrote:
>>
>> Hi Eric,
>>
>> Thanks for letting me know! I’ll take a look and see if I can figure this out.
>>
>> Regards,
>> Lang.
>>
>> Sent from my iPad
>>
>>> On Feb 24, 2020, at 10:29 AM, Erik Pilkington <erik.pilkington at gmail.com> wrote:
>>>
>>> Hi Lang,
>>>
>>> It looks like a test you added in this commit is failing on macOS: http://lab.llvm.org:8080/green/job/clang-stage2-cmake-RgSan/6977 <http://lab.llvm.org:8080/green/job/clang-stage2-cmake-RgSan/6977>
>>>
>>> Can you please take a look?
>>>
>>> Thanks!
>>> Erik
>>>
>>>
>>> ******************** TEST 'LLVM :: ExecutionEngine/OrcLazy/objc-minimal.ll' FAILED ********************
>>> Script:
>>> --
>>> : 'RUN: at line 2'; /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/lli -jit-kind=orc-lazy /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm-project/llvm/test/ExecutionEngine/OrcLazy/objc-minimal.ll
>>> --
>>> Exit Code: 1
>>>
>>> Command Output (stderr):
>>> --
>>>
>>> =================================================================
>>> ==96779==ERROR: LeakSanitizer: detected memory leaks
>>>
>>> Direct leak of 80 byte(s) in 1 object(s) allocated from:
>>> #0 0x1127fbef2 in wrap_calloc+0xa2 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x45ef2)
>>> #1 0x7fff5addac5d in allocateBuckets(unsigned int)+0x1d (libobjc.A.dylib:x86_64h+0x9c5d)
>>> #2 0x7fff5addabec in cache_t::reallocate(unsigned int, unsigned int)+0x2a (libobjc.A.dylib:x86_64h+0x9bec)
>>> #3 0x7fff5adda3a8 in cache_fill+0xc2 (libobjc.A.dylib:x86_64h+0x93a8)
>>> #4 0x7fff5add9b94 in lookUpImpOrForward+0x269 (libobjc.A.dylib:x86_64h+0x8b94)
>>> #5 0x7fff5add9493 in _objc_msgSend_uncached+0x43 (libobjc.A.dylib:x86_64h+0x8493)
>>> #6 0x1166f702a (<unknown module>)
>>> #7 0x10a89cd22 in runOrcLazyJIT(char const*) lli.cpp:991
>>> #8 0x10a890b79 in main lli.cpp:409
>>> #9 0x7fff5beb5084 in start+0x0 (libdyld.dylib:x86_64+0x17084)
>>>
>>> Direct leak of 80 byte(s) in 1 object(s) allocated from:
>>> #0 0x1127fbef2 in wrap_calloc+0xa2 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x45ef2)
>>> #1 0x7fff5addac5d in allocateBuckets(unsigned int)+0x1d (libobjc.A.dylib:x86_64h+0x9c5d)
>>> #2 0x7fff5addabec in cache_t::reallocate(unsigned int, unsigned int)+0x2a (libobjc.A.dylib:x86_64h+0x9bec)
>>> #3 0x7fff5adda3a8 in cache_fill+0xc2 (libobjc.A.dylib:x86_64h+0x93a8)
>>> #4 0x7fff5add9b94 in lookUpImpOrForward+0x269 (libobjc.A.dylib:x86_64h+0x8b94)
>>> #5 0x7fff5add9493 in _objc_msgSend_uncached+0x43 (libobjc.A.dylib:x86_64h+0x8493)
>>> #6 0x1166f701b (<unknown module>)
>>> #7 0x10a89cd22 in runOrcLazyJIT(char const*) lli.cpp:991
>>> #8 0x10a890b79 in main lli.cpp:409
>>> #9 0x7fff5beb5084 in start+0x0 (libdyld.dylib:x86_64+0x17084)
>>>
>>> Direct leak of 64 byte(s) in 1 object(s) allocated from:
>>> #0 0x1127fbef2 in wrap_calloc+0xa2 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x45ef2)
>>> #1 0x7fff5add6510 in realizeClass(objc_class*)+0x4c (libobjc.A.dylib:x86_64h+0x5510)
>>> #2 0x7fff5add65e2 in realizeClass(objc_class*)+0x11e (libobjc.A.dylib:x86_64h+0x55e2)
>>> #3 0x7fff5adee3f5 in objc_readClassPair+0x76 (libobjc.A.dylib:x86_64h+0x1d3f5)
>>> #4 0x10be4620c in llvm::orc::MachOJITDylibInitializers::registerObjCClasses() const MachOPlatform.cpp:133
>>> #5 0x10be246a7 in (anonymous namespace)::MachOPlatformSupport::initialize(llvm::orc::JITDylib&) LLJIT.cpp:525
>>> #6 0x10a8a73f7 in llvm::orc::LLJIT::initialize(llvm::orc::JITDylib&) LLJIT.h:141
>>> #7 0x10a89c514 in runOrcLazyJIT(char const*) lli.cpp:975
>>> #8 0x10a890b79 in main lli.cpp:409
>>> #9 0x7fff5beb5084 in start+0x0 (libdyld.dylib:x86_64+0x17084)
>>>
>>> Direct leak of 64 byte(s) in 1 object(s) allocated from:
>>> #0 0x1127fbef2 in wrap_calloc+0xa2 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x45ef2)
>>> #1 0x7fff5add6510 in realizeClass(objc_class*)+0x4c (libobjc.A.dylib:x86_64h+0x5510)
>>> #2 0x7fff5adee3f5 in objc_readClassPair+0x76 (libobjc.A.dylib:x86_64h+0x1d3f5)
>>> #3 0x10be4620c in llvm::orc::MachOJITDylibInitializers::registerObjCClasses() const MachOPlatform.cpp:133
>>> #4 0x10be246a7 in (anonymous namespace)::MachOPlatformSupport::initialize(llvm::orc::JITDylib&) LLJIT.cpp:525
>>> #5 0x10a8a73f7 in llvm::orc::LLJIT::initialize(llvm::orc::JITDylib&) LLJIT.h:141
>>> #6 0x10a89c514 in runOrcLazyJIT(char const*) lli.cpp:975
>>> #7 0x10a890b79 in main lli.cpp:409
>>> #8 0x7fff5beb5084 in start+0x0 (libdyld.dylib:x86_64+0x17084)
>>>
>>> Direct leak of 16 byte(s) in 1 object(s) allocated from:
>>> #0 0x1127fbef2 in wrap_calloc+0xa2 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x45ef2)
>>> #1 0x7fff5add883d in class_createInstance+0x52 (libobjc.A.dylib:x86_64h+0x783d)
>>> #2 0x7fff5addc595 in _objc_rootAlloc+0x2c (libobjc.A.dylib:x86_64h+0xb595)
>>> #3 0x1166f701b (<unknown module>)
>>> #4 0x10a89cd22 in runOrcLazyJIT(char const*) lli.cpp:991
>>> #5 0x10a890b79 in main lli.cpp:409
>>> #6 0x7fff5beb5084 in start+0x0 (libdyld.dylib:x86_64+0x17084)
>>>
>>> SUMMARY: AddressSanitizer: 304 byte(s) leaked in 5 allocation(s)
>>>> On Feb 19, 2020, at 2:03 PM, Lang Hames via llvm-commits <llvm-commits at lists.llvm.org <mailto:llvm-commits at lists.llvm.org>> wrote:
>>>>
>>>>
>>>> Author: Lang Hames
>>>> Date: 2020-02-19T13:59:32-08:00
>>>> New Revision: 85fb997659b55101b58e3545733e581bf4ed9cf2
>>>>
>>>> URL: https://github.com/llvm/llvm-project/commit/85fb997659b55101b58e3545733e581bf4ed9cf2 <https://github.com/llvm/llvm-project/commit/85fb997659b55101b58e3545733e581bf4ed9cf2>
>>>> DIFF: https://github.com/llvm/llvm-project/commit/85fb997659b55101b58e3545733e581bf4ed9cf2.diff <https://github.com/llvm/llvm-project/commit/85fb997659b55101b58e3545733e581bf4ed9cf2.diff>
>>>>
>>>> LOG: [ORC] Add generic initializer/deinitializer support.
>>>>
>>>> Initializers and deinitializers are used to implement C++ static constructors
>>>> and destructors, runtime registration for some languages (e.g. with the
>>>> Objective-C runtime for Objective-C/C++ code) and other tasks that would
>>>> typically be performed when a shared-object/dylib is loaded or unloaded by a
>>>> statically compiled program.
>>>>
>>>> MCJIT and ORC have historically provided limited support for discovering and
>>>> running initializers/deinitializers by scanning the llvm.global_ctors and
>>>> llvm.global_dtors variables and recording the functions to be run. This approach
>>>> suffers from several drawbacks: (1) It only works for IR inputs, not for object
>>>> files (including cached JIT'd objects). (2) It only works for initializers
>>>> described by llvm.global_ctors and llvm.global_dtors, however not all
>>>> initializers are described in this way (Objective-C, for example, describes
>>>> initializers via specially named metadata sections). (3) To make the
>>>> initializer/deinitializer functions described by llvm.global_ctors and
>>>> llvm.global_dtors searchable they must be promoted to extern linkage, polluting
>>>> the JIT symbol table (extra care must be taken to ensure this promotion does
>>>> not result in symbol name clashes).
>>>>
>>>> This patch introduces several interdependent changes to ORCv2 to support the
>>>> construction of new initialization schemes, and includes an implementation of a
>>>> backwards-compatible llvm.global_ctor/llvm.global_dtor scanning scheme, and a
>>>> MachO specific scheme that handles Objective-C runtime registration (if the
>>>> Objective-C runtime is available) enabling execution of LLVM IR compiled from
>>>> Objective-C and Swift.
>>>>
>>>> The major changes included in this patch are:
>>>>
>>>> (1) The MaterializationUnit and MaterializationResponsibility classes are
>>>> extended to describe an optional "initializer" symbol for the module (see the
>>>> getInitializerSymbol method on each class). The presence or absence of this
>>>> symbol indicates whether the module contains any initializers or
>>>> deinitializers. The initializer symbol otherwise behaves like any other:
>>>> searching for it triggers materialization.
>>>>
>>>> (2) A new Platform interface is introduced in llvm/ExecutionEngine/Orc/Core.h
>>>> which provides the following callback interface:
>>>>
>>>> - Error setupJITDylib(JITDylib &JD): Can be used to install standard symbols
>>>> in JITDylibs upon creation. E.g. __dso_handle.
>>>>
>>>> - Error notifyAdding(JITDylib &JD, const MaterializationUnit &MU): Generally
>>>> used to record initializer symbols.
>>>>
>>>> - Error notifyRemoving(JITDylib &JD, VModuleKey K): Used to notify a platform
>>>> that a module is being removed.
>>>>
>>>> Platform implementations can use these callbacks to track outstanding
>>>> initializers and implement a platform-specific approach for executing them. For
>>>> example, the MachOPlatform installs a plugin in the JIT linker to scan for both
>>>> __mod_inits sections (for C++ static constructors) and ObjC metadata sections.
>>>> If discovered, these are processed in the usual platform order: Objective-C
>>>> registration is carried out first, then static initializers are executed,
>>>> ensuring that calls to Objective-C from static initializers will be safe.
>>>>
>>>> This patch updates LLJIT to use the new scheme for initialization. Two
>>>> LLJIT::PlatformSupport classes are implemented: A GenericIR platform and a MachO
>>>> platform. The GenericIR platform implements a modified version of the previous
>>>> llvm.global-ctor scraping scheme to provide support for Windows and
>>>> Linux. LLJIT's MachO platform uses the MachOPlatform class to provide MachO
>>>> specific initialization as described above.
>>>>
>>>> Reviewers: sgraenitz, dblaikie
>>>>
>>>> Subscribers: mgorny, hiraditya, mgrang, ributzka, llvm-commits
>>>>
>>>> Tags: #llvm
>>>>
>>>> Differential Revision: https://reviews.llvm.org/D74300 <https://reviews.llvm.org/D74300>
>>>>
>>>> Added:
>>>> llvm/include/llvm/ExecutionEngine/Orc/MachOPlatform.h
>>>> llvm/include/llvm/ExecutionEngine/Orc/Mangling.h
>>>> llvm/lib/ExecutionEngine/Orc/MachOPlatform.cpp
>>>> llvm/lib/ExecutionEngine/Orc/Mangling.cpp
>>>> llvm/test/ExecutionEngine/OrcLazy/objc-minimal.ll
>>>>
>>>> Modified:
>>>> llvm/examples/Kaleidoscope/BuildingAJIT/Chapter1/KaleidoscopeJIT.h
>>>> llvm/examples/Kaleidoscope/BuildingAJIT/Chapter2/KaleidoscopeJIT.h
>>>> llvm/include/llvm/ExecutionEngine/JITLink/JITLink.h
>>>> llvm/include/llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h
>>>> llvm/include/llvm/ExecutionEngine/Orc/CompileUtils.h
>>>> llvm/include/llvm/ExecutionEngine/Orc/Core.h
>>>> llvm/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h
>>>> llvm/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h
>>>> llvm/include/llvm/ExecutionEngine/Orc/IRTransformLayer.h
>>>> llvm/include/llvm/ExecutionEngine/Orc/IndirectionUtils.h
>>>> llvm/include/llvm/ExecutionEngine/Orc/LLJIT.h
>>>> llvm/include/llvm/ExecutionEngine/Orc/Layer.h
>>>> llvm/include/llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h
>>>> llvm/include/llvm/ExecutionEngine/Orc/SymbolStringPool.h
>>>> llvm/lib/ExecutionEngine/Orc/CMakeLists.txt
>>>> llvm/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp
>>>> llvm/lib/ExecutionEngine/Orc/CompileUtils.cpp
>>>> llvm/lib/ExecutionEngine/Orc/Core.cpp
>>>> llvm/lib/ExecutionEngine/Orc/ExecutionUtils.cpp
>>>> llvm/lib/ExecutionEngine/Orc/IndirectionUtils.cpp
>>>> llvm/lib/ExecutionEngine/Orc/LLJIT.cpp
>>>> llvm/lib/ExecutionEngine/Orc/Layer.cpp
>>>> llvm/lib/ExecutionEngine/Orc/LazyReexports.cpp
>>>> llvm/lib/ExecutionEngine/Orc/ObjectLinkingLayer.cpp
>>>> llvm/lib/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.cpp
>>>> llvm/tools/lli/lli.cpp
>>>> llvm/tools/llvm-jitlink/llvm-jitlink.cpp
>>>> llvm/tools/llvm-jitlink/llvm-jitlink.h
>>>> llvm/unittests/ExecutionEngine/Orc/CoreAPIsTest.cpp
>>>> llvm/unittests/ExecutionEngine/Orc/OrcTestCommon.h
>>>> llvm/unittests/ExecutionEngine/Orc/RTDyldObjectLinkingLayerTest.cpp
>>>>
>>>> Removed:
>>>>
>>>>
>>>>
>>>> ################################################################################
>>>> diff --git a/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter1/KaleidoscopeJIT.h b/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter1/KaleidoscopeJIT.h
>>>> index b7404baf1ff0..adb6be436660 100644
>>>> --- a/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter1/KaleidoscopeJIT.h
>>>> +++ b/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter1/KaleidoscopeJIT.h
>>>> @@ -49,7 +49,7 @@ class KaleidoscopeJIT {
>>>> std::make_unique<ConcurrentIRCompiler>(std::move(JTMB))),
>>>> DL(std::move(DL)), Mangle(ES, this->DL),
>>>> Ctx(std::make_unique<LLVMContext>()),
>>>> - MainJD(ES.createJITDylib("<main>")) {
>>>> + MainJD(ES.createBareJITDylib("<main>")) {
>>>> MainJD.addGenerator(
>>>> cantFail(DynamicLibrarySearchGenerator::GetForCurrentProcess(
>>>> DL.getGlobalPrefix())));
>>>>
>>>> diff --git a/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter2/KaleidoscopeJIT.h b/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter2/KaleidoscopeJIT.h
>>>> index efb19349e3e1..28649ffe4910 100644
>>>> --- a/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter2/KaleidoscopeJIT.h
>>>> +++ b/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter2/KaleidoscopeJIT.h
>>>> @@ -55,7 +55,7 @@ class KaleidoscopeJIT {
>>>> std::make_unique<ConcurrentIRCompiler>(std::move(JTMB))),
>>>> OptimizeLayer(ES, CompileLayer, optimizeModule), DL(std::move(DL)),
>>>> Mangle(ES, this->DL), Ctx(std::make_unique<LLVMContext>()),
>>>> - MainJD(ES.createJITDylib("<main>")) {
>>>> + MainJD(ES.createBareJITDylib("<main>")) {
>>>> MainJD.addGenerator(
>>>> cantFail(DynamicLibrarySearchGenerator::GetForCurrentProcess(
>>>> DL.getGlobalPrefix())));
>>>>
>>>> diff --git a/llvm/include/llvm/ExecutionEngine/JITLink/JITLink.h b/llvm/include/llvm/ExecutionEngine/JITLink/JITLink.h
>>>> index 1a8f5cacbcd4..d673a89386da 100644
>>>> --- a/llvm/include/llvm/ExecutionEngine/JITLink/JITLink.h
>>>> +++ b/llvm/include/llvm/ExecutionEngine/JITLink/JITLink.h
>>>> @@ -488,6 +488,8 @@ class Symbol {
>>>>
>>>> /// Set the visibility for this Symbol.
>>>> void setScope(Scope S) {
>>>> + assert((!Name.empty() || S == Scope::Local) &&
>>>> + "Can not set anonymous symbol to non-local scope");
>>>> assert((S == Scope::Default || Base->isDefined() || Base->isAbsolute()) &&
>>>> "Invalid visibility for symbol type");
>>>> this->S = static_cast<uint8_t>(S);
>>>>
>>>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h b/llvm/include/llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h
>>>> index 65295f97e2c6..ef251ac302df 100644
>>>> --- a/llvm/include/llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h
>>>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h
>>>> @@ -94,6 +94,7 @@ class CompileOnDemandLayer : public IRLayer {
>>>>
>>>> /// Sets the ImplSymbolMap
>>>> void setImplMap(ImplSymbolMap *Imp);
>>>> +
>>>> /// Emits the given module. This should not be called by clients: it will be
>>>> /// called by the JIT when a definition added via the add method is requested.
>>>> void emit(MaterializationResponsibility R, ThreadSafeModule TSM) override;
>>>>
>>>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/CompileUtils.h b/llvm/include/llvm/ExecutionEngine/Orc/CompileUtils.h
>>>> index 218afda1b546..8376d163d57a 100644
>>>> --- a/llvm/include/llvm/ExecutionEngine/Orc/CompileUtils.h
>>>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/CompileUtils.h
>>>> @@ -30,7 +30,7 @@ namespace orc {
>>>>
>>>> class JITTargetMachineBuilder;
>>>>
>>>> -IRMaterializationUnit::ManglingOptions
>>>> +IRSymbolMapper::ManglingOptions
>>>> irManglingOptionsFromTargetOptions(const TargetOptions &Opts);
>>>>
>>>> /// Simple compile functor: Takes a single IR module and returns an ObjectFile.
>>>> @@ -52,7 +52,7 @@ class SimpleCompiler : public IRCompileLayer::IRCompiler {
>>>> Expected<CompileResult> operator()(Module &M) override;
>>>>
>>>> private:
>>>> - IRMaterializationUnit::ManglingOptions
>>>> + IRSymbolMapper::ManglingOptions
>>>> manglingOptionsForTargetMachine(const TargetMachine &TM);
>>>>
>>>> CompileResult tryToLoadFromObjectCache(const Module &M);
>>>>
>>>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/Core.h b/llvm/include/llvm/ExecutionEngine/Orc/Core.h
>>>> index 1a55801b9d87..94600f2286b8 100644
>>>> --- a/llvm/include/llvm/ExecutionEngine/Orc/Core.h
>>>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/Core.h
>>>> @@ -14,11 +14,11 @@
>>>> #define LLVM_EXECUTIONENGINE_ORC_CORE_H
>>>>
>>>> #include "llvm/ADT/BitmaskEnum.h"
>>>> +#include "llvm/ADT/DenseSet.h"
>>>> #include "llvm/ADT/FunctionExtras.h"
>>>> #include "llvm/ExecutionEngine/JITSymbol.h"
>>>> #include "llvm/ExecutionEngine/Orc/SymbolStringPool.h"
>>>> #include "llvm/ExecutionEngine/OrcV1Deprecation.h"
>>>> -#include "llvm/IR/Module.h"
>>>> #include "llvm/Support/Debug.h"
>>>>
>>>> #include <memory>
>>>> @@ -456,6 +456,11 @@ class MaterializationResponsibility {
>>>> /// before using.
>>>> const SymbolFlagsMap &getSymbols() const { return SymbolFlags; }
>>>>
>>>> + /// Returns the initialization pseudo-symbol, if any. This symbol will also
>>>> + /// be present in the SymbolFlagsMap for this MaterializationResponsibility
>>>> + /// object.
>>>> + const SymbolStringPtr &getInitializerSymbol() const { return InitSymbol; }
>>>> +
>>>> /// Returns the names of any symbols covered by this
>>>> /// MaterializationResponsibility object that have queries pending. This
>>>> /// information can be used to return responsibility for unrequested symbols
>>>> @@ -532,10 +537,15 @@ class MaterializationResponsibility {
>>>> /// Create a MaterializationResponsibility for the given JITDylib and
>>>> /// initial symbols.
>>>> MaterializationResponsibility(JITDylib &JD, SymbolFlagsMap SymbolFlags,
>>>> - VModuleKey K);
>>>> + SymbolStringPtr InitSymbol, VModuleKey K)
>>>> + : JD(JD), SymbolFlags(std::move(SymbolFlags)),
>>>> + InitSymbol(std::move(InitSymbol)), K(std::move(K)) {
>>>> + assert(!this->SymbolFlags.empty() && "Materializing nothing?");
>>>> + }
>>>>
>>>> JITDylib &JD;
>>>> SymbolFlagsMap SymbolFlags;
>>>> + SymbolStringPtr InitSymbol;
>>>> VModuleKey K;
>>>> };
>>>>
>>>> @@ -549,8 +559,13 @@ class MaterializationResponsibility {
>>>> /// stronger definition is added or already present.
>>>> class MaterializationUnit {
>>>> public:
>>>> - MaterializationUnit(SymbolFlagsMap InitalSymbolFlags, VModuleKey K)
>>>> - : SymbolFlags(std::move(InitalSymbolFlags)), K(std::move(K)) {}
>>>> + MaterializationUnit(SymbolFlagsMap InitalSymbolFlags,
>>>> + SymbolStringPtr InitSymbol, VModuleKey K)
>>>> + : SymbolFlags(std::move(InitalSymbolFlags)),
>>>> + InitSymbol(std::move(InitSymbol)), K(std::move(K)) {
>>>> + assert((!this->InitSymbol || this->SymbolFlags.count(this->InitSymbol)) &&
>>>> + "If set, InitSymbol should appear in InitialSymbolFlags map");
>>>> + }
>>>>
>>>> virtual ~MaterializationUnit() {}
>>>>
>>>> @@ -561,12 +576,15 @@ class MaterializationUnit {
>>>> /// Return the set of symbols that this source provides.
>>>> const SymbolFlagsMap &getSymbols() const { return SymbolFlags; }
>>>>
>>>> + /// Returns the initialization symbol for this MaterializationUnit (if any).
>>>> + const SymbolStringPtr &getInitializerSymbol() const { return InitSymbol; }
>>>> +
>>>> /// Called by materialization dispatchers (see
>>>> /// ExecutionSession::DispatchMaterializationFunction) to trigger
>>>> /// materialization of this MaterializationUnit.
>>>> void doMaterialize(JITDylib &JD) {
>>>> - materialize(MaterializationResponsibility(JD, std::move(SymbolFlags),
>>>> - std::move(K)));
>>>> + materialize(MaterializationResponsibility(
>>>> + JD, std::move(SymbolFlags), std::move(InitSymbol), std::move(K)));
>>>> }
>>>>
>>>> /// Called by JITDylibs to notify MaterializationUnits that the given symbol
>>>> @@ -578,6 +596,7 @@ class MaterializationUnit {
>>>>
>>>> protected:
>>>> SymbolFlagsMap SymbolFlags;
>>>> + SymbolStringPtr InitSymbol;
>>>> VModuleKey K;
>>>>
>>>> private:
>>>> @@ -774,6 +793,7 @@ class AsynchronousSymbolQuery {
>>>> class JITDylib {
>>>> friend class AsynchronousSymbolQuery;
>>>> friend class ExecutionSession;
>>>> + friend class Platform;
>>>> friend class MaterializationResponsibility;
>>>> public:
>>>> /// Definition generators can be attached to JITDylibs to generate new
>>>> @@ -1054,6 +1074,35 @@ class JITDylib {
>>>> JITDylibSearchOrder SearchOrder;
>>>> };
>>>>
>>>> +/// Platforms set up standard symbols and mediate interactions between dynamic
>>>> +/// initializers (e.g. C++ static constructors) and ExecutionSession state.
>>>> +/// Note that Platforms do not automatically run initializers: clients are still
>>>> +/// responsible for doing this.
>>>> +class Platform {
>>>> +public:
>>>> + virtual ~Platform();
>>>> +
>>>> + /// This method will be called outside the session lock each time a JITDylib
>>>> + /// is created (unless it is created with EmptyJITDylib set) to allow the
>>>> + /// Platform to install any JITDylib specific standard symbols (e.g
>>>> + /// __dso_handle).
>>>> + virtual Error setupJITDylib(JITDylib &JD) = 0;
>>>> +
>>>> + /// This method will be called under the ExecutionSession lock each time a
>>>> + /// MaterializationUnit is added to a JITDylib.
>>>> + virtual Error notifyAdding(JITDylib &JD, const MaterializationUnit &MU) = 0;
>>>> +
>>>> + /// This method will be called under the ExecutionSession lock when a
>>>> + /// VModuleKey is removed.
>>>> + virtual Error notifyRemoving(JITDylib &JD, VModuleKey K) = 0;
>>>> +
>>>> + /// A utility function for looking up initializer symbols. Performs a blocking
>>>> + /// lookup for the given symbols in each of the given JITDylibs.
>>>> + static Expected<DenseMap<JITDylib *, SymbolMap>>
>>>> + lookupInitSymbols(ExecutionSession &ES,
>>>> + const DenseMap<JITDylib *, SymbolLookupSet> &InitSyms);
>>>> +};
>>>> +
>>>> /// An ExecutionSession represents a running JIT program.
>>>> class ExecutionSession {
>>>> // FIXME: Remove this when we remove the old ORC layers.
>>>> @@ -1078,6 +1127,13 @@ class ExecutionSession {
>>>> /// Returns a shared_ptr to the SymbolStringPool for this ExecutionSession.
>>>> std::shared_ptr<SymbolStringPool> getSymbolStringPool() const { return SSP; }
>>>>
>>>> + /// Set the Platform for this ExecutionSession.
>>>> + void setPlatform(std::unique_ptr<Platform> P) { this->P = std::move(P); }
>>>> +
>>>> + /// Get the Platform for this session.
>>>> + /// Will return null if no Platform has been set for this ExecutionSession.
>>>> + Platform *getPlatform() { return P.get(); }
>>>> +
>>>> /// Run the given lambda with the session mutex locked.
>>>> template <typename Func> decltype(auto) runSessionLocked(Func &&F) {
>>>> std::lock_guard<std::recursive_mutex> Lock(SessionMutex);
>>>> @@ -1088,12 +1144,26 @@ class ExecutionSession {
>>>> /// Ownership of JITDylib remains within Execution Session
>>>> JITDylib *getJITDylibByName(StringRef Name);
>>>>
>>>> + /// Add a new bare JITDylib to this ExecutionSession.
>>>> + ///
>>>> + /// The JITDylib Name is required to be unique. Clients should verify that
>>>> + /// names are not being re-used (E.g. by calling getJITDylibByName) if names
>>>> + /// are based on user input.
>>>> + ///
>>>> + /// This call does not install any library code or symbols into the newly
>>>> + /// created JITDylib. The client is responsible for all configuration.
>>>> + JITDylib &createBareJITDylib(std::string Name);
>>>> +
>>>> /// Add a new JITDylib to this ExecutionSession.
>>>> ///
>>>> /// The JITDylib Name is required to be unique. Clients should verify that
>>>> /// names are not being re-used (e.g. by calling getJITDylibByName) if names
>>>> /// are based on user input.
>>>> - JITDylib &createJITDylib(std::string Name);
>>>> + ///
>>>> + /// If a Platform is attached then Platform::setupJITDylib will be called to
>>>> + /// install standard platform symbols (e.g. standard library interposes).
>>>> + /// If no Platform is attached this call is equivalent to createBareJITDylib.
>>>> + Expected<JITDylib &> createJITDylib(std::string Name);
>>>>
>>>> /// Allocate a module key for a new module to add to the JIT.
>>>> VModuleKey allocateVModule() {
>>>> @@ -1177,20 +1247,23 @@ class ExecutionSession {
>>>> /// Convenience version of blocking lookup.
>>>> /// Searches each of the JITDylibs in the search order in turn for the given
>>>> /// symbol.
>>>> - Expected<JITEvaluatedSymbol> lookup(const JITDylibSearchOrder &SearchOrder,
>>>> - SymbolStringPtr Symbol);
>>>> + Expected<JITEvaluatedSymbol>
>>>> + lookup(const JITDylibSearchOrder &SearchOrder, SymbolStringPtr Symbol,
>>>> + SymbolState RequiredState = SymbolState::Ready);
>>>>
>>>> /// Convenience version of blocking lookup.
>>>> /// Searches each of the JITDylibs in the search order in turn for the given
>>>> /// symbol. The search will not find non-exported symbols.
>>>> - Expected<JITEvaluatedSymbol> lookup(ArrayRef<JITDylib *> SearchOrder,
>>>> - SymbolStringPtr Symbol);
>>>> + Expected<JITEvaluatedSymbol>
>>>> + lookup(ArrayRef<JITDylib *> SearchOrder, SymbolStringPtr Symbol,
>>>> + SymbolState RequiredState = SymbolState::Ready);
>>>>
>>>> /// Convenience version of blocking lookup.
>>>> /// Searches each of the JITDylibs in the search order in turn for the given
>>>> /// symbol. The search will not find non-exported symbols.
>>>> - Expected<JITEvaluatedSymbol> lookup(ArrayRef<JITDylib *> SearchOrder,
>>>> - StringRef Symbol);
>>>> + Expected<JITEvaluatedSymbol>
>>>> + lookup(ArrayRef<JITDylib *> SearchOrder, StringRef Symbol,
>>>> + SymbolState RequiredState = SymbolState::Ready);
>>>>
>>>> /// Materialize the given unit.
>>>> void dispatchMaterialization(JITDylib &JD,
>>>> @@ -1221,6 +1294,7 @@ class ExecutionSession {
>>>>
>>>> mutable std::recursive_mutex SessionMutex;
>>>> std::shared_ptr<SymbolStringPool> SSP;
>>>> + std::unique_ptr<Platform> P;
>>>> VModuleKey LastKey = 0;
>>>> ErrorReporter ReportError = logErrorsToStdErr;
>>>> DispatchMaterializationFunction DispatchMaterialization =
>>>> @@ -1256,6 +1330,11 @@ Error JITDylib::define(std::unique_ptr<MaterializationUnitType> &&MU) {
>>>> if (auto Err = defineImpl(*MU))
>>>> return Err;
>>>>
>>>> + if (auto *P = ES.getPlatform()) {
>>>> + if (auto Err = P->notifyAdding(*this, *MU))
>>>> + return Err;
>>>> + }
>>>> +
>>>> /// defineImpl succeeded.
>>>> auto UMI = std::make_shared<UnmaterializedInfo>(std::move(MU));
>>>> for (auto &KV : UMI->MU->getSymbols())
>>>> @@ -1273,6 +1352,11 @@ Error JITDylib::define(std::unique_ptr<MaterializationUnitType> &MU) {
>>>> if (auto Err = defineImpl(*MU))
>>>> return Err;
>>>>
>>>> + if (auto *P = ES.getPlatform()) {
>>>> + if (auto Err = P->notifyAdding(*this, *MU))
>>>> + return Err;
>>>> + }
>>>> +
>>>> /// defineImpl succeeded.
>>>> auto UMI = std::make_shared<UnmaterializedInfo>(std::move(MU));
>>>> for (auto &KV : UMI->MU->getSymbols())
>>>> @@ -1305,18 +1389,6 @@ class ReexportsGenerator : public JITDylib::DefinitionGenerator {
>>>> SymbolPredicate Allow;
>>>> };
>>>>
>>>> -/// Mangles symbol names then uniques them in the context of an
>>>> -/// ExecutionSession.
>>>> -class MangleAndInterner {
>>>> -public:
>>>> - MangleAndInterner(ExecutionSession &ES, const DataLayout &DL);
>>>> - SymbolStringPtr operator()(StringRef Name);
>>>> -
>>>> -private:
>>>> - ExecutionSession &ES;
>>>> - const DataLayout &DL;
>>>> -};
>>>> -
>>>> } // End namespace orc
>>>> } // End namespace llvm
>>>>
>>>>
>>>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h b/llvm/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h
>>>> index f7255c5af845..49ab0d628b0b 100644
>>>> --- a/llvm/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h
>>>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h
>>>> @@ -17,6 +17,7 @@
>>>> #include "llvm/ADT/iterator_range.h"
>>>> #include "llvm/ExecutionEngine/JITSymbol.h"
>>>> #include "llvm/ExecutionEngine/Orc/Core.h"
>>>> +#include "llvm/ExecutionEngine/Orc/Mangling.h"
>>>> #include "llvm/ExecutionEngine/Orc/OrcError.h"
>>>> #include "llvm/ExecutionEngine/RuntimeDyld.h"
>>>> #include "llvm/Object/Archive.h"
>>>> @@ -104,6 +105,53 @@ iterator_range<CtorDtorIterator> getConstructors(const Module &M);
>>>> /// array.
>>>> iterator_range<CtorDtorIterator> getDestructors(const Module &M);
>>>>
>>>> +/// This iterator provides a convenient way to iterate over GlobalValues that
>>>> +/// have initialization effects.
>>>> +class StaticInitGVIterator {
>>>> +public:
>>>> + StaticInitGVIterator() = default;
>>>> +
>>>> + StaticInitGVIterator(Module &M)
>>>> + : I(M.global_values().begin()), E(M.global_values().end()),
>>>> + ObjFmt(Triple(M.getTargetTriple()).getObjectFormat()) {
>>>> + if (I != E) {
>>>> + if (!isStaticInitGlobal(*I))
>>>> + moveToNextStaticInitGlobal();
>>>> + } else
>>>> + I = E = Module::global_value_iterator();
>>>> + }
>>>> +
>>>> + bool operator==(const StaticInitGVIterator &O) const { return I == O.I; }
>>>> + bool operator!=(const StaticInitGVIterator &O) const { return I != O.I; }
>>>> +
>>>> + StaticInitGVIterator &operator++() {
>>>> + assert(I != E && "Increment past end of range");
>>>> + moveToNextStaticInitGlobal();
>>>> + return *this;
>>>> + }
>>>> +
>>>> + GlobalValue &operator*() { return *I; }
>>>> +
>>>> +private:
>>>> + bool isStaticInitGlobal(GlobalValue &GV);
>>>> + void moveToNextStaticInitGlobal() {
>>>> + ++I;
>>>> + while (I != E && !isStaticInitGlobal(*I))
>>>> + ++I;
>>>> + if (I == E)
>>>> + I = E = Module::global_value_iterator();
>>>> + }
>>>> +
>>>> + Module::global_value_iterator I, E;
>>>> + Triple::ObjectFormatType ObjFmt;
>>>> +};
>>>> +
>>>> +/// Create an iterator range over the GlobalValues that contribute to static
>>>> +/// initialization.
>>>> +inline iterator_range<StaticInitGVIterator> getStaticInitGVs(Module &M) {
>>>> + return make_range(StaticInitGVIterator(M), StaticInitGVIterator());
>>>> +}
>>>> +
>>>> /// Convenience class for recording constructor/destructor names for
>>>> /// later execution.
>>>> template <typename JITLayerT>
>>>> @@ -246,6 +294,22 @@ class LocalCXXRuntimeOverrides : public LocalCXXRuntimeOverridesBase {
>>>> Error enable(JITDylib &JD, MangleAndInterner &Mangler);
>>>> };
>>>>
>>>> +/// An interface for Itanium __cxa_atexit interposer implementations.
>>>> +class ItaniumCXAAtExitSupport {
>>>> +public:
>>>> + struct AtExitRecord {
>>>> + void (*F)(void *);
>>>> + void *Ctx;
>>>> + };
>>>> +
>>>> + void registerAtExit(void (*F)(void *), void *Ctx, void *DSOHandle);
>>>> + void runAtExits(void *DSOHandle);
>>>> +
>>>> +private:
>>>> + std::mutex AtExitsMutex;
>>>> + DenseMap<void *, std::vector<AtExitRecord>> AtExitRecords;
>>>> +};
>>>> +
>>>> /// A utility class to expose symbols found via dlsym to the JIT.
>>>> ///
>>>> /// If an instance of this class is attached to a JITDylib as a fallback
>>>>
>>>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h b/llvm/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h
>>>> index bb8270fe80a3..eb74d283f043 100644
>>>> --- a/llvm/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h
>>>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h
>>>> @@ -31,18 +31,18 @@ class IRCompileLayer : public IRLayer {
>>>> public:
>>>> class IRCompiler {
>>>> public:
>>>> - IRCompiler(IRMaterializationUnit::ManglingOptions MO) : MO(std::move(MO)) {}
>>>> + IRCompiler(IRSymbolMapper::ManglingOptions MO) : MO(std::move(MO)) {}
>>>> virtual ~IRCompiler();
>>>> - const IRMaterializationUnit::ManglingOptions &getManglingOptions() const {
>>>> + const IRSymbolMapper::ManglingOptions &getManglingOptions() const {
>>>> return MO;
>>>> }
>>>> virtual Expected<std::unique_ptr<MemoryBuffer>> operator()(Module &M) = 0;
>>>>
>>>> protected:
>>>> - IRMaterializationUnit::ManglingOptions &manglingOptions() { return MO; }
>>>> + IRSymbolMapper::ManglingOptions &manglingOptions() { return MO; }
>>>>
>>>> private:
>>>> - IRMaterializationUnit::ManglingOptions MO;
>>>> + IRSymbolMapper::ManglingOptions MO;
>>>> };
>>>>
>>>> using NotifyCompiledFunction =
>>>> @@ -61,7 +61,7 @@ class IRCompileLayer : public IRLayer {
>>>> mutable std::mutex IRLayerMutex;
>>>> ObjectLayer &BaseLayer;
>>>> std::unique_ptr<IRCompiler> Compile;
>>>> - const IRMaterializationUnit::ManglingOptions *ManglingOpts;
>>>> + const IRSymbolMapper::ManglingOptions *ManglingOpts;
>>>> NotifyCompiledFunction NotifyCompiled = NotifyCompiledFunction();
>>>> };
>>>>
>>>>
>>>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/IRTransformLayer.h b/llvm/include/llvm/ExecutionEngine/Orc/IRTransformLayer.h
>>>> index b71e5b339711..296d74ae6b86 100644
>>>> --- a/llvm/include/llvm/ExecutionEngine/Orc/IRTransformLayer.h
>>>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/IRTransformLayer.h
>>>> @@ -28,7 +28,7 @@ namespace orc {
>>>> class IRTransformLayer : public IRLayer {
>>>> public:
>>>> using TransformFunction = std::function<Expected<ThreadSafeModule>(
>>>> - ThreadSafeModule, const MaterializationResponsibility &R)>;
>>>> + ThreadSafeModule, MaterializationResponsibility &R)>;
>>>>
>>>> IRTransformLayer(ExecutionSession &ES, IRLayer &BaseLayer,
>>>> TransformFunction Transform = identityTransform);
>>>> @@ -39,9 +39,8 @@ class IRTransformLayer : public IRLayer {
>>>>
>>>> void emit(MaterializationResponsibility R, ThreadSafeModule TSM) override;
>>>>
>>>> - static ThreadSafeModule
>>>> - identityTransform(ThreadSafeModule TSM,
>>>> - const MaterializationResponsibility &R) {
>>>> + static ThreadSafeModule identityTransform(ThreadSafeModule TSM,
>>>> + MaterializationResponsibility &R) {
>>>> return TSM;
>>>> }
>>>>
>>>>
>>>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/IndirectionUtils.h b/llvm/include/llvm/ExecutionEngine/Orc/IndirectionUtils.h
>>>> index a9ab3a630a64..880c8ff44c5f 100644
>>>> --- a/llvm/include/llvm/ExecutionEngine/Orc/IndirectionUtils.h
>>>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/IndirectionUtils.h
>>>> @@ -201,7 +201,7 @@ class JITCompileCallbackManager {
>>>> ExecutionSession &ES,
>>>> JITTargetAddress ErrorHandlerAddress)
>>>> : TP(std::move(TP)), ES(ES),
>>>> - CallbacksJD(ES.createJITDylib("<Callbacks>")),
>>>> + CallbacksJD(ES.createBareJITDylib("<Callbacks>")),
>>>> ErrorHandlerAddress(ErrorHandlerAddress) {}
>>>>
>>>> void setTrampolinePool(std::unique_ptr<TrampolinePool> TP) {
>>>>
>>>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/LLJIT.h b/llvm/include/llvm/ExecutionEngine/Orc/LLJIT.h
>>>> index 3374a29f04df..c0b5012f801f 100644
>>>> --- a/llvm/include/llvm/ExecutionEngine/Orc/LLJIT.h
>>>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/LLJIT.h
>>>> @@ -35,7 +35,23 @@ class LLLazyJITBuilderState;
>>>> class LLJIT {
>>>> template <typename, typename, typename> friend class LLJITBuilderSetters;
>>>>
>>>> + friend void setUpGenericLLVMIRPlatform(LLJIT &J);
>>>> +
>>>> public:
>>>> + /// Initializer support for LLJIT.
>>>> + class PlatformSupport {
>>>> + public:
>>>> + virtual ~PlatformSupport();
>>>> +
>>>> + virtual Error initialize(JITDylib &JD) = 0;
>>>> +
>>>> + virtual Error deinitialize(JITDylib &JD) = 0;
>>>> +
>>>> + protected:
>>>> + static void setInitTransform(LLJIT &J,
>>>> + IRTransformLayer::TransformFunction T);
>>>> + };
>>>> +
>>>> static Expected<std::unique_ptr<LLJIT>> Create(LLJITBuilderState &S);
>>>>
>>>> /// Destruct this instance. If a multi-threaded instance, waits for all
>>>> @@ -52,7 +68,7 @@ class LLJIT {
>>>> const DataLayout &getDataLayout() const { return DL; }
>>>>
>>>> /// Returns a reference to the JITDylib representing the JIT'd main program.
>>>> - JITDylib &getMainJITDylib() { return Main; }
>>>> + JITDylib &getMainJITDylib() { return *Main; }
>>>>
>>>> /// Returns the JITDylib with the given name, or nullptr if no JITDylib with
>>>> /// that name exists.
>>>> @@ -66,7 +82,7 @@ class LLJIT {
>>>> /// input or elsewhere in the environment then the client should check
>>>> /// (e.g. by calling getJITDylibByName) that the given name is not already in
>>>> /// use.
>>>> - JITDylib &createJITDylib(std::string Name) {
>>>> + Expected<JITDylib &> createJITDylib(std::string Name) {
>>>> return ES->createJITDylib(std::move(Name));
>>>> }
>>>>
>>>> @@ -78,7 +94,7 @@ class LLJIT {
>>>>
>>>> /// Adds an IR module to the Main JITDylib.
>>>> Error addIRModule(ThreadSafeModule TSM) {
>>>> - return addIRModule(Main, std::move(TSM));
>>>> + return addIRModule(*Main, std::move(TSM));
>>>> }
>>>>
>>>> /// Adds an object file to the given JITDylib.
>>>> @@ -86,7 +102,7 @@ class LLJIT {
>>>>
>>>> /// Adds an object file to the given JITDylib.
>>>> Error addObjectFile(std::unique_ptr<MemoryBuffer> Obj) {
>>>> - return addObjectFile(Main, std::move(Obj));
>>>> + return addObjectFile(*Main, std::move(Obj));
>>>> }
>>>>
>>>> /// Look up a symbol in JITDylib JD by the symbol's linker-mangled name (to
>>>> @@ -98,7 +114,7 @@ class LLJIT {
>>>> /// (to look up symbols based on their IR name use the lookup function
>>>> /// instead).
>>>> Expected<JITEvaluatedSymbol> lookupLinkerMangled(StringRef Name) {
>>>> - return lookupLinkerMangled(Main, Name);
>>>> + return lookupLinkerMangled(*Main, Name);
>>>> }
>>>>
>>>> /// Look up a symbol in JITDylib JD based on its IR symbol name.
>>>> @@ -108,14 +124,28 @@ class LLJIT {
>>>>
>>>> /// Look up a symbol in the main JITDylib based on its IR symbol name.
>>>> Expected<JITEvaluatedSymbol> lookup(StringRef UnmangledName) {
>>>> - return lookup(Main, UnmangledName);
>>>> + return lookup(*Main, UnmangledName);
>>>> }
>>>>
>>>> - /// Runs all not-yet-run static constructors.
>>>> - Error runConstructors() { return CtorRunner.run(); }
>>>> + /// Set the PlatformSupport instance.
>>>> + void setPlatformSupport(std::unique_ptr<PlatformSupport> PS) {
>>>> + this->PS = std::move(PS);
>>>> + }
>>>>
>>>> - /// Runs all not-yet-run static destructors.
>>>> - Error runDestructors() { return DtorRunner.run(); }
>>>> + /// Get the PlatformSupport instance.
>>>> + PlatformSupport *getPlatformSupport() { return PS.get(); }
>>>> +
>>>> + /// Run the initializers for the given JITDylib.
>>>> + Error initialize(JITDylib &JD) {
>>>> + assert(PS && "PlatformSupport must be set to run initializers.");
>>>> + return PS->initialize(JD);
>>>> + }
>>>> +
>>>> + /// Run the deinitializers for the given JITDylib.
>>>> + Error deinitialize(JITDylib &JD) {
>>>> + assert(PS && "PlatformSupport must be set to run initializers.");
>>>> + return PS->deinitialize(JD);
>>>> + }
>>>>
>>>> /// Returns a reference to the ObjLinkingLayer
>>>> ObjectLayer &getObjLinkingLayer() { return *ObjLinkingLayer; }
>>>> @@ -126,6 +156,9 @@ class LLJIT {
>>>> /// Returns a reference to the IR transform layer.
>>>> IRTransformLayer &getIRTransformLayer() { return *TransformLayer; }
>>>>
>>>> + /// Returns a reference to the IR compile layer.
>>>> + IRCompileLayer &getIRCompileLayer() { return *CompileLayer; }
>>>> +
>>>> protected:
>>>> static std::unique_ptr<ObjectLayer>
>>>> createObjectLinkingLayer(LLJITBuilderState &S, ExecutionSession &ES);
>>>> @@ -143,7 +176,9 @@ class LLJIT {
>>>> void recordCtorDtors(Module &M);
>>>>
>>>> std::unique_ptr<ExecutionSession> ES;
>>>> - JITDylib &Main;
>>>> + std::unique_ptr<PlatformSupport> PS;
>>>> +
>>>> + JITDylib *Main = nullptr;
>>>>
>>>> DataLayout DL;
>>>> Triple TT;
>>>> @@ -153,8 +188,7 @@ class LLJIT {
>>>> ObjectTransformLayer ObjTransformLayer;
>>>> std::unique_ptr<IRCompileLayer> CompileLayer;
>>>> std::unique_ptr<IRTransformLayer> TransformLayer;
>>>> -
>>>> - CtorDtorRunner CtorRunner, DtorRunner;
>>>> + std::unique_ptr<IRTransformLayer> InitHelperTransformLayer;
>>>> };
>>>>
>>>> /// An extended version of LLJIT that supports lazy function-at-a-time
>>>> @@ -175,7 +209,7 @@ class LLLazyJIT : public LLJIT {
>>>>
>>>> /// Add a module to be lazily compiled to the main JITDylib.
>>>> Error addLazyIRModule(ThreadSafeModule M) {
>>>> - return addLazyIRModule(Main, std::move(M));
>>>> + return addLazyIRModule(*Main, std::move(M));
>>>> }
>>>>
>>>> private:
>>>> @@ -196,10 +230,14 @@ class LLJITBuilderState {
>>>> std::function<Expected<std::unique_ptr<IRCompileLayer::IRCompiler>>(
>>>> JITTargetMachineBuilder JTMB)>;
>>>>
>>>> + using PlatformSetupFunction = std::function<Error(LLJIT &J)>;
>>>> +
>>>> std::unique_ptr<ExecutionSession> ES;
>>>> Optional<JITTargetMachineBuilder> JTMB;
>>>> + Optional<DataLayout> DL;
>>>> ObjectLinkingLayerCreator CreateObjectLinkingLayer;
>>>> CompileFunctionCreator CreateCompileFunction;
>>>> + PlatformSetupFunction SetUpPlatform;
>>>> unsigned NumCompileThreads = 0;
>>>>
>>>> /// Called prior to JIT class construcion to fix up defaults.
>>>> @@ -224,6 +262,13 @@ class LLJITBuilderSetters {
>>>> return impl().JTMB;
>>>> }
>>>>
>>>> + /// Set a DataLayout for this instance. If no data layout is specified then
>>>> + /// the target's default data layout will be used.
>>>> + SetterImpl &setDataLayout(Optional<DataLayout> DL) {
>>>> + impl().DL = std::move(DL);
>>>> + return impl();
>>>> + }
>>>> +
>>>> /// Set an ObjectLinkingLayer creation function.
>>>> ///
>>>> /// If this method is not called, a default creation function will be used
>>>> @@ -246,6 +291,16 @@ class LLJITBuilderSetters {
>>>> return impl();
>>>> }
>>>>
>>>> + /// Set up an PlatformSetupFunction.
>>>> + ///
>>>> + /// If this method is not called then setUpGenericLLVMIRPlatform
>>>> + /// will be used to configure the JIT's platform support.
>>>> + SetterImpl &
>>>> + setPlatformSetUp(LLJITBuilderState::PlatformSetupFunction SetUpPlatform) {
>>>> + impl().SetUpPlatform = std::move(SetUpPlatform);
>>>> + return impl();
>>>> + }
>>>> +
>>>> /// Set the number of compile threads to use.
>>>> ///
>>>> /// If set to zero, compilation will be performed on the execution thread when
>>>> @@ -334,6 +389,26 @@ class LLLazyJITBuilder
>>>> public LLLazyJITBuilderSetters<LLLazyJIT, LLLazyJITBuilder,
>>>> LLLazyJITBuilderState> {};
>>>>
>>>> +/// Configure the LLJIT instance to scrape modules for llvm.global_ctors and
>>>> +/// llvm.global_dtors variables and (if present) build initialization and
>>>> +/// deinitialization functions. Platform specific initialization configurations
>>>> +/// should be preferred where available.
>>>> +void setUpGenericLLVMIRPlatform(LLJIT &J);
>>>> +
>>>> +/// Configure the LLJIT instance to use MachOPlatform support.
>>>> +///
>>>> +/// Warning: MachOPlatform *requires* that LLJIT be configured to use
>>>> +/// ObjectLinkingLayer (default on platforms supported by JITLink). If
>>>> +/// MachOPlatform is used with RTDyldObjectLinkingLayer it will result in
>>>> +/// undefined behavior).
>>>> +///
>>>> +/// MachOPlatform installs an ObjectLinkingLayer plugin to scrape initializers
>>>> +/// from the __mod_inits section. It also provides interposes for the dlfcn
>>>> +/// functions (dlopen, dlclose, dlsym, dlerror) that work for JITDylibs as
>>>> +/// well as regular libraries (JITDylibs will be preferenced, so make sure
>>>> +/// your JITDylib names do not shadow any real library paths).
>>>> +Error setUpMachOPlatform(LLJIT &J);
>>>> +
>>>> } // End namespace orc
>>>> } // End namespace llvm
>>>>
>>>>
>>>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/Layer.h b/llvm/include/llvm/ExecutionEngine/Orc/Layer.h
>>>> index 95e32b2431a0..e843d0f56245 100644
>>>> --- a/llvm/include/llvm/ExecutionEngine/Orc/Layer.h
>>>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/Layer.h
>>>> @@ -14,6 +14,7 @@
>>>> #define LLVM_EXECUTIONENGINE_ORC_LAYER_H
>>>>
>>>> #include "llvm/ExecutionEngine/Orc/Core.h"
>>>> +#include "llvm/ExecutionEngine/Orc/Mangling.h"
>>>> #include "llvm/ExecutionEngine/Orc/ThreadSafeModule.h"
>>>> #include "llvm/IR/Module.h"
>>>> #include "llvm/Support/MemoryBuffer.h"
>>>> @@ -27,15 +28,12 @@ namespace orc {
>>>> /// their linkage is changed to available-externally.
>>>> class IRMaterializationUnit : public MaterializationUnit {
>>>> public:
>>>> - struct ManglingOptions {
>>>> - bool EmulatedTLS = false;
>>>> - };
>>>> -
>>>> using SymbolNameToDefinitionMap = std::map<SymbolStringPtr, GlobalValue *>;
>>>>
>>>> /// Create an IRMaterializationLayer. Scans the module to build the
>>>> /// SymbolFlags and SymbolToDefinition maps.
>>>> - IRMaterializationUnit(ExecutionSession &ES, const ManglingOptions &MO,
>>>> + IRMaterializationUnit(ExecutionSession &ES,
>>>> + const IRSymbolMapper::ManglingOptions &MO,
>>>> ThreadSafeModule TSM, VModuleKey K);
>>>>
>>>> /// Create an IRMaterializationLayer from a module, and pre-existing
>>>> @@ -44,12 +42,13 @@ class IRMaterializationUnit : public MaterializationUnit {
>>>> /// This constructor is useful for delegating work from one
>>>> /// IRMaterializationUnit to another.
>>>> IRMaterializationUnit(ThreadSafeModule TSM, VModuleKey K,
>>>> - SymbolFlagsMap SymbolFlags,
>>>> + SymbolFlagsMap SymbolFlags, SymbolStringPtr InitSymbol,
>>>> SymbolNameToDefinitionMap SymbolToDefinition);
>>>>
>>>> /// Return the ModuleIdentifier as the name for this MaterializationUnit.
>>>> StringRef getName() const override;
>>>>
>>>> + /// Return a reference to the contained ThreadSafeModule.
>>>> const ThreadSafeModule &getModule() const { return TSM; }
>>>>
>>>> protected:
>>>> @@ -57,14 +56,16 @@ class IRMaterializationUnit : public MaterializationUnit {
>>>> SymbolNameToDefinitionMap SymbolToDefinition;
>>>>
>>>> private:
>>>> + static SymbolStringPtr getInitSymbol(ExecutionSession &ES,
>>>> + const ThreadSafeModule &TSM);
>>>> +
>>>> void discard(const JITDylib &JD, const SymbolStringPtr &Name) override;
>>>> };
>>>>
>>>> /// Interface for layers that accept LLVM IR.
>>>> class IRLayer {
>>>> public:
>>>> - IRLayer(ExecutionSession &ES,
>>>> - const IRMaterializationUnit::ManglingOptions *&MO)
>>>> + IRLayer(ExecutionSession &ES, const IRSymbolMapper::ManglingOptions *&MO)
>>>> : ES(ES), MO(MO) {}
>>>>
>>>> virtual ~IRLayer();
>>>> @@ -73,7 +74,7 @@ class IRLayer {
>>>> ExecutionSession &getExecutionSession() { return ES; }
>>>>
>>>> /// Get the mangling options for this layer.
>>>> - const IRMaterializationUnit::ManglingOptions *&getManglingOptions() const {
>>>> + const IRSymbolMapper::ManglingOptions *&getManglingOptions() const {
>>>> return MO;
>>>> }
>>>>
>>>> @@ -104,14 +105,15 @@ class IRLayer {
>>>> private:
>>>> bool CloneToNewContextOnEmit = false;
>>>> ExecutionSession &ES;
>>>> - const IRMaterializationUnit::ManglingOptions *&MO;
>>>> + const IRSymbolMapper::ManglingOptions *&MO;
>>>> };
>>>>
>>>> /// MaterializationUnit that materializes modules by calling the 'emit' method
>>>> /// on the given IRLayer.
>>>> class BasicIRLayerMaterializationUnit : public IRMaterializationUnit {
>>>> public:
>>>> - BasicIRLayerMaterializationUnit(IRLayer &L, const ManglingOptions &MO,
>>>> + BasicIRLayerMaterializationUnit(IRLayer &L,
>>>> + const IRSymbolMapper::ManglingOptions &MO,
>>>> ThreadSafeModule TSM, VModuleKey K);
>>>>
>>>> private:
>>>> @@ -153,7 +155,8 @@ class BasicObjectLayerMaterializationUnit : public MaterializationUnit {
>>>>
>>>> BasicObjectLayerMaterializationUnit(ObjectLayer &L, VModuleKey K,
>>>> std::unique_ptr<MemoryBuffer> O,
>>>> - SymbolFlagsMap SymbolFlags);
>>>> + SymbolFlagsMap SymbolFlags,
>>>> + SymbolStringPtr InitSymbol);
>>>>
>>>> /// Return the buffer's identifier as the name for this MaterializationUnit.
>>>> StringRef getName() const override;
>>>> @@ -167,12 +170,6 @@ class BasicObjectLayerMaterializationUnit : public MaterializationUnit {
>>>> std::unique_ptr<MemoryBuffer> O;
>>>> };
>>>>
>>>> -/// Returns a SymbolFlagsMap for the object file represented by the given
>>>> -/// buffer, or an error if the buffer does not contain a valid object file.
>>>> -// FIXME: Maybe move to Core.h?
>>>> -Expected<SymbolFlagsMap> getObjectSymbolFlags(ExecutionSession &ES,
>>>> - MemoryBufferRef ObjBuffer);
>>>> -
>>>> } // End namespace orc
>>>> } // End namespace llvm
>>>>
>>>>
>>>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/MachOPlatform.h b/llvm/include/llvm/ExecutionEngine/Orc/MachOPlatform.h
>>>> new file mode 100644
>>>> index 000000000000..92c1a04921b8
>>>> --- /dev/null
>>>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/MachOPlatform.h
>>>> @@ -0,0 +1,147 @@
>>>> +//===-- MachOPlatform.h - Utilities for executing MachO in Orc --*- C++ -*-===//
>>>> +//
>>>> +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
>>>> +// See https://llvm.org/LICENSE.txt <https://llvm.org/LICENSE.txt> for license information.
>>>> +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
>>>> +//
>>>> +//===----------------------------------------------------------------------===//
>>>> +//
>>>> +// Utilities for executing JIT'd MachO in Orc.
>>>> +//
>>>> +//===----------------------------------------------------------------------===//
>>>> +
>>>> +#ifndef LLVM_EXECUTIONENGINE_ORC_MACHOPLATFORM_H
>>>> +#define LLVM_EXECUTIONENGINE_ORC_MACHOPLATFORM_H
>>>> +
>>>> +#include "llvm/ADT/StringRef.h"
>>>> +#include "llvm/ExecutionEngine/Orc/Core.h"
>>>> +#include "llvm/ExecutionEngine/Orc/ExecutionUtils.h"
>>>> +#include "llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h"
>>>> +
>>>> +#include <future>
>>>> +#include <thread>
>>>> +#include <vector>
>>>> +
>>>> +namespace llvm {
>>>> +namespace orc {
>>>> +
>>>> +/// Enable registration of JIT'd ObjC classes and selectors.
>>>> +Error enableObjCRegistration(const char *PathToLibObjC);
>>>> +bool objCRegistrationEnabled();
>>>> +
>>>> +class MachOJITDylibInitializers {
>>>> +public:
>>>> + struct SectionExtent {
>>>> + SectionExtent() = default;
>>>> + SectionExtent(JITTargetAddress Address, uint64_t NumPtrs)
>>>> + : Address(Address), NumPtrs(NumPtrs) {}
>>>> + JITTargetAddress Address = 0;
>>>> + uint64_t NumPtrs = 0;
>>>> + };
>>>> +
>>>> + void setObjCImageInfoAddr(JITTargetAddress ObjCImageInfoAddr) {
>>>> + this->ObjCImageInfoAddr = ObjCImageInfoAddr;
>>>> + }
>>>> +
>>>> + void addModInitsSection(SectionExtent ModInit) {
>>>> + ModInitSections.push_back(std::move(ModInit));
>>>> + }
>>>> +
>>>> + void addObjCSelRefsSection(SectionExtent ObjCSelRefs) {
>>>> + ObjCSelRefsSections.push_back(std::move(ObjCSelRefs));
>>>> + }
>>>> +
>>>> + void addObjCClassListSection(SectionExtent ObjCClassList) {
>>>> + ObjCClassListSections.push_back(std::move(ObjCClassList));
>>>> + }
>>>> +
>>>> + void runModInits() const;
>>>> + void registerObjCSelectors() const;
>>>> + Error registerObjCClasses() const;
>>>> +
>>>> + void dump() const;
>>>> +
>>>> +private:
>>>> + using RawPointerSectionList = std::vector<SectionExtent>;
>>>> +
>>>> + JITTargetAddress ObjCImageInfoAddr;
>>>> + RawPointerSectionList ModInitSections;
>>>> + RawPointerSectionList ObjCSelRefsSections;
>>>> + RawPointerSectionList ObjCClassListSections;
>>>> +};
>>>> +
>>>> +class MachOJITDylibDeinitializers {};
>>>> +
>>>> +/// Mediates between MachO initialization and ExecutionSession state.
>>>> +class MachOPlatform : public Platform {
>>>> +public:
>>>> + using InitializerSequence =
>>>> + std::vector<std::pair<JITDylib *, MachOJITDylibInitializers>>;
>>>> +
>>>> + using DeinitializerSequence =
>>>> + std::vector<std::pair<JITDylib *, MachOJITDylibDeinitializers>>;
>>>> +
>>>> + MachOPlatform(ExecutionSession &ES, ObjectLinkingLayer &ObjLinkingLayer,
>>>> + std::unique_ptr<MemoryBuffer> StandardSymbolsObject);
>>>> +
>>>> + ExecutionSession &getExecutionSession() const { return ES; }
>>>> +
>>>> + Error setupJITDylib(JITDylib &JD) override;
>>>> + Error notifyAdding(JITDylib &JD, const MaterializationUnit &MU) override;
>>>> + Error notifyRemoving(JITDylib &JD, VModuleKey K) override;
>>>> +
>>>> + Expected<InitializerSequence> getInitializerSequence(JITDylib &JD);
>>>> +
>>>> + Expected<DeinitializerSequence> getDeinitializerSequence(JITDylib &JD);
>>>> +
>>>> +private:
>>>> + // This ObjectLinkingLayer plugin scans JITLink graphs for __mod_init_func,
>>>> + // __objc_classlist and __sel_ref sections and records their extents so that
>>>> + // they can be run in the target process.
>>>> + class InitScraperPlugin : public ObjectLinkingLayer::Plugin {
>>>> + public:
>>>> + InitScraperPlugin(MachOPlatform &MP) : MP(MP) {}
>>>> +
>>>> + void modifyPassConfig(MaterializationResponsibility &MR, const Triple &TT,
>>>> + jitlink::PassConfiguration &Config) override;
>>>> +
>>>> + LocalDependenciesMap getSyntheticSymbolLocalDependencies(
>>>> + MaterializationResponsibility &MR) override;
>>>> +
>>>> + private:
>>>> + using InitSymbolDepMap =
>>>> + DenseMap<MaterializationResponsibility *, JITLinkSymbolVector>;
>>>> +
>>>> + void preserveInitSectionIfPresent(JITLinkSymbolVector &Syms,
>>>> + jitlink::LinkGraph &G,
>>>> + StringRef SectionName);
>>>> +
>>>> + Error processObjCImageInfo(jitlink::LinkGraph &G,
>>>> + MaterializationResponsibility &MR);
>>>> +
>>>> + std::mutex InitScraperMutex;
>>>> + MachOPlatform &MP;
>>>> + DenseMap<JITDylib *, std::pair<uint32_t, uint32_t>> ObjCImageInfos;
>>>> + InitSymbolDepMap InitSymbolDeps;
>>>> + };
>>>> +
>>>> + static std::vector<JITDylib *> getDFSLinkOrder(JITDylib &JD);
>>>> +
>>>> + void registerInitInfo(JITDylib &JD, JITTargetAddress ObjCImageInfoAddr,
>>>> + MachOJITDylibInitializers::SectionExtent ModInits,
>>>> + MachOJITDylibInitializers::SectionExtent ObjCSelRefs,
>>>> + MachOJITDylibInitializers::SectionExtent ObjCClassList);
>>>> +
>>>> + std::mutex PlatformMutex;
>>>> + ExecutionSession &ES;
>>>> + ObjectLinkingLayer &ObjLinkingLayer;
>>>> + std::unique_ptr<MemoryBuffer> StandardSymbolsObject;
>>>> +
>>>> + DenseMap<JITDylib *, SymbolLookupSet> RegisteredInitSymbols;
>>>> + DenseMap<JITDylib *, MachOJITDylibInitializers> InitSeqs;
>>>> +};
>>>> +
>>>> +} // end namespace orc
>>>> +} // end namespace llvm
>>>> +
>>>> +#endif // LLVM_EXECUTIONENGINE_ORC_MACHOPLATFORM_H
>>>>
>>>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/Mangling.h b/llvm/include/llvm/ExecutionEngine/Orc/Mangling.h
>>>> new file mode 100644
>>>> index 000000000000..e0f770a601fb
>>>> --- /dev/null
>>>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/Mangling.h
>>>> @@ -0,0 +1,66 @@
>>>> +//===------ Mangling.h -- Name Mangling Utilities for ORC -------*- C++ -*-===//
>>>> +//
>>>> +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
>>>> +// See https://llvm.org/LICENSE.txt <https://llvm.org/LICENSE.txt> for license information.
>>>> +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
>>>> +//
>>>> +//===----------------------------------------------------------------------===//
>>>> +//
>>>> +// Name mangling utilities for ORC.
>>>> +//
>>>> +//===----------------------------------------------------------------------===//
>>>> +
>>>> +#ifndef LLVM_EXECUTIONENGINE_ORC_MANGLING_H
>>>> +#define LLVM_EXECUTIONENGINE_ORC_MANGLING_H
>>>> +
>>>> +#include "llvm/ExecutionEngine/Orc/Core.h"
>>>> +#include "llvm/ExecutionEngine/Orc/ThreadSafeModule.h"
>>>> +#include "llvm/IR/Module.h"
>>>> +#include "llvm/Support/MemoryBuffer.h"
>>>> +
>>>> +namespace llvm {
>>>> +namespace orc {
>>>> +
>>>> +/// Mangles symbol names then uniques them in the context of an
>>>> +/// ExecutionSession.
>>>> +class MangleAndInterner {
>>>> +public:
>>>> + MangleAndInterner(ExecutionSession &ES, const DataLayout &DL);
>>>> + SymbolStringPtr operator()(StringRef Name);
>>>> +
>>>> +private:
>>>> + ExecutionSession &ES;
>>>> + const DataLayout &DL;
>>>> +};
>>>> +
>>>> +/// Maps IR global values to their linker symbol names / flags.
>>>> +///
>>>> +/// This utility can be used when adding new IR globals in the JIT.
>>>> +class IRSymbolMapper {
>>>> +public:
>>>> + struct ManglingOptions {
>>>> + bool EmulatedTLS = false;
>>>> + };
>>>> +
>>>> + using SymbolNameToDefinitionMap = std::map<SymbolStringPtr, GlobalValue *>;
>>>> +
>>>> + /// Add mangled symbols for the given GlobalValues to SymbolFlags.
>>>> + /// If a SymbolToDefinitionMap pointer is supplied then it will be populated
>>>> + /// with Name-to-GlobalValue* mappings. Note that this mapping is not
>>>> + /// necessarily one-to-one: thread-local GlobalValues, for example, may
>>>> + /// produce more than one symbol, in which case the map will contain duplicate
>>>> + /// values.
>>>> + static void add(ExecutionSession &ES, const ManglingOptions &MO,
>>>> + ArrayRef<GlobalValue *> GVs, SymbolFlagsMap &SymbolFlags,
>>>> + SymbolNameToDefinitionMap *SymbolToDefinition = nullptr);
>>>> +};
>>>> +
>>>> +/// Returns a SymbolFlagsMap for the object file represented by the given
>>>> +/// buffer, or an error if the buffer does not contain a valid object file.
>>>> +Expected<std::pair<SymbolFlagsMap, SymbolStringPtr>>
>>>> +getObjectSymbolInfo(ExecutionSession &ES, MemoryBufferRef ObjBuffer);
>>>> +
>>>> +} // End namespace orc
>>>> +} // End namespace llvm
>>>> +
>>>> +#endif // LLVM_EXECUTIONENGINE_ORC_MANGLING_H
>>>>
>>>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h b/llvm/include/llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h
>>>> index 50d25f18891e..1b4c03bd2cfd 100644
>>>> --- a/llvm/include/llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h
>>>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h
>>>> @@ -35,6 +35,7 @@ namespace llvm {
>>>>
>>>> namespace jitlink {
>>>> class EHFrameRegistrar;
>>>> +class Symbol;
>>>> } // namespace jitlink
>>>>
>>>> namespace object {
>>>> @@ -59,10 +60,14 @@ class ObjectLinkingLayer : public ObjectLayer {
>>>> /// configured.
>>>> class Plugin {
>>>> public:
>>>> + using JITLinkSymbolVector = std::vector<const jitlink::Symbol *>;
>>>> + using LocalDependenciesMap = DenseMap<SymbolStringPtr, JITLinkSymbolVector>;
>>>> +
>>>> virtual ~Plugin();
>>>> virtual void modifyPassConfig(MaterializationResponsibility &MR,
>>>> const Triple &TT,
>>>> jitlink::PassConfiguration &Config) {}
>>>> +
>>>> virtual void notifyLoaded(MaterializationResponsibility &MR) {}
>>>> virtual Error notifyEmitted(MaterializationResponsibility &MR) {
>>>> return Error::success();
>>>> @@ -71,6 +76,15 @@ class ObjectLinkingLayer : public ObjectLayer {
>>>> return Error::success();
>>>> }
>>>> virtual Error notifyRemovingAllModules() { return Error::success(); }
>>>> +
>>>> + /// Return any dependencies that synthetic symbols (e.g. init symbols)
>>>> + /// have on locally scoped jitlink::Symbols. This is used by the
>>>> + /// ObjectLinkingLayer to update the dependencies for the synthetic
>>>> + /// symbols.
>>>> + virtual LocalDependenciesMap
>>>> + getSyntheticSymbolLocalDependencies(MaterializationResponsibility &MR) {
>>>> + return LocalDependenciesMap();
>>>> + }
>>>> };
>>>>
>>>> using ReturnObjectBufferFunction =
>>>>
>>>> diff --git a/llvm/include/llvm/ExecutionEngine/Orc/SymbolStringPool.h b/llvm/include/llvm/ExecutionEngine/Orc/SymbolStringPool.h
>>>> index c354f6c3559c..c8462dc3d2c8 100644
>>>> --- a/llvm/include/llvm/ExecutionEngine/Orc/SymbolStringPool.h
>>>> +++ b/llvm/include/llvm/ExecutionEngine/Orc/SymbolStringPool.h
>>>> @@ -53,6 +53,7 @@ class SymbolStringPtr {
>>>>
>>>> public:
>>>> SymbolStringPtr() = default;
>>>> + SymbolStringPtr(nullptr_t) {}
>>>> SymbolStringPtr(const SymbolStringPtr &Other)
>>>> : S(Other.S) {
>>>> if (isRealPoolEntry(S))
>>>> @@ -85,6 +86,8 @@ class SymbolStringPtr {
>>>> --S->getValue();
>>>> }
>>>>
>>>> + explicit operator bool() const { return S; }
>>>> +
>>>> StringRef operator*() const { return S->first(); }
>>>>
>>>> friend bool operator==(const SymbolStringPtr &LHS,
>>>>
>>>> diff --git a/llvm/lib/ExecutionEngine/Orc/CMakeLists.txt b/llvm/lib/ExecutionEngine/Orc/CMakeLists.txt
>>>> index e3a7a67c8a0e..7eb2742d7b53 100644
>>>> --- a/llvm/lib/ExecutionEngine/Orc/CMakeLists.txt
>>>> +++ b/llvm/lib/ExecutionEngine/Orc/CMakeLists.txt
>>>> @@ -12,6 +12,8 @@ add_llvm_component_library(LLVMOrcJIT
>>>> Legacy.cpp
>>>> Layer.cpp
>>>> LLJIT.cpp
>>>> + MachOPlatform.cpp
>>>> + Mangling.cpp
>>>> NullResolver.cpp
>>>> ObjectLinkingLayer.cpp
>>>> ObjectTransformLayer.cpp
>>>>
>>>> diff --git a/llvm/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp b/llvm/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp
>>>> index 29d18b6e4d7b..f22ae01a2080 100644
>>>> --- a/llvm/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp
>>>> +++ b/llvm/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp
>>>> @@ -7,6 +7,7 @@
>>>> //===----------------------------------------------------------------------===//
>>>>
>>>> #include "llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h"
>>>> +#include "llvm/ExecutionEngine/Orc/ExecutionUtils.h"
>>>> #include "llvm/IR/Mangler.h"
>>>> #include "llvm/IR/Module.h"
>>>>
>>>> @@ -68,18 +69,18 @@ namespace orc {
>>>> class PartitioningIRMaterializationUnit : public IRMaterializationUnit {
>>>> public:
>>>> PartitioningIRMaterializationUnit(ExecutionSession &ES,
>>>> - const ManglingOptions &MO,
>>>> + const IRSymbolMapper::ManglingOptions &MO,
>>>> ThreadSafeModule TSM, VModuleKey K,
>>>> CompileOnDemandLayer &Parent)
>>>> : IRMaterializationUnit(ES, MO, std::move(TSM), std::move(K)),
>>>> Parent(Parent) {}
>>>>
>>>> PartitioningIRMaterializationUnit(
>>>> - ThreadSafeModule TSM, SymbolFlagsMap SymbolFlags,
>>>> - SymbolNameToDefinitionMap SymbolToDefinition,
>>>> + ThreadSafeModule TSM, VModuleKey K, SymbolFlagsMap SymbolFlags,
>>>> + SymbolStringPtr InitSymbol, SymbolNameToDefinitionMap SymbolToDefinition,
>>>> CompileOnDemandLayer &Parent)
>>>> : IRMaterializationUnit(std::move(TSM), std::move(K),
>>>> - std::move(SymbolFlags),
>>>> + std::move(SymbolFlags), std::move(InitSymbol),
>>>> std::move(SymbolToDefinition)),
>>>> Parent(Parent) {}
>>>>
>>>> @@ -172,21 +173,23 @@ CompileOnDemandLayer::getPerDylibResources(JITDylib &TargetD) {
>>>> auto I = DylibResources.find(&TargetD);
>>>> if (I == DylibResources.end()) {
>>>> auto &ImplD =
>>>> - getExecutionSession().createJITDylib(TargetD.getName() + ".impl");
>>>> + getExecutionSession().createBareJITDylib(TargetD.getName() + ".impl");
>>>> + JITDylibSearchOrder NewSearchOrder;
>>>> TargetD.withSearchOrderDo(
>>>> [&](const JITDylibSearchOrder &TargetSearchOrder) {
>>>> - auto NewSearchOrder = TargetSearchOrder;
>>>> - assert(
>>>> - !NewSearchOrder.empty() &&
>>>> - NewSearchOrder.front().first == &TargetD &&
>>>> - NewSearchOrder.front().second ==
>>>> - JITDylibLookupFlags::MatchAllSymbols &&
>>>> - "TargetD must be at the front of its own search order and match "
>>>> - "non-exported symbol");
>>>> - NewSearchOrder.insert(std::next(NewSearchOrder.begin()),
>>>> - {&ImplD, JITDylibLookupFlags::MatchAllSymbols});
>>>> - ImplD.setSearchOrder(std::move(NewSearchOrder), false);
>>>> + NewSearchOrder = TargetSearchOrder;
>>>> });
>>>> +
>>>> + assert(
>>>> + !NewSearchOrder.empty() && NewSearchOrder.front().first == &TargetD &&
>>>> + NewSearchOrder.front().second == JITDylibLookupFlags::MatchAllSymbols &&
>>>> + "TargetD must be at the front of its own search order and match "
>>>> + "non-exported symbol");
>>>> + NewSearchOrder.insert(std::next(NewSearchOrder.begin()),
>>>> + {&ImplD, JITDylibLookupFlags::MatchAllSymbols});
>>>> + ImplD.setSearchOrder(NewSearchOrder, false);
>>>> + TargetD.setSearchOrder(std::move(NewSearchOrder), false);
>>>> +
>>>> PerDylibResources PDR(ImplD, BuildIndirectStubsManager());
>>>> I = DylibResources.insert(std::make_pair(&TargetD, std::move(PDR))).first;
>>>> }
>>>> @@ -251,8 +254,15 @@ void CompileOnDemandLayer::emitPartition(
>>>> auto &ES = getExecutionSession();
>>>> GlobalValueSet RequestedGVs;
>>>> for (auto &Name : R.getRequestedSymbols()) {
>>>> - assert(Defs.count(Name) && "No definition for symbol");
>>>> - RequestedGVs.insert(Defs[Name]);
>>>> + if (Name == R.getInitializerSymbol())
>>>> + TSM.withModuleDo([&](Module &M) {
>>>> + for (auto &GV : getStaticInitGVs(M))
>>>> + RequestedGVs.insert(&GV);
>>>> + });
>>>> + else {
>>>> + assert(Defs.count(Name) && "No definition for symbol");
>>>> + RequestedGVs.insert(Defs[Name]);
>>>> + }
>>>> }
>>>>
>>>> /// Perform partitioning with the context lock held, since the partition
>>>> @@ -272,7 +282,8 @@ void CompileOnDemandLayer::emitPartition(
>>>> // If the partition is empty, return the whole module to the symbol table.
>>>> if (GVsToExtract->empty()) {
>>>> R.replace(std::make_unique<PartitioningIRMaterializationUnit>(
>>>> - std::move(TSM), R.getSymbols(), std::move(Defs), *this));
>>>> + std::move(TSM), R.getVModuleKey(), R.getSymbols(),
>>>> + R.getInitializerSymbol(), std::move(Defs), *this));
>>>> return;
>>>> }
>>>>
>>>>
>>>> diff --git a/llvm/lib/ExecutionEngine/Orc/CompileUtils.cpp b/llvm/lib/ExecutionEngine/Orc/CompileUtils.cpp
>>>> index 160e5ba50311..f8efed15edea 100644
>>>> --- a/llvm/lib/ExecutionEngine/Orc/CompileUtils.cpp
>>>> +++ b/llvm/lib/ExecutionEngine/Orc/CompileUtils.cpp
>>>> @@ -24,9 +24,9 @@
>>>> namespace llvm {
>>>> namespace orc {
>>>>
>>>> -IRMaterializationUnit::ManglingOptions
>>>> +IRSymbolMapper::ManglingOptions
>>>> irManglingOptionsFromTargetOptions(const TargetOptions &Opts) {
>>>> - IRMaterializationUnit::ManglingOptions MO;
>>>> + IRSymbolMapper::ManglingOptions MO;
>>>>
>>>> MO.EmulatedTLS = Opts.EmulatedTLS;
>>>>
>>>>
>>>> diff --git a/llvm/lib/ExecutionEngine/Orc/Core.cpp b/llvm/lib/ExecutionEngine/Orc/Core.cpp
>>>> index a7af34a21926..6a0df10f7c3f 100644
>>>> --- a/llvm/lib/ExecutionEngine/Orc/Core.cpp
>>>> +++ b/llvm/lib/ExecutionEngine/Orc/Core.cpp
>>>> @@ -11,7 +11,6 @@
>>>> #include "llvm/ADT/STLExtras.h"
>>>> #include "llvm/Config/llvm-config.h"
>>>> #include "llvm/ExecutionEngine/Orc/OrcError.h"
>>>> -#include "llvm/IR/Mangler.h"
>>>> #include "llvm/Support/CommandLine.h"
>>>> #include "llvm/Support/Debug.h"
>>>> #include "llvm/Support/Format.h"
>>>> @@ -421,12 +420,6 @@ void AsynchronousSymbolQuery::detach() {
>>>> QueryRegistrations.clear();
>>>> }
>>>>
>>>> -MaterializationResponsibility::MaterializationResponsibility(
>>>> - JITDylib &JD, SymbolFlagsMap SymbolFlags, VModuleKey K)
>>>> - : JD(JD), SymbolFlags(std::move(SymbolFlags)), K(std::move(K)) {
>>>> - assert(!this->SymbolFlags.empty() && "Materializing nothing?");
>>>> -}
>>>> -
>>>> MaterializationResponsibility::~MaterializationResponsibility() {
>>>> assert(SymbolFlags.empty() &&
>>>> "All symbols should have been explicitly materialized or failed");
>>>> @@ -501,9 +494,13 @@ void MaterializationResponsibility::failMaterialization() {
>>>>
>>>> void MaterializationResponsibility::replace(
>>>> std::unique_ptr<MaterializationUnit> MU) {
>>>> +
>>>> for (auto &KV : MU->getSymbols())
>>>> SymbolFlags.erase(KV.first);
>>>>
>>>> + if (MU->getInitializerSymbol() == InitSymbol)
>>>> + InitSymbol = nullptr;
>>>> +
>>>> LLVM_DEBUG(JD.getExecutionSession().runSessionLocked([&]() {
>>>> dbgs() << "In " << JD.getName() << " replacing symbols with " << *MU
>>>> << "\n";
>>>> @@ -519,6 +516,7 @@ MaterializationResponsibility::delegate(const SymbolNameSet &Symbols,
>>>> if (NewKey == VModuleKey())
>>>> NewKey = K;
>>>>
>>>> + SymbolStringPtr DelegatedInitSymbol;
>>>> SymbolFlagsMap DelegatedFlags;
>>>>
>>>> for (auto &Name : Symbols) {
>>>> @@ -528,10 +526,14 @@ MaterializationResponsibility::delegate(const SymbolNameSet &Symbols,
>>>> "instance");
>>>>
>>>> DelegatedFlags[Name] = std::move(I->second);
>>>> + if (Name == InitSymbol)
>>>> + std::swap(InitSymbol, DelegatedInitSymbol);
>>>> +
>>>> SymbolFlags.erase(I);
>>>> }
>>>>
>>>> return MaterializationResponsibility(JD, std::move(DelegatedFlags),
>>>> + std::move(DelegatedInitSymbol),
>>>> std::move(NewKey));
>>>> }
>>>>
>>>> @@ -550,7 +552,7 @@ void MaterializationResponsibility::addDependenciesForAll(
>>>>
>>>> AbsoluteSymbolsMaterializationUnit::AbsoluteSymbolsMaterializationUnit(
>>>> SymbolMap Symbols, VModuleKey K)
>>>> - : MaterializationUnit(extractFlags(Symbols), std::move(K)),
>>>> + : MaterializationUnit(extractFlags(Symbols), nullptr, std::move(K)),
>>>> Symbols(std::move(Symbols)) {}
>>>>
>>>> StringRef AbsoluteSymbolsMaterializationUnit::getName() const {
>>>> @@ -581,7 +583,7 @@ AbsoluteSymbolsMaterializationUnit::extractFlags(const SymbolMap &Symbols) {
>>>> ReExportsMaterializationUnit::ReExportsMaterializationUnit(
>>>> JITDylib *SourceJD, JITDylibLookupFlags SourceJDLookupFlags,
>>>> SymbolAliasMap Aliases, VModuleKey K)
>>>> - : MaterializationUnit(extractFlags(Aliases), std::move(K)),
>>>> + : MaterializationUnit(extractFlags(Aliases), nullptr, std::move(K)),
>>>> SourceJD(SourceJD), SourceJDLookupFlags(SourceJDLookupFlags),
>>>> Aliases(std::move(Aliases)) {}
>>>>
>>>> @@ -972,7 +974,7 @@ void JITDylib::addDependencies(const SymbolStringPtr &Name,
>>>> // Assert that this symbol exists and has not reached the ready state
>>>> // already.
>>>> assert(OtherSymI != OtherJITDylib.Symbols.end() &&
>>>> - (OtherSymI->second.getState() != SymbolState::Ready &&
>>>> + (OtherSymI->second.getState() < SymbolState::Ready &&
>>>> "Dependency on emitted/ready symbol"));
>>>> #endif
>>>>
>>>> @@ -1101,6 +1103,7 @@ Error JITDylib::resolve(const SymbolMap &Resolved) {
>>>> Error JITDylib::emit(const SymbolFlagsMap &Emitted) {
>>>> AsynchronousSymbolQuerySet CompletedQueries;
>>>> SymbolNameSet SymbolsInErrorState;
>>>> + DenseMap<JITDylib *, SymbolNameVector> ReadySymbols;
>>>>
>>>> ES.runSessionLocked([&, this]() {
>>>> std::vector<SymbolTable::iterator> Worklist;
>>>> @@ -1145,6 +1148,7 @@ Error JITDylib::emit(const SymbolFlagsMap &Emitted) {
>>>> // dependencies) then notify any pending queries.
>>>> for (auto &KV : MI.Dependants) {
>>>> auto &DependantJD = *KV.first;
>>>> + auto &DependantJDReadySymbols = ReadySymbols[&DependantJD];
>>>> for (auto &DependantName : KV.second) {
>>>> auto DependantMII =
>>>> DependantJD.MaterializingInfos.find(DependantName);
>>>> @@ -1184,6 +1188,7 @@ Error JITDylib::emit(const SymbolFlagsMap &Emitted) {
>>>> // Since this dependant is now ready, we erase its MaterializingInfo
>>>> // and update its materializing state.
>>>> DependantSymEntry.setState(SymbolState::Ready);
>>>> + DependantJDReadySymbols.push_back(DependantName);
>>>>
>>>> for (auto &Q : DependantMI.takeQueriesMeeting(SymbolState::Ready)) {
>>>> Q->notifySymbolMetRequiredState(
>>>> @@ -1192,22 +1197,21 @@ Error JITDylib::emit(const SymbolFlagsMap &Emitted) {
>>>> CompletedQueries.insert(Q);
>>>> Q->removeQueryDependence(DependantJD, DependantName);
>>>> }
>>>> -
>>>> - DependantJD.MaterializingInfos.erase(DependantMII);
>>>> }
>>>> }
>>>> }
>>>>
>>>> + auto &ThisJDReadySymbols = ReadySymbols[this];
>>>> MI.Dependants.clear();
>>>> if (MI.UnemittedDependencies.empty()) {
>>>> SymI->second.setState(SymbolState::Ready);
>>>> + ThisJDReadySymbols.push_back(Name);
>>>> for (auto &Q : MI.takeQueriesMeeting(SymbolState::Ready)) {
>>>> Q->notifySymbolMetRequiredState(Name, SymI->second.getSymbol());
>>>> if (Q->isComplete())
>>>> CompletedQueries.insert(Q);
>>>> Q->removeQueryDependence(*this, Name);
>>>> }
>>>> - MaterializingInfos.erase(MII);
>>>> }
>>>> }
>>>> });
>>>> @@ -1882,6 +1886,57 @@ void JITDylib::transferEmittedNodeDependencies(
>>>> }
>>>> }
>>>>
>>>> +Platform::~Platform() {}
>>>> +
>>>> +Expected<DenseMap<JITDylib *, SymbolMap>> Platform::lookupInitSymbols(
>>>> + ExecutionSession &ES,
>>>> + const DenseMap<JITDylib *, SymbolLookupSet> &InitSyms) {
>>>> +
>>>> + DenseMap<JITDylib *, SymbolMap> CompoundResult;
>>>> + Error CompoundErr = Error::success();
>>>> + std::mutex LookupMutex;
>>>> + std::condition_variable CV;
>>>> + uint64_t Count = InitSyms.size();
>>>> +
>>>> + LLVM_DEBUG({
>>>> + dbgs() << "Issuing init-symbol lookup:\n";
>>>> + for (auto &KV : InitSyms)
>>>> + dbgs() << " " << KV.first->getName() << ": " << KV.second << "\n";
>>>> + });
>>>> +
>>>> + for (auto &KV : InitSyms) {
>>>> + auto *JD = KV.first;
>>>> + auto Names = std::move(KV.second);
>>>> + ES.lookup(
>>>> + LookupKind::Static,
>>>> + JITDylibSearchOrder({{JD, JITDylibLookupFlags::MatchAllSymbols}}),
>>>> + std::move(Names), SymbolState::Ready,
>>>> + [&, JD](Expected<SymbolMap> Result) {
>>>> + {
>>>> + std::lock_guard<std::mutex> Lock(LookupMutex);
>>>> + --Count;
>>>> + if (Result) {
>>>> + assert(!CompoundResult.count(JD) &&
>>>> + "Duplicate JITDylib in lookup?");
>>>> + CompoundResult[JD] = std::move(*Result);
>>>> + } else
>>>> + CompoundErr =
>>>> + joinErrors(std::move(CompoundErr), Result.takeError());
>>>> + }
>>>> + CV.notify_one();
>>>> + },
>>>> + NoDependenciesToRegister);
>>>> + }
>>>> +
>>>> + std::unique_lock<std::mutex> Lock(LookupMutex);
>>>> + CV.wait(Lock, [&] { return Count == 0 || CompoundErr; });
>>>> +
>>>> + if (CompoundErr)
>>>> + return std::move(CompoundErr);
>>>> +
>>>> + return std::move(CompoundResult);
>>>> +}
>>>> +
>>>> ExecutionSession::ExecutionSession(std::shared_ptr<SymbolStringPool> SSP)
>>>> : SSP(SSP ? std::move(SSP) : std::make_shared<SymbolStringPool>()) {
>>>> }
>>>> @@ -1895,7 +1950,7 @@ JITDylib *ExecutionSession::getJITDylibByName(StringRef Name) {
>>>> });
>>>> }
>>>>
>>>> -JITDylib &ExecutionSession::createJITDylib(std::string Name) {
>>>> +JITDylib &ExecutionSession::createBareJITDylib(std::string Name) {
>>>> assert(!getJITDylibByName(Name) && "JITDylib with that name already exists");
>>>> return runSessionLocked([&, this]() -> JITDylib & {
>>>> JDs.push_back(
>>>> @@ -1904,6 +1959,14 @@ JITDylib &ExecutionSession::createJITDylib(std::string Name) {
>>>> });
>>>> }
>>>>
>>>> +Expected<JITDylib &> ExecutionSession::createJITDylib(std::string Name) {
>>>> + auto &JD = createBareJITDylib(Name);
>>>> + if (P)
>>>> + if (auto Err = P->setupJITDylib(JD))
>>>> + return std::move(Err);
>>>> + return JD;
>>>> +}
>>>> +
>>>> void ExecutionSession::legacyFailQuery(AsynchronousSymbolQuery &Q, Error Err) {
>>>> assert(!!Err && "Error should be in failure state");
>>>>
>>>> @@ -2144,11 +2207,11 @@ ExecutionSession::lookup(const JITDylibSearchOrder &SearchOrder,
>>>>
>>>> Expected<JITEvaluatedSymbol>
>>>> ExecutionSession::lookup(const JITDylibSearchOrder &SearchOrder,
>>>> - SymbolStringPtr Name) {
>>>> + SymbolStringPtr Name, SymbolState RequiredState) {
>>>> SymbolLookupSet Names({Name});
>>>>
>>>> if (auto ResultMap = lookup(SearchOrder, std::move(Names), LookupKind::Static,
>>>> - SymbolState::Ready, NoDependenciesToRegister)) {
>>>> + RequiredState, NoDependenciesToRegister)) {
>>>> assert(ResultMap->size() == 1 && "Unexpected number of results");
>>>> assert(ResultMap->count(Name) && "Missing result for symbol");
>>>> return std::move(ResultMap->begin()->second);
>>>> @@ -2157,14 +2220,15 @@ ExecutionSession::lookup(const JITDylibSearchOrder &SearchOrder,
>>>> }
>>>>
>>>> Expected<JITEvaluatedSymbol>
>>>> -ExecutionSession::lookup(ArrayRef<JITDylib *> SearchOrder,
>>>> - SymbolStringPtr Name) {
>>>> - return lookup(makeJITDylibSearchOrder(SearchOrder), Name);
>>>> +ExecutionSession::lookup(ArrayRef<JITDylib *> SearchOrder, SymbolStringPtr Name,
>>>> + SymbolState RequiredState) {
>>>> + return lookup(makeJITDylibSearchOrder(SearchOrder), Name, RequiredState);
>>>> }
>>>>
>>>> Expected<JITEvaluatedSymbol>
>>>> -ExecutionSession::lookup(ArrayRef<JITDylib *> SearchOrder, StringRef Name) {
>>>> - return lookup(SearchOrder, intern(Name));
>>>> +ExecutionSession::lookup(ArrayRef<JITDylib *> SearchOrder, StringRef Name,
>>>> + SymbolState RequiredState) {
>>>> + return lookup(SearchOrder, intern(Name), RequiredState);
>>>> }
>>>>
>>>> void ExecutionSession::dump(raw_ostream &OS) {
>>>> @@ -2195,17 +2259,5 @@ void ExecutionSession::runOutstandingMUs() {
>>>> }
>>>> }
>>>>
>>>> -MangleAndInterner::MangleAndInterner(ExecutionSession &ES, const DataLayout &DL)
>>>> - : ES(ES), DL(DL) {}
>>>> -
>>>> -SymbolStringPtr MangleAndInterner::operator()(StringRef Name) {
>>>> - std::string MangledName;
>>>> - {
>>>> - raw_string_ostream MangledNameStream(MangledName);
>>>> - Mangler::getNameWithPrefix(MangledNameStream, Name, DL);
>>>> - }
>>>> - return ES.intern(MangledName);
>>>> -}
>>>> -
>>>> } // End namespace orc.
>>>> } // End namespace llvm.
>>>>
>>>> diff --git a/llvm/lib/ExecutionEngine/Orc/ExecutionUtils.cpp b/llvm/lib/ExecutionEngine/Orc/ExecutionUtils.cpp
>>>> index 3d97fe9eeab1..a98445a2295e 100644
>>>> --- a/llvm/lib/ExecutionEngine/Orc/ExecutionUtils.cpp
>>>> +++ b/llvm/lib/ExecutionEngine/Orc/ExecutionUtils.cpp
>>>> @@ -113,6 +113,26 @@ iterator_range<CtorDtorIterator> getDestructors(const Module &M) {
>>>> CtorDtorIterator(DtorsList, true));
>>>> }
>>>>
>>>> +bool StaticInitGVIterator::isStaticInitGlobal(GlobalValue &GV) {
>>>> + if (GV.isDeclaration())
>>>> + return false;
>>>> +
>>>> + if (GV.hasName() && (GV.getName() == "llvm.global_ctors" ||
>>>> + GV.getName() == "llvm.global_dtors"))
>>>> + return true;
>>>> +
>>>> + if (ObjFmt == Triple::MachO) {
>>>> + // FIXME: These section checks are too strict: We should match first and
>>>> + // second word split by comma.
>>>> + if (GV.hasSection() &&
>>>> + (GV.getSection().startswith("__DATA,__objc_classlist") ||
>>>> + GV.getSection().startswith("__DATA,__objc_selrefs")))
>>>> + return true;
>>>> + }
>>>> +
>>>> + return false;
>>>> +}
>>>> +
>>>> void CtorDtorRunner::add(iterator_range<CtorDtorIterator> CtorDtors) {
>>>> if (CtorDtors.empty())
>>>> return;
>>>> @@ -198,6 +218,30 @@ Error LocalCXXRuntimeOverrides::enable(JITDylib &JD,
>>>> return JD.define(absoluteSymbols(std::move(RuntimeInterposes)));
>>>> }
>>>>
>>>> +void ItaniumCXAAtExitSupport::registerAtExit(void (*F)(void *), void *Ctx,
>>>> + void *DSOHandle) {
>>>> + std::lock_guard<std::mutex> Lock(AtExitsMutex);
>>>> + AtExitRecords[DSOHandle].push_back({F, Ctx});
>>>> +}
>>>> +
>>>> +void ItaniumCXAAtExitSupport::runAtExits(void *DSOHandle) {
>>>> + std::vector<AtExitRecord> AtExitsToRun;
>>>> +
>>>> + {
>>>> + std::lock_guard<std::mutex> Lock(AtExitsMutex);
>>>> + auto I = AtExitRecords.find(DSOHandle);
>>>> + if (I != AtExitRecords.end()) {
>>>> + AtExitsToRun = std::move(I->second);
>>>> + AtExitRecords.erase(I);
>>>> + }
>>>> + }
>>>> +
>>>> + while (!AtExitsToRun.empty()) {
>>>> + AtExitsToRun.back().F(AtExitsToRun.back().Ctx);
>>>> + AtExitsToRun.pop_back();
>>>> + }
>>>> +}
>>>> +
>>>> DynamicLibrarySearchGenerator::DynamicLibrarySearchGenerator(
>>>> sys::DynamicLibrary Dylib, char GlobalPrefix, SymbolPredicate Allow)
>>>> : Dylib(std::move(Dylib)), Allow(std::move(Allow)),
>>>>
>>>> diff --git a/llvm/lib/ExecutionEngine/Orc/IndirectionUtils.cpp b/llvm/lib/ExecutionEngine/Orc/IndirectionUtils.cpp
>>>> index 1ac9a58aeaef..1e991d8f58f2 100644
>>>> --- a/llvm/lib/ExecutionEngine/Orc/IndirectionUtils.cpp
>>>> +++ b/llvm/lib/ExecutionEngine/Orc/IndirectionUtils.cpp
>>>> @@ -28,7 +28,7 @@ class CompileCallbackMaterializationUnit : public orc::MaterializationUnit {
>>>> CompileCallbackMaterializationUnit(SymbolStringPtr Name,
>>>> CompileFunction Compile, VModuleKey K)
>>>> : MaterializationUnit(SymbolFlagsMap({{Name, JITSymbolFlags::Exported}}),
>>>> - std::move(K)),
>>>> + nullptr, std::move(K)),
>>>> Name(std::move(Name)), Compile(std::move(Compile)) {}
>>>>
>>>> StringRef getName() const override { return "<Compile Callbacks>"; }
>>>>
>>>> diff --git a/llvm/lib/ExecutionEngine/Orc/LLJIT.cpp b/llvm/lib/ExecutionEngine/Orc/LLJIT.cpp
>>>> index 4218ca4e481f..5b2f86823355 100644
>>>> --- a/llvm/lib/ExecutionEngine/Orc/LLJIT.cpp
>>>> +++ b/llvm/lib/ExecutionEngine/Orc/LLJIT.cpp
>>>> @@ -8,15 +8,827 @@
>>>>
>>>> #include "llvm/ExecutionEngine/Orc/LLJIT.h"
>>>> #include "llvm/ExecutionEngine/JITLink/JITLinkMemoryManager.h"
>>>> +#include "llvm/ExecutionEngine/Orc/MachOPlatform.h"
>>>> #include "llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h"
>>>> #include "llvm/ExecutionEngine/Orc/OrcError.h"
>>>> #include "llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h"
>>>> #include "llvm/ExecutionEngine/SectionMemoryManager.h"
>>>> +#include "llvm/IR/GlobalVariable.h"
>>>> +#include "llvm/IR/IRBuilder.h"
>>>> #include "llvm/IR/Mangler.h"
>>>> +#include "llvm/IR/Module.h"
>>>> +#include "llvm/Support/DynamicLibrary.h"
>>>> +
>>>> +#include <map>
>>>> +
>>>> +using namespace llvm;
>>>> +using namespace llvm::orc;
>>>> +
>>>> +namespace {
>>>> +
>>>> +/// Add a reference to the __dso_handle global to the given module.
>>>> +/// Returns a reference to the __dso_handle IR decl.
>>>> +GlobalVariable *addDSOHandleDecl(Module &M) {
>>>> + auto DSOHandleTy = StructType::create(M.getContext(), "lljit.dso_handle");
>>>> + return new GlobalVariable(M, DSOHandleTy, true, GlobalValue::ExternalLinkage,
>>>> + nullptr, "__dso_handle");
>>>> +}
>>>> +
>>>> +/// Adds helper function decls and wrapper functions that call the helper with
>>>> +/// some additional prefix arguments.
>>>> +///
>>>> +/// E.g. For wrapper "foo" with type i8(i8, i64), helper "bar", and prefix
>>>> +/// args i32 4 and i16 12345, this function will add:
>>>> +///
>>>> +/// declare i8 @bar(i32, i16, i8, i64)
>>>> +///
>>>> +/// define i8 @foo(i8, i64) {
>>>> +/// entry:
>>>> +/// %2 = call i8 @bar(i32 4, i16 12345, i8 %0, i64 %1)
>>>> +/// ret i8 %2
>>>> +/// }
>>>> +///
>>>> +Function *addHelperAndWrapper(Module &M, StringRef WrapperName,
>>>> + FunctionType *WrapperFnType,
>>>> + GlobalValue::VisibilityTypes WrapperVisibility,
>>>> + StringRef HelperName,
>>>> + ArrayRef<Value *> HelperPrefixArgs) {
>>>> + std::vector<Type *> HelperArgTypes;
>>>> + for (auto *Arg : HelperPrefixArgs)
>>>> + HelperArgTypes.push_back(Arg->getType());
>>>> + for (auto *T : WrapperFnType->params())
>>>> + HelperArgTypes.push_back(T);
>>>> + auto *HelperFnType =
>>>> + FunctionType::get(WrapperFnType->getReturnType(), HelperArgTypes, false);
>>>> + auto *HelperFn = Function::Create(HelperFnType, GlobalValue::ExternalLinkage,
>>>> + HelperName, M);
>>>> +
>>>> + auto *WrapperFn = Function::Create(
>>>> + WrapperFnType, GlobalValue::ExternalLinkage, WrapperName, M);
>>>> + WrapperFn->setVisibility(WrapperVisibility);
>>>> +
>>>> + auto *EntryBlock = BasicBlock::Create(M.getContext(), "entry", WrapperFn);
>>>> + IRBuilder<> IB(EntryBlock);
>>>> +
>>>> + std::vector<Value *> HelperArgs;
>>>> + for (auto *Arg : HelperPrefixArgs)
>>>> + HelperArgs.push_back(Arg);
>>>> + for (auto &Arg : WrapperFn->args())
>>>> + HelperArgs.push_back(&Arg);
>>>> + auto *HelperResult = IB.CreateCall(HelperFn, HelperArgs);
>>>> + if (HelperFn->getReturnType()->isVoidTy())
>>>> + IB.CreateRetVoid();
>>>> + else
>>>> + IB.CreateRet(HelperResult);
>>>> +
>>>> + return WrapperFn;
>>>> +}
>>>> +
>>>> +class GenericLLVMIRPlatformSupport;
>>>> +
>>>> +/// orc::Platform component of Generic LLVM IR Platform support.
>>>> +/// Just forwards calls to the GenericLLVMIRPlatformSupport class below.
>>>> +class GenericLLVMIRPlatform : public Platform {
>>>> +public:
>>>> + GenericLLVMIRPlatform(GenericLLVMIRPlatformSupport &S) : S(S) {}
>>>> + Error setupJITDylib(JITDylib &JD) override;
>>>> + Error notifyAdding(JITDylib &JD, const MaterializationUnit &MU) override;
>>>> + Error notifyRemoving(JITDylib &JD, VModuleKey K) override {
>>>> + // Noop -- Nothing to do (yet).
>>>> + return Error::success();
>>>> + }
>>>> +
>>>> +private:
>>>> + GenericLLVMIRPlatformSupport &S;
>>>> +};
>>>> +
>>>> +/// This transform parses llvm.global_ctors to produce a single initialization
>>>> +/// function for the module, records the function, then deletes
>>>> +/// llvm.global_ctors.
>>>> +class GlobalCtorDtorScraper {
>>>> +public:
>>>> + GlobalCtorDtorScraper(GenericLLVMIRPlatformSupport &PS) : PS(PS) {}
>>>> + Expected<ThreadSafeModule> operator()(ThreadSafeModule TSM,
>>>> + MaterializationResponsibility &R);
>>>> +
>>>> +private:
>>>> + GenericLLVMIRPlatformSupport &PS;
>>>> +};
>>>> +
>>>> +/// Generic IR Platform Support
>>>> +///
>>>> +/// Scrapes llvm.global_ctors and llvm.global_dtors and replaces them with
>>>> +/// specially named 'init' and 'deinit'. Injects definitions / interposes for
>>>> +/// some runtime API, including __cxa_atexit, dlopen, and dlclose.
>>>> +class GenericLLVMIRPlatformSupport : public LLJIT::PlatformSupport {
>>>> +public:
>>>> + // GenericLLVMIRPlatform &P) : P(P) {
>>>> + GenericLLVMIRPlatformSupport(LLJIT &J) : J(J) {
>>>> +
>>>> + getExecutionSession().setPlatform(
>>>> + std::make_unique<GenericLLVMIRPlatform>(*this));
>>>> +
>>>> + setInitTransform(J, GlobalCtorDtorScraper(*this));
>>>> +
>>>> + MangleAndInterner Mangle(getExecutionSession(), J.getDataLayout());
>>>> + SymbolMap StdInterposes;
>>>> +
>>>> + StdInterposes[Mangle("__lljit.platform_support_instance")] =
>>>> + JITEvaluatedSymbol(pointerToJITTargetAddress(this), JITSymbolFlags());
>>>> + StdInterposes[Mangle("__lljit.cxa_atexit_helper")] = JITEvaluatedSymbol(
>>>> + pointerToJITTargetAddress(registerAtExitHelper), JITSymbolFlags());
>>>> + StdInterposes[Mangle("__lljit.run_atexits_helper")] = JITEvaluatedSymbol(
>>>> + pointerToJITTargetAddress(runAtExitsHelper), JITSymbolFlags());
>>>> +
>>>> + cantFail(
>>>> + J.getMainJITDylib().define(absoluteSymbols(std::move(StdInterposes))));
>>>> + cantFail(setupJITDylib(J.getMainJITDylib()));
>>>> + cantFail(J.addIRModule(J.getMainJITDylib(), createPlatformRuntimeModule()));
>>>> + }
>>>> +
>>>> + ExecutionSession &getExecutionSession() { return J.getExecutionSession(); }
>>>> +
>>>> + /// Adds a module that defines the __dso_handle global.
>>>> + Error setupJITDylib(JITDylib &JD) {
>>>> + auto Ctx = std::make_unique<LLVMContext>();
>>>> + auto M = std::make_unique<Module>("__standard_lib", *Ctx);
>>>> + M->setDataLayout(J.getDataLayout());
>>>> +
>>>> + auto *Int64Ty = Type::getInt64Ty(*Ctx);
>>>> + auto *DSOHandle = new GlobalVariable(
>>>> + *M, Int64Ty, true, GlobalValue::ExternalLinkage,
>>>> + ConstantInt::get(Int64Ty, reinterpret_cast<uintptr_t>(&JD)),
>>>> + "__dso_handle");
>>>> + DSOHandle->setVisibility(GlobalValue::HiddenVisibility);
>>>> + DSOHandle->setInitializer(
>>>> + ConstantInt::get(Int64Ty, pointerToJITTargetAddress(&JD)));
>>>> + return J.addIRModule(JD, ThreadSafeModule(std::move(M), std::move(Ctx)));
>>>> + }
>>>> +
>>>> + Error notifyAdding(JITDylib &JD, const MaterializationUnit &MU) {
>>>> + std::lock_guard<std::mutex> Lock(PlatformSupportMutex);
>>>> + if (auto &InitSym = MU.getInitializerSymbol())
>>>> + InitSymbols[&JD].add(InitSym);
>>>> + return Error::success();
>>>> + }
>>>> +
>>>> + Error initialize(JITDylib &JD) override {
>>>> + if (auto Initializers = getInitializers(JD)) {
>>>> + for (auto InitFnAddr : *Initializers) {
>>>> + auto *InitFn = jitTargetAddressToFunction<void (*)()>(InitFnAddr);
>>>> + InitFn();
>>>> + }
>>>> + } else
>>>> + return Initializers.takeError();
>>>> + return Error::success();
>>>> + }
>>>> +
>>>> + Error deinitialize(JITDylib &JD) override {
>>>> + if (auto Deinitializers = getDeinitializers(JD)) {
>>>> + for (auto DeinitFnAddr : *Deinitializers) {
>>>> + auto *DeinitFn = jitTargetAddressToFunction<void (*)()>(DeinitFnAddr);
>>>> + DeinitFn();
>>>> + }
>>>> + } else
>>>> + return Deinitializers.takeError();
>>>> +
>>>> + return Error::success();
>>>> + }
>>>> +
>>>> + void registerInitFunc(JITDylib &JD, SymbolStringPtr InitName) {
>>>> + std::lock_guard<std::mutex> Lock(PlatformSupportMutex);
>>>> + InitFunctions[&JD].add(InitName);
>>>> + }
>>>> +
>>>> +private:
>>>> + Expected<std::vector<JITTargetAddress>> getInitializers(JITDylib &JD) {
>>>> + if (auto Err = issueInitLookups(JD))
>>>> + return std::move(Err);
>>>> +
>>>> + DenseMap<JITDylib *, SymbolLookupSet> LookupSymbols;
>>>> + std::vector<JITDylib *> DFSLinkOrder;
>>>> +
>>>> + {
>>>> + std::lock_guard<std::mutex> Lock(PlatformSupportMutex);
>>>> + DFSLinkOrder = getDFSLinkOrder(JD);
>>>> +
>>>> + for (auto *NextJD : DFSLinkOrder) {
>>>> + auto IFItr = InitFunctions.find(NextJD);
>>>> + if (IFItr != InitFunctions.end()) {
>>>> + LookupSymbols[NextJD] = std::move(IFItr->second);
>>>> + InitFunctions.erase(IFItr);
>>>> + }
>>>> + }
>>>> + }
>>>> +
>>>> + auto &ES = getExecutionSession();
>>>> + auto LookupResult = Platform::lookupInitSymbols(ES, LookupSymbols);
>>>> +
>>>> + if (!LookupResult)
>>>> + return LookupResult.takeError();
>>>> +
>>>> + std::vector<JITTargetAddress> Initializers;
>>>> + while (!DFSLinkOrder.empty()) {
>>>> + auto &NextJD = *DFSLinkOrder.back();
>>>> + DFSLinkOrder.pop_back();
>>>> + auto InitsItr = LookupResult->find(&NextJD);
>>>> + if (InitsItr == LookupResult->end())
>>>> + continue;
>>>> + for (auto &KV : InitsItr->second)
>>>> + Initializers.push_back(KV.second.getAddress());
>>>> + }
>>>> +
>>>> + return Initializers;
>>>> + }
>>>> +
>>>> + Expected<std::vector<JITTargetAddress>> getDeinitializers(JITDylib &JD) {
>>>> + auto &ES = getExecutionSession();
>>>> +
>>>> + MangleAndInterner Mangle(getExecutionSession(), J.getDataLayout());
>>>> + auto LLJITRunAtExits = Mangle("__lljit_run_atexits");
>>>> +
>>>> + DenseMap<JITDylib *, SymbolLookupSet> LookupSymbols;
>>>> + std::vector<JITDylib *> DFSLinkOrder;
>>>> +
>>>> + {
>>>> + std::lock_guard<std::mutex> Lock(PlatformSupportMutex);
>>>> + DFSLinkOrder = getDFSLinkOrder(JD);
>>>> +
>>>> + for (auto *NextJD : DFSLinkOrder) {
>>>> + auto &JDLookupSymbols = LookupSymbols[NextJD];
>>>> + auto DIFItr = DeInitFunctions.find(NextJD);
>>>> + if (DIFItr != DeInitFunctions.end()) {
>>>> + LookupSymbols[NextJD] = std::move(DIFItr->second);
>>>> + DeInitFunctions.erase(DIFItr);
>>>> + }
>>>> + JDLookupSymbols.add(LLJITRunAtExits,
>>>> + SymbolLookupFlags::WeaklyReferencedSymbol);
>>>> + }
>>>> + }
>>>> +
>>>> + auto LookupResult = Platform::lookupInitSymbols(ES, LookupSymbols);
>>>> +
>>>> + if (!LookupResult)
>>>> + return LookupResult.takeError();
>>>> +
>>>> + std::vector<JITTargetAddress> DeInitializers;
>>>> + for (auto *NextJD : DFSLinkOrder) {
>>>> + auto DeInitsItr = LookupResult->find(NextJD);
>>>> + assert(DeInitsItr != LookupResult->end() &&
>>>> + "Every JD should have at least __lljit_run_atexits");
>>>> +
>>>> + auto RunAtExitsItr = DeInitsItr->second.find(LLJITRunAtExits);
>>>> + if (RunAtExitsItr != DeInitsItr->second.end())
>>>> + DeInitializers.push_back(RunAtExitsItr->second.getAddress());
>>>> +
>>>> + for (auto &KV : DeInitsItr->second)
>>>> + if (KV.first != LLJITRunAtExits)
>>>> + DeInitializers.push_back(KV.second.getAddress());
>>>> + }
>>>> +
>>>> + return DeInitializers;
>>>> + }
>>>> +
>>>> + // Returns a DFS traversal order of the JITDylibs reachable (via
>>>> + // links-against edges) from JD, starting with JD itself.
>>>> + static std::vector<JITDylib *> getDFSLinkOrder(JITDylib &JD) {
>>>> + std::vector<JITDylib *> DFSLinkOrder;
>>>> + std::vector<JITDylib *> WorkStack({&JD});
>>>> + DenseSet<JITDylib *> Visited;
>>>> +
>>>> + while (!WorkStack.empty()) {
>>>> + auto &NextJD = *WorkStack.back();
>>>> + WorkStack.pop_back();
>>>> + if (Visited.count(&NextJD))
>>>> + continue;
>>>> + Visited.insert(&NextJD);
>>>> + DFSLinkOrder.push_back(&NextJD);
>>>> + NextJD.withSearchOrderDo([&](const JITDylibSearchOrder &SearchOrder) {
>>>> + for (auto &KV : SearchOrder)
>>>> + WorkStack.push_back(KV.first);
>>>> + });
>>>> + }
>>>> +
>>>> + return DFSLinkOrder;
>>>> + }
>>>> +
>>>> + /// Issue lookups for all init symbols required to initialize JD (and any
>>>> + /// JITDylibs that it depends on).
>>>> + Error issueInitLookups(JITDylib &JD) {
>>>> + DenseMap<JITDylib *, SymbolLookupSet> RequiredInitSymbols;
>>>> +
>>>> + {
>>>> + std::lock_guard<std::mutex> Lock(PlatformSupportMutex);
>>>> +
>>>> + auto DFSLinkOrder = getDFSLinkOrder(JD);
>>>> +
>>>> + for (auto *NextJD : DFSLinkOrder) {
>>>> + auto ISItr = InitSymbols.find(NextJD);
>>>> + if (ISItr != InitSymbols.end()) {
>>>> + RequiredInitSymbols[NextJD] = std::move(ISItr->second);
>>>> + InitSymbols.erase(ISItr);
>>>> + }
>>>> + }
>>>> + }
>>>> +
>>>> + return Platform::lookupInitSymbols(getExecutionSession(),
>>>> + RequiredInitSymbols)
>>>> + .takeError();
>>>> + }
>>>> +
>>>> + static void registerAtExitHelper(void *Self, void (*F)(void *), void *Ctx,
>>>> + void *DSOHandle) {
>>>> + static_cast<GenericLLVMIRPlatformSupport *>(Self)->AtExitMgr.registerAtExit(
>>>> + F, Ctx, DSOHandle);
>>>> + }
>>>> +
>>>> + static void runAtExitsHelper(void *Self, void *DSOHandle) {
>>>> + static_cast<GenericLLVMIRPlatformSupport *>(Self)->AtExitMgr.runAtExits(
>>>> + DSOHandle);
>>>> + }
>>>> +
>>>> + // Constructs an LLVM IR module containing platform runtime globals,
>>>> + // functions, and interposes.
>>>> + ThreadSafeModule createPlatformRuntimeModule() {
>>>> + auto Ctx = std::make_unique<LLVMContext>();
>>>> + auto M = std::make_unique<Module>("__standard_lib", *Ctx);
>>>> + M->setDataLayout(J.getDataLayout());
>>>> +
>>>> + auto *GenericIRPlatformSupportTy =
>>>> + StructType::create(*Ctx, "lljit.GenericLLJITIRPlatformSupport");
>>>> +
>>>> + auto *PlatformInstanceDecl = new GlobalVariable(
>>>> + *M, GenericIRPlatformSupportTy, true, GlobalValue::ExternalLinkage,
>>>> + nullptr, "__lljit.platform_support_instance");
>>>> +
>>>> + auto *DSOHandleDecl = addDSOHandleDecl(*M);
>>>> +
>>>> + auto *Int8Ty = Type::getInt8Ty(*Ctx);
>>>> + auto *IntTy = Type::getIntNTy(*Ctx, sizeof(int) * CHAR_BIT);
>>>> + auto *VoidTy = Type::getVoidTy(*Ctx);
>>>> + auto *BytePtrTy = PointerType::getUnqual(Int8Ty);
>>>> + auto *AtExitCallbackTy = FunctionType::get(VoidTy, {BytePtrTy}, false);
>>>> + auto *AtExitCallbackPtrTy = PointerType::getUnqual(AtExitCallbackTy);
>>>> +
>>>> + addHelperAndWrapper(
>>>> + *M, "__cxa_atexit",
>>>> + FunctionType::get(IntTy, {AtExitCallbackPtrTy, BytePtrTy, BytePtrTy},
>>>> + false),
>>>> + GlobalValue::HiddenVisibility, "__lljit.cxa_atexit_helper",
>>>> + {PlatformInstanceDecl});
>>>> +
>>>> + addHelperAndWrapper(
>>>> + *M, "__lljit_run_atexits", FunctionType::get(VoidTy, {}, false),
>>>> + GlobalValue::HiddenVisibility, "__lljit.run_atexits_helper",
>>>> + {PlatformInstanceDecl, DSOHandleDecl});
>>>> +
>>>> + return ThreadSafeModule(std::move(M), std::move(Ctx));
>>>> + }
>>>> +
>>>> + std::mutex PlatformSupportMutex;
>>>> + LLJIT &J;
>>>> + DenseMap<JITDylib *, SymbolLookupSet> InitSymbols;
>>>> + DenseMap<JITDylib *, SymbolLookupSet> InitFunctions;
>>>> + DenseMap<JITDylib *, SymbolLookupSet> DeInitFunctions;
>>>> + ItaniumCXAAtExitSupport AtExitMgr;
>>>> +};
>>>> +
>>>> +Error GenericLLVMIRPlatform::setupJITDylib(JITDylib &JD) {
>>>> + return S.setupJITDylib(JD);
>>>> +}
>>>> +
>>>> +Error GenericLLVMIRPlatform::notifyAdding(JITDylib &JD,
>>>> + const MaterializationUnit &MU) {
>>>> + return S.notifyAdding(JD, MU);
>>>> +}
>>>> +
>>>> +Expected<ThreadSafeModule>
>>>> +GlobalCtorDtorScraper::operator()(ThreadSafeModule TSM,
>>>> + MaterializationResponsibility &R) {
>>>> + auto Err = TSM.withModuleDo([&](Module &M) -> Error {
>>>> + auto &Ctx = M.getContext();
>>>> + auto *GlobalCtors = M.getNamedGlobal("llvm.global_ctors");
>>>> +
>>>> + // If there's no llvm.global_ctors or it's just a decl then skip.
>>>> + if (!GlobalCtors || GlobalCtors->isDeclaration())
>>>> + return Error::success();
>>>> +
>>>> + std::string InitFunctionName;
>>>> + raw_string_ostream(InitFunctionName)
>>>> + << "__orc_init." << M.getModuleIdentifier();
>>>> +
>>>> + MangleAndInterner Mangle(PS.getExecutionSession(), M.getDataLayout());
>>>> + auto InternedName = Mangle(InitFunctionName);
>>>> + if (auto Err =
>>>> + R.defineMaterializing({{InternedName, JITSymbolFlags::Callable}}))
>>>> + return Err;
>>>> +
>>>> + auto *InitFunc =
>>>> + Function::Create(FunctionType::get(Type::getVoidTy(Ctx), {}, false),
>>>> + GlobalValue::ExternalLinkage, InitFunctionName, &M);
>>>> + InitFunc->setVisibility(GlobalValue::HiddenVisibility);
>>>> + std::vector<std::pair<Function *, unsigned>> Inits;
>>>> + for (auto E : getConstructors(M))
>>>> + Inits.push_back(std::make_pair(E.Func, E.Priority));
>>>> + llvm::sort(Inits, [](const std::pair<Function *, unsigned> &LHS,
>>>> + const std::pair<Function *, unsigned> &RHS) {
>>>> + return LHS.first < RHS.first;
>>>> + });
>>>> + auto *EntryBlock = BasicBlock::Create(Ctx, "entry", InitFunc);
>>>> + IRBuilder<> IB(EntryBlock);
>>>> + for (auto &KV : Inits)
>>>> + IB.CreateCall(KV.first);
>>>> + IB.CreateRetVoid();
>>>> +
>>>> + PS.registerInitFunc(R.getTargetJITDylib(), InternedName);
>>>> + GlobalCtors->eraseFromParent();
>>>> + return Error::success();
>>>> + });
>>>> +
>>>> + if (Err)
>>>> + return std::move(Err);
>>>> +
>>>> + return TSM;
>>>> +}
>>>> +
>>>> +class MachOPlatformSupport : public LLJIT::PlatformSupport {
>>>> +public:
>>>> + using DLOpenType = void *(*)(const char *Name, int Mode);
>>>> + using DLCloseType = int (*)(void *Handle);
>>>> + using DLSymType = void *(*)(void *Handle, const char *Name);
>>>> + using DLErrorType = const char *(*)();
>>>> +
>>>> + struct DlFcnValues {
>>>> + Optional<void *> RTLDDefault;
>>>> + DLOpenType dlopen = nullptr;
>>>> + DLCloseType dlclose = nullptr;
>>>> + DLSymType dlsym = nullptr;
>>>> + DLErrorType dlerror = nullptr;
>>>> + };
>>>> +
>>>> + static Expected<std::unique_ptr<MachOPlatformSupport>>
>>>> + Create(LLJIT &J, JITDylib &PlatformJITDylib) {
>>>> +
>>>> + // Make process symbols visible.
>>>> + {
>>>> + std::string ErrMsg;
>>>> + auto Lib = sys::DynamicLibrary::getPermanentLibrary(nullptr, &ErrMsg);
>>>> + if (!Lib.isValid())
>>>> + return make_error<StringError>(std::move(ErrMsg),
>>>> + inconvertibleErrorCode());
>>>> + }
>>>> +
>>>> + DlFcnValues DlFcn;
>>>> +
>>>> + // Add support for RTLDDefault on known platforms.
>>>> +#ifdef __APPLE__
>>>> + DlFcn.RTLDDefault = reinterpret_cast<void *>(-2);
>>>> +#endif // __APPLE__
>>>> +
>>>> + if (auto Err = hookUpFunction(DlFcn.dlopen, "dlopen"))
>>>> + return std::move(Err);
>>>> + if (auto Err = hookUpFunction(DlFcn.dlclose, "dlclose"))
>>>> + return std::move(Err);
>>>> + if (auto Err = hookUpFunction(DlFcn.dlsym, "dlsym"))
>>>> + return std::move(Err);
>>>> + if (auto Err = hookUpFunction(DlFcn.dlerror, "dlerror"))
>>>> + return std::move(Err);
>>>> +
>>>> + std::unique_ptr<MachOPlatformSupport> MP(
>>>> + new MachOPlatformSupport(J, PlatformJITDylib, DlFcn));
>>>> + return std::move(MP);
>>>> + }
>>>> +
>>>> + Error initialize(JITDylib &JD) override {
>>>> + if (auto InitSeq = MP.getInitializerSequence(JD)) {
>>>> + for (auto &KV : *InitSeq) {
>>>> + KV.second.registerObjCSelectors();
>>>> + if (auto Err = KV.second.registerObjCClasses()) {
>>>> + // FIXME: Roll back registrations on error?
>>>> + return Err;
>>>> + }
>>>> + }
>>>> + for (auto &KV : *InitSeq)
>>>> + KV.second.runModInits();
>>>> + } else
>>>> + return InitSeq.takeError();
>>>> + return Error::success();
>>>> + }
>>>> +
>>>> + Error deinitialize(JITDylib &JD) override {
>>>> + auto &ES = J.getExecutionSession();
>>>> + if (auto DeinitSeq = MP.getDeinitializerSequence(JD)) {
>>>> + for (auto &KV : *DeinitSeq) {
>>>> + auto DSOHandleName = ES.intern("___dso_handle");
>>>> +
>>>> + // FIXME: Run DeInits here.
>>>> + auto Result = ES.lookup(
>>>> + {{KV.first, JITDylibLookupFlags::MatchAllSymbols}},
>>>> + SymbolLookupSet(DSOHandleName,
>>>> + SymbolLookupFlags::WeaklyReferencedSymbol));
>>>> + if (!Result)
>>>> + return Result.takeError();
>>>> + if (Result->empty())
>>>> + continue;
>>>> + assert(Result->count(DSOHandleName) &&
>>>> + "Result does not contain __dso_handle");
>>>> + auto *DSOHandle = jitTargetAddressToPointer<void *>(
>>>> + Result->begin()->second.getAddress());
>>>> + AtExitMgr.runAtExits(DSOHandle);
>>>> + }
>>>> + } else
>>>> + return DeinitSeq.takeError();
>>>> + return Error::success();
>>>> + }
>>>> +
>>>> +private:
>>>> + template <typename FunctionPtrTy>
>>>> + static Error hookUpFunction(FunctionPtrTy &Fn, const char *Name) {
>>>> + if (auto *FnAddr = sys::DynamicLibrary::SearchForAddressOfSymbol(Name)) {
>>>> + Fn = reinterpret_cast<FunctionPtrTy>(Fn);
>>>> + return Error::success();
>>>> + }
>>>> +
>>>> + return make_error<StringError>((Twine("Can not enable MachO JIT Platform: "
>>>> + "missing function: ") +
>>>> + Name)
>>>> + .str(),
>>>> + inconvertibleErrorCode());
>>>> + }
>>>> +
>>>> + MachOPlatformSupport(LLJIT &J, JITDylib &PlatformJITDylib, DlFcnValues DlFcn)
>>>> + : J(J), MP(setupPlatform(J)), DlFcn(std::move(DlFcn)) {
>>>> +
>>>> + MangleAndInterner Mangle(J.getExecutionSession(), J.getDataLayout());
>>>> + SymbolMap HelperSymbols;
>>>> +
>>>> + // platform and atexit helpers.
>>>> + HelperSymbols[Mangle("__lljit.platform_support_instance")] =
>>>> + JITEvaluatedSymbol(pointerToJITTargetAddress(this), JITSymbolFlags());
>>>> + HelperSymbols[Mangle("__lljit.cxa_atexit_helper")] = JITEvaluatedSymbol(
>>>> + pointerToJITTargetAddress(registerAtExitHelper), JITSymbolFlags());
>>>> + HelperSymbols[Mangle("__lljit.run_atexits_helper")] = JITEvaluatedSymbol(
>>>> + pointerToJITTargetAddress(runAtExitsHelper), JITSymbolFlags());
>>>> +
>>>> + // dlfcn helpers.
>>>> + HelperSymbols[Mangle("__lljit.dlopen_helper")] = JITEvaluatedSymbol(
>>>> + pointerToJITTargetAddress(dlopenHelper), JITSymbolFlags());
>>>> + HelperSymbols[Mangle("__lljit.dlclose_helper")] = JITEvaluatedSymbol(
>>>> + pointerToJITTargetAddress(dlcloseHelper), JITSymbolFlags());
>>>> + HelperSymbols[Mangle("__lljit.dlsym_helper")] = JITEvaluatedSymbol(
>>>> + pointerToJITTargetAddress(dlsymHelper), JITSymbolFlags());
>>>> + HelperSymbols[Mangle("__lljit.dlerror_helper")] = JITEvaluatedSymbol(
>>>> + pointerToJITTargetAddress(dlerrorHelper), JITSymbolFlags());
>>>> +
>>>> + cantFail(
>>>> + PlatformJITDylib.define(absoluteSymbols(std::move(HelperSymbols))));
>>>> + cantFail(MP.setupJITDylib(J.getMainJITDylib()));
>>>> + cantFail(J.addIRModule(PlatformJITDylib, createPlatformRuntimeModule()));
>>>> + }
>>>> +
>>>> + static MachOPlatform &setupPlatform(LLJIT &J) {
>>>> + auto Tmp = std::make_unique<MachOPlatform>(
>>>> + J.getExecutionSession(),
>>>> + static_cast<ObjectLinkingLayer &>(J.getObjLinkingLayer()),
>>>> + createStandardSymbolsObject(J));
>>>> + auto &MP = *Tmp;
>>>> + J.getExecutionSession().setPlatform(std::move(Tmp));
>>>> + return MP;
>>>> + }
>>>> +
>>>> + static std::unique_ptr<MemoryBuffer> createStandardSymbolsObject(LLJIT &J) {
>>>> + LLVMContext Ctx;
>>>> + Module M("__standard_symbols", Ctx);
>>>> + M.setDataLayout(J.getDataLayout());
>>>> +
>>>> + auto *Int64Ty = Type::getInt64Ty(Ctx);
>>>> +
>>>> + auto *DSOHandle =
>>>> + new GlobalVariable(M, Int64Ty, true, GlobalValue::ExternalLinkage,
>>>> + ConstantInt::get(Int64Ty, 0), "__dso_handle");
>>>> + DSOHandle->setVisibility(GlobalValue::HiddenVisibility);
>>>> +
>>>> + return cantFail(J.getIRCompileLayer().getCompiler()(M));
>>>> + }
>>>> +
>>>> + ThreadSafeModule createPlatformRuntimeModule() {
>>>> + auto Ctx = std::make_unique<LLVMContext>();
>>>> + auto M = std::make_unique<Module>("__standard_lib", *Ctx);
>>>> + M->setDataLayout(J.getDataLayout());
>>>> +
>>>> + auto *MachOPlatformSupportTy =
>>>> + StructType::create(*Ctx, "lljit.MachOPlatformSupport");
>>>> +
>>>> + auto *PlatformInstanceDecl = new GlobalVariable(
>>>> + *M, MachOPlatformSupportTy, true, GlobalValue::ExternalLinkage, nullptr,
>>>> + "__lljit.platform_support_instance");
>>>> +
>>>> + auto *Int8Ty = Type::getInt8Ty(*Ctx);
>>>> + auto *IntTy = Type::getIntNTy(*Ctx, sizeof(int) * CHAR_BIT);
>>>> + auto *VoidTy = Type::getVoidTy(*Ctx);
>>>> + auto *BytePtrTy = PointerType::getUnqual(Int8Ty);
>>>> + auto *AtExitCallbackTy = FunctionType::get(VoidTy, {BytePtrTy}, false);
>>>> + auto *AtExitCallbackPtrTy = PointerType::getUnqual(AtExitCallbackTy);
>>>> +
>>>> + addHelperAndWrapper(
>>>> + *M, "__cxa_atexit",
>>>> + FunctionType::get(IntTy, {AtExitCallbackPtrTy, BytePtrTy, BytePtrTy},
>>>> + false),
>>>> + GlobalValue::DefaultVisibility, "__lljit.cxa_atexit_helper",
>>>> + {PlatformInstanceDecl});
>>>> +
>>>> + addHelperAndWrapper(*M, "dlopen",
>>>> + FunctionType::get(BytePtrTy, {BytePtrTy, IntTy}, false),
>>>> + GlobalValue::DefaultVisibility, "__lljit.dlopen_helper",
>>>> + {PlatformInstanceDecl});
>>>> +
>>>> + addHelperAndWrapper(*M, "dlclose",
>>>> + FunctionType::get(IntTy, {BytePtrTy}, false),
>>>> + GlobalValue::DefaultVisibility,
>>>> + "__lljit.dlclose_helper", {PlatformInstanceDecl});
>>>> +
>>>> + addHelperAndWrapper(
>>>> + *M, "dlsym",
>>>> + FunctionType::get(BytePtrTy, {BytePtrTy, BytePtrTy}, false),
>>>> + GlobalValue::DefaultVisibility, "__lljit.dlsym_helper",
>>>> + {PlatformInstanceDecl});
>>>> +
>>>> + addHelperAndWrapper(*M, "dlerror", FunctionType::get(BytePtrTy, {}, false),
>>>> + GlobalValue::DefaultVisibility,
>>>> + "__lljit.dlerror_helper", {PlatformInstanceDecl});
>>>> +
>>>> + return ThreadSafeModule(std::move(M), std::move(Ctx));
>>>> + }
>>>> +
>>>> + static void registerAtExitHelper(void *Self, void (*F)(void *), void *Ctx,
>>>> + void *DSOHandle) {
>>>> + static_cast<MachOPlatformSupport *>(Self)->AtExitMgr.registerAtExit(
>>>> + F, Ctx, DSOHandle);
>>>> + }
>>>> +
>>>> + static void runAtExitsHelper(void *Self, void *DSOHandle) {
>>>> + static_cast<MachOPlatformSupport *>(Self)->AtExitMgr.runAtExits(DSOHandle);
>>>> + }
>>>> +
>>>> + void *jit_dlopen(const char *Path, int Mode) {
>>>> + JITDylib *JDToOpen = nullptr;
>>>> + // FIXME: Do the right thing with Mode flags.
>>>> + {
>>>> + std::lock_guard<std::mutex> Lock(PlatformSupportMutex);
>>>> +
>>>> + // Clear any existing error messages.
>>>> + dlErrorMsgs.erase(std::this_thread::get_id());
>>>> +
>>>> + if (auto *JD = J.getExecutionSession().getJITDylibByName(Path)) {
>>>> + auto I = JDRefCounts.find(JD);
>>>> + if (I != JDRefCounts.end()) {
>>>> + ++I->second;
>>>> + return JD;
>>>> + }
>>>> +
>>>> + JDRefCounts[JD] = 1;
>>>> + JDToOpen = JD;
>>>> + }
>>>> + }
>>>> +
>>>> + if (JDToOpen) {
>>>> + if (auto Err = initialize(*JDToOpen)) {
>>>> + recordError(std::move(Err));
>>>> + return 0;
>>>> + }
>>>> + }
>>>> +
>>>> + // Fall through to dlopen if no JITDylib found for Path.
>>>> + return DlFcn.dlopen(Path, Mode);
>>>> + }
>>>> +
>>>> + static void *dlopenHelper(void *Self, const char *Path, int Mode) {
>>>> + return static_cast<MachOPlatformSupport *>(Self)->jit_dlopen(Path, Mode);
>>>> + }
>>>> +
>>>> + int jit_dlclose(void *Handle) {
>>>> + JITDylib *JDToClose = nullptr;
>>>> +
>>>> + {
>>>> + std::lock_guard<std::mutex> Lock(PlatformSupportMutex);
>>>> +
>>>> + // Clear any existing error messages.
>>>> + dlErrorMsgs.erase(std::this_thread::get_id());
>>>> +
>>>> + auto I = JDRefCounts.find(Handle);
>>>> + if (I != JDRefCounts.end()) {
>>>> + --I->second;
>>>> + if (I->second == 0) {
>>>> + JDRefCounts.erase(I);
>>>> + JDToClose = static_cast<JITDylib *>(Handle);
>>>> + } else
>>>> + return 0;
>>>> + }
>>>> + }
>>>> +
>>>> + if (JDToClose) {
>>>> + if (auto Err = deinitialize(*JDToClose)) {
>>>> + recordError(std::move(Err));
>>>> + return -1;
>>>> + }
>>>> + return 0;
>>>> + }
>>>> +
>>>> + // Fall through to dlclose if no JITDylib found for Path.
>>>> + return DlFcn.dlclose(Handle);
>>>> + }
>>>> +
>>>> + static int dlcloseHelper(void *Self, void *Handle) {
>>>> + return static_cast<MachOPlatformSupport *>(Self)->jit_dlclose(Handle);
>>>> + }
>>>> +
>>>> + void *jit_dlsym(void *Handle, const char *Name) {
>>>> + JITDylibSearchOrder JITSymSearchOrder;
>>>> +
>>>> + // FIXME: RTLD_NEXT, RTLD_SELF not supported.
>>>> + {
>>>> + std::lock_guard<std::mutex> Lock(PlatformSupportMutex);
>>>> +
>>>> + // Clear any existing error messages.
>>>> + dlErrorMsgs.erase(std::this_thread::get_id());
>>>> +
>>>> + if (JDRefCounts.count(Handle)) {
>>>> + JITSymSearchOrder.push_back(
>>>> + {static_cast<JITDylib *>(Handle),
>>>> + JITDylibLookupFlags::MatchExportedSymbolsOnly});
>>>> + } else if (Handle == DlFcn.RTLDDefault) {
>>>> + for (auto &KV : JDRefCounts)
>>>> + JITSymSearchOrder.push_back(
>>>> + {static_cast<JITDylib *>(KV.first),
>>>> + JITDylibLookupFlags::MatchExportedSymbolsOnly});
>>>> + }
>>>> + }
>>>> +
>>>> + if (!JITSymSearchOrder.empty()) {
>>>> + MangleAndInterner Mangle(J.getExecutionSession(), J.getDataLayout());
>>>> + auto MangledName = Mangle(Name);
>>>> + SymbolLookupSet Syms(MangledName,
>>>> + SymbolLookupFlags::WeaklyReferencedSymbol);
>>>> + if (auto Result = J.getExecutionSession().lookup(JITSymSearchOrder, Syms,
>>>> + LookupKind::DLSym)) {
>>>> + auto I = Result->find(MangledName);
>>>> + if (I != Result->end())
>>>> + return jitTargetAddressToPointer<void *>(I->second.getAddress());
>>>> + } else {
>>>> + recordError(Result.takeError());
>>>> + return 0;
>>>> + }
>>>> + }
>>>> +
>>>> + // Fall through to dlsym.
>>>> + return DlFcn.dlsym(Handle, Name);
>>>> + }
>>>> +
>>>> + static void *dlsymHelper(void *Self, void *Handle, const char *Name) {
>>>> + return static_cast<MachOPlatformSupport *>(Self)->jit_dlsym(Handle, Name);
>>>> + }
>>>> +
>>>> + const char *jit_dlerror() {
>>>> + {
>>>> + std::lock_guard<std::mutex> Lock(PlatformSupportMutex);
>>>> + auto I = dlErrorMsgs.find(std::this_thread::get_id());
>>>> + if (I != dlErrorMsgs.end())
>>>> + return I->second->c_str();
>>>> + }
>>>> + return DlFcn.dlerror();
>>>> + }
>>>> +
>>>> + static const char *dlerrorHelper(void *Self) {
>>>> + return static_cast<MachOPlatformSupport *>(Self)->jit_dlerror();
>>>> + }
>>>> +
>>>> + void recordError(Error Err) {
>>>> + std::lock_guard<std::mutex> Lock(PlatformSupportMutex);
>>>> + dlErrorMsgs[std::this_thread::get_id()] =
>>>> + std::make_unique<std::string>(toString(std::move(Err)));
>>>> + }
>>>> +
>>>> + std::mutex PlatformSupportMutex;
>>>> + LLJIT &J;
>>>> + MachOPlatform &MP;
>>>> + DlFcnValues DlFcn;
>>>> + ItaniumCXAAtExitSupport AtExitMgr;
>>>> + DenseMap<void *, unsigned> JDRefCounts;
>>>> + std::map<std::thread::id, std::unique_ptr<std::string>> dlErrorMsgs;
>>>> +};
>>>> +
>>>> +} // end anonymous namespace
>>>>
>>>> namespace llvm {
>>>> namespace orc {
>>>>
>>>> +void LLJIT::PlatformSupport::setInitTransform(
>>>> + LLJIT &J, IRTransformLayer::TransformFunction T) {
>>>> + J.InitHelperTransformLayer->setTransform(std::move(T));
>>>> +}
>>>> +
>>>> +LLJIT::PlatformSupport::~PlatformSupport() {}
>>>> +
>>>> Error LLJITBuilderState::prepareForConstruction() {
>>>>
>>>> if (!JTMB) {
>>>> @@ -57,7 +869,7 @@ LLJIT::~LLJIT() {
>>>> Error LLJIT::defineAbsolute(StringRef Name, JITEvaluatedSymbol Sym) {
>>>> auto InternedName = ES->intern(Name);
>>>> SymbolMap Symbols({{InternedName, Sym}});
>>>> - return Main.define(absoluteSymbols(std::move(Symbols)));
>>>> + return Main->define(absoluteSymbols(std::move(Symbols)));
>>>> }
>>>>
>>>> Error LLJIT::addIRModule(JITDylib &JD, ThreadSafeModule TSM) {
>>>> @@ -67,7 +879,8 @@ Error LLJIT::addIRModule(JITDylib &JD, ThreadSafeModule TSM) {
>>>> TSM.withModuleDo([&](Module &M) { return applyDataLayout(M); }))
>>>> return Err;
>>>>
>>>> - return TransformLayer->add(JD, std::move(TSM), ES->allocateVModule());
>>>> + return InitHelperTransformLayer->add(JD, std::move(TSM),
>>>> + ES->allocateVModule());
>>>> }
>>>>
>>>> Error LLJIT::addObjectFile(JITDylib &JD, std::unique_ptr<MemoryBuffer> Obj) {
>>>> @@ -128,16 +941,23 @@ LLJIT::createCompileFunction(LLJITBuilderState &S,
>>>> }
>>>>
>>>> LLJIT::LLJIT(LLJITBuilderState &S, Error &Err)
>>>> - : ES(S.ES ? std::move(S.ES) : std::make_unique<ExecutionSession>()),
>>>> - Main(this->ES->createJITDylib("<main>")), DL(""),
>>>> - TT(S.JTMB->getTargetTriple()),
>>>> + : ES(S.ES ? std::move(S.ES) : std::make_unique<ExecutionSession>()), Main(),
>>>> + DL(""), TT(S.JTMB->getTargetTriple()),
>>>> ObjLinkingLayer(createObjectLinkingLayer(S, *ES)),
>>>> - ObjTransformLayer(*this->ES, *ObjLinkingLayer), CtorRunner(Main),
>>>> - DtorRunner(Main) {
>>>> + ObjTransformLayer(*this->ES, *ObjLinkingLayer) {
>>>>
>>>> ErrorAsOutParameter _(&Err);
>>>>
>>>> - if (auto DLOrErr = S.JTMB->getDefaultDataLayoutForTarget())
>>>> + if (auto MainOrErr = this->ES->createJITDylib("main"))
>>>> + Main = &*MainOrErr;
>>>> + else {
>>>> + Err = MainOrErr.takeError();
>>>> + return;
>>>> + }
>>>> +
>>>> + if (S.DL)
>>>> + DL = std::move(*S.DL);
>>>> + else if (auto DLOrErr = S.JTMB->getDefaultDataLayoutForTarget())
>>>> DL = std::move(*DLOrErr);
>>>> else {
>>>> Err = DLOrErr.takeError();
>>>> @@ -153,10 +973,12 @@ LLJIT::LLJIT(LLJITBuilderState &S, Error &Err)
>>>> CompileLayer = std::make_unique<IRCompileLayer>(
>>>> *ES, ObjTransformLayer, std::move(*CompileFunction));
>>>> TransformLayer = std::make_unique<IRTransformLayer>(*ES, *CompileLayer);
>>>> + InitHelperTransformLayer =
>>>> + std::make_unique<IRTransformLayer>(*ES, *TransformLayer);
>>>> }
>>>>
>>>> if (S.NumCompileThreads > 0) {
>>>> - TransformLayer->setCloneToNewContextOnEmit(true);
>>>> + InitHelperTransformLayer->setCloneToNewContextOnEmit(true);
>>>> CompileThreads =
>>>> std::make_unique<ThreadPool>(hardware_concurrency(S.NumCompileThreads));
>>>> ES->setDispatchMaterialization(
>>>> @@ -167,6 +989,11 @@ LLJIT::LLJIT(LLJITBuilderState &S, Error &Err)
>>>> CompileThreads->async(std::move(Work));
>>>> });
>>>> }
>>>> +
>>>> + if (S.SetUpPlatform)
>>>> + Err = S.SetUpPlatform(*this);
>>>> + else
>>>> + setUpGenericLLVMIRPlatform(*this);
>>>> }
>>>>
>>>> std::string LLJIT::mangle(StringRef UnmangledName) {
>>>> @@ -184,15 +1011,24 @@ Error LLJIT::applyDataLayout(Module &M) {
>>>>
>>>> if (M.getDataLayout() != DL)
>>>> return make_error<StringError>(
>>>> - "Added modules have incompatible data layouts",
>>>> + "Added modules have incompatible data layouts: " +
>>>> + M.getDataLayout().getStringRepresentation() + " (module) vs " +
>>>> + DL.getStringRepresentation() + " (jit)",
>>>> inconvertibleErrorCode());
>>>>
>>>> return Error::success();
>>>> }
>>>>
>>>> -void LLJIT::recordCtorDtors(Module &M) {
>>>> - CtorRunner.add(getConstructors(M));
>>>> - DtorRunner.add(getDestructors(M));
>>>> +void setUpGenericLLVMIRPlatform(LLJIT &J) {
>>>> + J.setPlatformSupport(std::make_unique<GenericLLVMIRPlatformSupport>(J));
>>>> +}
>>>> +
>>>> +Error setUpMachOPlatform(LLJIT &J) {
>>>> + auto MP = MachOPlatformSupport::Create(J, J.getMainJITDylib());
>>>> + if (!MP)
>>>> + return MP.takeError();
>>>> + J.setPlatformSupport(std::move(*MP));
>>>> + return Error::success();
>>>> }
>>>>
>>>> Error LLLazyJITBuilderState::prepareForConstruction() {
>>>> @@ -205,13 +1041,8 @@ Error LLLazyJITBuilderState::prepareForConstruction() {
>>>> Error LLLazyJIT::addLazyIRModule(JITDylib &JD, ThreadSafeModule TSM) {
>>>> assert(TSM && "Can not add null module");
>>>>
>>>> - if (auto Err = TSM.withModuleDo([&](Module &M) -> Error {
>>>> - if (auto Err = applyDataLayout(M))
>>>> - return Err;
>>>> -
>>>> - recordCtorDtors(M);
>>>> - return Error::success();
>>>> - }))
>>>> + if (auto Err = TSM.withModuleDo(
>>>> + [&](Module &M) -> Error { return applyDataLayout(M); }))
>>>> return Err;
>>>>
>>>> return CODLayer->add(JD, std::move(TSM), ES->allocateVModule());
>>>> @@ -256,7 +1087,7 @@ LLLazyJIT::LLLazyJIT(LLLazyJITBuilderState &S, Error &Err) : LLJIT(S, Err) {
>>>>
>>>> // Create the COD layer.
>>>> CODLayer = std::make_unique<CompileOnDemandLayer>(
>>>> - *ES, *TransformLayer, *LCTMgr, std::move(ISMBuilder));
>>>> + *ES, *InitHelperTransformLayer, *LCTMgr, std::move(ISMBuilder));
>>>>
>>>> if (S.NumCompileThreads > 0)
>>>> CODLayer->setCloneToNewContextOnEmit(true);
>>>>
>>>> diff --git a/llvm/lib/ExecutionEngine/Orc/Layer.cpp b/llvm/lib/ExecutionEngine/Orc/Layer.cpp
>>>> index ebc7801f11ff..63b04a0562bd 100644
>>>> --- a/llvm/lib/ExecutionEngine/Orc/Layer.cpp
>>>> +++ b/llvm/lib/ExecutionEngine/Orc/Layer.cpp
>>>> @@ -7,7 +7,10 @@
>>>> //===----------------------------------------------------------------------===//
>>>>
>>>> #include "llvm/ExecutionEngine/Orc/Layer.h"
>>>> +
>>>> +#include "llvm/ExecutionEngine/Orc/ExecutionUtils.h"
>>>> #include "llvm/IR/Constants.h"
>>>> +#include "llvm/Object/MachO.h"
>>>> #include "llvm/Object/ObjectFile.h"
>>>> #include "llvm/Support/Debug.h"
>>>>
>>>> @@ -23,10 +26,11 @@ Error IRLayer::add(JITDylib &JD, ThreadSafeModule TSM, VModuleKey K) {
>>>> *this, *getManglingOptions(), std::move(TSM), std::move(K)));
>>>> }
>>>>
>>>> -IRMaterializationUnit::IRMaterializationUnit(ExecutionSession &ES,
>>>> - const ManglingOptions &MO,
>>>> - ThreadSafeModule TSM, VModuleKey K)
>>>> - : MaterializationUnit(SymbolFlagsMap(), std::move(K)), TSM(std::move(TSM)) {
>>>> +IRMaterializationUnit::IRMaterializationUnit(
>>>> + ExecutionSession &ES, const IRSymbolMapper::ManglingOptions &MO,
>>>> + ThreadSafeModule TSM, VModuleKey K)
>>>> + : MaterializationUnit(SymbolFlagsMap(), nullptr, std::move(K)),
>>>> + TSM(std::move(TSM)) {
>>>>
>>>> assert(this->TSM && "Module must not be null");
>>>>
>>>> @@ -34,6 +38,7 @@ IRMaterializationUnit::IRMaterializationUnit(ExecutionSession &ES,
>>>> this->TSM.withModuleDo([&](Module &M) {
>>>> for (auto &G : M.global_values()) {
>>>> // Skip globals that don't generate symbols.
>>>> +
>>>> if (!G.hasName() || G.isDeclaration() || G.hasLocalLinkage() ||
>>>> G.hasAvailableExternallyLinkage() || G.hasAppendingLinkage())
>>>> continue;
>>>> @@ -72,13 +77,23 @@ IRMaterializationUnit::IRMaterializationUnit(ExecutionSession &ES,
>>>> SymbolFlags[MangledName] = JITSymbolFlags::fromGlobalValue(G);
>>>> SymbolToDefinition[MangledName] = &G;
>>>> }
>>>> +
>>>> + // If we need an init symbol for this module then create one.
>>>> + if (!llvm::empty(getStaticInitGVs(M))) {
>>>> + std::string InitSymbolName;
>>>> + raw_string_ostream(InitSymbolName)
>>>> + << "$." << M.getModuleIdentifier() << ".__inits";
>>>> + InitSymbol = ES.intern(InitSymbolName);
>>>> + SymbolFlags[InitSymbol] = JITSymbolFlags();
>>>> + }
>>>> });
>>>> }
>>>>
>>>> IRMaterializationUnit::IRMaterializationUnit(
>>>> ThreadSafeModule TSM, VModuleKey K, SymbolFlagsMap SymbolFlags,
>>>> - SymbolNameToDefinitionMap SymbolToDefinition)
>>>> - : MaterializationUnit(std::move(SymbolFlags), std::move(K)),
>>>> + SymbolStringPtr InitSymbol, SymbolNameToDefinitionMap SymbolToDefinition)
>>>> + : MaterializationUnit(std::move(SymbolFlags), std::move(InitSymbol),
>>>> + std::move(K)),
>>>> TSM(std::move(TSM)), SymbolToDefinition(std::move(SymbolToDefinition)) {}
>>>>
>>>> StringRef IRMaterializationUnit::getName() const {
>>>> @@ -105,7 +120,8 @@ void IRMaterializationUnit::discard(const JITDylib &JD,
>>>> }
>>>>
>>>> BasicIRLayerMaterializationUnit::BasicIRLayerMaterializationUnit(
>>>> - IRLayer &L, const ManglingOptions &MO, ThreadSafeModule TSM, VModuleKey K)
>>>> + IRLayer &L, const IRSymbolMapper::ManglingOptions &MO, ThreadSafeModule TSM,
>>>> + VModuleKey K)
>>>> : IRMaterializationUnit(L.getExecutionSession(), MO, std::move(TSM),
>>>> std::move(K)),
>>>> L(L), K(std::move(K)) {}
>>>> @@ -150,22 +166,26 @@ Error ObjectLayer::add(JITDylib &JD, std::unique_ptr<MemoryBuffer> O,
>>>> Expected<std::unique_ptr<BasicObjectLayerMaterializationUnit>>
>>>> BasicObjectLayerMaterializationUnit::Create(ObjectLayer &L, VModuleKey K,
>>>> std::unique_ptr<MemoryBuffer> O) {
>>>> - auto SymbolFlags =
>>>> - getObjectSymbolFlags(L.getExecutionSession(), O->getMemBufferRef());
>>>> + auto ObjSymInfo =
>>>> + getObjectSymbolInfo(L.getExecutionSession(), O->getMemBufferRef());
>>>> +
>>>> + if (!ObjSymInfo)
>>>> + return ObjSymInfo.takeError();
>>>>
>>>> - if (!SymbolFlags)
>>>> - return SymbolFlags.takeError();
>>>> + auto &SymbolFlags = ObjSymInfo->first;
>>>> + auto &InitSymbol = ObjSymInfo->second;
>>>>
>>>> return std::unique_ptr<BasicObjectLayerMaterializationUnit>(
>>>> - new BasicObjectLayerMaterializationUnit(L, K, std::move(O),
>>>> - std::move(*SymbolFlags)));
>>>> + new BasicObjectLayerMaterializationUnit(
>>>> + L, K, std::move(O), std::move(SymbolFlags), std::move(InitSymbol)));
>>>> }
>>>>
>>>> BasicObjectLayerMaterializationUnit::BasicObjectLayerMaterializationUnit(
>>>> ObjectLayer &L, VModuleKey K, std::unique_ptr<MemoryBuffer> O,
>>>> - SymbolFlagsMap SymbolFlags)
>>>> - : MaterializationUnit(std::move(SymbolFlags), std::move(K)), L(L),
>>>> - O(std::move(O)) {}
>>>> + SymbolFlagsMap SymbolFlags, SymbolStringPtr InitSymbol)
>>>> + : MaterializationUnit(std::move(SymbolFlags), std::move(InitSymbol),
>>>> + std::move(K)),
>>>> + L(L), O(std::move(O)) {}
>>>>
>>>> StringRef BasicObjectLayerMaterializationUnit::getName() const {
>>>> if (O)
>>>> @@ -184,35 +204,5 @@ void BasicObjectLayerMaterializationUnit::discard(const JITDylib &JD,
>>>> // filter to pass to the object layer along with the object itself.
>>>> }
>>>>
>>>> -Expected<SymbolFlagsMap> getObjectSymbolFlags(ExecutionSession &ES,
>>>> - MemoryBufferRef ObjBuffer) {
>>>> - auto Obj = object::ObjectFile::createObjectFile(ObjBuffer);
>>>> -
>>>> - if (!Obj)
>>>> - return Obj.takeError();
>>>> -
>>>> - SymbolFlagsMap SymbolFlags;
>>>> - for (auto &Sym : (*Obj)->symbols()) {
>>>> - // Skip symbols not defined in this object file.
>>>> - if (Sym.getFlags() & object::BasicSymbolRef::SF_Undefined)
>>>> - continue;
>>>> -
>>>> - // Skip symbols that are not global.
>>>> - if (!(Sym.getFlags() & object::BasicSymbolRef::SF_Global))
>>>> - continue;
>>>> -
>>>> - auto Name = Sym.getName();
>>>> - if (!Name)
>>>> - return Name.takeError();
>>>> - auto InternedName = ES.intern(*Name);
>>>> - auto SymFlags = JITSymbolFlags::fromObjectSymbol(Sym);
>>>> - if (!SymFlags)
>>>> - return SymFlags.takeError();
>>>> - SymbolFlags[InternedName] = std::move(*SymFlags);
>>>> - }
>>>> -
>>>> - return SymbolFlags;
>>>> -}
>>>> -
>>>> } // End namespace orc.
>>>> } // End namespace llvm.
>>>>
>>>> diff --git a/llvm/lib/ExecutionEngine/Orc/LazyReexports.cpp b/llvm/lib/ExecutionEngine/Orc/LazyReexports.cpp
>>>> index 97f36ede713f..27d08e18270f 100644
>>>> --- a/llvm/lib/ExecutionEngine/Orc/LazyReexports.cpp
>>>> +++ b/llvm/lib/ExecutionEngine/Orc/LazyReexports.cpp
>>>> @@ -51,7 +51,7 @@ LazyCallThroughManager::callThroughToSymbol(JITTargetAddress TrampolineAddr) {
>>>>
>>>> auto LookupResult = ES.lookup(
>>>> makeJITDylibSearchOrder(SourceJD, JITDylibLookupFlags::MatchAllSymbols),
>>>> - SymbolName);
>>>> + SymbolName, SymbolState::Ready);
>>>>
>>>> if (!LookupResult) {
>>>> ES.reportError(LookupResult.takeError());
>>>> @@ -123,10 +123,9 @@ LazyReexportsMaterializationUnit::LazyReexportsMaterializationUnit(
>>>> LazyCallThroughManager &LCTManager, IndirectStubsManager &ISManager,
>>>> JITDylib &SourceJD, SymbolAliasMap CallableAliases, ImplSymbolMap *SrcJDLoc,
>>>> VModuleKey K)
>>>> - : MaterializationUnit(extractFlags(CallableAliases), std::move(K)),
>>>> + : MaterializationUnit(extractFlags(CallableAliases), nullptr, std::move(K)),
>>>> LCTManager(LCTManager), ISManager(ISManager), SourceJD(SourceJD),
>>>> - CallableAliases(std::move(CallableAliases)),
>>>> - AliaseeTable(SrcJDLoc) {}
>>>> + CallableAliases(std::move(CallableAliases)), AliaseeTable(SrcJDLoc) {}
>>>>
>>>> StringRef LazyReexportsMaterializationUnit::getName() const {
>>>> return "<Lazy Reexports>";
>>>>
>>>> diff --git a/llvm/lib/ExecutionEngine/Orc/MachOPlatform.cpp b/llvm/lib/ExecutionEngine/Orc/MachOPlatform.cpp
>>>> new file mode 100644
>>>> index 000000000000..4618d53315ef
>>>> --- /dev/null
>>>> +++ b/llvm/lib/ExecutionEngine/Orc/MachOPlatform.cpp
>>>> @@ -0,0 +1,459 @@
>>>> +//===------ MachOPlatform.cpp - Utilities for executing MachO in Orc ------===//
>>>> +//
>>>> +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
>>>> +// See https://llvm.org/LICENSE.txt <https://llvm.org/LICENSE.txt> for license information.
>>>> +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
>>>> +//
>>>> +//===----------------------------------------------------------------------===//
>>>> +
>>>> +#include "llvm/ExecutionEngine/Orc/MachOPlatform.h"
>>>> +
>>>> +#include "llvm/BinaryFormat/MachO.h"
>>>> +#include "llvm/Support/BinaryByteStream.h"
>>>> +
>>>> +namespace {
>>>> +
>>>> +struct objc_class;
>>>> +struct objc_image_info;
>>>> +struct objc_object;
>>>> +struct objc_selector;
>>>> +
>>>> +using Class = objc_class *;
>>>> +using id = objc_object *;
>>>> +using SEL = objc_selector *;
>>>> +
>>>> +using ObjCMsgSendTy = id (*)(id, SEL, ...);
>>>> +using ObjCReadClassPairTy = Class (*)(Class, const objc_image_info *);
>>>> +using SelRegisterNameTy = SEL (*)(const char *);
>>>> +
>>>> +enum class ObjCRegistrationAPI { Uninitialized, Unavailable, Initialized };
>>>> +
>>>> +ObjCRegistrationAPI ObjCRegistrationAPIState =
>>>> + ObjCRegistrationAPI::Uninitialized;
>>>> +ObjCMsgSendTy objc_msgSend = nullptr;
>>>> +ObjCReadClassPairTy objc_readClassPair = nullptr;
>>>> +SelRegisterNameTy sel_registerName = nullptr;
>>>> +
>>>> +} // end anonymous namespace
>>>> +
>>>> +namespace llvm {
>>>> +namespace orc {
>>>> +
>>>> +template <typename FnTy>
>>>> +static Error setUpObjCRegAPIFunc(FnTy &Target, sys::DynamicLibrary &LibObjC,
>>>> + const char *Name) {
>>>> + if (void *Addr = LibObjC.getAddressOfSymbol(Name))
>>>> + Target = reinterpret_cast<FnTy>(Addr);
>>>> + else
>>>> + return make_error<StringError>(
>>>> + (Twine("Could not find address for ") + Name).str(),
>>>> + inconvertibleErrorCode());
>>>> + return Error::success();
>>>> +}
>>>> +
>>>> +Error enableObjCRegistration(const char *PathToLibObjC) {
>>>> + // If we've already tried to initialize then just bail out.
>>>> + if (ObjCRegistrationAPIState != ObjCRegistrationAPI::Uninitialized)
>>>> + return Error::success();
>>>> +
>>>> + ObjCRegistrationAPIState = ObjCRegistrationAPI::Unavailable;
>>>> +
>>>> + std::string ErrMsg;
>>>> + auto LibObjC =
>>>> + sys::DynamicLibrary::getPermanentLibrary(PathToLibObjC, &ErrMsg);
>>>> +
>>>> + if (!LibObjC.isValid())
>>>> + return make_error<StringError>(std::move(ErrMsg), inconvertibleErrorCode());
>>>> +
>>>> + if (auto Err = setUpObjCRegAPIFunc(objc_msgSend, LibObjC, "objc_msgSend"))
>>>> + return Err;
>>>> + if (auto Err = setUpObjCRegAPIFunc(objc_readClassPair, LibObjC,
>>>> + "objc_readClassPair"))
>>>> + return Err;
>>>> + if (auto Err =
>>>> + setUpObjCRegAPIFunc(sel_registerName, LibObjC, "sel_registerName"))
>>>> + return Err;
>>>> +
>>>> + ObjCRegistrationAPIState = ObjCRegistrationAPI::Initialized;
>>>> + return Error::success();
>>>> +}
>>>> +
>>>> +bool objcRegistrationEnabled() {
>>>> + return ObjCRegistrationAPIState == ObjCRegistrationAPI::Initialized;
>>>> +}
>>>> +
>>>> +void MachOJITDylibInitializers::runModInits() const {
>>>> + for (const auto &ModInit : ModInitSections) {
>>>> + for (uint64_t I = 0; I != ModInit.NumPtrs; ++I) {
>>>> + auto *InitializerAddr = jitTargetAddressToPointer<uintptr_t *>(
>>>> + ModInit.Address + (I * sizeof(uintptr_t)));
>>>> + auto *Initializer =
>>>> + jitTargetAddressToFunction<void (*)()>(*InitializerAddr);
>>>> + Initializer();
>>>> + }
>>>> + }
>>>> +}
>>>> +
>>>> +void MachOJITDylibInitializers::registerObjCSelectors() const {
>>>> + assert(objcRegistrationEnabled() && "ObjC registration not enabled.");
>>>> +
>>>> + for (const auto &ObjCSelRefs : ObjCSelRefsSections) {
>>>> + for (uint64_t I = 0; I != ObjCSelRefs.NumPtrs; ++I) {
>>>> + auto SelEntryAddr = ObjCSelRefs.Address + (I * sizeof(uintptr_t));
>>>> + const auto *SelName =
>>>> + *jitTargetAddressToPointer<const char **>(SelEntryAddr);
>>>> + auto Sel = sel_registerName(SelName);
>>>> + *jitTargetAddressToPointer<SEL *>(SelEntryAddr) = Sel;
>>>> + }
>>>> + }
>>>> +}
>>>> +
>>>> +Error MachOJITDylibInitializers::registerObjCClasses() const {
>>>> + assert(objcRegistrationEnabled() && "ObjC registration not enabled.");
>>>> +
>>>> + struct ObjCClassCompiled {
>>>> + void *Metaclass;
>>>> + void *Parent;
>>>> + void *Cache1;
>>>> + void *Cache2;
>>>> + void *Data;
>>>> + };
>>>> +
>>>> + auto *ImageInfo =
>>>> + jitTargetAddressToPointer<const objc_image_info *>(ObjCImageInfoAddr);
>>>> + auto ClassSelector = sel_registerName("class");
>>>> +
>>>> + for (const auto &ObjCClassList : ObjCClassListSections) {
>>>> + for (uint64_t I = 0; I != ObjCClassList.NumPtrs; ++I) {
>>>> + auto ClassPtrAddr = ObjCClassList.Address + (I * sizeof(uintptr_t));
>>>> + auto Cls = *jitTargetAddressToPointer<Class *>(ClassPtrAddr);
>>>> + auto *ClassCompiled =
>>>> + *jitTargetAddressToPointer<ObjCClassCompiled **>(ClassPtrAddr);
>>>> + objc_msgSend(reinterpret_cast<id>(ClassCompiled->Parent), ClassSelector);
>>>> + auto Registered = objc_readClassPair(Cls, ImageInfo);
>>>> +
>>>> + // FIXME: Improve diagnostic by reporting the failed class's name.
>>>> + if (Registered != Cls)
>>>> + return make_error<StringError>("Unable to register Objective-C class",
>>>> + inconvertibleErrorCode());
>>>> + }
>>>> + }
>>>> + return Error::success();
>>>> +}
>>>> +
>>>> +void MachOJITDylibInitializers::dump() const {
>>>> + for (auto &Extent : ModInitSections)
>>>> + dbgs() << formatv("{0:x16}", Extent.Address) << " -- "
>>>> + << formatv("{0:x16}", Extent.Address + 8 * Extent.NumPtrs) << "\n";
>>>> +}
>>>> +
>>>> +MachOPlatform::MachOPlatform(
>>>> + ExecutionSession &ES, ObjectLinkingLayer &ObjLinkingLayer,
>>>> + std::unique_ptr<MemoryBuffer> StandardSymbolsObject)
>>>> + : ES(ES), ObjLinkingLayer(ObjLinkingLayer),
>>>> + StandardSymbolsObject(std::move(StandardSymbolsObject)) {
>>>> + ObjLinkingLayer.addPlugin(std::make_unique<InitScraperPlugin>(*this));
>>>> +}
>>>> +
>>>> +Error MachOPlatform::setupJITDylib(JITDylib &JD) {
>>>> + auto ObjBuffer = MemoryBuffer::getMemBuffer(
>>>> + StandardSymbolsObject->getMemBufferRef(), false);
>>>> + return ObjLinkingLayer.add(JD, std::move(ObjBuffer));
>>>> +}
>>>> +
>>>> +Error MachOPlatform::notifyAdding(JITDylib &JD, const MaterializationUnit &MU) {
>>>> + const auto &InitSym = MU.getInitializerSymbol();
>>>> + if (!InitSym)
>>>> + return Error::success();
>>>> +
>>>> + std::lock_guard<std::mutex> Lock(PlatformMutex);
>>>> + RegisteredInitSymbols[&JD].add(InitSym);
>>>> + return Error::success();
>>>> +}
>>>> +
>>>> +Error MachOPlatform::notifyRemoving(JITDylib &JD, VModuleKey K) {
>>>> + llvm_unreachable("Not supported yet");
>>>> +}
>>>> +
>>>> +Expected<MachOPlatform::InitializerSequence>
>>>> +MachOPlatform::getInitializerSequence(JITDylib &JD) {
>>>> +
>>>> + std::vector<JITDylib *> DFSLinkOrder;
>>>> +
>>>> + while (true) {
>>>> + // Lock the platform while we search for any initializer symbols to
>>>> + // look up.
>>>> + DenseMap<JITDylib *, SymbolLookupSet> NewInitSymbols;
>>>> + {
>>>> + std::lock_guard<std::mutex> Lock(PlatformMutex);
>>>> + DFSLinkOrder = getDFSLinkOrder(JD);
>>>> +
>>>> + for (auto *InitJD : DFSLinkOrder) {
>>>> + auto RISItr = RegisteredInitSymbols.find(InitJD);
>>>> + if (RISItr != RegisteredInitSymbols.end()) {
>>>> + NewInitSymbols[InitJD] = std::move(RISItr->second);
>>>> + RegisteredInitSymbols.erase(RISItr);
>>>> + }
>>>> + }
>>>> + }
>>>> +
>>>> + if (NewInitSymbols.empty())
>>>> + break;
>>>> +
>>>> + // Outside the lock, issue the lookup.
>>>> + if (auto R = lookupInitSymbols(JD.getExecutionSession(), NewInitSymbols))
>>>> + ; // Nothing to do in the success case.
>>>> + else
>>>> + return R.takeError();
>>>> + }
>>>> +
>>>> + // Lock again to collect the initializers.
>>>> + InitializerSequence FullInitSeq;
>>>> + {
>>>> + std::lock_guard<std::mutex> Lock(PlatformMutex);
>>>> + for (auto *InitJD : reverse(DFSLinkOrder)) {
>>>> + auto ISItr = InitSeqs.find(InitJD);
>>>> + if (ISItr != InitSeqs.end()) {
>>>> + FullInitSeq.emplace_back(InitJD, std::move(ISItr->second));
>>>> + InitSeqs.erase(ISItr);
>>>> + }
>>>> + }
>>>> + }
>>>> +
>>>> + return FullInitSeq;
>>>> +}
>>>> +
>>>> +Expected<MachOPlatform::DeinitializerSequence>
>>>> +MachOPlatform::getDeinitializerSequence(JITDylib &JD) {
>>>> + std::vector<JITDylib *> DFSLinkOrder = getDFSLinkOrder(JD);
>>>> +
>>>> + DeinitializerSequence FullDeinitSeq;
>>>> + {
>>>> + std::lock_guard<std::mutex> Lock(PlatformMutex);
>>>> + for (auto *DeinitJD : DFSLinkOrder) {
>>>> + FullDeinitSeq.emplace_back(DeinitJD, MachOJITDylibDeinitializers());
>>>> + }
>>>> + }
>>>> +
>>>> + return FullDeinitSeq;
>>>> +}
>>>> +
>>>> +std::vector<JITDylib *> MachOPlatform::getDFSLinkOrder(JITDylib &JD) {
>>>> + std::vector<JITDylib *> Result, WorkStack({&JD});
>>>> + DenseSet<JITDylib *> Visited;
>>>> +
>>>> + while (!WorkStack.empty()) {
>>>> + auto *NextJD = WorkStack.back();
>>>> + WorkStack.pop_back();
>>>> + if (Visited.count(NextJD))
>>>> + continue;
>>>> + Visited.insert(NextJD);
>>>> + Result.push_back(NextJD);
>>>> + NextJD->withSearchOrderDo([&](const JITDylibSearchOrder &SO) {
>>>> + for (auto &KV : SO)
>>>> + WorkStack.push_back(KV.first);
>>>> + });
>>>> + }
>>>> +
>>>> + return Result;
>>>> +}
>>>> +
>>>> +void MachOPlatform::registerInitInfo(
>>>> + JITDylib &JD, JITTargetAddress ObjCImageInfoAddr,
>>>> + MachOJITDylibInitializers::SectionExtent ModInits,
>>>> + MachOJITDylibInitializers::SectionExtent ObjCSelRefs,
>>>> + MachOJITDylibInitializers::SectionExtent ObjCClassList) {
>>>> + std::lock_guard<std::mutex> Lock(PlatformMutex);
>>>> +
>>>> + auto &InitSeq = InitSeqs[&JD];
>>>> +
>>>> + InitSeq.setObjCImageInfoAddr(ObjCImageInfoAddr);
>>>> +
>>>> + if (ModInits.Address)
>>>> + InitSeq.addModInitsSection(std::move(ModInits));
>>>> +
>>>> + if (ObjCSelRefs.Address)
>>>> + InitSeq.addObjCSelRefsSection(std::move(ObjCSelRefs));
>>>> +
>>>> + if (ObjCClassList.Address)
>>>> + InitSeq.addObjCClassListSection(std::move(ObjCClassList));
>>>> +}
>>>> +
>>>> +static Expected<MachOJITDylibInitializers::SectionExtent>
>>>> +getSectionExtent(jitlink::LinkGraph &G, StringRef SectionName) {
>>>> + auto *Sec = G.findSectionByName(SectionName);
>>>> + if (!Sec)
>>>> + return MachOJITDylibInitializers::SectionExtent();
>>>> + jitlink::SectionRange R(*Sec);
>>>> + if (R.getSize() % G.getPointerSize() != 0)
>>>> + return make_error<StringError>(SectionName + " section size is not a "
>>>> + "multiple of the pointer size",
>>>> + inconvertibleErrorCode());
>>>> + return MachOJITDylibInitializers::SectionExtent(
>>>> + R.getStart(), R.getSize() / G.getPointerSize());
>>>> +}
>>>> +
>>>> +void MachOPlatform::InitScraperPlugin::modifyPassConfig(
>>>> + MaterializationResponsibility &MR, const Triple &TT,
>>>> + jitlink::PassConfiguration &Config) {
>>>> +
>>>> + Config.PrePrunePasses.push_back([this, &MR](jitlink::LinkGraph &G) -> Error {
>>>> + JITLinkSymbolVector InitSectionSymbols;
>>>> + preserveInitSectionIfPresent(InitSectionSymbols, G, "__mod_init_func");
>>>> + preserveInitSectionIfPresent(InitSectionSymbols, G, "__objc_selrefs");
>>>> + preserveInitSectionIfPresent(InitSectionSymbols, G, "__objc_classlist");
>>>> +
>>>> + if (!InitSymbolDeps.empty()) {
>>>> + std::lock_guard<std::mutex> Lock(InitScraperMutex);
>>>> + InitSymbolDeps[&MR] = std::move(InitSectionSymbols);
>>>> + }
>>>> +
>>>> + if (auto Err = processObjCImageInfo(G, MR))
>>>> + return Err;
>>>> +
>>>> + return Error::success();
>>>> + });
>>>> +
>>>> + Config.PostFixupPasses.push_back([this, &JD = MR.getTargetJITDylib()](
>>>> + jitlink::LinkGraph &G) -> Error {
>>>> + MachOJITDylibInitializers::SectionExtent ModInits, ObjCSelRefs,
>>>> + ObjCClassList;
>>>> +
>>>> + JITTargetAddress ObjCImageInfoAddr = 0;
>>>> + if (auto *ObjCImageInfoSec = G.findSectionByName("__objc_image_info")) {
>>>> + if (auto Addr = jitlink::SectionRange(*ObjCImageInfoSec).getStart()) {
>>>> + ObjCImageInfoAddr = Addr;
>>>> + dbgs() << "Recorded __objc_imageinfo @ " << formatv("{0:x16}", Addr);
>>>> + }
>>>> + }
>>>> +
>>>> + // Record __mod_init_func.
>>>> + if (auto ModInitsOrErr = getSectionExtent(G, "__mod_init_func"))
>>>> + ModInits = std::move(*ModInitsOrErr);
>>>> + else
>>>> + return ModInitsOrErr.takeError();
>>>> +
>>>> + // Record __objc_selrefs.
>>>> + if (auto ObjCSelRefsOrErr = getSectionExtent(G, "__objc_selrefs"))
>>>> + ObjCSelRefs = std::move(*ObjCSelRefsOrErr);
>>>> + else
>>>> + return ObjCSelRefsOrErr.takeError();
>>>> +
>>>> + // Record __objc_classlist.
>>>> + if (auto ObjCClassListOrErr = getSectionExtent(G, "__objc_classlist"))
>>>> + ObjCClassList = std::move(*ObjCClassListOrErr);
>>>> + else
>>>> + return ObjCClassListOrErr.takeError();
>>>> +
>>>> + MP.registerInitInfo(JD, ObjCImageInfoAddr, std::move(ModInits),
>>>> + std::move(ObjCSelRefs), std::move(ObjCClassList));
>>>> +
>>>> + return Error::success();
>>>> + });
>>>> +}
>>>> +
>>>> +ObjectLinkingLayer::Plugin::LocalDependenciesMap
>>>> +MachOPlatform::InitScraperPlugin::getSyntheticSymbolLocalDependencies(
>>>> + MaterializationResponsibility &MR) {
>>>> + std::lock_guard<std::mutex> Lock(InitScraperMutex);
>>>> + auto I = InitSymbolDeps.find(&MR);
>>>> + if (I != InitSymbolDeps.end()) {
>>>> + LocalDependenciesMap Result;
>>>> + Result[MR.getInitializerSymbol()] = std::move(I->second);
>>>> + InitSymbolDeps.erase(&MR);
>>>> + return Result;
>>>> + }
>>>> + return LocalDependenciesMap();
>>>> +}
>>>> +
>>>> +void MachOPlatform::InitScraperPlugin::preserveInitSectionIfPresent(
>>>> + JITLinkSymbolVector &Symbols, jitlink::LinkGraph &G,
>>>> + StringRef SectionName) {
>>>> + if (auto *Sec = G.findSectionByName(SectionName)) {
>>>> + auto SecBlocks = Sec->blocks();
>>>> + if (!llvm::empty(SecBlocks))
>>>> + Symbols.push_back(
>>>> + &G.addAnonymousSymbol(**SecBlocks.begin(), 0, 0, false, true));
>>>> + }
>>>> +}
>>>> +
>>>> +Error MachOPlatform::InitScraperPlugin::processObjCImageInfo(
>>>> + jitlink::LinkGraph &G, MaterializationResponsibility &MR) {
>>>> +
>>>> + // If there's an ObjC imagine info then either
>>>> + // (1) It's the first __objc_imageinfo we've seen in this JITDylib. In
>>>> + // this case we name and record it.
>>>> + // OR
>>>> + // (2) We already have a recorded __objc_imageinfo for this JITDylib,
>>>> + // in which case we just verify it.
>>>> + auto *ObjCImageInfo = G.findSectionByName("__objc_imageinfo");
>>>> + if (!ObjCImageInfo)
>>>> + return Error::success();
>>>> +
>>>> + auto ObjCImageInfoBlocks = ObjCImageInfo->blocks();
>>>> +
>>>> + // Check that the section is not empty if present.
>>>> + if (llvm::empty(ObjCImageInfoBlocks))
>>>> + return make_error<StringError>("Empty __objc_imageinfo section in " +
>>>> + G.getName(),
>>>> + inconvertibleErrorCode());
>>>> +
>>>> + // Check that there's only one block in the section.
>>>> + if (std::next(ObjCImageInfoBlocks.begin()) != ObjCImageInfoBlocks.end())
>>>> + return make_error<StringError>("Multiple blocks in __objc_imageinfo "
>>>> + "section in " +
>>>> + G.getName(),
>>>> + inconvertibleErrorCode());
>>>> +
>>>> + // Check that the __objc_imageinfo section is unreferenced.
>>>> + // FIXME: We could optimize this check if Symbols had a ref-count.
>>>> + for (auto &Sec : G.sections()) {
>>>> + if (&Sec != ObjCImageInfo)
>>>> + for (auto *B : Sec.blocks())
>>>> + for (auto &E : B->edges())
>>>> + if (E.getTarget().isDefined() &&
>>>> + &E.getTarget().getBlock().getSection() == ObjCImageInfo)
>>>> + return make_error<StringError>("__objc_imageinfo is referenced "
>>>> + "within file " +
>>>> + G.getName(),
>>>> + inconvertibleErrorCode());
>>>> + }
>>>> +
>>>> + auto &ObjCImageInfoBlock = **ObjCImageInfoBlocks.begin();
>>>> + auto *ObjCImageInfoData = ObjCImageInfoBlock.getContent().data();
>>>> + auto Version = support::endian::read32(ObjCImageInfoData, G.getEndianness());
>>>> + auto Flags =
>>>> + support::endian::read32(ObjCImageInfoData + 4, G.getEndianness());
>>>> +
>>>> + // Lock the mutex while we verify / update the ObjCImageInfos map.
>>>> + std::lock_guard<std::mutex> Lock(InitScraperMutex);
>>>> +
>>>> + auto ObjCImageInfoItr = ObjCImageInfos.find(&MR.getTargetJITDylib());
>>>> + if (ObjCImageInfoItr != ObjCImageInfos.end()) {
>>>> + // We've already registered an __objc_imageinfo section. Verify the
>>>> + // content of this new section matches, then delete it.
>>>> + if (ObjCImageInfoItr->second.first != Version)
>>>> + return make_error<StringError>(
>>>> + "ObjC version in " + G.getName() +
>>>> + " does not match first registered version",
>>>> + inconvertibleErrorCode());
>>>> + if (ObjCImageInfoItr->second.second != Flags)
>>>> + return make_error<StringError>("ObjC flags in " + G.getName() +
>>>> + " do not match first registered flags",
>>>> + inconvertibleErrorCode());
>>>> +
>>>> + // __objc_imageinfo is valid. Delete the block.
>>>> + for (auto *S : ObjCImageInfo->symbols())
>>>> + G.removeDefinedSymbol(*S);
>>>> + G.removeBlock(ObjCImageInfoBlock);
>>>> + } else {
>>>> + // We haven't registered an __objc_imageinfo section yet. Register and
>>>> + // move on. The section should already be marked no-dead-strip.
>>>> + ObjCImageInfos[&MR.getTargetJITDylib()] = std::make_pair(Version, Flags);
>>>> + }
>>>> +
>>>> + return Error::success();
>>>> +}
>>>> +
>>>> +} // End namespace orc.
>>>> +} // End namespace llvm.
>>>>
>>>> diff --git a/llvm/lib/ExecutionEngine/Orc/Mangling.cpp b/llvm/lib/ExecutionEngine/Orc/Mangling.cpp
>>>> new file mode 100644
>>>> index 000000000000..7ec7869d00b5
>>>> --- /dev/null
>>>> +++ b/llvm/lib/ExecutionEngine/Orc/Mangling.cpp
>>>> @@ -0,0 +1,131 @@
>>>> +//===----------- Mangling.cpp -- Name Mangling Utilities for ORC ----------===//
>>>> +//
>>>> +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
>>>> +// See https://llvm.org/LICENSE.txt <https://llvm.org/LICENSE.txt> for license information.
>>>> +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
>>>> +//
>>>> +//===----------------------------------------------------------------------===//
>>>> +
>>>> +#include "llvm/ExecutionEngine/Orc/Mangling.h"
>>>> +#include "llvm/IR/Constants.h"
>>>> +#include "llvm/IR/Mangler.h"
>>>> +#include "llvm/Object/MachO.h"
>>>> +#include "llvm/Object/ObjectFile.h"
>>>> +#include "llvm/Support/Debug.h"
>>>> +
>>>> +#define DEBUG_TYPE "orc"
>>>> +
>>>> +namespace llvm {
>>>> +namespace orc {
>>>> +
>>>> +MangleAndInterner::MangleAndInterner(ExecutionSession &ES, const DataLayout &DL)
>>>> + : ES(ES), DL(DL) {}
>>>> +
>>>> +SymbolStringPtr MangleAndInterner::operator()(StringRef Name) {
>>>> + std::string MangledName;
>>>> + {
>>>> + raw_string_ostream MangledNameStream(MangledName);
>>>> + Mangler::getNameWithPrefix(MangledNameStream, Name, DL);
>>>> + }
>>>> + return ES.intern(MangledName);
>>>> +}
>>>> +
>>>> +void IRSymbolMapper::add(ExecutionSession &ES, const ManglingOptions &MO,
>>>> + ArrayRef<GlobalValue *> GVs,
>>>> + SymbolFlagsMap &SymbolFlags,
>>>> + SymbolNameToDefinitionMap *SymbolToDefinition) {
>>>> + if (GVs.empty())
>>>> + return;
>>>> +
>>>> + MangleAndInterner Mangle(ES, GVs[0]->getParent()->getDataLayout());
>>>> + for (auto *G : GVs) {
>>>> + assert(G && "GVs cannot contain null elements");
>>>> + if (!G->hasName() || G->isDeclaration() || G->hasLocalLinkage() ||
>>>> + G->hasAvailableExternallyLinkage() || G->hasAppendingLinkage())
>>>> + continue;
>>>> +
>>>> + if (G->isThreadLocal() && MO.EmulatedTLS) {
>>>> + auto *GV = cast<GlobalVariable>(G);
>>>> +
>>>> + auto Flags = JITSymbolFlags::fromGlobalValue(*GV);
>>>> +
>>>> + auto EmuTLSV = Mangle(("__emutls_v." + GV->getName()).str());
>>>> + SymbolFlags[EmuTLSV] = Flags;
>>>> + if (SymbolToDefinition)
>>>> + (*SymbolToDefinition)[EmuTLSV] = GV;
>>>> +
>>>> + // If this GV has a non-zero initializer we'll need to emit an
>>>> + // __emutls.t symbol too.
>>>> + if (GV->hasInitializer()) {
>>>> + const auto *InitVal = GV->getInitializer();
>>>> +
>>>> + // Skip zero-initializers.
>>>> + if (isa<ConstantAggregateZero>(InitVal))
>>>> + continue;
>>>> + const auto *InitIntValue = dyn_cast<ConstantInt>(InitVal);
>>>> + if (InitIntValue && InitIntValue->isZero())
>>>> + continue;
>>>> +
>>>> + auto EmuTLST = Mangle(("__emutls_t." + GV->getName()).str());
>>>> + SymbolFlags[EmuTLST] = Flags;
>>>> + if (SymbolToDefinition)
>>>> + (*SymbolToDefinition)[EmuTLST] = GV;
>>>> + }
>>>> + continue;
>>>> + }
>>>> +
>>>> + // Otherwise we just need a normal linker mangling.
>>>> + auto MangledName = Mangle(G->getName());
>>>> + SymbolFlags[MangledName] = JITSymbolFlags::fromGlobalValue(*G);
>>>> + if (SymbolToDefinition)
>>>> + (*SymbolToDefinition)[MangledName] = G;
>>>> + }
>>>> +}
>>>> +
>>>> +Expected<std::pair<SymbolFlagsMap, SymbolStringPtr>>
>>>> +getObjectSymbolInfo(ExecutionSession &ES, MemoryBufferRef ObjBuffer) {
>>>> + auto Obj = object::ObjectFile::createObjectFile(ObjBuffer);
>>>> +
>>>> + if (!Obj)
>>>> + return Obj.takeError();
>>>> +
>>>> + SymbolFlagsMap SymbolFlags;
>>>> + for (auto &Sym : (*Obj)->symbols()) {
>>>> + // Skip symbols not defined in this object file.
>>>> + if (Sym.getFlags() & object::BasicSymbolRef::SF_Undefined)
>>>> + continue;
>>>> +
>>>> + // Skip symbols that are not global.
>>>> + if (!(Sym.getFlags() & object::BasicSymbolRef::SF_Global))
>>>> + continue;
>>>> +
>>>> + auto Name = Sym.getName();
>>>> + if (!Name)
>>>> + return Name.takeError();
>>>> + auto InternedName = ES.intern(*Name);
>>>> + auto SymFlags = JITSymbolFlags::fromObjectSymbol(Sym);
>>>> + if (!SymFlags)
>>>> + return SymFlags.takeError();
>>>> + SymbolFlags[InternedName] = std::move(*SymFlags);
>>>> + }
>>>> +
>>>> + SymbolStringPtr InitSymbol;
>>>> +
>>>> + if (auto *MachOObj = dyn_cast<object::MachOObjectFile>(Obj->get())) {
>>>> + for (auto &Sec : MachOObj->sections()) {
>>>> + auto SecType = MachOObj->getSectionType(Sec);
>>>> + if ((SecType & MachO::SECTION_TYPE) == MachO::S_MOD_INIT_FUNC_POINTERS) {
>>>> + std::string InitSymString;
>>>> + raw_string_ostream(InitSymString)
>>>> + << "$." << ObjBuffer.getBufferIdentifier() << ".__inits";
>>>> + InitSymbol = ES.intern(InitSymString);
>>>> + break;
>>>> + }
>>>> + }
>>>> + }
>>>> +
>>>> + return std::make_pair(std::move(SymbolFlags), std::move(InitSymbol));
>>>> +}
>>>> +
>>>> +} // End namespace orc.
>>>> +} // End namespace llvm.
>>>>
>>>> diff --git a/llvm/lib/ExecutionEngine/Orc/ObjectLinkingLayer.cpp b/llvm/lib/ExecutionEngine/Orc/ObjectLinkingLayer.cpp
>>>> index 6575b0a6ccb2..db88ec8517bf 100644
>>>> --- a/llvm/lib/ExecutionEngine/Orc/ObjectLinkingLayer.cpp
>>>> +++ b/llvm/lib/ExecutionEngine/Orc/ObjectLinkingLayer.cpp
>>>> @@ -144,6 +144,10 @@ class ObjectLinkingLayerJITLinkContext final : public JITLinkContext {
>>>> if (!ExtraSymbolsToClaim.empty())
>>>> if (auto Err = MR.defineMaterializing(ExtraSymbolsToClaim))
>>>> return notifyFailed(std::move(Err));
>>>> +
>>>> + if (const auto &InitSym = MR.getInitializerSymbol())
>>>> + InternedResult[InitSym] = JITEvaluatedSymbol();
>>>> +
>>>> if (auto Err = MR.notifyResolved(InternedResult)) {
>>>> Layer.getExecutionSession().reportError(std::move(Err));
>>>> MR.failMaterialization();
>>>> @@ -184,8 +188,12 @@ class ObjectLinkingLayerJITLinkContext final : public JITLinkContext {
>>>> }
>>>>
>>>> private:
>>>> - using JITLinkSymbolSet = DenseSet<const Symbol *>;
>>>> - using LocalToNamedDependenciesMap = DenseMap<const Symbol *, JITLinkSymbolSet>;
>>>> + struct LocalSymbolNamedDependencies {
>>>> + SymbolNameSet Internal, External;
>>>> + };
>>>> +
>>>> + using LocalSymbolNamedDependenciesMap =
>>>> + DenseMap<const Symbol *, LocalSymbolNamedDependencies>;
>>>>
>>>> Error externalizeWeakAndCommonSymbols(LinkGraph &G) {
>>>> auto &ES = Layer.getExecutionSession();
>>>> @@ -216,6 +224,7 @@ class ObjectLinkingLayerJITLinkContext final : public JITLinkContext {
>>>> auto &ES = MR.getTargetJITDylib().getExecutionSession();
>>>> auto LocalDeps = computeLocalDeps(G);
>>>>
>>>> + // Compute dependencies for symbols defined in the JITLink graph.
>>>> for (auto *Sym : G.defined_symbols()) {
>>>>
>>>> // Skip local symbols: we do not track dependencies for these.
>>>> @@ -239,15 +248,12 @@ class ObjectLinkingLayerJITLinkContext final : public JITLinkContext {
>>>> assert(TargetSym.isDefined() &&
>>>> "local symbols must be defined");
>>>> auto I = LocalDeps.find(&TargetSym);
>>>> - if (I != LocalDeps.end())
>>>> - for (auto &S : I->second) {
>>>> - assert(S->hasName() &&
>>>> - "LocalDeps should only contain named values");
>>>> - if (S->isExternal())
>>>> - ExternalSymDeps.insert(ES.intern(S->getName()));
>>>> - else if (S != Sym)
>>>> - InternalSymDeps.insert(ES.intern(S->getName()));
>>>> - }
>>>> + if (I != LocalDeps.end()) {
>>>> + for (auto &S : I->second.External)
>>>> + ExternalSymDeps.insert(S);
>>>> + for (auto &S : I->second.Internal)
>>>> + InternalSymDeps.insert(S);
>>>> + }
>>>> }
>>>> }
>>>>
>>>> @@ -261,11 +267,33 @@ class ObjectLinkingLayerJITLinkContext final : public JITLinkContext {
>>>> InternalNamedSymbolDeps[SymName] = std::move(InternalSymDeps);
>>>> }
>>>>
>>>> + for (auto &P : Layer.Plugins) {
>>>> + auto SyntheticLocalDeps = P->getSyntheticSymbolLocalDependencies(MR);
>>>> + if (SyntheticLocalDeps.empty())
>>>> + continue;
>>>> +
>>>> + for (auto &KV : SyntheticLocalDeps) {
>>>> + auto &Name = KV.first;
>>>> + auto &LocalDepsForName = KV.second;
>>>> + for (auto *Local : LocalDepsForName) {
>>>> + assert(Local->getScope() == Scope::Local &&
>>>> + "Dependence on non-local symbol");
>>>> + auto LocalNamedDepsItr = LocalDeps.find(Local);
>>>> + if (LocalNamedDepsItr == LocalDeps.end())
>>>> + continue;
>>>> + for (auto &S : LocalNamedDepsItr->second.Internal)
>>>> + InternalNamedSymbolDeps[Name].insert(S);
>>>> + for (auto &S : LocalNamedDepsItr->second.External)
>>>> + ExternalNamedSymbolDeps[Name].insert(S);
>>>> + }
>>>> + }
>>>> + }
>>>> +
>>>> return Error::success();
>>>> }
>>>>
>>>> - LocalToNamedDependenciesMap computeLocalDeps(LinkGraph &G) {
>>>> - LocalToNamedDependenciesMap DepMap;
>>>> + LocalSymbolNamedDependenciesMap computeLocalDeps(LinkGraph &G) {
>>>> + DenseMap<jitlink::Symbol *, DenseSet<jitlink::Symbol *>> DepMap;
>>>>
>>>> // For all local symbols:
>>>> // (1) Add their named dependencies.
>>>> @@ -319,7 +347,26 @@ class ObjectLinkingLayerJITLinkContext final : public JITLinkContext {
>>>> }
>>>> } while (Changed);
>>>>
>>>> - return DepMap;
>>>> + // Intern the results to produce a mapping of jitlink::Symbol* to internal
>>>> + // and external symbol names.
>>>> + auto &ES = Layer.getExecutionSession();
>>>> + LocalSymbolNamedDependenciesMap Result;
>>>> + for (auto &KV : DepMap) {
>>>> + auto *Local = KV.first;
>>>> + assert(Local->getScope() == Scope::Local &&
>>>> + "DepMap keys should all be local symbols");
>>>> + auto &LocalNamedDeps = Result[Local];
>>>> + for (auto *Named : KV.second) {
>>>> + assert(Named->getScope() != Scope::Local &&
>>>> + "DepMap values should all be non-local symbol sets");
>>>> + if (Named->isExternal())
>>>> + LocalNamedDeps.External.insert(ES.intern(Named->getName()));
>>>> + else
>>>> + LocalNamedDeps.Internal.insert(ES.intern(Named->getName()));
>>>> + }
>>>> + }
>>>> +
>>>> + return Result;
>>>> }
>>>>
>>>> void registerDependencies(const SymbolDependenceMap &QueryDeps) {
>>>>
>>>> diff --git a/llvm/lib/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.cpp b/llvm/lib/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.cpp
>>>> index ff8289a264c8..0eaf13c6e3c8 100644
>>>> --- a/llvm/lib/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.cpp
>>>> +++ b/llvm/lib/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.cpp
>>>> @@ -88,7 +88,8 @@ RTDyldObjectLinkingLayer::~RTDyldObjectLinkingLayer() {
>>>> void RTDyldObjectLinkingLayer::emit(MaterializationResponsibility R,
>>>> std::unique_ptr<MemoryBuffer> O) {
>>>> assert(O && "Object must not be null");
>>>> -
>>>> + dbgs() << "Emitting via RTDyldObjectLinkingLayer:\n"
>>>> + << R.getSymbols() << "\n";
>>>> // This method launches an asynchronous link step that will fulfill our
>>>> // materialization responsibility. We need to switch R to be heap
>>>> // allocated before that happens so it can live as long as the asynchronous
>>>> @@ -229,6 +230,9 @@ Error RTDyldObjectLinkingLayer::onObjLoad(
>>>> Symbols.erase(KV.first);
>>>> }
>>>>
>>>> + if (const auto &InitSym = R.getInitializerSymbol())
>>>> + Symbols[InitSym] = JITEvaluatedSymbol();
>>>> +
>>>> if (auto Err = R.notifyResolved(Symbols)) {
>>>> R.failMaterialization();
>>>> return Err;
>>>>
>>>> diff --git a/llvm/test/ExecutionEngine/OrcLazy/objc-minimal.ll b/llvm/test/ExecutionEngine/OrcLazy/objc-minimal.ll
>>>> new file mode 100644
>>>> index 000000000000..8a940573dcb9
>>>> --- /dev/null
>>>> +++ b/llvm/test/ExecutionEngine/OrcLazy/objc-minimal.ll
>>>> @@ -0,0 +1,73 @@
>>>> +; REQUIRES: system-darwin
>>>> +; RUN: lli -jit-kind=orc-lazy %s
>>>> +;
>>>> +; Sanity check MachO Platform support: Call a no-op method (returning int 0) on
>>>> +; an ObjC object. This test checks that we correctly auto-identify this as a
>>>> +; MachO target, configure MachOPlatform support, and correctly register the
>>>> +; class metadata and method selector with the Objective-C runtime.
>>>> +
>>>> +source_filename = "objc-minimal.mm"
>>>> +target datalayout = "e-m:o-i64:64-f80:128-n8:16:32:64-S128"
>>>> +target triple = "x86_64-apple-macosx10.14.0"
>>>> +
>>>> +%0 = type opaque
>>>> +%struct._objc_cache = type opaque
>>>> +%struct._class_t = type { %struct._class_t*, %struct._class_t*, %struct._objc_cache*, i8* (i8*, i8*)**, %struct._class_ro_t* }
>>>> +%struct._class_ro_t = type { i32, i32, i32, i8*, i8*, %struct.__method_list_t*, %struct._objc_protocol_list*, %struct._ivar_list_t*, i8*, %struct._prop_list_t* }
>>>> +%struct.__method_list_t = type { i32, i32, [0 x %struct._objc_method] }
>>>> +%struct._objc_method = type { i8*, i8*, i8* }
>>>> +%struct._objc_protocol_list = type { i64, [0 x %struct._protocol_t*] }
>>>> +%struct._protocol_t = type { i8*, i8*, %struct._objc_protocol_list*, %struct.__method_list_t*, %struct.__method_list_t*, %struct.__method_list_t*, %struct.__method_list_t*, %struct._prop_list_t*, i32, i32, i8**, i8*, %struct._prop_list_t* }
>>>> +%struct._ivar_list_t = type { i32, i32, [0 x %struct._ivar_t] }
>>>> +%struct._ivar_t = type { i64*, i8*, i8*, i32, i32 }
>>>> +%struct._prop_list_t = type { i32, i32, [0 x %struct._prop_t] }
>>>> +%struct._prop_t = type { i8*, i8* }
>>>> +
>>>> + at _objc_empty_cache = external global %struct._objc_cache
>>>> +@"OBJC_METACLASS_$_NSObject" = external global %struct._class_t
>>>> + at OBJC_CLASS_NAME_ = private unnamed_addr constant [4 x i8] c"Foo\00", section "__TEXT,__objc_classname,cstring_literals", align 1
>>>> +@"\01l_OBJC_METACLASS_RO_$_Foo" = private global %struct._class_ro_t { i32 1, i32 40, i32 40, i8* null, i8* getelementptr inbounds ([4 x i8], [4 x i8]* @OBJC_CLASS_NAME_, i32 0, i32 0), %struct.__method_list_t* null, %struct._objc_protocol_list* null, %struct._ivar_list_t* null, i8* null, %struct._prop_list_t* null }, section "__DATA, __objc_const", align 8
>>>> +@"OBJC_METACLASS_$_Foo" = global %struct._class_t { %struct._class_t* @"OBJC_METACLASS_$_NSObject", %struct._class_t* @"OBJC_METACLASS_$_NSObject", %struct._objc_cache* @_objc_empty_cache, i8* (i8*, i8*)** null, %struct._class_ro_t* @"\01l_OBJC_METACLASS_RO_$_Foo" }, section "__DATA, __objc_data", align 8
>>>> +@"OBJC_CLASS_$_NSObject" = external global %struct._class_t
>>>> + at OBJC_METH_VAR_NAME_ = private unnamed_addr constant [4 x i8] c"foo\00", section "__TEXT,__objc_methname,cstring_literals", align 1
>>>> + at OBJC_METH_VAR_TYPE_ = private unnamed_addr constant [8 x i8] c"i16 at 0:8\00", section "__TEXT,__objc_methtype,cstring_literals", align 1
>>>> +@"\01l_OBJC_$_INSTANCE_METHODS_Foo" = private global { i32, i32, [1 x %struct._objc_method] } { i32 24, i32 1, [1 x %struct._objc_method] [%struct._objc_method { i8* getelementptr inbounds ([4 x i8], [4 x i8]* @OBJC_METH_VAR_NAME_, i32 0, i32 0), i8* getelementptr inbounds ([8 x i8], [8 x i8]* @OBJC_METH_VAR_TYPE_, i32 0, i32 0), i8* bitcast (i32 (%0*, i8*)* @"\01-[Foo foo]" to i8*) }] }, section "__DATA, __objc_const", align 8
>>>> +@"\01l_OBJC_CLASS_RO_$_Foo" = private global %struct._class_ro_t { i32 0, i32 8, i32 8, i8* null, i8* getelementptr inbounds ([4 x i8], [4 x i8]* @OBJC_CLASS_NAME_, i32 0, i32 0), %struct.__method_list_t* bitcast ({ i32, i32, [1 x %struct._objc_method] }* @"\01l_OBJC_$_INSTANCE_METHODS_Foo" to %struct.__method_list_t*), %struct._objc_protocol_list* null, %struct._ivar_list_t* null, i8* null, %struct._prop_list_t* null }, section "__DATA, __objc_const", align 8
>>>> +@"OBJC_CLASS_$_Foo" = global %struct._class_t { %struct._class_t* @"OBJC_METACLASS_$_Foo", %struct._class_t* @"OBJC_CLASS_$_NSObject", %struct._objc_cache* @_objc_empty_cache, i8* (i8*, i8*)** null, %struct._class_ro_t* @"\01l_OBJC_CLASS_RO_$_Foo" }, section "__DATA, __objc_data", align 8
>>>> +@"OBJC_CLASSLIST_REFERENCES_$_" = private global %struct._class_t* @"OBJC_CLASS_$_Foo", section "__DATA,__objc_classrefs,regular,no_dead_strip", align 8
>>>> + at OBJC_METH_VAR_NAME_.1 = private unnamed_addr constant [5 x i8] c"init\00", section "__TEXT,__objc_methname,cstring_literals", align 1
>>>> + at OBJC_SELECTOR_REFERENCES_ = private externally_initialized global i8* getelementptr inbounds ([5 x i8], [5 x i8]* @OBJC_METH_VAR_NAME_.1, i64 0, i64 0), section "__DATA,__objc_selrefs,literal_pointers,no_dead_strip", align 8
>>>> + at OBJC_SELECTOR_REFERENCES_.2 = private externally_initialized global i8* getelementptr inbounds ([4 x i8], [4 x i8]* @OBJC_METH_VAR_NAME_, i64 0, i64 0), section "__DATA,__objc_selrefs,literal_pointers,no_dead_strip", align 8
>>>> +@"OBJC_LABEL_CLASS_$" = private global [1 x i8*] [i8* bitcast (%struct._class_t* @"OBJC_CLASS_$_Foo" to i8*)], section "__DATA,__objc_classlist,regular,no_dead_strip", align 8
>>>> + at llvm.compiler.used = appending global [9 x i8*] [i8* bitcast ({ i32, i32, [1 x %struct._objc_method] }* @"\01l_OBJC_$_INSTANCE_METHODS_Foo" to i8*), i8* bitcast (%struct._class_t** @"OBJC_CLASSLIST_REFERENCES_$_" to i8*), i8* getelementptr inbounds ([4 x i8], [4 x i8]* @OBJC_CLASS_NAME_, i32 0, i32 0), i8* bitcast ([1 x i8*]* @"OBJC_LABEL_CLASS_$" to i8*), i8* getelementptr inbounds ([4 x i8], [4 x i8]* @OBJC_METH_VAR_NAME_, i32 0, i32 0), i8* getelementptr inbounds ([5 x i8], [5 x i8]* @OBJC_METH_VAR_NAME_.1, i32 0, i32 0), i8* getelementptr inbounds ([8 x i8], [8 x i8]* @OBJC_METH_VAR_TYPE_, i32 0, i32 0), i8* bitcast (i8** @OBJC_SELECTOR_REFERENCES_ to i8*), i8* bitcast (i8** @OBJC_SELECTOR_REFERENCES_.2 to i8*)], section "llvm.metadata"
>>>> +
>>>> +; Function Attrs: noinline norecurse nounwind readnone ssp uwtable
>>>> +define internal i32 @"\01-[Foo foo]"(%0* nocapture readnone, i8* nocapture readnone) {
>>>> + ret i32 0
>>>> +}
>>>> +
>>>> +; Function Attrs: noinline norecurse ssp uwtable
>>>> +define i32 @main(i32, i8** nocapture readnone) local_unnamed_addr {
>>>> + %3 = load i8*, i8** bitcast (%struct._class_t** @"OBJC_CLASSLIST_REFERENCES_$_" to i8**), align 8
>>>> + %4 = tail call i8* @objc_alloc(i8* %3)
>>>> + %5 = load i8*, i8** @OBJC_SELECTOR_REFERENCES_, align 8
>>>> + %6 = tail call i8* bitcast (i8* (i8*, i8*, ...)* @objc_msgSend to i8* (i8*, i8*)*)(i8* %4, i8* %5)
>>>> + %7 = load i8*, i8** @OBJC_SELECTOR_REFERENCES_.2, align 8
>>>> + %8 = tail call i32 bitcast (i8* (i8*, i8*, ...)* @objc_msgSend to i32 (i8*, i8*)*)(i8* %6, i8* %7)
>>>> + ret i32 %8
>>>> +}
>>>> +
>>>> +declare i8* @objc_alloc(i8*) local_unnamed_addr
>>>> +
>>>> +; Function Attrs: nonlazybind
>>>> +declare i8* @objc_msgSend(i8*, i8*, ...) local_unnamed_addr
>>>> +
>>>> +!llvm.module.flags = !{!0, !1, !2, !3, !4, !5, !6 }
>>>> +
>>>> +!0 = !{i32 2, !"SDK Version", [2 x i32] [i32 10, i32 15]}
>>>> +!1 = !{i32 1, !"Objective-C Version", i32 2}
>>>> +!2 = !{i32 1, !"Objective-C Image Info Version", i32 0}
>>>> +!3 = !{i32 1, !"Objective-C Image Info Section", !"__DATA,__objc_imageinfo,regular,no_dead_strip"}
>>>> +!4 = !{i32 4, !"Objective-C Garbage Collection", i32 0}
>>>> +!5 = !{i32 1, !"Objective-C Class Properties", i32 64}
>>>> +!6 = !{i32 1, !"wchar_size", i32 4}
>>>>
>>>> diff --git a/llvm/tools/lli/lli.cpp b/llvm/tools/lli/lli.cpp
>>>> index e7251b12f7fc..55b9557bddfe 100644
>>>> --- a/llvm/tools/lli/lli.cpp
>>>> +++ b/llvm/tools/lli/lli.cpp
>>>> @@ -24,9 +24,11 @@
>>>> #include "llvm/ExecutionEngine/JITEventListener.h"
>>>> #include "llvm/ExecutionEngine/MCJIT.h"
>>>> #include "llvm/ExecutionEngine/ObjectCache.h"
>>>> +#include "llvm/ExecutionEngine/Orc/DebugUtils.h"
>>>> #include "llvm/ExecutionEngine/Orc/ExecutionUtils.h"
>>>> #include "llvm/ExecutionEngine/Orc/JITTargetMachineBuilder.h"
>>>> #include "llvm/ExecutionEngine/Orc/LLJIT.h"
>>>> +#include "llvm/ExecutionEngine/Orc/MachOPlatform.h"
>>>> #include "llvm/ExecutionEngine/Orc/OrcRemoteTargetClient.h"
>>>> #include "llvm/ExecutionEngine/OrcMCJITReplacement.h"
>>>> #include "llvm/ExecutionEngine/SectionMemoryManager.h"
>>>> @@ -206,6 +208,19 @@ namespace {
>>>> cl::desc("Do not resolve lli process symbols in JIT'd code"),
>>>> cl::init(false));
>>>>
>>>> + enum class LLJITPlatform { DetectHost, GenericIR, MachO };
>>>> +
>>>> + cl::opt<LLJITPlatform>
>>>> + Platform("lljit-platform", cl::desc("Platform to use with LLJIT"),
>>>> + cl::init(LLJITPlatform::DetectHost),
>>>> + cl::values(clEnumValN(LLJITPlatform::DetectHost, "DetectHost",
>>>> + "Select based on JIT target triple"),
>>>> + clEnumValN(LLJITPlatform::GenericIR, "GenericIR",
>>>> + "Use LLJITGenericIRPlatform"),
>>>> + clEnumValN(LLJITPlatform::MachO, "MachO",
>>>> + "Use LLJITMachOPlatform")),
>>>> + cl::Hidden);
>>>> +
>>>> enum class DumpKind {
>>>> NoDump,
>>>> DumpFuncsToStdOut,
>>>> @@ -772,16 +787,20 @@ int runOrcLazyJIT(const char *ProgName) {
>>>> if (!MainModule)
>>>> reportError(Err, ProgName);
>>>>
>>>> - const auto &TT = MainModule->getTargetTriple();
>>>> + Triple TT(MainModule->getTargetTriple());
>>>> orc::LLLazyJITBuilder Builder;
>>>>
>>>> Builder.setJITTargetMachineBuilder(
>>>> - TT.empty() ? ExitOnErr(orc::JITTargetMachineBuilder::detectHost())
>>>> - : orc::JITTargetMachineBuilder(Triple(TT)));
>>>> + MainModule->getTargetTriple().empty()
>>>> + ? ExitOnErr(orc::JITTargetMachineBuilder::detectHost())
>>>> + : orc::JITTargetMachineBuilder(TT));
>>>>
>>>> if (!MArch.empty())
>>>> Builder.getJITTargetMachineBuilder()->getTargetTriple().setArchName(MArch);
>>>>
>>>> + if (!MainModule->getDataLayout().isDefault())
>>>> + Builder.setDataLayout(MainModule->getDataLayout());
>>>> +
>>>> Builder.getJITTargetMachineBuilder()
>>>> ->setCPU(getCPUStr())
>>>> .addFeatures(getFeatureList())
>>>> @@ -796,6 +815,29 @@ int runOrcLazyJIT(const char *ProgName) {
>>>> pointerToJITTargetAddress(exitOnLazyCallThroughFailure));
>>>> Builder.setNumCompileThreads(LazyJITCompileThreads);
>>>>
>>>> + // Set up LLJIT platform.
>>>> + {
>>>> + LLJITPlatform P = Platform;
>>>> + if (P == LLJITPlatform::DetectHost) {
>>>> + if (TT.isOSBinFormatMachO())
>>>> + P = LLJITPlatform::MachO;
>>>> + else
>>>> + P = LLJITPlatform::GenericIR;
>>>> + }
>>>> +
>>>> + switch (P) {
>>>> + case LLJITPlatform::GenericIR:
>>>> + // Nothing to do: LLJITBuilder will use this by default.
>>>> + break;
>>>> + case LLJITPlatform::MachO:
>>>> + Builder.setPlatformSetUp(orc::setUpMachOPlatform);
>>>> + ExitOnErr(orc::enableObjCRegistration("libobjc.dylib"));
>>>> + break;
>>>> + default:
>>>> + llvm_unreachable("Unrecognized platform value");
>>>> + }
>>>> + }
>>>> +
>>>> auto J = ExitOnErr(Builder.create());
>>>>
>>>> if (PerModuleLazy)
>>>> @@ -828,9 +870,6 @@ int runOrcLazyJIT(const char *ProgName) {
>>>> return Name != MainName;
>>>> })));
>>>>
>>>> - orc::LocalCXXRuntimeOverrides CXXRuntimeOverrides;
>>>> - ExitOnErr(CXXRuntimeOverrides.enable(J->getMainJITDylib(), Mangle));
>>>> -
>>>> // Add the main module.
>>>> ExitOnErr(
>>>> J->addLazyIRModule(orc::ThreadSafeModule(std::move(MainModule), TSCtx)));
>>>> @@ -845,8 +884,11 @@ int runOrcLazyJIT(const char *ProgName) {
>>>> for (auto JDItr = JITDylibs.begin(), JDEnd = JITDylibs.end();
>>>> JDItr != JDEnd; ++JDItr) {
>>>> orc::JITDylib *JD = J->getJITDylibByName(*JDItr);
>>>> - if (!JD)
>>>> - JD = &J->createJITDylib(*JDItr);
>>>> + if (!JD) {
>>>> + JD = &ExitOnErr(J->createJITDylib(*JDItr));
>>>> + J->getMainJITDylib().addToSearchOrder(*JD);
>>>> + JD->addToSearchOrder(J->getMainJITDylib());
>>>> + }
>>>> IdxToDylib[JITDylibs.getPosition(JDItr - JITDylibs.begin())] = JD;
>>>> }
>>>>
>>>> @@ -882,7 +924,7 @@ int runOrcLazyJIT(const char *ProgName) {
>>>> }
>>>>
>>>> // Run any static constructors.
>>>> - ExitOnErr(J->runConstructors());
>>>> + ExitOnErr(J->initialize(J->getMainJITDylib()));
>>>>
>>>> // Run any -thread-entry points.
>>>> std::vector<std::thread> AltEntryThreads;
>>>> @@ -907,8 +949,7 @@ int runOrcLazyJIT(const char *ProgName) {
>>>> AltEntryThread.join();
>>>>
>>>> // Run destructors.
>>>> - ExitOnErr(J->runDestructors());
>>>> - CXXRuntimeOverrides.runDestructors();
>>>> + ExitOnErr(J->deinitialize(J->getMainJITDylib()));
>>>>
>>>> return Result;
>>>> }
>>>>
>>>> diff --git a/llvm/tools/llvm-jitlink/llvm-jitlink.cpp b/llvm/tools/llvm-jitlink/llvm-jitlink.cpp
>>>> index 26bcf46d60a6..7765d53f402d 100644
>>>> --- a/llvm/tools/llvm-jitlink/llvm-jitlink.cpp
>>>> +++ b/llvm/tools/llvm-jitlink/llvm-jitlink.cpp
>>>> @@ -396,9 +396,18 @@ static std::unique_ptr<jitlink::JITLinkMemoryManager> createMemoryManager() {
>>>> return std::make_unique<jitlink::InProcessMemoryManager>();
>>>> }
>>>>
>>>> -Session::Session(Triple TT)
>>>> - : MainJD(ES.createJITDylib("<main>")), ObjLayer(ES, createMemoryManager()),
>>>> - TT(std::move(TT)) {
>>>> +Expected<std::unique_ptr<Session>> Session::Create(Triple TT) {
>>>> + Error Err = Error::success();
>>>> + std::unique_ptr<Session> S(new Session(std::move(TT), Err));
>>>> + if (Err)
>>>> + return std::move(Err);
>>>> + return std::move(S);
>>>> +}
>>>> +
>>>> +// FIXME: Move to createJITDylib if/when we start using Platform support in
>>>> +// llvm-jitlink.
>>>> +Session::Session(Triple TT, Error &Err)
>>>> + : ObjLayer(ES, createMemoryManager()), TT(std::move(TT)) {
>>>>
>>>> /// Local ObjectLinkingLayer::Plugin class to forward modifyPassConfig to the
>>>> /// Session.
>>>> @@ -414,6 +423,15 @@ Session::Session(Triple TT)
>>>> Session &S;
>>>> };
>>>>
>>>> + ErrorAsOutParameter _(&Err);
>>>> +
>>>> + if (auto MainJDOrErr = ES.createJITDylib("main"))
>>>> + MainJD = &*MainJDOrErr;
>>>> + else {
>>>> + Err = MainJDOrErr.takeError();
>>>> + return;
>>>> + }
>>>> +
>>>> if (!NoExec && !TT.isOSWindows())
>>>> ObjLayer.addPlugin(std::make_unique<EHFrameRegistrationPlugin>(
>>>> InProcessEHFrameRegistrar::getInstance()));
>>>> @@ -561,7 +579,7 @@ Error loadProcessSymbols(Session &S) {
>>>> auto FilterMainEntryPoint = [InternedEntryPointName](SymbolStringPtr Name) {
>>>> return Name != InternedEntryPointName;
>>>> };
>>>> - S.MainJD.addGenerator(
>>>> + S.MainJD->addGenerator(
>>>> ExitOnErr(orc::DynamicLibrarySearchGenerator::GetForCurrentProcess(
>>>> GlobalPrefix, FilterMainEntryPoint)));
>>>>
>>>> @@ -590,20 +608,22 @@ Error loadObjects(Session &S) {
>>>> LLVM_DEBUG(dbgs() << "Creating JITDylibs...\n");
>>>> {
>>>> // Create a "main" JITLinkDylib.
>>>> - IdxToJLD[0] = &S.MainJD;
>>>> - S.JDSearchOrder.push_back(&S.MainJD);
>>>> - LLVM_DEBUG(dbgs() << " 0: " << S.MainJD.getName() << "\n");
>>>> + IdxToJLD[0] = S.MainJD;
>>>> + S.JDSearchOrder.push_back(S.MainJD);
>>>> + LLVM_DEBUG(dbgs() << " 0: " << S.MainJD->getName() << "\n");
>>>>
>>>> // Add any extra JITLinkDylibs from the command line.
>>>> std::string JDNamePrefix("lib");
>>>> for (auto JLDItr = JITLinkDylibs.begin(), JLDEnd = JITLinkDylibs.end();
>>>> JLDItr != JLDEnd; ++JLDItr) {
>>>> - auto &JD = S.ES.createJITDylib(JDNamePrefix + *JLDItr);
>>>> + auto JD = S.ES.createJITDylib(JDNamePrefix + *JLDItr);
>>>> + if (!JD)
>>>> + return JD.takeError();
>>>> unsigned JDIdx =
>>>> JITLinkDylibs.getPosition(JLDItr - JITLinkDylibs.begin());
>>>> - IdxToJLD[JDIdx] = &JD;
>>>> - S.JDSearchOrder.push_back(&JD);
>>>> - LLVM_DEBUG(dbgs() << " " << JDIdx << ": " << JD.getName() << "\n");
>>>> + IdxToJLD[JDIdx] = &*JD;
>>>> + S.JDSearchOrder.push_back(&*JD);
>>>> + LLVM_DEBUG(dbgs() << " " << JDIdx << ": " << JD->getName() << "\n");
>>>> }
>>>>
>>>> // Set every dylib to link against every other, in command line order.
>>>> @@ -790,32 +810,32 @@ int main(int argc, char *argv[]) {
>>>> std::unique_ptr<JITLinkTimers> Timers =
>>>> ShowTimes ? std::make_unique<JITLinkTimers>() : nullptr;
>>>>
>>>> - Session S(getFirstFileTriple());
>>>> + auto S = ExitOnErr(Session::Create(getFirstFileTriple()));
>>>>
>>>> - ExitOnErr(sanitizeArguments(S));
>>>> + ExitOnErr(sanitizeArguments(*S));
>>>>
>>>> if (!NoProcessSymbols)
>>>> - ExitOnErr(loadProcessSymbols(S));
>>>> + ExitOnErr(loadProcessSymbols(*S));
>>>> ExitOnErr(loadDylibs());
>>>>
>>>>
>>>> {
>>>> TimeRegion TR(Timers ? &Timers->LoadObjectsTimer : nullptr);
>>>> - ExitOnErr(loadObjects(S));
>>>> + ExitOnErr(loadObjects(*S));
>>>> }
>>>>
>>>> JITEvaluatedSymbol EntryPoint = 0;
>>>> {
>>>> TimeRegion TR(Timers ? &Timers->LinkTimer : nullptr);
>>>> - EntryPoint = ExitOnErr(getMainEntryPoint(S));
>>>> + EntryPoint = ExitOnErr(getMainEntryPoint(*S));
>>>> }
>>>>
>>>> if (ShowAddrs)
>>>> - S.dumpSessionInfo(outs());
>>>> + S->dumpSessionInfo(outs());
>>>>
>>>> - ExitOnErr(runChecks(S));
>>>> + ExitOnErr(runChecks(*S));
>>>>
>>>> - dumpSessionStats(S);
>>>> + dumpSessionStats(*S);
>>>>
>>>> if (NoExec)
>>>> return 0;
>>>>
>>>> diff --git a/llvm/tools/llvm-jitlink/llvm-jitlink.h b/llvm/tools/llvm-jitlink/llvm-jitlink.h
>>>> index 0f92d7605012..c888baec9adf 100644
>>>> --- a/llvm/tools/llvm-jitlink/llvm-jitlink.h
>>>> +++ b/llvm/tools/llvm-jitlink/llvm-jitlink.h
>>>> @@ -26,12 +26,13 @@ namespace llvm {
>>>>
>>>> struct Session {
>>>> orc::ExecutionSession ES;
>>>> - orc::JITDylib &MainJD;
>>>> + orc::JITDylib *MainJD;
>>>> orc::ObjectLinkingLayer ObjLayer;
>>>> std::vector<orc::JITDylib *> JDSearchOrder;
>>>> Triple TT;
>>>>
>>>> Session(Triple TT);
>>>> + static Expected<std::unique_ptr<Session>> Create(Triple TT);
>>>> void dumpSessionInfo(raw_ostream &OS);
>>>> void modifyPassConfig(const Triple &FTT,
>>>> jitlink::PassConfiguration &PassConfig);
>>>> @@ -63,6 +64,9 @@ struct Session {
>>>> FileInfoMap FileInfos;
>>>> uint64_t SizeBeforePruning = 0;
>>>> uint64_t SizeAfterFixups = 0;
>>>> +
>>>> +private:
>>>> + Session(Triple TT, Error &Err);
>>>> };
>>>>
>>>> Error registerMachOStubsAndGOT(Session &S, jitlink::LinkGraph &G);
>>>>
>>>> diff --git a/llvm/unittests/ExecutionEngine/Orc/CoreAPIsTest.cpp b/llvm/unittests/ExecutionEngine/Orc/CoreAPIsTest.cpp
>>>> index 68a6d2ed2ca2..171ff37f0069 100644
>>>> --- a/llvm/unittests/ExecutionEngine/Orc/CoreAPIsTest.cpp
>>>> +++ b/llvm/unittests/ExecutionEngine/Orc/CoreAPIsTest.cpp
>>>> @@ -113,6 +113,7 @@ TEST_F(CoreAPIsStandardTest, RemoveSymbolsTest) {
>>>> cantFail(R.notifyResolved({{Bar, BarSym}}));
>>>> cantFail(R.notifyEmitted());
>>>> },
>>>> + nullptr,
>>>> [&](const JITDylib &JD, const SymbolStringPtr &Name) {
>>>> EXPECT_EQ(Name, Bar) << "Expected \"Bar\" to be discarded";
>>>> if (Name == Bar)
>>>> @@ -126,6 +127,7 @@ TEST_F(CoreAPIsStandardTest, RemoveSymbolsTest) {
>>>> cantFail(JD.define(std::make_unique<SimpleMaterializationUnit>(
>>>> SymbolFlagsMap({{Baz, BazSym.getFlags()}}),
>>>> [&](MaterializationResponsibility R) { BazR.emplace(std::move(R)); },
>>>> + nullptr,
>>>> [](const JITDylib &JD, const SymbolStringPtr &Name) {
>>>> ADD_FAILURE() << "\"Baz\" discarded unexpectedly";
>>>> })));
>>>> @@ -176,7 +178,7 @@ TEST_F(CoreAPIsStandardTest, RemoveSymbolsTest) {
>>>> TEST_F(CoreAPIsStandardTest, ChainedJITDylibLookup) {
>>>> cantFail(JD.define(absoluteSymbols({{Foo, FooSym}})));
>>>>
>>>> - auto &JD2 = ES.createJITDylib("JD2");
>>>> + auto &JD2 = ES.createBareJITDylib("JD2");
>>>>
>>>> bool OnCompletionRun = false;
>>>>
>>>> @@ -198,7 +200,7 @@ TEST_F(CoreAPIsStandardTest, LookupWithHiddenSymbols) {
>>>>
>>>> cantFail(JD.define(absoluteSymbols({{Foo, FooSym}, {Bar, BarHiddenSym}})));
>>>>
>>>> - auto &JD2 = ES.createJITDylib("JD2");
>>>> + auto &JD2 = ES.createBareJITDylib("JD2");
>>>> cantFail(JD2.define(absoluteSymbols({{Bar, QuxSym}})));
>>>>
>>>> /// Try a blocking lookup.
>>>> @@ -307,7 +309,7 @@ TEST_F(CoreAPIsStandardTest, TestBasicReExports) {
>>>> // JITDylib works.
>>>> cantFail(JD.define(absoluteSymbols({{Foo, FooSym}})));
>>>>
>>>> - auto &JD2 = ES.createJITDylib("JD2");
>>>> + auto &JD2 = ES.createBareJITDylib("JD2");
>>>>
>>>> cantFail(JD2.define(reexports(JD, {{Bar, {Foo, BarSym.getFlags()}}})));
>>>>
>>>> @@ -332,7 +334,7 @@ TEST_F(CoreAPIsStandardTest, TestThatReExportsDontUnnecessarilyMaterialize) {
>>>>
>>>> cantFail(JD.define(BarMU));
>>>>
>>>> - auto &JD2 = ES.createJITDylib("JD2");
>>>> + auto &JD2 = ES.createBareJITDylib("JD2");
>>>>
>>>> cantFail(JD2.define(reexports(
>>>> JD, {{Baz, {Foo, BazSym.getFlags()}}, {Qux, {Bar, QuxSym.getFlags()}}})));
>>>> @@ -347,7 +349,7 @@ TEST_F(CoreAPIsStandardTest, TestThatReExportsDontUnnecessarilyMaterialize) {
>>>> TEST_F(CoreAPIsStandardTest, TestReexportsGenerator) {
>>>> // Test that a re-exports generator can dynamically generate reexports.
>>>>
>>>> - auto &JD2 = ES.createJITDylib("JD2");
>>>> + auto &JD2 = ES.createBareJITDylib("JD2");
>>>> cantFail(JD2.define(absoluteSymbols({{Foo, FooSym}, {Bar, BarSym}})));
>>>>
>>>> auto Filter = [this](SymbolStringPtr Name) { return Name != Bar; };
>>>> @@ -838,6 +840,7 @@ TEST_F(CoreAPIsStandardTest, DropMaterializerWhenEmpty) {
>>>> [](MaterializationResponsibility R) {
>>>> llvm_unreachable("Unexpected call to materialize");
>>>> },
>>>> + nullptr,
>>>> [&](const JITDylib &JD, SymbolStringPtr Name) {
>>>> EXPECT_TRUE(Name == Foo || Name == Bar)
>>>> << "Discard of unexpected symbol?";
>>>> @@ -872,6 +875,7 @@ TEST_F(CoreAPIsStandardTest, AddAndMaterializeLazySymbol) {
>>>> cantFail(R.notifyEmitted());
>>>> FooMaterialized = true;
>>>> },
>>>> + nullptr,
>>>> [&](const JITDylib &JD, SymbolStringPtr Name) {
>>>> EXPECT_EQ(Name, Bar) << "Expected Name to be Bar";
>>>> BarDiscarded = true;
>>>> @@ -920,6 +924,7 @@ TEST_F(CoreAPIsStandardTest, TestBasicWeakSymbolMaterialization) {
>>>> ADD_FAILURE() << "Attempt to materialize Bar from the wrong unit";
>>>> R.failMaterialization();
>>>> },
>>>> + nullptr,
>>>> [&](const JITDylib &JD, SymbolStringPtr Name) {
>>>> EXPECT_EQ(Name, Bar) << "Expected \"Bar\" to be discarded";
>>>> DuplicateBarDiscarded = true;
>>>>
>>>> diff --git a/llvm/unittests/ExecutionEngine/Orc/OrcTestCommon.h b/llvm/unittests/ExecutionEngine/Orc/OrcTestCommon.h
>>>> index d82012b5a853..b25851d8f796 100644
>>>> --- a/llvm/unittests/ExecutionEngine/Orc/OrcTestCommon.h
>>>> +++ b/llvm/unittests/ExecutionEngine/Orc/OrcTestCommon.h
>>>> @@ -47,7 +47,7 @@ class CoreAPIsBasedStandardTest : public testing::Test {
>>>> protected:
>>>> std::shared_ptr<SymbolStringPool> SSP = std::make_shared<SymbolStringPool>();
>>>> ExecutionSession ES{SSP};
>>>> - JITDylib &JD = ES.createJITDylib("JD");
>>>> + JITDylib &JD = ES.createBareJITDylib("JD");
>>>> SymbolStringPtr Foo = ES.intern("foo");
>>>> SymbolStringPtr Bar = ES.intern("bar");
>>>> SymbolStringPtr Baz = ES.intern("baz");
>>>> @@ -93,9 +93,11 @@ class SimpleMaterializationUnit : public orc::MaterializationUnit {
>>>>
>>>> SimpleMaterializationUnit(
>>>> orc::SymbolFlagsMap SymbolFlags, MaterializeFunction Materialize,
>>>> + orc::SymbolStringPtr InitSym = nullptr,
>>>> DiscardFunction Discard = DiscardFunction(),
>>>> DestructorFunction Destructor = DestructorFunction())
>>>> - : MaterializationUnit(std::move(SymbolFlags), orc::VModuleKey()),
>>>> + : MaterializationUnit(std::move(SymbolFlags), std::move(InitSym),
>>>> + orc::VModuleKey()),
>>>> Materialize(std::move(Materialize)), Discard(std::move(Discard)),
>>>> Destructor(std::move(Destructor)) {}
>>>>
>>>>
>>>> diff --git a/llvm/unittests/ExecutionEngine/Orc/RTDyldObjectLinkingLayerTest.cpp b/llvm/unittests/ExecutionEngine/Orc/RTDyldObjectLinkingLayerTest.cpp
>>>> index 0c66841e9af0..9a50571b1969 100644
>>>> --- a/llvm/unittests/ExecutionEngine/Orc/RTDyldObjectLinkingLayerTest.cpp
>>>> +++ b/llvm/unittests/ExecutionEngine/Orc/RTDyldObjectLinkingLayerTest.cpp
>>>> @@ -50,7 +50,7 @@ static bool testSetProcessAllSections(std::unique_ptr<MemoryBuffer> Obj,
>>>> bool DebugSectionSeen = false;
>>>>
>>>> ExecutionSession ES;
>>>> - auto &JD = ES.createJITDylib("main");
>>>> + auto &JD = ES.createBareJITDylib("main");
>>>> auto Foo = ES.intern("foo");
>>>>
>>>> RTDyldObjectLinkingLayer ObjLayer(ES, [&DebugSectionSeen]() {
>>>> @@ -151,7 +151,7 @@ TEST(RTDyldObjectLinkingLayerTest, TestOverrideObjectFlags) {
>>>>
>>>> // Create a simple stack and set the override flags option.
>>>> ExecutionSession ES;
>>>> - auto &JD = ES.createJITDylib("main");
>>>> + auto &JD = ES.createBareJITDylib("main");
>>>> auto Foo = ES.intern("foo");
>>>> RTDyldObjectLinkingLayer ObjLayer(
>>>> ES, []() { return std::make_unique<SectionMemoryManager>(); });
>>>> @@ -218,7 +218,7 @@ TEST(RTDyldObjectLinkingLayerTest, TestAutoClaimResponsibilityForSymbols) {
>>>>
>>>> // Create a simple stack and set the override flags option.
>>>> ExecutionSession ES;
>>>> - auto &JD = ES.createJITDylib("main");
>>>> + auto &JD = ES.createBareJITDylib("main");
>>>> auto Foo = ES.intern("foo");
>>>> RTDyldObjectLinkingLayer ObjLayer(
>>>> ES, []() { return std::make_unique<SectionMemoryManager>(); });
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> llvm-commits mailing list
>>>> llvm-commits at lists.llvm.org <mailto:llvm-commits at lists.llvm.org>
>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20200226/ebdfb1fb/attachment-0001.html>
More information about the llvm-commits
mailing list