[llvm] 0aec49c - [ORC] Add support for resource tracking/removal (removable code).
Lang Hames via llvm-commits
llvm-commits at lists.llvm.org
Sun Oct 18 21:22:57 PDT 2020
Author: Lang Hames
Date: 2020-10-18T21:02:54-07:00
New Revision: 0aec49c8531bc5282b095730d34681455826bc2c
URL: https://github.com/llvm/llvm-project/commit/0aec49c8531bc5282b095730d34681455826bc2c
DIFF: https://github.com/llvm/llvm-project/commit/0aec49c8531bc5282b095730d34681455826bc2c.diff
LOG: [ORC] Add support for resource tracking/removal (removable code).
This patch introduces new APIs to support resource tracking and removal in Orc.
It is intended as a thread-safe generalization of the removeModule concept from
OrcV1.
Clients can now create ResourceTracker objects (using
JITDylib::createResourceTracker) to track resources for each MaterializationUnit
(code, data, aliases, absolute symbols, etc.) added to the JIT. Every
MaterializationUnit will be associated with a ResourceTracker, and
ResourceTrackers can be re-used for multiple MaterializationUnits. Each JITDylib
has a default ResourceTracker that will be used for MaterializationUnits added
to that JITDylib if no ResourceTracker is explicitly specified.
Two operations can be performed on ResourceTrackers: transferTo and remove. The
transferTo operation transfers tracking of the resources to a different
ResourceTracker object, allowing ResourceTrackers to be merged to reduce
administrative overhead (the source tracker is invalidated in the process). The
remove operation removes all resources associated with a ResourceTracker,
including any symbols defined by MaterializationUnits associated with the
tracker, and also invalidates the tracker. These operations are thread safe, and
should work regardless of the the state of the MaterializationUnits. In the case
of resource transfer any existing resources associated with the source tracker
will be transferred to the destination tracker, and all future resources for
those units will be automatically associated with the destination tracker. In
the case of resource removal all already-allocated resources will be
deallocated, any if any program representations associated with the tracker have
not been compiled yet they will be destroyed. If any program representations are
currently being compiled then they will be prevented from completing: their
MaterializationResponsibility will return errors on any attempt to update the
JIT state.
Clients (usually Layer writers) wishing to track resources can implement the
ResourceManager API to receive notifications when ResourceTrackers are
transferred or removed. The MaterializationResponsibility::withResourceKeyDo
method can be used to create associations between the key for a ResourceTracker
and an allocated resource in a thread-safe way.
RTDyldObjectLinkingLayer and ObjectLinkingLayer are updated to use the
ResourceManager API to enable tracking and removal of memory allocated by the
JIT linker.
The new JITDylib::clear method can be used to trigger removal of every
ResourceTracker associated with the JITDylib (note that this will only
remove resources for the JITDylib, it does not run static destructors).
This patch includes unit tests showing basic usage. A follow-up patch will
update the Kaleidoscope and BuildingAJIT tutorial series to OrcV2 and will
use this API to release code associated with anonymous expressions.
Added:
llvm/unittests/ExecutionEngine/Orc/ResourceTrackerTest.cpp
Modified:
llvm/examples/OrcV2Examples/LLJITWithObjectLinkingLayerPlugin/LLJITWithObjectLinkingLayerPlugin.cpp
llvm/include/llvm/ExecutionEngine/Orc/Core.h
llvm/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h
llvm/include/llvm/ExecutionEngine/Orc/Layer.h
llvm/include/llvm/ExecutionEngine/Orc/LazyReexports.h
llvm/include/llvm/ExecutionEngine/Orc/MachOPlatform.h
llvm/include/llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h
llvm/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h
llvm/include/llvm/ExecutionEngine/RuntimeDyld.h
llvm/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp
llvm/lib/ExecutionEngine/Orc/Core.cpp
llvm/lib/ExecutionEngine/Orc/ExecutionUtils.cpp
llvm/lib/ExecutionEngine/Orc/IRCompileLayer.cpp
llvm/lib/ExecutionEngine/Orc/IndirectionUtils.cpp
llvm/lib/ExecutionEngine/Orc/LLJIT.cpp
llvm/lib/ExecutionEngine/Orc/Layer.cpp
llvm/lib/ExecutionEngine/Orc/LazyReexports.cpp
llvm/lib/ExecutionEngine/Orc/MachOPlatform.cpp
llvm/lib/ExecutionEngine/Orc/ObjectLinkingLayer.cpp
llvm/lib/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.cpp
llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp
llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldImpl.h
llvm/tools/llvm-jitlink/llvm-jitlink.cpp
llvm/tools/llvm-jitlink/llvm-jitlink.h
llvm/unittests/ExecutionEngine/Orc/CMakeLists.txt
llvm/unittests/ExecutionEngine/Orc/CoreAPIsTest.cpp
llvm/unittests/ExecutionEngine/Orc/OrcTestCommon.h
llvm/unittests/ExecutionEngine/Orc/RTDyldObjectLinkingLayerTest.cpp
Removed:
################################################################################
diff --git a/llvm/examples/OrcV2Examples/LLJITWithObjectLinkingLayerPlugin/LLJITWithObjectLinkingLayerPlugin.cpp b/llvm/examples/OrcV2Examples/LLJITWithObjectLinkingLayerPlugin/LLJITWithObjectLinkingLayerPlugin.cpp
index 9bb6daa01da7..f349ce900e61 100644
--- a/llvm/examples/OrcV2Examples/LLJITWithObjectLinkingLayerPlugin/LLJITWithObjectLinkingLayerPlugin.cpp
+++ b/llvm/examples/OrcV2Examples/LLJITWithObjectLinkingLayerPlugin/LLJITWithObjectLinkingLayerPlugin.cpp
@@ -72,6 +72,17 @@ class MyPlugin : public ObjectLinkingLayer::Plugin {
return Error::success();
}
+ Error notifyFailed(MaterializationResponsibility &MR) override {
+ return Error::success();
+ }
+
+ Error notifyRemovingResources(ResourceKey K) override {
+ return Error::success();
+ }
+
+ void notifyTransferringResources(ResourceKey DstKey,
+ ResourceKey SrcKey) override {}
+
private:
void printLinkGraph(jitlink::LinkGraph &G, StringRef Title) {
constexpr JITTargetAddress LineWidth = 16;
diff --git a/llvm/include/llvm/ExecutionEngine/Orc/Core.h b/llvm/include/llvm/ExecutionEngine/Orc/Core.h
index 9bb3678b7212..484015034b7a 100644
--- a/llvm/include/llvm/ExecutionEngine/Orc/Core.h
+++ b/llvm/include/llvm/ExecutionEngine/Orc/Core.h
@@ -16,11 +16,13 @@
#include "llvm/ADT/BitmaskEnum.h"
#include "llvm/ADT/DenseSet.h"
#include "llvm/ADT/FunctionExtras.h"
+#include "llvm/ADT/IntrusiveRefCntPtr.h"
#include "llvm/ExecutionEngine/JITSymbol.h"
#include "llvm/ExecutionEngine/Orc/SymbolStringPool.h"
#include "llvm/ExecutionEngine/OrcV1Deprecation.h"
#include "llvm/Support/Debug.h"
+#include <atomic>
#include <memory>
#include <vector>
@@ -33,11 +35,66 @@ class ExecutionSession;
class MaterializationUnit;
class MaterializationResponsibility;
class JITDylib;
+class ResourceTracker;
+
enum class SymbolState : uint8_t;
-/// VModuleKey provides a unique identifier (allocated and managed by
-/// ExecutionSessions) for a module added to the JIT.
-using VModuleKey = uint64_t;
+using ResourceTrackerSP = IntrusiveRefCntPtr<ResourceTracker>;
+using JITDylibSP = IntrusiveRefCntPtr<JITDylib>;
+
+using ResourceKey = uintptr_t;
+
+/// API to remove / transfer ownership of JIT resources.
+class ResourceTracker : public ThreadSafeRefCountedBase<ResourceTracker> {
+private:
+ friend class ExecutionSession;
+ friend class JITDylib;
+ friend class MaterializationResponsibility;
+
+public:
+ ResourceTracker(const ResourceTracker &) = delete;
+ ResourceTracker &operator=(const ResourceTracker &) = delete;
+ ResourceTracker(ResourceTracker &&) = delete;
+ ResourceTracker &operator=(ResourceTracker &) = delete;
+
+ ~ResourceTracker();
+
+ /// Return the JITDylib targeted by this tracker.
+ JITDylib &getJITDylib() const {
+ return *reinterpret_cast<JITDylib *>(JDAndFlag.load() &
+ ~static_cast<uintptr_t>(1));
+ }
+
+ /// Remove all resources associated with this key.
+ Error remove();
+
+ /// Transfer all resources associated with this key to the given
+ /// tracker, which must target the same JITDylib as this one.
+ void transferTo(ResourceTracker &DstRT);
+
+ /// Return true if this tracker has become defunct.
+ bool isDefunct() const { return JDAndFlag.load() & 0x1; }
+
+ /// Returns the key associated with this tracker.
+ /// This method should not be used except for debug logging: there is no
+ /// guarantee that the returned value will remain valid.
+ ResourceKey getKeyUnsafe() const { return reinterpret_cast<uintptr_t>(this); }
+
+private:
+ ResourceTracker(JITDylibSP JD);
+
+ void makeDefunct();
+
+ std::atomic_uintptr_t JDAndFlag;
+};
+
+/// Listens for ResourceTracker operations.
+class ResourceManager {
+public:
+ virtual ~ResourceManager();
+ virtual Error handleRemoveResources(ResourceKey K) = 0;
+ virtual void handleTransferResources(ResourceKey DstK, ResourceKey SrcK) = 0;
+};
/// A set of symbol names (represented by SymbolStringPtrs for
// efficiency).
@@ -318,6 +375,18 @@ using RegisterDependenciesFunction =
/// are no dependants to register with.
extern RegisterDependenciesFunction NoDependenciesToRegister;
+class ResourceTrackerDefunct : public ErrorInfo<ResourceTrackerDefunct> {
+public:
+ static char ID;
+
+ ResourceTrackerDefunct(ResourceTrackerSP RT);
+ std::error_code convertToErrorCode() const override;
+ void log(raw_ostream &OS) const override;
+
+private:
+ ResourceTrackerSP RT;
+};
+
/// Used to notify a JITDylib that the given set of symbols failed to
/// materialize.
class FailedToMaterialize : public ErrorInfo<FailedToMaterialize> {
@@ -408,7 +477,8 @@ class UnexpectedSymbolDefinitions : public ErrorInfo<UnexpectedSymbolDefinitions
/// emit symbols, or abandon materialization by notifying any unmaterialized
/// symbols of an error.
class MaterializationResponsibility {
- friend class MaterializationUnit;
+ friend class ExecutionSession;
+
public:
MaterializationResponsibility(MaterializationResponsibility &&) = delete;
MaterializationResponsibility &
@@ -419,12 +489,15 @@ class MaterializationResponsibility {
/// emitted or notified of an error.
~MaterializationResponsibility();
+ /// Returns the ResourceTracker for this instance.
+ template <typename Func> Error withResourceKeyDo(Func &&F) const;
+
/// Returns the target JITDylib that these symbols are being materialized
/// into.
JITDylib &getTargetJITDylib() const { return *JD; }
- /// Returns the VModuleKey for this instance.
- VModuleKey getVModuleKey() const { return K; }
+ /// Returns the ExecutionSession for this instance.
+ ExecutionSession &getExecutionSession();
/// Returns the symbol flags map for this responsibility instance.
/// Note: The returned flags may have transient flags (Lazy, Materializing)
@@ -509,13 +582,13 @@ class MaterializationResponsibility {
/// materializers to break up work based on run-time information (e.g.
/// by introspecting which symbols have actually been looked up and
/// materializing only those).
- void replace(std::unique_ptr<MaterializationUnit> MU);
+ Error replace(std::unique_ptr<MaterializationUnit> MU);
/// Delegates responsibility for the given symbols to the returned
/// materialization responsibility. Useful for breaking up work between
/// threads, or
diff erent kinds of materialization processes.
- std::unique_ptr<MaterializationResponsibility>
- delegate(const SymbolNameSet &Symbols, VModuleKey NewKey = VModuleKey());
+ Expected<std::unique_ptr<MaterializationResponsibility>>
+ delegate(const SymbolNameSet &Symbols);
void addDependencies(const SymbolStringPtr &Name,
const SymbolDependenceMap &Dependencies);
@@ -526,19 +599,17 @@ class MaterializationResponsibility {
private:
/// Create a MaterializationResponsibility for the given JITDylib and
/// initial symbols.
- MaterializationResponsibility(std::shared_ptr<JITDylib> JD,
- SymbolFlagsMap SymbolFlags,
- SymbolStringPtr InitSymbol, VModuleKey K)
+ MaterializationResponsibility(JITDylibSP JD, SymbolFlagsMap SymbolFlags,
+ SymbolStringPtr InitSymbol)
: JD(std::move(JD)), SymbolFlags(std::move(SymbolFlags)),
- InitSymbol(std::move(InitSymbol)), K(std::move(K)) {
- assert(this->JD && "Cannot initialize with null JD");
+ InitSymbol(std::move(InitSymbol)) {
+ assert(this->JD && "Cannot initialize with null JITDylib");
assert(!this->SymbolFlags.empty() && "Materializing nothing?");
}
- std::shared_ptr<JITDylib> JD;
+ JITDylibSP JD;
SymbolFlagsMap SymbolFlags;
SymbolStringPtr InitSymbol;
- VModuleKey K;
};
/// A MaterializationUnit represents a set of symbol definitions that can
@@ -555,9 +626,9 @@ class MaterializationUnit {
public:
MaterializationUnit(SymbolFlagsMap InitalSymbolFlags,
- SymbolStringPtr InitSymbol, VModuleKey K)
+ SymbolStringPtr InitSymbol)
: SymbolFlags(std::move(InitalSymbolFlags)),
- InitSymbol(std::move(InitSymbol)), K(std::move(K)) {
+ InitSymbol(std::move(InitSymbol)) {
assert((!this->InitSymbol || this->SymbolFlags.count(this->InitSymbol)) &&
"If set, InitSymbol should appear in InitialSymbolFlags map");
}
@@ -590,18 +661,10 @@ class MaterializationUnit {
protected:
SymbolFlagsMap SymbolFlags;
SymbolStringPtr InitSymbol;
- VModuleKey K;
private:
virtual void anchor();
- std::unique_ptr<MaterializationResponsibility>
- createMaterializationResponsibility(std::shared_ptr<JITDylib> JD) {
- return std::unique_ptr<MaterializationResponsibility>(
- new MaterializationResponsibility(std::move(JD), std::move(SymbolFlags),
- std::move(InitSymbol), K));
- }
-
/// Implementations of this method should discard the given symbol
/// from the source (e.g. if the source is an LLVM IR Module and the
/// symbol is a function, delete the function body or mark it available
@@ -609,16 +672,13 @@ class MaterializationUnit {
virtual void discard(const JITDylib &JD, const SymbolStringPtr &Name) = 0;
};
-using MaterializationUnitList =
- std::vector<std::unique_ptr<MaterializationUnit>>;
-
/// A MaterializationUnit implementation for pre-existing absolute symbols.
///
/// All symbols will be resolved and marked ready as soon as the unit is
/// materialized.
class AbsoluteSymbolsMaterializationUnit : public MaterializationUnit {
public:
- AbsoluteSymbolsMaterializationUnit(SymbolMap Symbols, VModuleKey K);
+ AbsoluteSymbolsMaterializationUnit(SymbolMap Symbols);
StringRef getName() const override;
@@ -641,9 +701,9 @@ class AbsoluteSymbolsMaterializationUnit : public MaterializationUnit {
/// \endcode
///
inline std::unique_ptr<AbsoluteSymbolsMaterializationUnit>
-absoluteSymbols(SymbolMap Symbols, VModuleKey K = VModuleKey()) {
+absoluteSymbols(SymbolMap Symbols) {
return std::make_unique<AbsoluteSymbolsMaterializationUnit>(
- std::move(Symbols), std::move(K));
+ std::move(Symbols));
}
/// A materialization unit for symbol aliases. Allows existing symbols to be
@@ -660,7 +720,7 @@ class ReExportsMaterializationUnit : public MaterializationUnit {
/// resolved.
ReExportsMaterializationUnit(JITDylib *SourceJD,
JITDylibLookupFlags SourceJDLookupFlags,
- SymbolAliasMap Aliases, VModuleKey K);
+ SymbolAliasMap Aliases);
StringRef getName() const override;
@@ -686,10 +746,9 @@ class ReExportsMaterializationUnit : public MaterializationUnit {
/// return Err;
/// \endcode
inline std::unique_ptr<ReExportsMaterializationUnit>
-symbolAliases(SymbolAliasMap Aliases, VModuleKey K = VModuleKey()) {
+symbolAliases(SymbolAliasMap Aliases) {
return std::make_unique<ReExportsMaterializationUnit>(
- nullptr, JITDylibLookupFlags::MatchAllSymbols, std::move(Aliases),
- std::move(K));
+ nullptr, JITDylibLookupFlags::MatchAllSymbols, std::move(Aliases));
}
/// Create a materialization unit for re-exporting symbols from another JITDylib
@@ -698,10 +757,9 @@ symbolAliases(SymbolAliasMap Aliases, VModuleKey K = VModuleKey()) {
inline std::unique_ptr<ReExportsMaterializationUnit>
reexports(JITDylib &SourceJD, SymbolAliasMap Aliases,
JITDylibLookupFlags SourceJDLookupFlags =
- JITDylibLookupFlags::MatchExportedSymbolsOnly,
- VModuleKey K = VModuleKey()) {
+ JITDylibLookupFlags::MatchExportedSymbolsOnly) {
return std::make_unique<ReExportsMaterializationUnit>(
- &SourceJD, SourceJDLookupFlags, std::move(Aliases), std::move(K));
+ &SourceJD, SourceJDLookupFlags, std::move(Aliases));
}
/// Build a SymbolAliasMap for the common case where you want to re-export
@@ -727,6 +785,7 @@ class AsynchronousSymbolQuery {
friend class ExecutionSession;
friend class JITDylib;
friend class JITSymbolResolverAdapter;
+ friend class MaterializationResponsibility;
public:
/// Create a query for the given symbols. The NotifyComplete
@@ -776,7 +835,7 @@ class AsynchronousSymbolQuery {
/// their addresses may be used as keys for resource management.
/// JITDylib state changes must be made via an ExecutionSession to guarantee
/// that they are synchronized with respect to other JITDylib operations.
-class JITDylib : public std::enable_shared_from_this<JITDylib> {
+class JITDylib : public ThreadSafeRefCountedBase<JITDylib> {
friend class AsynchronousSymbolQuery;
friend class ExecutionSession;
friend class Platform;
@@ -813,6 +872,21 @@ class JITDylib : public std::enable_shared_from_this<JITDylib> {
/// Get a reference to the ExecutionSession for this JITDylib.
ExecutionSession &getExecutionSession() const { return ES; }
+ /// Calls remove on all trackers currently associated with this JITDylib.
+ /// Does not run static deinits.
+ ///
+ /// Note that removal happens outside the session lock, so new code may be
+ /// added concurrently while the clear is underway, and the newly added
+ /// code will *not* be cleared. Adding new code concurrently with a clear
+ /// is usually a bug and should be avoided.
+ Error clear();
+
+ /// Get the default resource tracker for this JITDylib.
+ ResourceTrackerSP getDefaultResourceTracker();
+
+ /// Create a resource tracker for this JITDylib.
+ ResourceTrackerSP createResourceTracker();
+
/// Adds a definition generator to this JITDylib and returns a referenece to
/// it.
///
@@ -873,10 +947,13 @@ class JITDylib : public std::enable_shared_from_this<JITDylib> {
/// Define all symbols provided by the materialization unit to be part of this
/// JITDylib.
///
+ /// If RT is not specified then the default resource tracker will be used.
+ ///
/// This overload always takes ownership of the MaterializationUnit. If any
/// errors occur, the MaterializationUnit consumed.
template <typename MaterializationUnitType>
- Error define(std::unique_ptr<MaterializationUnitType> &&MU);
+ Error define(std::unique_ptr<MaterializationUnitType> &&MU,
+ ResourceTrackerSP RT = nullptr);
/// Define all symbols provided by the materialization unit to be part of this
/// JITDylib.
@@ -886,7 +963,8 @@ class JITDylib : public std::enable_shared_from_this<JITDylib> {
/// may allow the caller to modify the MaterializationUnit to correct the
/// issue, then re-call define.
template <typename MaterializationUnitType>
- Error define(std::unique_ptr<MaterializationUnitType> &MU);
+ Error define(std::unique_ptr<MaterializationUnitType> &MU,
+ ResourceTrackerSP RT = nullptr);
/// Tries to remove the given symbols.
///
@@ -913,37 +991,41 @@ class JITDylib : public std::enable_shared_from_this<JITDylib> {
/// Returns the given JITDylibs and all of their transitive dependencies in
/// DFS order (based on linkage relationships). Each JITDylib will appear
/// only once.
- static std::vector<std::shared_ptr<JITDylib>>
- getDFSLinkOrder(ArrayRef<std::shared_ptr<JITDylib>> JDs);
+ static std::vector<JITDylibSP> getDFSLinkOrder(ArrayRef<JITDylibSP> JDs);
/// Returns the given JITDylibs and all of their transitive dependensies in
/// reverse DFS order (based on linkage relationships). Each JITDylib will
/// appear only once.
- static std::vector<std::shared_ptr<JITDylib>>
- getReverseDFSLinkOrder(ArrayRef<std::shared_ptr<JITDylib>> JDs);
+ static std::vector<JITDylibSP>
+ getReverseDFSLinkOrder(ArrayRef<JITDylibSP> JDs);
/// Return this JITDylib and its transitive dependencies in DFS order
/// based on linkage relationships.
- std::vector<std::shared_ptr<JITDylib>> getDFSLinkOrder();
+ std::vector<JITDylibSP> getDFSLinkOrder();
/// Rteurn this JITDylib and its transitive dependencies in reverse DFS order
/// based on linkage relationships.
- std::vector<std::shared_ptr<JITDylib>> getReverseDFSLinkOrder();
+ std::vector<JITDylibSP> getReverseDFSLinkOrder();
private:
using AsynchronousSymbolQueryList =
std::vector<std::shared_ptr<AsynchronousSymbolQuery>>;
struct UnmaterializedInfo {
- UnmaterializedInfo(std::unique_ptr<MaterializationUnit> MU)
- : MU(std::move(MU)) {}
+ UnmaterializedInfo(std::unique_ptr<MaterializationUnit> MU,
+ ResourceTracker *RT)
+ : MU(std::move(MU)), RT(RT) {}
std::unique_ptr<MaterializationUnit> MU;
+ ResourceTracker *RT;
};
using UnmaterializedInfosMap =
DenseMap<SymbolStringPtr, std::shared_ptr<UnmaterializedInfo>>;
+ using UnmaterializedInfosList =
+ std::vector<std::shared_ptr<UnmaterializedInfo>>;
+
struct MaterializingInfo {
SymbolDependenceMap Dependants;
SymbolDependenceMap UnemittedDependencies;
@@ -1010,18 +1092,27 @@ class JITDylib : public std::enable_shared_from_this<JITDylib> {
JITDylib(ExecutionSession &ES, std::string Name);
+ ResourceTrackerSP getTracker(MaterializationResponsibility &MR);
+ std::pair<AsynchronousSymbolQuerySet, std::shared_ptr<SymbolDependenceMap>>
+ removeTracker(ResourceTracker &RT);
+
+ void transferTracker(ResourceTracker &DstRT, ResourceTracker &SrcRT);
+
Error defineImpl(MaterializationUnit &MU);
+ void installMaterializationUnit(std::unique_ptr<MaterializationUnit> MU,
+ ResourceTracker &RT);
+
void lookupFlagsImpl(SymbolFlagsMap &Result, LookupKind K,
JITDylibLookupFlags JDLookupFlags,
SymbolLookupSet &Unresolved);
- Error lodgeQuery(MaterializationUnitList &MUs,
+ Error lodgeQuery(UnmaterializedInfosList &UMIs,
std::shared_ptr<AsynchronousSymbolQuery> &Q, LookupKind K,
JITDylibLookupFlags JDLookupFlags,
SymbolLookupSet &Unresolved);
- Error lodgeQueryImpl(MaterializationUnitList &MUs,
+ Error lodgeQueryImpl(UnmaterializedInfosList &UMIs,
std::shared_ptr<AsynchronousSymbolQuery> &Q,
LookupKind K, JITDylibLookupFlags JDLookupFlags,
SymbolLookupSet &Unresolved);
@@ -1035,20 +1126,30 @@ class JITDylib : public std::enable_shared_from_this<JITDylib> {
Expected<SymbolFlagsMap> defineMaterializing(SymbolFlagsMap SymbolFlags);
- void replace(std::unique_ptr<MaterializationUnit> MU);
+ Error replace(MaterializationResponsibility &FromMR,
+ std::unique_ptr<MaterializationUnit> MU);
+
+ Expected<std::unique_ptr<MaterializationResponsibility>>
+ delegate(MaterializationResponsibility &FromMR, SymbolFlagsMap SymbolFlags,
+ SymbolStringPtr InitSymbol);
SymbolNameSet getRequestedSymbols(const SymbolFlagsMap &SymbolFlags) const;
void addDependencies(const SymbolStringPtr &Name,
const SymbolDependenceMap &Dependants);
- Error resolve(const SymbolMap &Resolved);
+ Error resolve(MaterializationResponsibility &MR, const SymbolMap &Resolved);
+
+ Error emit(MaterializationResponsibility &MR, const SymbolFlagsMap &Emitted);
- Error emit(const SymbolFlagsMap &Emitted);
+ void unlinkMaterializationResponsibility(MaterializationResponsibility &MR);
using FailedSymbolsWorklist =
std::vector<std::pair<JITDylib *, SymbolStringPtr>>;
- static void notifyFailed(FailedSymbolsWorklist FailedSymbols);
+
+ static std::pair<AsynchronousSymbolQuerySet,
+ std::shared_ptr<SymbolDependenceMap>>
+ failSymbols(FailedSymbolsWorklist);
ExecutionSession &ES;
std::string JITDylibName;
@@ -1058,6 +1159,11 @@ class JITDylib : public std::enable_shared_from_this<JITDylib> {
MaterializingInfosMap MaterializingInfos;
std::vector<std::unique_ptr<DefinitionGenerator>> DefGenerators;
JITDylibSearchOrder LinkOrder;
+ ResourceTrackerSP DefaultTracker;
+
+ // Map trackers to sets of symbols tracked.
+ DenseMap<ResourceTracker *, SymbolNameVector> TrackerSymbols;
+ DenseMap<MaterializationResponsibility *, ResourceTracker *> MRTrackers;
};
/// Platforms set up standard symbols and mediate interactions between dynamic
@@ -1076,11 +1182,12 @@ class Platform {
/// This method will be called under the ExecutionSession lock each time a
/// MaterializationUnit is added to a JITDylib.
- virtual Error notifyAdding(JITDylib &JD, const MaterializationUnit &MU) = 0;
+ virtual Error notifyAdding(ResourceTracker &RT,
+ const MaterializationUnit &MU) = 0;
/// This method will be called under the ExecutionSession lock when a
- /// VModuleKey is removed.
- virtual Error notifyRemoving(JITDylib &JD, VModuleKey K) = 0;
+ /// ResourceTracker is removed.
+ virtual Error notifyRemoving(ResourceTracker &RT) = 0;
/// A utility function for looking up initializer symbols. Performs a blocking
/// lookup for the given symbols in each of the given JITDylibs.
@@ -1093,6 +1200,7 @@ class Platform {
class ExecutionSession {
// FIXME: Remove this when we remove the old ORC layers.
friend class JITDylib;
+ friend class ResourceTracker;
public:
/// For reporting errors.
@@ -1108,6 +1216,9 @@ class ExecutionSession {
/// SymbolStringPools may be shared between ExecutionSessions.
ExecutionSession(std::shared_ptr<SymbolStringPool> SSP = nullptr);
+ /// End the session. Closes all JITDylibs.
+ Error endSession();
+
/// Add a symbol name to the SymbolStringPool and return a pointer to it.
SymbolStringPtr intern(StringRef SymName) { return SSP->intern(SymName); }
@@ -1127,6 +1238,14 @@ class ExecutionSession {
return F();
}
+ /// Register the given ResourceManager with this ExecutionSession.
+ /// Managers will be notified of events in reverse order of registration.
+ void registerResourceManager(ResourceManager &RM);
+
+ /// Deregister the given ResourceManager with this ExecutionSession.
+ /// Manager must have been previously registered.
+ void deregisterResourceManager(ResourceManager &RM);
+
/// Return a pointer to the "name" JITDylib.
/// Ownership of JITDylib remains within Execution Session
JITDylib *getJITDylibByName(StringRef Name);
@@ -1152,17 +1271,6 @@ class ExecutionSession {
/// If no Platform is attached this call is equivalent to createBareJITDylib.
Expected<JITDylib &> createJITDylib(std::string Name);
- /// Allocate a module key for a new module to add to the JIT.
- VModuleKey allocateVModule() {
- return runSessionLocked([this]() { return ++LastKey; });
- }
-
- /// Return a module key to the ExecutionSession so that it can be
- /// re-used. This should only be done once all resources associated
- /// with the original key have been released.
- void releaseVModule(VModuleKey Key) { /* FIXME: Recycle keys */
- }
-
/// Set the error reporter function.
ExecutionSession &setErrorReporter(ErrorReporter ReportError) {
this->ReportError = std::move(ReportError);
@@ -1265,19 +1373,37 @@ class ExecutionSession {
void runOutstandingMUs();
+ static std::unique_ptr<MaterializationResponsibility>
+ createMaterializationResponsibility(ResourceTracker &RT,
+ SymbolFlagsMap Symbols,
+ SymbolStringPtr InitSymbol) {
+ auto &JD = RT.getJITDylib();
+ std::unique_ptr<MaterializationResponsibility> MR(
+ new MaterializationResponsibility(&JD, std::move(Symbols),
+ std::move(InitSymbol)));
+ JD.MRTrackers[MR.get()] = &RT;
+ return MR;
+ }
+
+ Error removeResourceTracker(ResourceTracker &RT);
+ void transferResourceTracker(ResourceTracker &DstRT, ResourceTracker &SrcRT);
+ void destroyResourceTracker(ResourceTracker &RT);
+
#ifndef NDEBUG
void dumpDispatchInfo(JITDylib &JD, MaterializationUnit &MU);
#endif // NDEBUG
mutable std::recursive_mutex SessionMutex;
+ bool SessionOpen = true;
std::shared_ptr<SymbolStringPool> SSP;
std::unique_ptr<Platform> P;
- VModuleKey LastKey = 0;
ErrorReporter ReportError = logErrorsToStdErr;
DispatchMaterializationFunction DispatchMaterialization =
materializeOnCurrentThread;
- std::vector<std::shared_ptr<JITDylib>> JDs;
+ std::vector<ResourceManager *> ResourceManagers;
+
+ std::vector<JITDylibSP> JDs;
// FIXME: Remove this (and runOutstandingMUs) once the linking layer works
// with callbacks from asynchronous queries.
@@ -1287,6 +1413,22 @@ class ExecutionSession {
OutstandingMUs;
};
+inline ExecutionSession &MaterializationResponsibility::getExecutionSession() {
+ return JD->getExecutionSession();
+}
+
+template <typename Func>
+Error MaterializationResponsibility::withResourceKeyDo(Func &&F) const {
+ return JD->getExecutionSession().runSessionLocked([&]() -> Error {
+ auto I = JD->MRTrackers.find(this);
+ assert(I != JD->MRTrackers.end() && "No tracker for this MR");
+ if (I->second->isDefunct())
+ return make_error<ResourceTrackerDefunct>(I->second);
+ F(I->second->getKeyUnsafe());
+ return Error::success();
+ });
+}
+
template <typename GeneratorT>
GeneratorT &JITDylib::addGenerator(std::unique_ptr<GeneratorT> DefGenerator) {
auto &G = *DefGenerator;
@@ -1302,7 +1444,8 @@ auto JITDylib::withLinkOrderDo(Func &&F)
}
template <typename MaterializationUnitType>
-Error JITDylib::define(std::unique_ptr<MaterializationUnitType> &&MU) {
+Error JITDylib::define(std::unique_ptr<MaterializationUnitType> &&MU,
+ ResourceTrackerSP RT) {
assert(MU && "Can not define with a null MU");
if (MU->getSymbols().empty()) {
@@ -1321,22 +1464,22 @@ Error JITDylib::define(std::unique_ptr<MaterializationUnitType> &&MU) {
if (auto Err = defineImpl(*MU))
return Err;
+ if (!RT)
+ RT = getDefaultResourceTracker();
+
if (auto *P = ES.getPlatform()) {
- if (auto Err = P->notifyAdding(*this, *MU))
+ if (auto Err = P->notifyAdding(*RT, *MU))
return Err;
}
- /// defineImpl succeeded.
- auto UMI = std::make_shared<UnmaterializedInfo>(std::move(MU));
- for (auto &KV : UMI->MU->getSymbols())
- UnmaterializedInfos[KV.first] = UMI;
-
+ installMaterializationUnit(std::move(MU), *RT);
return Error::success();
});
}
template <typename MaterializationUnitType>
-Error JITDylib::define(std::unique_ptr<MaterializationUnitType> &MU) {
+Error JITDylib::define(std::unique_ptr<MaterializationUnitType> &MU,
+ ResourceTrackerSP RT) {
assert(MU && "Can not define with a null MU");
if (MU->getSymbols().empty()) {
@@ -1355,16 +1498,15 @@ Error JITDylib::define(std::unique_ptr<MaterializationUnitType> &MU) {
if (auto Err = defineImpl(*MU))
return Err;
+ if (!RT)
+ RT = getDefaultResourceTracker();
+
if (auto *P = ES.getPlatform()) {
- if (auto Err = P->notifyAdding(*this, *MU))
+ if (auto Err = P->notifyAdding(*RT, *MU))
return Err;
}
- /// defineImpl succeeded.
- auto UMI = std::make_shared<UnmaterializedInfo>(std::move(MU));
- for (auto &KV : UMI->MU->getSymbols())
- UnmaterializedInfos[KV.first] = UMI;
-
+ installMaterializationUnit(std::move(MU), *RT);
return Error::success();
});
}
diff --git a/llvm/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h b/llvm/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h
index 81c39eab8779..f8fdb171bbf9 100644
--- a/llvm/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h
+++ b/llvm/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h
@@ -45,8 +45,8 @@ class IRCompileLayer : public IRLayer {
IRSymbolMapper::ManglingOptions MO;
};
- using NotifyCompiledFunction =
- std::function<void(VModuleKey K, ThreadSafeModule TSM)>;
+ using NotifyCompiledFunction = std::function<void(
+ MaterializationResponsibility &R, ThreadSafeModule TSM)>;
IRCompileLayer(ExecutionSession &ES, ObjectLayer &BaseLayer,
std::unique_ptr<IRCompiler> Compile);
diff --git a/llvm/include/llvm/ExecutionEngine/Orc/Layer.h b/llvm/include/llvm/ExecutionEngine/Orc/Layer.h
index c8a41199760d..f9cc15583b42 100644
--- a/llvm/include/llvm/ExecutionEngine/Orc/Layer.h
+++ b/llvm/include/llvm/ExecutionEngine/Orc/Layer.h
@@ -34,15 +34,15 @@ class IRMaterializationUnit : public MaterializationUnit {
/// SymbolFlags and SymbolToDefinition maps.
IRMaterializationUnit(ExecutionSession &ES,
const IRSymbolMapper::ManglingOptions &MO,
- ThreadSafeModule TSM, VModuleKey K);
+ ThreadSafeModule TSM);
/// Create an IRMaterializationLayer from a module, and pre-existing
/// SymbolFlags and SymbolToDefinition maps. The maps must provide
/// entries for each definition in M.
/// This constructor is useful for delegating work from one
/// IRMaterializationUnit to another.
- IRMaterializationUnit(ThreadSafeModule TSM, VModuleKey K,
- SymbolFlagsMap SymbolFlags, SymbolStringPtr InitSymbol,
+ IRMaterializationUnit(ThreadSafeModule TSM, SymbolFlagsMap SymbolFlags,
+ SymbolStringPtr InitSymbol,
SymbolNameToDefinitionMap SymbolToDefinition);
/// Return the ModuleIdentifier as the name for this MaterializationUnit.
@@ -94,10 +94,15 @@ class IRLayer {
/// Returns the current value of the CloneToNewContextOnEmit flag.
bool getCloneToNewContextOnEmit() const { return CloneToNewContextOnEmit; }
+ /// Add a MaterializatinoUnit representing the given IR to the JITDylib
+ /// targeted by the given tracker.
+ virtual Error add(ResourceTrackerSP RT, ThreadSafeModule TSM);
+
/// Adds a MaterializationUnit representing the given IR to the given
- /// JITDylib.
- virtual Error add(JITDylib &JD, ThreadSafeModule TSM,
- VModuleKey K = VModuleKey());
+ /// JITDylib. If RT is not specif
+ Error add(JITDylib &JD, ThreadSafeModule TSM) {
+ return add(JD.getDefaultResourceTracker(), std::move(TSM));
+ }
/// Emit should materialize the given IR.
virtual void emit(std::unique_ptr<MaterializationResponsibility> R,
@@ -115,13 +120,12 @@ class BasicIRLayerMaterializationUnit : public IRMaterializationUnit {
public:
BasicIRLayerMaterializationUnit(IRLayer &L,
const IRSymbolMapper::ManglingOptions &MO,
- ThreadSafeModule TSM, VModuleKey K);
+ ThreadSafeModule TSM);
private:
void materialize(std::unique_ptr<MaterializationResponsibility> R) override;
IRLayer &L;
- VModuleKey K;
};
/// Interface for Layers that accept object files.
@@ -135,8 +139,11 @@ class ObjectLayer {
/// Adds a MaterializationUnit representing the given IR to the given
/// JITDylib.
- virtual Error add(JITDylib &JD, std::unique_ptr<MemoryBuffer> O,
- VModuleKey K = VModuleKey());
+ virtual Error add(ResourceTrackerSP RT, std::unique_ptr<MemoryBuffer> O);
+
+ Error add(JITDylib &JD, std::unique_ptr<MemoryBuffer> O) {
+ return add(JD.getDefaultResourceTracker(), std::move(O));
+ }
/// Emit should materialize the given IR.
virtual void emit(std::unique_ptr<MaterializationResponsibility> R,
@@ -151,9 +158,9 @@ class ObjectLayer {
class BasicObjectLayerMaterializationUnit : public MaterializationUnit {
public:
static Expected<std::unique_ptr<BasicObjectLayerMaterializationUnit>>
- Create(ObjectLayer &L, VModuleKey K, std::unique_ptr<MemoryBuffer> O);
+ Create(ObjectLayer &L, std::unique_ptr<MemoryBuffer> O);
- BasicObjectLayerMaterializationUnit(ObjectLayer &L, VModuleKey K,
+ BasicObjectLayerMaterializationUnit(ObjectLayer &L,
std::unique_ptr<MemoryBuffer> O,
SymbolFlagsMap SymbolFlags,
SymbolStringPtr InitSymbol);
diff --git a/llvm/include/llvm/ExecutionEngine/Orc/LazyReexports.h b/llvm/include/llvm/ExecutionEngine/Orc/LazyReexports.h
index 63e3a80d87d8..e6a9d8945285 100644
--- a/llvm/include/llvm/ExecutionEngine/Orc/LazyReexports.h
+++ b/llvm/include/llvm/ExecutionEngine/Orc/LazyReexports.h
@@ -144,7 +144,7 @@ class LazyReexportsMaterializationUnit : public MaterializationUnit {
IndirectStubsManager &ISManager,
JITDylib &SourceJD,
SymbolAliasMap CallableAliases,
- ImplSymbolMap *SrcJDLoc, VModuleKey K);
+ ImplSymbolMap *SrcJDLoc);
StringRef getName() const override;
@@ -166,11 +166,10 @@ class LazyReexportsMaterializationUnit : public MaterializationUnit {
inline std::unique_ptr<LazyReexportsMaterializationUnit>
lazyReexports(LazyCallThroughManager &LCTManager,
IndirectStubsManager &ISManager, JITDylib &SourceJD,
- SymbolAliasMap CallableAliases, ImplSymbolMap *SrcJDLoc = nullptr,
- VModuleKey K = VModuleKey()) {
+ SymbolAliasMap CallableAliases,
+ ImplSymbolMap *SrcJDLoc = nullptr) {
return std::make_unique<LazyReexportsMaterializationUnit>(
- LCTManager, ISManager, SourceJD, std::move(CallableAliases), SrcJDLoc,
- std::move(K));
+ LCTManager, ISManager, SourceJD, std::move(CallableAliases), SrcJDLoc);
}
} // End namespace orc
diff --git a/llvm/include/llvm/ExecutionEngine/Orc/MachOPlatform.h b/llvm/include/llvm/ExecutionEngine/Orc/MachOPlatform.h
index 15cdc6bee3ce..90e1d4704f34 100644
--- a/llvm/include/llvm/ExecutionEngine/Orc/MachOPlatform.h
+++ b/llvm/include/llvm/ExecutionEngine/Orc/MachOPlatform.h
@@ -98,8 +98,9 @@ class MachOPlatform : public Platform {
ExecutionSession &getExecutionSession() const { return ES; }
Error setupJITDylib(JITDylib &JD) override;
- Error notifyAdding(JITDylib &JD, const MaterializationUnit &MU) override;
- Error notifyRemoving(JITDylib &JD, VModuleKey K) override;
+ Error notifyAdding(ResourceTracker &RT,
+ const MaterializationUnit &MU) override;
+ Error notifyRemoving(ResourceTracker &RT) override;
Expected<InitializerSequence> getInitializerSequence(JITDylib &JD);
@@ -119,6 +120,19 @@ class MachOPlatform : public Platform {
LocalDependenciesMap getSyntheticSymbolLocalDependencies(
MaterializationResponsibility &MR) override;
+ // FIXME: We should be tentatively tracking scraped sections and discarding
+ // if the MR fails.
+ Error notifyFailed(MaterializationResponsibility &MR) override {
+ return Error::success();
+ }
+
+ Error notifyRemovingResources(ResourceKey K) override {
+ return Error::success();
+ }
+
+ void notifyTransferringResources(ResourceKey DstKey,
+ ResourceKey SrcKey) override {}
+
private:
using InitSymbolDepMap =
DenseMap<MaterializationResponsibility *, JITLinkSymbolVector>;
diff --git a/llvm/include/llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h b/llvm/include/llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h
index cbcf3928be3d..b73217f09b54 100644
--- a/llvm/include/llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h
+++ b/llvm/include/llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h
@@ -51,7 +51,7 @@ class ObjectLinkingLayerJITLinkContext;
/// Clients can use this class to add relocatable object files to an
/// ExecutionSession, and it typically serves as the base layer (underneath
/// a compiling layer like IRCompileLayer) for the rest of the JIT.
-class ObjectLinkingLayer : public ObjectLayer {
+class ObjectLinkingLayer : public ObjectLayer, private ResourceManager {
friend class ObjectLinkingLayerJITLinkContext;
public:
@@ -72,10 +72,10 @@ class ObjectLinkingLayer : public ObjectLayer {
virtual Error notifyEmitted(MaterializationResponsibility &MR) {
return Error::success();
}
- virtual Error notifyRemovingModule(VModuleKey K) {
- return Error::success();
- }
- virtual Error notifyRemovingAllModules() { return Error::success(); }
+ virtual Error notifyFailed(MaterializationResponsibility &MR) = 0;
+ virtual Error notifyRemovingResources(ResourceKey K) = 0;
+ virtual void notifyTransferringResources(ResourceKey DstKey,
+ ResourceKey SrcKey) = 0;
/// Return any dependencies that synthetic symbols (e.g. init symbols)
/// have on locally scoped jitlink::Symbols. This is used by the
@@ -161,8 +161,8 @@ class ObjectLinkingLayer : public ObjectLayer {
void notifyLoaded(MaterializationResponsibility &MR);
Error notifyEmitted(MaterializationResponsibility &MR, AllocPtr Alloc);
- Error removeModule(VModuleKey K);
- Error removeAllModules();
+ Error handleRemoveResources(ResourceKey K) override;
+ void handleTransferResources(ResourceKey DstKey, ResourceKey SrcKey) override;
mutable std::mutex LayerMutex;
jitlink::JITLinkMemoryManager &MemMgr;
@@ -170,20 +170,22 @@ class ObjectLinkingLayer : public ObjectLayer {
bool OverrideObjectFlags = false;
bool AutoClaimObjectSymbols = false;
ReturnObjectBufferFunction ReturnObjectBuffer;
- DenseMap<VModuleKey, AllocPtr> TrackedAllocs;
- std::vector<AllocPtr> UntrackedAllocs;
+ DenseMap<ResourceKey, std::vector<AllocPtr>> Allocs;
std::vector<std::unique_ptr<Plugin>> Plugins;
};
class EHFrameRegistrationPlugin : public ObjectLinkingLayer::Plugin {
public:
EHFrameRegistrationPlugin(
+ ExecutionSession &ES,
std::unique_ptr<jitlink::EHFrameRegistrar> Registrar);
- Error notifyEmitted(MaterializationResponsibility &MR) override;
void modifyPassConfig(MaterializationResponsibility &MR, const Triple &TT,
jitlink::PassConfiguration &PassConfig) override;
- Error notifyRemovingModule(VModuleKey K) override;
- Error notifyRemovingAllModules() override;
+ Error notifyEmitted(MaterializationResponsibility &MR) override;
+ Error notifyFailed(MaterializationResponsibility &MR) override;
+ Error notifyRemovingResources(ResourceKey K) override;
+ void notifyTransferringResources(ResourceKey DstKey,
+ ResourceKey SrcKey) override;
private:
@@ -193,10 +195,10 @@ class EHFrameRegistrationPlugin : public ObjectLinkingLayer::Plugin {
};
std::mutex EHFramePluginMutex;
+ ExecutionSession &ES;
std::unique_ptr<jitlink::EHFrameRegistrar> Registrar;
DenseMap<MaterializationResponsibility *, EHFrameRange> InProcessLinks;
- DenseMap<VModuleKey, EHFrameRange> TrackedEHFrameRanges;
- std::vector<EHFrameRange> UntrackedEHFrameRanges;
+ DenseMap<ResourceKey, std::vector<EHFrameRange>> EHFrameRanges;
};
} // end namespace orc
diff --git a/llvm/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h b/llvm/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h
index eeec131503a1..7dfbf32b1ffa 100644
--- a/llvm/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h
+++ b/llvm/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h
@@ -35,16 +35,16 @@
namespace llvm {
namespace orc {
-class RTDyldObjectLinkingLayer : public ObjectLayer {
+class RTDyldObjectLinkingLayer : public ObjectLayer, private ResourceManager {
public:
/// Functor for receiving object-loaded notifications.
- using NotifyLoadedFunction =
- std::function<void(VModuleKey, const object::ObjectFile &Obj,
- const RuntimeDyld::LoadedObjectInfo &)>;
+ using NotifyLoadedFunction = std::function<void(
+ MaterializationResponsibility &R, const object::ObjectFile &Obj,
+ const RuntimeDyld::LoadedObjectInfo &)>;
/// Functor for receiving finalization notifications.
- using NotifyEmittedFunction =
- std::function<void(VModuleKey, std::unique_ptr<MemoryBuffer>)>;
+ using NotifyEmittedFunction = std::function<void(
+ MaterializationResponsibility &R, std::unique_ptr<MemoryBuffer>)>;
using GetMemoryManagerFunction =
std::function<std::unique_ptr<RuntimeDyld::MemoryManager>()>;
@@ -122,16 +122,23 @@ class RTDyldObjectLinkingLayer : public ObjectLayer {
void unregisterJITEventListener(JITEventListener &L);
private:
- Error onObjLoad(VModuleKey K, MaterializationResponsibility &R,
+ using MemoryManagerUP = std::unique_ptr<RuntimeDyld::MemoryManager>;
+
+ Error onObjLoad(MaterializationResponsibility &R,
const object::ObjectFile &Obj,
- RuntimeDyld::MemoryManager *MemMgr,
- std::unique_ptr<RuntimeDyld::LoadedObjectInfo> LoadedObjInfo,
+ RuntimeDyld::MemoryManager &MemMgr,
+ RuntimeDyld::LoadedObjectInfo &LoadedObjInfo,
std::map<StringRef, JITEvaluatedSymbol> Resolved,
std::set<StringRef> &InternalSymbols);
- void onObjEmit(VModuleKey K, MaterializationResponsibility &R,
+ void onObjEmit(MaterializationResponsibility &R,
object::OwningBinary<object::ObjectFile> O,
- RuntimeDyld::MemoryManager *MemMgr, Error Err);
+ std::unique_ptr<RuntimeDyld::MemoryManager> MemMgr,
+ std::unique_ptr<RuntimeDyld::LoadedObjectInfo> LoadedObjInfo,
+ Error Err);
+
+ Error handleRemoveResources(ResourceKey K) override;
+ void handleTransferResources(ResourceKey DstKey, ResourceKey SrcKey) override;
mutable std::mutex RTDyldLayerMutex;
GetMemoryManagerFunction GetMemoryManager;
@@ -140,11 +147,8 @@ class RTDyldObjectLinkingLayer : public ObjectLayer {
bool ProcessAllSections = false;
bool OverrideObjectFlags = false;
bool AutoClaimObjectSymbols = false;
- std::vector<std::unique_ptr<RuntimeDyld::MemoryManager>> MemMgrs;
+ DenseMap<ResourceKey, std::vector<MemoryManagerUP>> MemMgrs;
std::vector<JITEventListener *> EventListeners;
- DenseMap<RuntimeDyld::MemoryManager *,
- std::unique_ptr<RuntimeDyld::LoadedObjectInfo>>
- LoadedObjInfos;
};
} // end namespace orc
diff --git a/llvm/include/llvm/ExecutionEngine/RuntimeDyld.h b/llvm/include/llvm/ExecutionEngine/RuntimeDyld.h
index 1b3ce1127e4a..9b83092e653f 100644
--- a/llvm/include/llvm/ExecutionEngine/RuntimeDyld.h
+++ b/llvm/include/llvm/ExecutionEngine/RuntimeDyld.h
@@ -271,11 +271,11 @@ class RuntimeDyld {
object::OwningBinary<object::ObjectFile> O,
RuntimeDyld::MemoryManager &MemMgr, JITSymbolResolver &Resolver,
bool ProcessAllSections,
- unique_function<Error(const object::ObjectFile &Obj,
- std::unique_ptr<LoadedObjectInfo>,
+ unique_function<Error(const object::ObjectFile &Obj, LoadedObjectInfo &,
std::map<StringRef, JITEvaluatedSymbol>)>
OnLoaded,
- unique_function<void(object::OwningBinary<object::ObjectFile> O, Error)>
+ unique_function<void(object::OwningBinary<object::ObjectFile> O,
+ std::unique_ptr<LoadedObjectInfo>, Error)>
OnEmitted);
// RuntimeDyldImpl is the actual class. RuntimeDyld is just the public
@@ -298,10 +298,11 @@ void jitLinkForORC(
RuntimeDyld::MemoryManager &MemMgr, JITSymbolResolver &Resolver,
bool ProcessAllSections,
unique_function<Error(const object::ObjectFile &Obj,
- std::unique_ptr<RuntimeDyld::LoadedObjectInfo>,
+ RuntimeDyld::LoadedObjectInfo &,
std::map<StringRef, JITEvaluatedSymbol>)>
OnLoaded,
- unique_function<void(object::OwningBinary<object::ObjectFile>, Error)>
+ unique_function<void(object::OwningBinary<object::ObjectFile>,
+ std::unique_ptr<RuntimeDyld::LoadedObjectInfo>, Error)>
OnEmitted);
} // end namespace llvm
diff --git a/llvm/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp b/llvm/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp
index dfb0d06bdba3..68878f6729e9 100644
--- a/llvm/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp
+++ b/llvm/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp
@@ -73,17 +73,16 @@ class PartitioningIRMaterializationUnit : public IRMaterializationUnit {
public:
PartitioningIRMaterializationUnit(ExecutionSession &ES,
const IRSymbolMapper::ManglingOptions &MO,
- ThreadSafeModule TSM, VModuleKey K,
+ ThreadSafeModule TSM,
CompileOnDemandLayer &Parent)
- : IRMaterializationUnit(ES, MO, std::move(TSM), std::move(K)),
- Parent(Parent) {}
+ : IRMaterializationUnit(ES, MO, std::move(TSM)), Parent(Parent) {}
PartitioningIRMaterializationUnit(
- ThreadSafeModule TSM, VModuleKey K, SymbolFlagsMap SymbolFlags,
+ ThreadSafeModule TSM, SymbolFlagsMap SymbolFlags,
SymbolStringPtr InitSymbol, SymbolNameToDefinitionMap SymbolToDefinition,
CompileOnDemandLayer &Parent)
- : IRMaterializationUnit(std::move(TSM), std::move(K),
- std::move(SymbolFlags), std::move(InitSymbol),
+ : IRMaterializationUnit(std::move(TSM), std::move(SymbolFlags),
+ std::move(InitSymbol),
std::move(SymbolToDefinition)),
Parent(Parent) {}
@@ -158,19 +157,29 @@ void CompileOnDemandLayer::emit(
// implementation dylib.
if (auto Err = PDR.getImplDylib().define(
std::make_unique<PartitioningIRMaterializationUnit>(
- ES, *getManglingOptions(), std::move(TSM), R->getVModuleKey(),
- *this))) {
+ ES, *getManglingOptions(), std::move(TSM), *this))) {
ES.reportError(std::move(Err));
R->failMaterialization();
return;
}
if (!NonCallables.empty())
- R->replace(reexports(PDR.getImplDylib(), std::move(NonCallables),
- JITDylibLookupFlags::MatchAllSymbols));
- if (!Callables.empty())
- R->replace(lazyReexports(LCTMgr, PDR.getISManager(), PDR.getImplDylib(),
- std::move(Callables), AliaseeImpls));
+ if (auto Err =
+ R->replace(reexports(PDR.getImplDylib(), std::move(NonCallables),
+ JITDylibLookupFlags::MatchAllSymbols))) {
+ getExecutionSession().reportError(std::move(Err));
+ R->failMaterialization();
+ return;
+ }
+ if (!Callables.empty()) {
+ if (auto Err = R->replace(
+ lazyReexports(LCTMgr, PDR.getISManager(), PDR.getImplDylib(),
+ std::move(Callables), AliaseeImpls))) {
+ getExecutionSession().reportError(std::move(Err));
+ R->failMaterialization();
+ return;
+ }
+ }
}
CompileOnDemandLayer::PerDylibResources &
@@ -285,9 +294,14 @@ void CompileOnDemandLayer::emitPartition(
// If the partition is empty, return the whole module to the symbol table.
if (GVsToExtract->empty()) {
- R->replace(std::make_unique<PartitioningIRMaterializationUnit>(
- std::move(TSM), R->getVModuleKey(), R->getSymbols(),
- R->getInitializerSymbol(), std::move(Defs), *this));
+ if (auto Err =
+ R->replace(std::make_unique<PartitioningIRMaterializationUnit>(
+ std::move(TSM), R->getSymbols(), R->getInitializerSymbol(),
+ std::move(Defs), *this))) {
+ getExecutionSession().reportError(std::move(Err));
+ R->failMaterialization();
+ return;
+ }
return;
}
@@ -352,8 +366,12 @@ void CompileOnDemandLayer::emitPartition(
return;
}
- R->replace(std::make_unique<PartitioningIRMaterializationUnit>(
- ES, *getManglingOptions(), std::move(TSM), R->getVModuleKey(), *this));
+ if (auto Err = R->replace(std::make_unique<PartitioningIRMaterializationUnit>(
+ ES, *getManglingOptions(), std::move(TSM), *this))) {
+ ES.reportError(std::move(Err));
+ R->failMaterialization();
+ return;
+ }
BaseLayer.emit(std::move(R), std::move(*ExtractedTSM));
}
diff --git a/llvm/lib/ExecutionEngine/Orc/Core.cpp b/llvm/lib/ExecutionEngine/Orc/Core.cpp
index 9227f3f044ac..8709cc87d258 100644
--- a/llvm/lib/ExecutionEngine/Orc/Core.cpp
+++ b/llvm/lib/ExecutionEngine/Orc/Core.cpp
@@ -12,6 +12,7 @@
#include "llvm/Config/llvm-config.h"
#include "llvm/ExecutionEngine/Orc/DebugUtils.h"
#include "llvm/ExecutionEngine/Orc/OrcError.h"
+#include "llvm/Support/FormatVariadic.h"
#include <condition_variable>
#if LLVM_ENABLE_THREADS
@@ -23,6 +24,7 @@
namespace llvm {
namespace orc {
+char ResourceTrackerDefunct::ID = 0;
char FailedToMaterialize::ID = 0;
char SymbolsNotFound::ID = 0;
char SymbolsCouldNotBeRemoved::ID = 0;
@@ -34,6 +36,45 @@ RegisterDependenciesFunction NoDependenciesToRegister =
void MaterializationUnit::anchor() {}
+ResourceTracker::ResourceTracker(JITDylibSP JD) {
+ assert((reinterpret_cast<uintptr_t>(JD.get()) & 0x1) == 0 &&
+ "JITDylib must be two byte aligned");
+ JD->Retain();
+ JDAndFlag.store(reinterpret_cast<uintptr_t>(JD.get()));
+}
+
+ResourceTracker::~ResourceTracker() {
+ getJITDylib().getExecutionSession().destroyResourceTracker(*this);
+ getJITDylib().Release();
+}
+
+Error ResourceTracker::remove() {
+ return getJITDylib().getExecutionSession().removeResourceTracker(*this);
+}
+
+void ResourceTracker::transferTo(ResourceTracker &DstRT) {
+ getJITDylib().getExecutionSession().transferResourceTracker(DstRT, *this);
+}
+
+void ResourceTracker::makeDefunct() {
+ uintptr_t Val = JDAndFlag.load();
+ Val |= 0x1U;
+ JDAndFlag.store(Val);
+}
+
+ResourceManager::~ResourceManager() {}
+
+ResourceTrackerDefunct::ResourceTrackerDefunct(ResourceTrackerSP RT)
+ : RT(std::move(RT)) {}
+
+std::error_code ResourceTrackerDefunct::convertToErrorCode() const {
+ return orcError(OrcErrorCode::UnknownORCError);
+}
+
+void ResourceTrackerDefunct::log(raw_ostream &OS) const {
+ OS << "Resource tracker " << (void *)RT.get() << " became defunct";
+}
+
FailedToMaterialize::FailedToMaterialize(
std::shared_ptr<SymbolDependenceMap> Symbols)
: Symbols(std::move(Symbols)) {
@@ -182,6 +223,7 @@ void AsynchronousSymbolQuery::detach() {
MaterializationResponsibility::~MaterializationResponsibility() {
assert(SymbolFlags.empty() &&
"All symbols should have been explicitly materialized or failed");
+ JD->unlinkMaterializationResponsibility(*this);
}
SymbolNameSet MaterializationResponsibility::getRequestedSymbols() const {
@@ -205,7 +247,7 @@ Error MaterializationResponsibility::notifyResolved(const SymbolMap &Symbols) {
}
#endif
- return JD->resolve(Symbols);
+ return JD->resolve(*this, Symbols);
}
Error MaterializationResponsibility::notifyEmitted() {
@@ -214,7 +256,7 @@ Error MaterializationResponsibility::notifyEmitted() {
dbgs() << "In " << JD->getName() << " emitting " << SymbolFlags << "\n";
});
- if (auto Err = JD->emit(SymbolFlags))
+ if (auto Err = JD->emit(*this, SymbolFlags))
return Err;
SymbolFlags.clear();
@@ -250,16 +292,30 @@ void MaterializationResponsibility::failMaterialization() {
Worklist.push_back(std::make_pair(JD.get(), KV.first));
SymbolFlags.clear();
- JD->notifyFailed(std::move(Worklist));
+ if (Worklist.empty())
+ return;
+
+ auto &ES = JD->getExecutionSession();
+ JITDylib::AsynchronousSymbolQuerySet FailedQueries;
+ std::shared_ptr<SymbolDependenceMap> FailedSymbols;
+
+ ES.runSessionLocked([&]() {
+ auto RTI = JD->MRTrackers.find(this);
+ assert(RTI != JD->MRTrackers.end() && "No tracker for this");
+ if (RTI->second->isDefunct())
+ return;
+
+ std::tie(FailedQueries, FailedSymbols) =
+ JITDylib::failSymbols(std::move(Worklist));
+ });
+
+ for (auto &Q : FailedQueries)
+ Q->handleFailed(make_error<FailedToMaterialize>(FailedSymbols));
}
-void MaterializationResponsibility::replace(
+Error MaterializationResponsibility::replace(
std::unique_ptr<MaterializationUnit> MU) {
- // If the replacement MU is empty then return.
- if (MU->getSymbols().empty())
- return;
-
for (auto &KV : MU->getSymbols()) {
assert(SymbolFlags.count(KV.first) &&
"Replacing definition outside this responsibility set");
@@ -274,15 +330,11 @@ void MaterializationResponsibility::replace(
<< "\n";
}););
- JD->replace(std::move(MU));
+ return JD->replace(*this, std::move(MU));
}
-std::unique_ptr<MaterializationResponsibility>
-MaterializationResponsibility::delegate(const SymbolNameSet &Symbols,
- VModuleKey NewKey) {
-
- if (NewKey == VModuleKey())
- NewKey = K;
+Expected<std::unique_ptr<MaterializationResponsibility>>
+MaterializationResponsibility::delegate(const SymbolNameSet &Symbols) {
SymbolStringPtr DelegatedInitSymbol;
SymbolFlagsMap DelegatedFlags;
@@ -300,10 +352,8 @@ MaterializationResponsibility::delegate(const SymbolNameSet &Symbols,
SymbolFlags.erase(I);
}
- return std::unique_ptr<MaterializationResponsibility>(
- new MaterializationResponsibility(JD, std::move(DelegatedFlags),
- std::move(DelegatedInitSymbol),
- std::move(NewKey)));
+ return JD->delegate(*this, std::move(DelegatedFlags),
+ std::move(DelegatedInitSymbol));
}
void MaterializationResponsibility::addDependencies(
@@ -328,8 +378,8 @@ void MaterializationResponsibility::addDependenciesForAll(
}
AbsoluteSymbolsMaterializationUnit::AbsoluteSymbolsMaterializationUnit(
- SymbolMap Symbols, VModuleKey K)
- : MaterializationUnit(extractFlags(Symbols), nullptr, std::move(K)),
+ SymbolMap Symbols)
+ : MaterializationUnit(extractFlags(Symbols), nullptr),
Symbols(std::move(Symbols)) {}
StringRef AbsoluteSymbolsMaterializationUnit::getName() const {
@@ -359,10 +409,9 @@ AbsoluteSymbolsMaterializationUnit::extractFlags(const SymbolMap &Symbols) {
ReExportsMaterializationUnit::ReExportsMaterializationUnit(
JITDylib *SourceJD, JITDylibLookupFlags SourceJDLookupFlags,
- SymbolAliasMap Aliases, VModuleKey K)
- : MaterializationUnit(extractFlags(Aliases), nullptr, std::move(K)),
- SourceJD(SourceJD), SourceJDLookupFlags(SourceJDLookupFlags),
- Aliases(std::move(Aliases)) {}
+ SymbolAliasMap Aliases)
+ : MaterializationUnit(extractFlags(Aliases), nullptr), SourceJD(SourceJD),
+ SourceJDLookupFlags(SourceJDLookupFlags), Aliases(std::move(Aliases)) {}
StringRef ReExportsMaterializationUnit::getName() const {
return "<Reexports>";
@@ -397,10 +446,17 @@ void ReExportsMaterializationUnit::materialize(
});
if (!Aliases.empty()) {
- if (SourceJD)
- R->replace(reexports(*SourceJD, std::move(Aliases), SourceJDLookupFlags));
- else
- R->replace(symbolAliases(std::move(Aliases)));
+ auto Err = SourceJD ? R->replace(reexports(*SourceJD, std::move(Aliases),
+ SourceJDLookupFlags))
+ : R->replace(symbolAliases(std::move(Aliases)));
+
+ if (Err) {
+ // FIXME: Should this be reported / treated as failure to materialize?
+ // Or should this be treated as a sanctioned bailing-out?
+ ES.reportError(std::move(Err));
+ R->failMaterialization();
+ return;
+ }
}
// The OnResolveInfo struct will hold the aliases and responsibilty for each
@@ -450,8 +506,15 @@ void ReExportsMaterializationUnit::materialize(
assert(!QuerySymbols.empty() && "Alias cycle detected!");
- auto QueryInfo = std::make_shared<OnResolveInfo>(
- R->delegate(ResponsibilitySymbols), std::move(QueryAliases));
+ auto NewR = R->delegate(ResponsibilitySymbols);
+ if (!NewR) {
+ ES.reportError(NewR.takeError());
+ R->failMaterialization();
+ return;
+ }
+
+ auto QueryInfo = std::make_shared<OnResolveInfo>(std::move(*NewR),
+ std::move(QueryAliases));
QueryInfos.push_back(
make_pair(std::move(QuerySymbols), std::move(QueryInfo)));
}
@@ -592,6 +655,35 @@ Error ReexportsGenerator::tryToGenerate(LookupKind K, JITDylib &JD,
JITDylib::DefinitionGenerator::~DefinitionGenerator() {}
+Error JITDylib::clear() {
+ std::vector<ResourceTrackerSP> TrackersToRemove;
+ ES.runSessionLocked([&]() {
+ for (auto &KV : TrackerSymbols)
+ TrackersToRemove.push_back(KV.first);
+ TrackersToRemove.push_back(getDefaultResourceTracker());
+ });
+
+ Error Err = Error::success();
+ for (auto &RT : TrackersToRemove)
+ Err = joinErrors(std::move(Err), RT->remove());
+ return Err;
+}
+
+ResourceTrackerSP JITDylib::getDefaultResourceTracker() {
+ return ES.runSessionLocked([this] {
+ if (!DefaultTracker)
+ DefaultTracker = new ResourceTracker(this);
+ return DefaultTracker;
+ });
+}
+
+ResourceTrackerSP JITDylib::createResourceTracker() {
+ return ES.runSessionLocked([this] {
+ ResourceTrackerSP RT = new ResourceTracker(this);
+ return RT;
+ });
+}
+
void JITDylib::removeGenerator(DefinitionGenerator &G) {
ES.runSessionLocked([&]() {
auto I = std::find_if(DefGenerators.begin(), DefGenerators.end(),
@@ -652,11 +744,18 @@ JITDylib::defineMaterializing(SymbolFlagsMap SymbolFlags) {
});
}
-void JITDylib::replace(std::unique_ptr<MaterializationUnit> MU) {
+Error JITDylib::replace(MaterializationResponsibility &FromMR,
+ std::unique_ptr<MaterializationUnit> MU) {
assert(MU != nullptr && "Can not replace with a null MaterializationUnit");
+ std::unique_ptr<MaterializationUnit> MustRunMU;
+ std::unique_ptr<MaterializationResponsibility> MustRunMR;
- auto MustRunMU =
- ES.runSessionLocked([&, this]() -> std::unique_ptr<MaterializationUnit> {
+ auto Err =
+ ES.runSessionLocked([&, this]() -> Error {
+ auto RT = getTracker(FromMR);
+
+ if (RT->isDefunct())
+ return make_error<ResourceTrackerDefunct>(std::move(RT));
#ifndef NDEBUG
for (auto &KV : MU->getSymbols()) {
@@ -671,18 +770,27 @@ void JITDylib::replace(std::unique_ptr<MaterializationUnit> MU) {
}
#endif // NDEBUG
+ // If the tracker is defunct we need to bail out immediately.
+
// If any symbol has pending queries against it then we need to
// materialize MU immediately.
for (auto &KV : MU->getSymbols()) {
auto MII = MaterializingInfos.find(KV.first);
if (MII != MaterializingInfos.end()) {
- if (MII->second.hasQueriesPending())
- return std::move(MU);
+ if (MII->second.hasQueriesPending()) {
+ MustRunMR = ES.createMaterializationResponsibility(
+ *RT, std::move(MU->SymbolFlags), std::move(MU->InitSymbol));
+ MustRunMU = std::move(MU);
+ return Error::success();
+ }
}
}
// Otherwise, make MU responsible for all the symbols.
- auto UMI = std::make_shared<UnmaterializedInfo>(std::move(MU));
+ auto RTI = MRTrackers.find(&FromMR);
+ assert(RTI != MRTrackers.end() && "No tracker for FromMR");
+ auto UMI =
+ std::make_shared<UnmaterializedInfo>(std::move(MU), RTI->second);
for (auto &KV : UMI->MU->getSymbols()) {
auto SymI = Symbols.find(KV.first);
assert(SymI->second.getState() == SymbolState::Materializing &&
@@ -700,14 +808,36 @@ void JITDylib::replace(std::unique_ptr<MaterializationUnit> MU) {
UMIEntry = UMI;
}
- return nullptr;
+ return Error::success();
});
+ if (Err)
+ return Err;
+
if (MustRunMU) {
- auto MR =
- MustRunMU->createMaterializationResponsibility(shared_from_this());
- ES.dispatchMaterialization(std::move(MustRunMU), std::move(MR));
+ assert(MustRunMR && "MustRunMU set implies MustRunMR set");
+ ES.dispatchMaterialization(std::move(MustRunMU), std::move(MustRunMR));
+ } else {
+ assert(!MustRunMR && "MustRunMU unset implies MustRunMR unset");
}
+
+ return Error::success();
+}
+
+Expected<std::unique_ptr<MaterializationResponsibility>>
+JITDylib::delegate(MaterializationResponsibility &FromMR,
+ SymbolFlagsMap SymbolFlags, SymbolStringPtr InitSymbol) {
+
+ return ES.runSessionLocked(
+ [&]() -> Expected<std::unique_ptr<MaterializationResponsibility>> {
+ auto RT = getTracker(FromMR);
+
+ if (RT->isDefunct())
+ return make_error<ResourceTrackerDefunct>(std::move(RT));
+
+ return ES.createMaterializationResponsibility(
+ *RT, std::move(SymbolFlags), std::move(InitSymbol));
+ });
}
SymbolNameSet
@@ -808,89 +938,93 @@ void JITDylib::addDependencies(const SymbolStringPtr &Name,
Symbols[Name].setFlags(Symbols[Name].getFlags() | JITSymbolFlags::HasError);
}
-Error JITDylib::resolve(const SymbolMap &Resolved) {
- SymbolNameSet SymbolsInErrorState;
+Error JITDylib::resolve(MaterializationResponsibility &MR,
+ const SymbolMap &Resolved) {
AsynchronousSymbolQuerySet CompletedQueries;
- ES.runSessionLocked([&, this]() {
- struct WorklistEntry {
- SymbolTable::iterator SymI;
- JITEvaluatedSymbol ResolvedSym;
- };
-
- std::vector<WorklistEntry> Worklist;
- Worklist.reserve(Resolved.size());
+ if (auto Err = ES.runSessionLocked([&, this]() -> Error {
+ auto RTI = MRTrackers.find(&MR);
+ assert(RTI != MRTrackers.end() && "No resource tracker for MR?");
+ if (RTI->second->isDefunct())
+ return make_error<ResourceTrackerDefunct>(RTI->second);
- // Build worklist and check for any symbols in the error state.
- for (const auto &KV : Resolved) {
+ struct WorklistEntry {
+ SymbolTable::iterator SymI;
+ JITEvaluatedSymbol ResolvedSym;
+ };
- assert(!KV.second.getFlags().hasError() &&
- "Resolution result can not have error flag set");
+ SymbolNameSet SymbolsInErrorState;
+ std::vector<WorklistEntry> Worklist;
+ Worklist.reserve(Resolved.size());
- auto SymI = Symbols.find(KV.first);
+ // Build worklist and check for any symbols in the error state.
+ for (const auto &KV : Resolved) {
- assert(SymI != Symbols.end() && "Symbol not found");
- assert(!SymI->second.hasMaterializerAttached() &&
- "Resolving symbol with materializer attached?");
- assert(SymI->second.getState() == SymbolState::Materializing &&
- "Symbol should be materializing");
- assert(SymI->second.getAddress() == 0 &&
- "Symbol has already been resolved");
+ assert(!KV.second.getFlags().hasError() &&
+ "Resolution result can not have error flag set");
- if (SymI->second.getFlags().hasError())
- SymbolsInErrorState.insert(KV.first);
- else {
- auto Flags = KV.second.getFlags();
- Flags &= ~(JITSymbolFlags::Weak | JITSymbolFlags::Common);
- assert(Flags == (SymI->second.getFlags() &
- ~(JITSymbolFlags::Weak | JITSymbolFlags::Common)) &&
- "Resolved flags should match the declared flags");
-
- Worklist.push_back(
- {SymI, JITEvaluatedSymbol(KV.second.getAddress(), Flags)});
- }
- }
+ auto SymI = Symbols.find(KV.first);
- // If any symbols were in the error state then bail out.
- if (!SymbolsInErrorState.empty())
- return;
+ assert(SymI != Symbols.end() && "Symbol not found");
+ assert(!SymI->second.hasMaterializerAttached() &&
+ "Resolving symbol with materializer attached?");
+ assert(SymI->second.getState() == SymbolState::Materializing &&
+ "Symbol should be materializing");
+ assert(SymI->second.getAddress() == 0 &&
+ "Symbol has already been resolved");
+
+ if (SymI->second.getFlags().hasError())
+ SymbolsInErrorState.insert(KV.first);
+ else {
+ auto Flags = KV.second.getFlags();
+ Flags &= ~(JITSymbolFlags::Weak | JITSymbolFlags::Common);
+ assert(Flags ==
+ (SymI->second.getFlags() &
+ ~(JITSymbolFlags::Weak | JITSymbolFlags::Common)) &&
+ "Resolved flags should match the declared flags");
+
+ Worklist.push_back(
+ {SymI, JITEvaluatedSymbol(KV.second.getAddress(), Flags)});
+ }
+ }
- while (!Worklist.empty()) {
- auto SymI = Worklist.back().SymI;
- auto ResolvedSym = Worklist.back().ResolvedSym;
- Worklist.pop_back();
+ // If any symbols were in the error state then bail out.
+ if (!SymbolsInErrorState.empty()) {
+ auto FailedSymbolsDepMap = std::make_shared<SymbolDependenceMap>();
+ (*FailedSymbolsDepMap)[this] = std::move(SymbolsInErrorState);
+ return make_error<FailedToMaterialize>(
+ std::move(FailedSymbolsDepMap));
+ }
- auto &Name = SymI->first;
+ while (!Worklist.empty()) {
+ auto SymI = Worklist.back().SymI;
+ auto ResolvedSym = Worklist.back().ResolvedSym;
+ Worklist.pop_back();
- // Resolved symbols can not be weak: discard the weak flag.
- JITSymbolFlags ResolvedFlags = ResolvedSym.getFlags();
- SymI->second.setAddress(ResolvedSym.getAddress());
- SymI->second.setFlags(ResolvedFlags);
- SymI->second.setState(SymbolState::Resolved);
+ auto &Name = SymI->first;
- auto MII = MaterializingInfos.find(Name);
- if (MII == MaterializingInfos.end())
- continue;
+ // Resolved symbols can not be weak: discard the weak flag.
+ JITSymbolFlags ResolvedFlags = ResolvedSym.getFlags();
+ SymI->second.setAddress(ResolvedSym.getAddress());
+ SymI->second.setFlags(ResolvedFlags);
+ SymI->second.setState(SymbolState::Resolved);
- auto &MI = MII->second;
- for (auto &Q : MI.takeQueriesMeeting(SymbolState::Resolved)) {
- Q->notifySymbolMetRequiredState(Name, ResolvedSym);
- Q->removeQueryDependence(*this, Name);
- if (Q->isComplete())
- CompletedQueries.insert(std::move(Q));
- }
- }
- });
+ auto MII = MaterializingInfos.find(Name);
+ if (MII == MaterializingInfos.end())
+ continue;
- assert((SymbolsInErrorState.empty() || CompletedQueries.empty()) &&
- "Can't fail symbols and completed queries at the same time");
+ auto &MI = MII->second;
+ for (auto &Q : MI.takeQueriesMeeting(SymbolState::Resolved)) {
+ Q->notifySymbolMetRequiredState(Name, ResolvedSym);
+ Q->removeQueryDependence(*this, Name);
+ if (Q->isComplete())
+ CompletedQueries.insert(std::move(Q));
+ }
+ }
- // If we failed any symbols then return an error.
- if (!SymbolsInErrorState.empty()) {
- auto FailedSymbolsDepMap = std::make_shared<SymbolDependenceMap>();
- (*FailedSymbolsDepMap)[this] = std::move(SymbolsInErrorState);
- return make_error<FailedToMaterialize>(std::move(FailedSymbolsDepMap));
- }
+ return Error::success();
+ }))
+ return Err;
// Otherwise notify all the completed queries.
for (auto &Q : CompletedQueries) {
@@ -901,139 +1035,145 @@ Error JITDylib::resolve(const SymbolMap &Resolved) {
return Error::success();
}
-Error JITDylib::emit(const SymbolFlagsMap &Emitted) {
+Error JITDylib::emit(MaterializationResponsibility &MR,
+ const SymbolFlagsMap &Emitted) {
AsynchronousSymbolQuerySet CompletedQueries;
- SymbolNameSet SymbolsInErrorState;
DenseMap<JITDylib *, SymbolNameVector> ReadySymbols;
- ES.runSessionLocked([&, this]() {
- std::vector<SymbolTable::iterator> Worklist;
+ if (auto Err = ES.runSessionLocked([&, this]() -> Error {
+ auto RTI = MRTrackers.find(&MR);
+ assert(RTI != MRTrackers.end() && "No resource tracker for MR?");
+ if (RTI->second->isDefunct())
+ return make_error<ResourceTrackerDefunct>(RTI->second);
- // Scan to build worklist, record any symbols in the erorr state.
- for (const auto &KV : Emitted) {
- auto &Name = KV.first;
+ SymbolNameSet SymbolsInErrorState;
+ std::vector<SymbolTable::iterator> Worklist;
- auto SymI = Symbols.find(Name);
- assert(SymI != Symbols.end() && "No symbol table entry for Name");
+ // Scan to build worklist, record any symbols in the erorr state.
+ for (const auto &KV : Emitted) {
+ auto &Name = KV.first;
- if (SymI->second.getFlags().hasError())
- SymbolsInErrorState.insert(Name);
- else
- Worklist.push_back(SymI);
- }
+ auto SymI = Symbols.find(Name);
+ assert(SymI != Symbols.end() && "No symbol table entry for Name");
- // If any symbols were in the error state then bail out.
- if (!SymbolsInErrorState.empty())
- return;
+ if (SymI->second.getFlags().hasError())
+ SymbolsInErrorState.insert(Name);
+ else
+ Worklist.push_back(SymI);
+ }
- // Otherwise update dependencies and move to the emitted state.
- while (!Worklist.empty()) {
- auto SymI = Worklist.back();
- Worklist.pop_back();
+ // If any symbols were in the error state then bail out.
+ if (!SymbolsInErrorState.empty()) {
+ auto FailedSymbolsDepMap = std::make_shared<SymbolDependenceMap>();
+ (*FailedSymbolsDepMap)[this] = std::move(SymbolsInErrorState);
+ return make_error<FailedToMaterialize>(
+ std::move(FailedSymbolsDepMap));
+ }
- auto &Name = SymI->first;
- auto &SymEntry = SymI->second;
+ // Otherwise update dependencies and move to the emitted state.
+ while (!Worklist.empty()) {
+ auto SymI = Worklist.back();
+ Worklist.pop_back();
- // Move symbol to the emitted state.
- assert(((SymEntry.getFlags().hasMaterializationSideEffectsOnly() &&
- SymEntry.getState() == SymbolState::Materializing) ||
- SymEntry.getState() == SymbolState::Resolved) &&
- "Emitting from state other than Resolved");
- SymEntry.setState(SymbolState::Emitted);
+ auto &Name = SymI->first;
+ auto &SymEntry = SymI->second;
- auto MII = MaterializingInfos.find(Name);
+ // Move symbol to the emitted state.
+ assert(((SymEntry.getFlags().hasMaterializationSideEffectsOnly() &&
+ SymEntry.getState() == SymbolState::Materializing) ||
+ SymEntry.getState() == SymbolState::Resolved) &&
+ "Emitting from state other than Resolved");
+ SymEntry.setState(SymbolState::Emitted);
- // If this symbol has no MaterializingInfo then it's trivially ready.
- // Update its state and continue.
- if (MII == MaterializingInfos.end()) {
- SymEntry.setState(SymbolState::Ready);
- continue;
- }
+ auto MII = MaterializingInfos.find(Name);
+
+ // If this symbol has no MaterializingInfo then it's trivially ready.
+ // Update its state and continue.
+ if (MII == MaterializingInfos.end()) {
+ SymEntry.setState(SymbolState::Ready);
+ continue;
+ }
+
+ auto &MI = MII->second;
+
+ // For each dependant, transfer this node's emitted dependencies to
+ // it. If the dependant node is ready (i.e. has no unemitted
+ // dependencies) then notify any pending queries.
+ for (auto &KV : MI.Dependants) {
+ auto &DependantJD = *KV.first;
+ auto &DependantJDReadySymbols = ReadySymbols[&DependantJD];
+ for (auto &DependantName : KV.second) {
+ auto DependantMII =
+ DependantJD.MaterializingInfos.find(DependantName);
+ assert(DependantMII != DependantJD.MaterializingInfos.end() &&
+ "Dependant should have MaterializingInfo");
+
+ auto &DependantMI = DependantMII->second;
+
+ // Remove the dependant's dependency on this node.
+ assert(DependantMI.UnemittedDependencies.count(this) &&
+ "Dependant does not have an unemitted dependencies record "
+ "for "
+ "this JITDylib");
+ assert(DependantMI.UnemittedDependencies[this].count(Name) &&
+ "Dependant does not count this symbol as a dependency?");
+
+ DependantMI.UnemittedDependencies[this].erase(Name);
+ if (DependantMI.UnemittedDependencies[this].empty())
+ DependantMI.UnemittedDependencies.erase(this);
+
+ // Transfer unemitted dependencies from this node to the
+ // dependant.
+ DependantJD.transferEmittedNodeDependencies(DependantMI,
+ DependantName, MI);
+
+ auto DependantSymI = DependantJD.Symbols.find(DependantName);
+ assert(DependantSymI != DependantJD.Symbols.end() &&
+ "Dependant has no entry in the Symbols table");
+ auto &DependantSymEntry = DependantSymI->second;
+
+ // If the dependant is emitted and this node was the last of its
+ // unemitted dependencies then the dependant node is now ready, so
+ // notify any pending queries on the dependant node.
+ if (DependantSymEntry.getState() == SymbolState::Emitted &&
+ DependantMI.UnemittedDependencies.empty()) {
+ assert(DependantMI.Dependants.empty() &&
+ "Dependants should be empty by now");
+
+ // Since this dependant is now ready, we erase its
+ // MaterializingInfo and update its materializing state.
+ DependantSymEntry.setState(SymbolState::Ready);
+ DependantJDReadySymbols.push_back(DependantName);
+
+ for (auto &Q :
+ DependantMI.takeQueriesMeeting(SymbolState::Ready)) {
+ Q->notifySymbolMetRequiredState(
+ DependantName, DependantSymI->second.getSymbol());
+ if (Q->isComplete())
+ CompletedQueries.insert(Q);
+ Q->removeQueryDependence(DependantJD, DependantName);
+ }
+ }
+ }
+ }
- auto &MI = MII->second;
-
- // For each dependant, transfer this node's emitted dependencies to
- // it. If the dependant node is ready (i.e. has no unemitted
- // dependencies) then notify any pending queries.
- for (auto &KV : MI.Dependants) {
- auto &DependantJD = *KV.first;
- auto &DependantJDReadySymbols = ReadySymbols[&DependantJD];
- for (auto &DependantName : KV.second) {
- auto DependantMII =
- DependantJD.MaterializingInfos.find(DependantName);
- assert(DependantMII != DependantJD.MaterializingInfos.end() &&
- "Dependant should have MaterializingInfo");
-
- auto &DependantMI = DependantMII->second;
-
- // Remove the dependant's dependency on this node.
- assert(DependantMI.UnemittedDependencies.count(this) &&
- "Dependant does not have an unemitted dependencies record for "
- "this JITDylib");
- assert(DependantMI.UnemittedDependencies[this].count(Name) &&
- "Dependant does not count this symbol as a dependency?");
-
- DependantMI.UnemittedDependencies[this].erase(Name);
- if (DependantMI.UnemittedDependencies[this].empty())
- DependantMI.UnemittedDependencies.erase(this);
-
- // Transfer unemitted dependencies from this node to the dependant.
- DependantJD.transferEmittedNodeDependencies(DependantMI,
- DependantName, MI);
-
- auto DependantSymI = DependantJD.Symbols.find(DependantName);
- assert(DependantSymI != DependantJD.Symbols.end() &&
- "Dependant has no entry in the Symbols table");
- auto &DependantSymEntry = DependantSymI->second;
-
- // If the dependant is emitted and this node was the last of its
- // unemitted dependencies then the dependant node is now ready, so
- // notify any pending queries on the dependant node.
- if (DependantSymEntry.getState() == SymbolState::Emitted &&
- DependantMI.UnemittedDependencies.empty()) {
- assert(DependantMI.Dependants.empty() &&
- "Dependants should be empty by now");
-
- // Since this dependant is now ready, we erase its MaterializingInfo
- // and update its materializing state.
- DependantSymEntry.setState(SymbolState::Ready);
- DependantJDReadySymbols.push_back(DependantName);
-
- for (auto &Q : DependantMI.takeQueriesMeeting(SymbolState::Ready)) {
- Q->notifySymbolMetRequiredState(
- DependantName, DependantSymI->second.getSymbol());
+ auto &ThisJDReadySymbols = ReadySymbols[this];
+ MI.Dependants.clear();
+ if (MI.UnemittedDependencies.empty()) {
+ SymI->second.setState(SymbolState::Ready);
+ ThisJDReadySymbols.push_back(Name);
+ for (auto &Q : MI.takeQueriesMeeting(SymbolState::Ready)) {
+ Q->notifySymbolMetRequiredState(Name, SymI->second.getSymbol());
if (Q->isComplete())
CompletedQueries.insert(Q);
- Q->removeQueryDependence(DependantJD, DependantName);
+ Q->removeQueryDependence(*this, Name);
}
}
}
- }
-
- auto &ThisJDReadySymbols = ReadySymbols[this];
- MI.Dependants.clear();
- if (MI.UnemittedDependencies.empty()) {
- SymI->second.setState(SymbolState::Ready);
- ThisJDReadySymbols.push_back(Name);
- for (auto &Q : MI.takeQueriesMeeting(SymbolState::Ready)) {
- Q->notifySymbolMetRequiredState(Name, SymI->second.getSymbol());
- if (Q->isComplete())
- CompletedQueries.insert(Q);
- Q->removeQueryDependence(*this, Name);
- }
- }
- }
- });
- assert((SymbolsInErrorState.empty() || CompletedQueries.empty()) &&
- "Can't fail symbols and completed queries at the same time");
-
- // If we failed any symbols then return an error.
- if (!SymbolsInErrorState.empty()) {
- auto FailedSymbolsDepMap = std::make_shared<SymbolDependenceMap>();
- (*FailedSymbolsDepMap)[this] = std::move(SymbolsInErrorState);
- return make_error<FailedToMaterialize>(std::move(FailedSymbolsDepMap));
- }
+ return Error::success();
+ }))
+ return Err;
// Otherwise notify all the completed queries.
for (auto &Q : CompletedQueries) {
@@ -1044,120 +1184,122 @@ Error JITDylib::emit(const SymbolFlagsMap &Emitted) {
return Error::success();
}
-void JITDylib::notifyFailed(FailedSymbolsWorklist Worklist) {
- AsynchronousSymbolQuerySet FailedQueries;
- auto FailedSymbolsMap = std::make_shared<SymbolDependenceMap>();
-
- // Failing no symbols is a no-op.
- if (Worklist.empty())
- return;
-
- auto &ES = Worklist.front().first->getExecutionSession();
-
+void JITDylib::unlinkMaterializationResponsibility(
+ MaterializationResponsibility &MR) {
ES.runSessionLocked([&]() {
- while (!Worklist.empty()) {
- assert(Worklist.back().first && "Failed JITDylib can not be null");
- auto &JD = *Worklist.back().first;
- auto Name = std::move(Worklist.back().second);
- Worklist.pop_back();
-
- (*FailedSymbolsMap)[&JD].insert(Name);
-
- assert(JD.Symbols.count(Name) && "No symbol table entry for Name");
- auto &Sym = JD.Symbols[Name];
-
- // Move the symbol into the error state.
- // Note that this may be redundant: The symbol might already have been
- // moved to this state in response to the failure of a dependence.
- Sym.setFlags(Sym.getFlags() | JITSymbolFlags::HasError);
-
- // FIXME: Come up with a sane mapping of state to
- // presence-of-MaterializingInfo so that we can assert presence / absence
- // here, rather than testing it.
- auto MII = JD.MaterializingInfos.find(Name);
+ auto I = MRTrackers.find(&MR);
+ assert(I != MRTrackers.end() && "MaterializationResponsibility not linked");
+ MRTrackers.erase(I);
+ });
+}
- if (MII == JD.MaterializingInfos.end())
- continue;
+std::pair<JITDylib::AsynchronousSymbolQuerySet,
+ std::shared_ptr<SymbolDependenceMap>>
+JITDylib::failSymbols(FailedSymbolsWorklist Worklist) {
+ AsynchronousSymbolQuerySet FailedQueries;
+ auto FailedSymbolsMap = std::make_shared<SymbolDependenceMap>();
- auto &MI = MII->second;
-
- // Move all dependants to the error state and disconnect from them.
- for (auto &KV : MI.Dependants) {
- auto &DependantJD = *KV.first;
- for (auto &DependantName : KV.second) {
- assert(DependantJD.Symbols.count(DependantName) &&
- "No symbol table entry for DependantName");
- auto &DependantSym = DependantJD.Symbols[DependantName];
- DependantSym.setFlags(DependantSym.getFlags() |
- JITSymbolFlags::HasError);
-
- assert(DependantJD.MaterializingInfos.count(DependantName) &&
- "No MaterializingInfo for dependant");
- auto &DependantMI = DependantJD.MaterializingInfos[DependantName];
-
- auto UnemittedDepI = DependantMI.UnemittedDependencies.find(&JD);
- assert(UnemittedDepI != DependantMI.UnemittedDependencies.end() &&
- "No UnemittedDependencies entry for this JITDylib");
- assert(UnemittedDepI->second.count(Name) &&
- "No UnemittedDependencies entry for this symbol");
- UnemittedDepI->second.erase(Name);
- if (UnemittedDepI->second.empty())
- DependantMI.UnemittedDependencies.erase(UnemittedDepI);
-
- // If this symbol is already in the emitted state then we need to
- // take responsibility for failing its queries, so add it to the
- // worklist.
- if (DependantSym.getState() == SymbolState::Emitted) {
- assert(DependantMI.Dependants.empty() &&
- "Emitted symbol should not have dependants");
- Worklist.push_back(std::make_pair(&DependantJD, DependantName));
- }
- }
- }
- MI.Dependants.clear();
-
- // Disconnect from all unemitted depenencies.
- for (auto &KV : MI.UnemittedDependencies) {
- auto &UnemittedDepJD = *KV.first;
- for (auto &UnemittedDepName : KV.second) {
- auto UnemittedDepMII =
- UnemittedDepJD.MaterializingInfos.find(UnemittedDepName);
- assert(UnemittedDepMII != UnemittedDepJD.MaterializingInfos.end() &&
- "Missing MII for unemitted dependency");
- assert(UnemittedDepMII->second.Dependants.count(&JD) &&
- "JD not listed as a dependant of unemitted dependency");
- assert(UnemittedDepMII->second.Dependants[&JD].count(Name) &&
- "Name is not listed as a dependant of unemitted dependency");
- UnemittedDepMII->second.Dependants[&JD].erase(Name);
- if (UnemittedDepMII->second.Dependants[&JD].empty())
- UnemittedDepMII->second.Dependants.erase(&JD);
+ while (!Worklist.empty()) {
+ assert(Worklist.back().first && "Failed JITDylib can not be null");
+ auto &JD = *Worklist.back().first;
+ auto Name = std::move(Worklist.back().second);
+ Worklist.pop_back();
+
+ (*FailedSymbolsMap)[&JD].insert(Name);
+
+ assert(JD.Symbols.count(Name) && "No symbol table entry for Name");
+ auto &Sym = JD.Symbols[Name];
+
+ // Move the symbol into the error state.
+ // Note that this may be redundant: The symbol might already have been
+ // moved to this state in response to the failure of a dependence.
+ Sym.setFlags(Sym.getFlags() | JITSymbolFlags::HasError);
+
+ // FIXME: Come up with a sane mapping of state to
+ // presence-of-MaterializingInfo so that we can assert presence / absence
+ // here, rather than testing it.
+ auto MII = JD.MaterializingInfos.find(Name);
+
+ if (MII == JD.MaterializingInfos.end())
+ continue;
+
+ auto &MI = MII->second;
+
+ // Move all dependants to the error state and disconnect from them.
+ for (auto &KV : MI.Dependants) {
+ auto &DependantJD = *KV.first;
+ for (auto &DependantName : KV.second) {
+ assert(DependantJD.Symbols.count(DependantName) &&
+ "No symbol table entry for DependantName");
+ auto &DependantSym = DependantJD.Symbols[DependantName];
+ DependantSym.setFlags(DependantSym.getFlags() |
+ JITSymbolFlags::HasError);
+
+ assert(DependantJD.MaterializingInfos.count(DependantName) &&
+ "No MaterializingInfo for dependant");
+ auto &DependantMI = DependantJD.MaterializingInfos[DependantName];
+
+ auto UnemittedDepI = DependantMI.UnemittedDependencies.find(&JD);
+ assert(UnemittedDepI != DependantMI.UnemittedDependencies.end() &&
+ "No UnemittedDependencies entry for this JITDylib");
+ assert(UnemittedDepI->second.count(Name) &&
+ "No UnemittedDependencies entry for this symbol");
+ UnemittedDepI->second.erase(Name);
+ if (UnemittedDepI->second.empty())
+ DependantMI.UnemittedDependencies.erase(UnemittedDepI);
+
+ // If this symbol is already in the emitted state then we need to
+ // take responsibility for failing its queries, so add it to the
+ // worklist.
+ if (DependantSym.getState() == SymbolState::Emitted) {
+ assert(DependantMI.Dependants.empty() &&
+ "Emitted symbol should not have dependants");
+ Worklist.push_back(std::make_pair(&DependantJD, DependantName));
}
}
- MI.UnemittedDependencies.clear();
-
- // Collect queries to be failed for this MII.
- AsynchronousSymbolQueryList ToDetach;
- for (auto &Q : MII->second.pendingQueries()) {
- // Add the query to the list to be failed and detach it.
- FailedQueries.insert(Q);
- ToDetach.push_back(Q);
+ }
+ MI.Dependants.clear();
+
+ // Disconnect from all unemitted depenencies.
+ for (auto &KV : MI.UnemittedDependencies) {
+ auto &UnemittedDepJD = *KV.first;
+ for (auto &UnemittedDepName : KV.second) {
+ auto UnemittedDepMII =
+ UnemittedDepJD.MaterializingInfos.find(UnemittedDepName);
+ assert(UnemittedDepMII != UnemittedDepJD.MaterializingInfos.end() &&
+ "Missing MII for unemitted dependency");
+ assert(UnemittedDepMII->second.Dependants.count(&JD) &&
+ "JD not listed as a dependant of unemitted dependency");
+ assert(UnemittedDepMII->second.Dependants[&JD].count(Name) &&
+ "Name is not listed as a dependant of unemitted dependency");
+ UnemittedDepMII->second.Dependants[&JD].erase(Name);
+ if (UnemittedDepMII->second.Dependants[&JD].empty())
+ UnemittedDepMII->second.Dependants.erase(&JD);
}
- for (auto &Q : ToDetach)
- Q->detach();
-
- assert(MI.Dependants.empty() &&
- "Can not delete MaterializingInfo with dependants still attached");
- assert(MI.UnemittedDependencies.empty() &&
- "Can not delete MaterializingInfo with unemitted dependencies "
- "still attached");
- assert(!MI.hasQueriesPending() &&
- "Can not delete MaterializingInfo with queries pending");
- JD.MaterializingInfos.erase(MII);
}
- });
+ MI.UnemittedDependencies.clear();
+
+ // Collect queries to be failed for this MII.
+ AsynchronousSymbolQueryList ToDetach;
+ for (auto &Q : MII->second.pendingQueries()) {
+ // Add the query to the list to be failed and detach it.
+ FailedQueries.insert(Q);
+ ToDetach.push_back(Q);
+ }
+ for (auto &Q : ToDetach)
+ Q->detach();
- for (auto &Q : FailedQueries)
- Q->handleFailed(make_error<FailedToMaterialize>(FailedSymbolsMap));
+ assert(MI.Dependants.empty() &&
+ "Can not delete MaterializingInfo with dependants still attached");
+ assert(MI.UnemittedDependencies.empty() &&
+ "Can not delete MaterializingInfo with unemitted dependencies "
+ "still attached");
+ assert(!MI.hasQueriesPending() &&
+ "Can not delete MaterializingInfo with queries pending");
+ JD.MaterializingInfos.erase(MII);
+ }
+
+ return std::make_pair(std::move(FailedQueries), std::move(FailedSymbolsMap));
}
void JITDylib::setLinkOrder(JITDylibSearchOrder NewLinkOrder,
@@ -1298,13 +1440,13 @@ void JITDylib::lookupFlagsImpl(SymbolFlagsMap &Result, LookupKind K,
});
}
-Error JITDylib::lodgeQuery(MaterializationUnitList &MUs,
+Error JITDylib::lodgeQuery(UnmaterializedInfosList &UMIs,
std::shared_ptr<AsynchronousSymbolQuery> &Q,
LookupKind K, JITDylibLookupFlags JDLookupFlags,
SymbolLookupSet &Unresolved) {
assert(Q && "Query can not be null");
- if (auto Err = lodgeQueryImpl(MUs, Q, K, JDLookupFlags, Unresolved))
+ if (auto Err = lodgeQueryImpl(UMIs, Q, K, JDLookupFlags, Unresolved))
return Err;
// Run any definition generators.
@@ -1321,13 +1463,13 @@ Error JITDylib::lodgeQuery(MaterializationUnitList &MUs,
// Lodge query. This can not fail as any new definitions were added
// by the generator under the session locked. Since they can't have
// started materializing yet they can not have failed.
- cantFail(lodgeQueryImpl(MUs, Q, K, JDLookupFlags, Unresolved));
+ cantFail(lodgeQueryImpl(UMIs, Q, K, JDLookupFlags, Unresolved));
}
return Error::success();
}
-Error JITDylib::lodgeQueryImpl(MaterializationUnitList &MUs,
+Error JITDylib::lodgeQueryImpl(UnmaterializedInfosList &UMIs,
std::shared_ptr<AsynchronousSymbolQuery> &Q,
LookupKind K, JITDylibLookupFlags JDLookupFlags,
SymbolLookupSet &Unresolved) {
@@ -1343,6 +1485,8 @@ Error JITDylib::lodgeQueryImpl(MaterializationUnitList &MUs,
// If we match against a materialization-side-effects only symbol then
// make sure it is weakly-referenced. Otherwise bail out with an error.
+ // FIXME: Use a "materialization-side-effecs-only symbols must be weakly
+ // referenced" specific error here to reduce confusion.
if (SymI->second.getFlags().hasMaterializationSideEffectsOnly() &&
SymLookupFlags != SymbolLookupFlags::WeaklyReferencedSymbol)
return make_error<SymbolsNotFound>(SymbolNameVector({Name}));
@@ -1376,12 +1520,14 @@ Error JITDylib::lodgeQueryImpl(MaterializationUnitList &MUs,
auto UMII = UnmaterializedInfos.find(Name);
assert(UMII != UnmaterializedInfos.end() &&
"Lazy symbol should have UnmaterializedInfo");
- auto MU = std::move(UMII->second->MU);
- assert(MU != nullptr && "Materializer should not be null");
+
+ auto UMI = UMII->second;
+ assert(UMI->MU && "Materializer should not be null");
+ assert(UMI->RT && "Tracker should not be null");
// Move all symbols associated with this MaterializationUnit into
// materializing state.
- for (auto &KV : MU->getSymbols()) {
+ for (auto &KV : UMI->MU->getSymbols()) {
auto SymK = Symbols.find(KV.first);
SymK->second.setMaterializerAttached(false);
SymK->second.setState(SymbolState::Materializing);
@@ -1389,7 +1535,7 @@ Error JITDylib::lodgeQueryImpl(MaterializationUnitList &MUs,
}
// Add MU to the list of MaterializationUnits to be materialized.
- MUs.push_back(std::move(MU));
+ UMIs.push_back(std::move(UMI));
}
// Add the query to the PendingQueries list and continue, deleting the
@@ -1492,6 +1638,138 @@ JITDylib::JITDylib(ExecutionSession &ES, std::string Name)
LinkOrder.push_back({this, JITDylibLookupFlags::MatchAllSymbols});
}
+ResourceTrackerSP JITDylib::getTracker(MaterializationResponsibility &MR) {
+ auto I = MRTrackers.find(&MR);
+ assert(I != MRTrackers.end() && "MR is not linked");
+ assert(I->second && "Linked tracker is null");
+ return I->second;
+}
+
+std::pair<JITDylib::AsynchronousSymbolQuerySet,
+ std::shared_ptr<SymbolDependenceMap>>
+JITDylib::removeTracker(ResourceTracker &RT) {
+ // Note: Should be called under the session lock.
+
+ SymbolNameVector SymbolsToRemove;
+ std::vector<std::pair<JITDylib *, SymbolStringPtr>> SymbolsToFail;
+
+ if (&RT == DefaultTracker.get()) {
+ SymbolNameSet TrackedSymbols;
+ for (auto &KV : TrackerSymbols)
+ for (auto &Sym : KV.second)
+ TrackedSymbols.insert(Sym);
+
+ for (auto &KV : Symbols) {
+ auto &Sym = KV.first;
+ if (!TrackedSymbols.count(Sym))
+ SymbolsToRemove.push_back(Sym);
+ }
+
+ DefaultTracker.reset();
+ } else {
+ /// Check for a non-default tracker.
+ auto I = TrackerSymbols.find(&RT);
+ if (I != TrackerSymbols.end()) {
+ SymbolsToRemove = std::move(I->second);
+ TrackerSymbols.erase(I);
+ }
+ // ... if not found this tracker was already defunct. Nothing to do.
+ }
+
+ for (auto &Sym : SymbolsToRemove) {
+ auto I = Symbols.find(Sym);
+ assert(I != Symbols.end() && "Symbol not in symbol table");
+
+ // If there is a MaterializingInfo then collect any queries to fail.
+ auto MII = MaterializingInfos.find(Sym);
+ if (MII != MaterializingInfos.end())
+ SymbolsToFail.push_back({this, Sym});
+ }
+
+ AsynchronousSymbolQuerySet QueriesToFail;
+ auto Result = failSymbols(std::move(SymbolsToFail));
+
+ // Removed symbols should be taken out of the table altogether.
+ for (auto &Sym : SymbolsToRemove) {
+ auto I = Symbols.find(Sym);
+ assert(I != Symbols.end() && "Symbol not present in table");
+
+ // Remove Materializer if present.
+ if (I->second.hasMaterializerAttached()) {
+ // FIXME: Should this discard the symbols?
+ UnmaterializedInfos.erase(Sym);
+ } else {
+ assert(!UnmaterializedInfos.count(Sym) &&
+ "Symbol has materializer attached");
+ }
+
+ Symbols.erase(I);
+ }
+
+ return Result;
+}
+
+void JITDylib::transferTracker(ResourceTracker &DstRT, ResourceTracker &SrcRT) {
+ assert(&DstRT != &SrcRT && "No-op transfers shouldn't call transferTracker");
+ assert(&DstRT.getJITDylib() == this && "DstRT is not for this JITDylib");
+ assert(&SrcRT.getJITDylib() == this && "SrcRT is not for this JITDylib");
+
+ // Update trackers for any not-yet materialized units.
+ for (auto &KV : UnmaterializedInfos) {
+ if (KV.second->RT == &SrcRT)
+ KV.second->RT = &DstRT;
+ }
+
+ // Update trackers for any active materialization responsibilities.
+ for (auto &KV : MRTrackers) {
+ if (KV.second == &SrcRT)
+ KV.second = &DstRT;
+ }
+
+ // If we're transfering to the default tracker we just need to delete the
+ // tracked symbols for the source tracker.
+ if (&DstRT == DefaultTracker.get()) {
+ TrackerSymbols.erase(&SrcRT);
+ return;
+ }
+
+ // If we're transferring from the default tracker we need to find all
+ // currently untracked symbols.
+ if (&SrcRT == DefaultTracker.get()) {
+ assert(!TrackerSymbols.count(&SrcRT) &&
+ "Default tracker should not appear in TrackerSymbols");
+
+ SymbolNameVector SymbolsToTrack;
+
+ SymbolNameSet CurrentlyTrackedSymbols;
+ for (auto &KV : TrackerSymbols)
+ for (auto &Sym : KV.second)
+ CurrentlyTrackedSymbols.insert(Sym);
+
+ for (auto &KV : Symbols) {
+ auto &Sym = KV.first;
+ if (!CurrentlyTrackedSymbols.count(Sym))
+ SymbolsToTrack.push_back(Sym);
+ }
+
+ TrackerSymbols[&DstRT] = std::move(SymbolsToTrack);
+ return;
+ }
+
+ auto &DstTrackedSymbols = TrackerSymbols[&DstRT];
+
+ // Finally if neither SrtRT or DstRT are the default tracker then
+ // just append DstRT's tracked symbols to SrtRT's.
+ auto SI = TrackerSymbols.find(&SrcRT);
+ if (SI == TrackerSymbols.end())
+ return;
+
+ DstTrackedSymbols.reserve(DstTrackedSymbols.size() + SI->second.size());
+ for (auto &Sym : SI->second)
+ DstTrackedSymbols.push_back(std::move(Sym));
+ TrackerSymbols.erase(SI);
+}
+
Error JITDylib::defineImpl(MaterializationUnit &MU) {
LLVM_DEBUG({ dbgs() << " " << MU.getSymbols() << "\n"; });
@@ -1559,6 +1837,22 @@ Error JITDylib::defineImpl(MaterializationUnit &MU) {
return Error::success();
}
+void JITDylib::installMaterializationUnit(
+ std::unique_ptr<MaterializationUnit> MU, ResourceTracker &RT) {
+
+ /// defineImpl succeeded.
+ if (&RT != DefaultTracker.get()) {
+ auto &TS = TrackerSymbols[&RT];
+ TS.reserve(TS.size() + MU->getSymbols().size());
+ for (auto &KV : MU->getSymbols())
+ TS.push_back(KV.first);
+ }
+
+ auto UMI = std::make_shared<UnmaterializedInfo>(std::move(MU), &RT);
+ for (auto &KV : UMI->MU->getSymbols())
+ UnmaterializedInfos[KV.first] = UMI;
+}
+
void JITDylib::detachQueryHelper(AsynchronousSymbolQuery &Q,
const SymbolNameSet &QuerySymbols) {
for (auto &QuerySymbol : QuerySymbols) {
@@ -1647,7 +1941,39 @@ Expected<DenseMap<JITDylib *, SymbolMap>> Platform::lookupInitSymbols(
}
ExecutionSession::ExecutionSession(std::shared_ptr<SymbolStringPool> SSP)
- : SSP(SSP ? std::move(SSP) : std::make_shared<SymbolStringPool>()) {
+ : SSP(SSP ? std::move(SSP) : std::make_shared<SymbolStringPool>()) {}
+
+Error ExecutionSession::endSession() {
+ LLVM_DEBUG(dbgs() << "Ending ExecutionSession " << this << "\n");
+
+ std::vector<JITDylibSP> JITDylibsToClose = runSessionLocked([&] {
+ SessionOpen = false;
+ return std::move(JDs);
+ });
+
+ // TODO: notifiy platform? run static deinits?
+
+ Error Err = Error::success();
+ for (auto &JD : JITDylibsToClose)
+ Err = joinErrors(std::move(Err), JD->clear());
+ return Err;
+}
+
+void ExecutionSession::registerResourceManager(ResourceManager &RM) {
+ runSessionLocked([&] { ResourceManagers.push_back(&RM); });
+}
+
+void ExecutionSession::deregisterResourceManager(ResourceManager &RM) {
+ runSessionLocked([&] {
+ assert(!ResourceManagers.empty() && "No managers registered");
+ if (ResourceManagers.back() == &RM)
+ ResourceManagers.pop_back();
+ else {
+ auto I = llvm::find(ResourceManagers, &RM);
+ assert(I != ResourceManagers.end() && "RM not registered");
+ ResourceManagers.erase(I);
+ }
+ });
}
JITDylib *ExecutionSession::getJITDylibByName(StringRef Name) {
@@ -1662,8 +1988,7 @@ JITDylib *ExecutionSession::getJITDylibByName(StringRef Name) {
JITDylib &ExecutionSession::createBareJITDylib(std::string Name) {
assert(!getJITDylibByName(Name) && "JITDylib with that name already exists");
return runSessionLocked([&, this]() -> JITDylib & {
- JDs.push_back(
- std::shared_ptr<JITDylib>(new JITDylib(*this, std::move(Name))));
+ JDs.push_back(new JITDylib(*this, std::move(Name)));
return *JDs.back();
});
}
@@ -1676,22 +2001,21 @@ Expected<JITDylib &> ExecutionSession::createJITDylib(std::string Name) {
return JD;
}
-std::vector<std::shared_ptr<JITDylib>>
-JITDylib::getDFSLinkOrder(ArrayRef<std::shared_ptr<JITDylib>> JDs) {
+std::vector<JITDylibSP> JITDylib::getDFSLinkOrder(ArrayRef<JITDylibSP> JDs) {
if (JDs.empty())
return {};
auto &ES = JDs.front()->getExecutionSession();
return ES.runSessionLocked([&]() {
DenseSet<JITDylib *> Visited;
- std::vector<std::shared_ptr<JITDylib>> Result;
+ std::vector<JITDylibSP> Result;
for (auto &JD : JDs) {
if (Visited.count(JD.get()))
continue;
- SmallVector<std::shared_ptr<JITDylib>, 64> WorkStack;
+ SmallVector<JITDylibSP, 64> WorkStack;
WorkStack.push_back(JD);
Visited.insert(JD.get());
@@ -1704,7 +2028,7 @@ JITDylib::getDFSLinkOrder(ArrayRef<std::shared_ptr<JITDylib>> JDs) {
if (Visited.count(&JD))
continue;
Visited.insert(&JD);
- WorkStack.push_back(JD.shared_from_this());
+ WorkStack.push_back(&JD);
}
}
}
@@ -1712,19 +2036,19 @@ JITDylib::getDFSLinkOrder(ArrayRef<std::shared_ptr<JITDylib>> JDs) {
});
}
-std::vector<std::shared_ptr<JITDylib>>
-JITDylib::getReverseDFSLinkOrder(ArrayRef<std::shared_ptr<JITDylib>> JDs) {
+std::vector<JITDylibSP>
+JITDylib::getReverseDFSLinkOrder(ArrayRef<JITDylibSP> JDs) {
auto Tmp = getDFSLinkOrder(JDs);
std::reverse(Tmp.begin(), Tmp.end());
return Tmp;
}
-std::vector<std::shared_ptr<JITDylib>> JITDylib::getDFSLinkOrder() {
- return getDFSLinkOrder({shared_from_this()});
+std::vector<JITDylibSP> JITDylib::getDFSLinkOrder() {
+ return getDFSLinkOrder({this});
}
-std::vector<std::shared_ptr<JITDylib>> JITDylib::getReverseDFSLinkOrder() {
- return getReverseDFSLinkOrder({shared_from_this()});
+std::vector<JITDylibSP> JITDylib::getReverseDFSLinkOrder() {
+ return getReverseDFSLinkOrder({this});
}
void ExecutionSession::lookup(
@@ -1746,12 +2070,13 @@ void ExecutionSession::lookup(
runOutstandingMUs();
auto Unresolved = std::move(Symbols);
- std::map<JITDylib *, MaterializationUnitList> CollectedMUsMap;
auto Q = std::make_shared<AsynchronousSymbolQuery>(Unresolved, RequiredState,
std::move(NotifyComplete));
bool QueryComplete = false;
auto LodgingErr = runSessionLocked([&]() -> Error {
+ DenseMap<JITDylib *, JITDylib::UnmaterializedInfosList> CollectedMUsMap;
+
auto LodgeQuery = [&]() -> Error {
for (auto &KV : SearchOrder) {
assert(KV.first && "JITDylibList entries must not be null");
@@ -1788,14 +2113,44 @@ void ExecutionSession::lookup(
Q->detach();
// Replace the MUs.
- for (auto &KV : CollectedMUsMap)
- for (auto &MU : KV.second)
- KV.first->replace(std::move(MU));
+ for (auto &KV : CollectedMUsMap) {
+ auto &JD = *KV.first;
+ for (auto &UMI : KV.second)
+ for (auto &KV2 : UMI->MU->getSymbols()) {
+ assert(!JD.UnmaterializedInfos.count(KV2.first) &&
+ "Unexpected materializer in map");
+ auto SymI = JD.Symbols.find(KV2.first);
+ assert(SymI != JD.Symbols.end() && "Missing symbol entry");
+ assert(SymI->second.getState() == SymbolState::Materializing &&
+ "Can not replace symbol that is not materializing");
+ assert(!SymI->second.hasMaterializerAttached() &&
+ "MaterializerAttached flag should not be set");
+ SymI->second.setMaterializerAttached(true);
+ JD.UnmaterializedInfos[KV2.first] = UMI;
+ }
+ }
return Err;
}
+ // Query lodged successfully. Add MRs to the OutstandingMUs list.
+
+ // Move the MUs to the OutstandingMUs list.
+ {
+ std::lock_guard<std::recursive_mutex> Lock(OutstandingMUsMutex);
- // Query lodged successfully.
+ for (auto &KV : CollectedMUsMap) {
+ auto &JD = *KV.first;
+ for (auto &UMI : KV.second) {
+ std::unique_ptr<MaterializationResponsibility> MR(
+ new MaterializationResponsibility(
+ &JD, std::move(UMI->MU->SymbolFlags),
+ std::move(UMI->MU->InitSymbol)));
+ JD.MRTrackers[MR.get()] = UMI->RT;
+ OutstandingMUs.push_back(
+ std::make_pair(std::move(UMI->MU), std::move(MR)));
+ }
+ }
+ }
// Record whether this query is fully ready / resolved. We will use
// this to call handleFullyResolved/handleFullyReady outside the session
@@ -1817,19 +2172,6 @@ void ExecutionSession::lookup(
if (QueryComplete)
Q->handleComplete();
- // Move the MUs to the OutstandingMUs list, then materialize.
- {
- std::lock_guard<std::recursive_mutex> Lock(OutstandingMUsMutex);
-
- for (auto &KV : CollectedMUsMap) {
- auto JD = KV.first->shared_from_this();
- for (auto &MU : KV.second) {
- auto MR = MU->createMaterializationResponsibility(JD);
- OutstandingMUs.push_back(std::make_pair(std::move(MU), std::move(MR)));
- }
- }
- }
-
runOutstandingMUs();
}
@@ -1942,6 +2284,70 @@ void ExecutionSession::runOutstandingMUs() {
}
}
+Error ExecutionSession::removeResourceTracker(ResourceTracker &RT) {
+ LLVM_DEBUG({
+ dbgs() << "In " << RT.getJITDylib().getName() << " removing tracker "
+ << formatv("{0:x}", RT.getKeyUnsafe()) << "\n";
+ });
+ std::vector<ResourceManager *> CurrentResourceManagers;
+
+ JITDylib::AsynchronousSymbolQuerySet QueriesToFail;
+ std::shared_ptr<SymbolDependenceMap> FailedSymbols;
+
+ runSessionLocked([&] {
+ CurrentResourceManagers = ResourceManagers;
+ RT.makeDefunct();
+ std::tie(QueriesToFail, FailedSymbols) = RT.getJITDylib().removeTracker(RT);
+ });
+
+ Error Err = Error::success();
+
+ for (auto *L : reverse(CurrentResourceManagers))
+ Err =
+ joinErrors(std::move(Err), L->handleRemoveResources(RT.getKeyUnsafe()));
+
+ for (auto &Q : QueriesToFail)
+ Q->handleFailed(make_error<FailedToMaterialize>(FailedSymbols));
+
+ return Err;
+}
+
+void ExecutionSession::transferResourceTracker(ResourceTracker &DstRT,
+ ResourceTracker &SrcRT) {
+ LLVM_DEBUG({
+ dbgs() << "In " << SrcRT.getJITDylib().getName()
+ << " transfering resources from tracker "
+ << formatv("{0:x}", SrcRT.getKeyUnsafe()) << " to tracker "
+ << formatv("{0:x}", DstRT.getKeyUnsafe()) << "\n";
+ });
+
+ // No-op transfers are allowed and do not invalidate the source.
+ if (&DstRT == &SrcRT)
+ return;
+
+ assert(&DstRT.getJITDylib() == &SrcRT.getJITDylib() &&
+ "Can't transfer resources between JITDylibs");
+ runSessionLocked([&]() {
+ SrcRT.makeDefunct();
+ auto &JD = DstRT.getJITDylib();
+ JD.transferTracker(DstRT, SrcRT);
+ for (auto *L : reverse(ResourceManagers))
+ L->handleTransferResources(DstRT.getKeyUnsafe(), SrcRT.getKeyUnsafe());
+ });
+}
+
+void ExecutionSession::destroyResourceTracker(ResourceTracker &RT) {
+ runSessionLocked([&]() {
+ LLVM_DEBUG({
+ dbgs() << "In " << RT.getJITDylib().getName() << " destroying tracker "
+ << formatv("{0:x}", RT.getKeyUnsafe()) << "\n";
+ });
+ if (!RT.isDefunct())
+ transferResourceTracker(*RT.getJITDylib().getDefaultResourceTracker(),
+ RT);
+ });
+}
+
#ifndef NDEBUG
void ExecutionSession::dumpDispatchInfo(JITDylib &JD, MaterializationUnit &MU) {
runSessionLocked([&]() {
diff --git a/llvm/lib/ExecutionEngine/Orc/ExecutionUtils.cpp b/llvm/lib/ExecutionEngine/Orc/ExecutionUtils.cpp
index 278f492f0ebe..0052149c4b27 100644
--- a/llvm/lib/ExecutionEngine/Orc/ExecutionUtils.cpp
+++ b/llvm/lib/ExecutionEngine/Orc/ExecutionUtils.cpp
@@ -397,8 +397,7 @@ Error StaticLibraryDefinitionGenerator::tryToGenerate(
MemoryBufferRef ChildBufferRef(ChildBufferInfo.first,
ChildBufferInfo.second);
- if (auto Err = L.add(JD, MemoryBuffer::getMemBuffer(ChildBufferRef, false),
- VModuleKey()))
+ if (auto Err = L.add(JD, MemoryBuffer::getMemBuffer(ChildBufferRef, false)))
return Err;
}
diff --git a/llvm/lib/ExecutionEngine/Orc/IRCompileLayer.cpp b/llvm/lib/ExecutionEngine/Orc/IRCompileLayer.cpp
index c6f687027972..aadc437c80c4 100644
--- a/llvm/lib/ExecutionEngine/Orc/IRCompileLayer.cpp
+++ b/llvm/lib/ExecutionEngine/Orc/IRCompileLayer.cpp
@@ -33,7 +33,7 @@ void IRCompileLayer::emit(std::unique_ptr<MaterializationResponsibility> R,
{
std::lock_guard<std::mutex> Lock(IRLayerMutex);
if (NotifyCompiled)
- NotifyCompiled(R->getVModuleKey(), std::move(TSM));
+ NotifyCompiled(*R, std::move(TSM));
else
TSM = ThreadSafeModule();
}
diff --git a/llvm/lib/ExecutionEngine/Orc/IndirectionUtils.cpp b/llvm/lib/ExecutionEngine/Orc/IndirectionUtils.cpp
index 7d57ed5a3a04..373f67dabe70 100644
--- a/llvm/lib/ExecutionEngine/Orc/IndirectionUtils.cpp
+++ b/llvm/lib/ExecutionEngine/Orc/IndirectionUtils.cpp
@@ -25,9 +25,9 @@ class CompileCallbackMaterializationUnit : public orc::MaterializationUnit {
using CompileFunction = JITCompileCallbackManager::CompileFunction;
CompileCallbackMaterializationUnit(SymbolStringPtr Name,
- CompileFunction Compile, VModuleKey K)
+ CompileFunction Compile)
: MaterializationUnit(SymbolFlagsMap({{Name, JITSymbolFlags::Exported}}),
- nullptr, std::move(K)),
+ nullptr),
Name(std::move(Name)), Compile(std::move(Compile)) {}
StringRef getName() const override { return "<Compile Callbacks>"; }
@@ -65,10 +65,9 @@ JITCompileCallbackManager::getCompileCallback(CompileFunction Compile) {
std::lock_guard<std::mutex> Lock(CCMgrMutex);
AddrToSymbol[*TrampolineAddr] = CallbackName;
- cantFail(CallbacksJD.define(
- std::make_unique<CompileCallbackMaterializationUnit>(
- std::move(CallbackName), std::move(Compile),
- ES.allocateVModule())));
+ cantFail(
+ CallbacksJD.define(std::make_unique<CompileCallbackMaterializationUnit>(
+ std::move(CallbackName), std::move(Compile))));
return *TrampolineAddr;
} else
return TrampolineAddr.takeError();
diff --git a/llvm/lib/ExecutionEngine/Orc/LLJIT.cpp b/llvm/lib/ExecutionEngine/Orc/LLJIT.cpp
index 81f500d66bc2..5716fe25ca3e 100644
--- a/llvm/lib/ExecutionEngine/Orc/LLJIT.cpp
+++ b/llvm/lib/ExecutionEngine/Orc/LLJIT.cpp
@@ -88,8 +88,9 @@ class GenericLLVMIRPlatform : public Platform {
public:
GenericLLVMIRPlatform(GenericLLVMIRPlatformSupport &S) : S(S) {}
Error setupJITDylib(JITDylib &JD) override;
- Error notifyAdding(JITDylib &JD, const MaterializationUnit &MU) override;
- Error notifyRemoving(JITDylib &JD, VModuleKey K) override {
+ Error notifyAdding(ResourceTracker &RT,
+ const MaterializationUnit &MU) override;
+ Error notifyRemoving(ResourceTracker &RT) override {
// Noop -- Nothing to do (yet).
return Error::success();
}
@@ -187,7 +188,8 @@ class GenericLLVMIRPlatformSupport : public LLJIT::PlatformSupport {
return J.addIRModule(JD, ThreadSafeModule(std::move(M), std::move(Ctx)));
}
- Error notifyAdding(JITDylib &JD, const MaterializationUnit &MU) {
+ Error notifyAdding(ResourceTracker &RT, const MaterializationUnit &MU) {
+ auto &JD = RT.getJITDylib();
if (auto &InitSym = MU.getInitializerSymbol())
InitSymbols[&JD].add(InitSym, SymbolLookupFlags::WeaklyReferencedSymbol);
else {
@@ -261,7 +263,7 @@ class GenericLLVMIRPlatformSupport : public LLJIT::PlatformSupport {
return std::move(Err);
DenseMap<JITDylib *, SymbolLookupSet> LookupSymbols;
- std::vector<std::shared_ptr<JITDylib>> DFSLinkOrder;
+ std::vector<JITDylibSP> DFSLinkOrder;
getExecutionSession().runSessionLocked([&]() {
DFSLinkOrder = JD.getDFSLinkOrder();
@@ -311,7 +313,7 @@ class GenericLLVMIRPlatformSupport : public LLJIT::PlatformSupport {
auto LLJITRunAtExits = J.mangleAndIntern("__lljit_run_atexits");
DenseMap<JITDylib *, SymbolLookupSet> LookupSymbols;
- std::vector<std::shared_ptr<JITDylib>> DFSLinkOrder;
+ std::vector<JITDylibSP> DFSLinkOrder;
ES.runSessionLocked([&]() {
DFSLinkOrder = JD.getDFSLinkOrder();
@@ -365,7 +367,7 @@ class GenericLLVMIRPlatformSupport : public LLJIT::PlatformSupport {
/// JITDylibs that it depends on).
Error issueInitLookups(JITDylib &JD) {
DenseMap<JITDylib *, SymbolLookupSet> RequiredInitSymbols;
- std::vector<std::shared_ptr<JITDylib>> DFSLinkOrder;
+ std::vector<JITDylibSP> DFSLinkOrder;
getExecutionSession().runSessionLocked([&]() {
DFSLinkOrder = JD.getDFSLinkOrder();
@@ -446,9 +448,9 @@ Error GenericLLVMIRPlatform::setupJITDylib(JITDylib &JD) {
return S.setupJITDylib(JD);
}
-Error GenericLLVMIRPlatform::notifyAdding(JITDylib &JD,
+Error GenericLLVMIRPlatform::notifyAdding(ResourceTracker &RT,
const MaterializationUnit &MU) {
- return S.notifyAdding(JD, MU);
+ return S.notifyAdding(RT, MU);
}
Expected<ThreadSafeModule>
@@ -961,7 +963,7 @@ Error LLJITBuilderState::prepareForConstruction() {
ObjLinkingLayer = std::make_unique<ObjectLinkingLayer>(
ES, std::make_unique<jitlink::InProcessMemoryManager>());
ObjLinkingLayer->addPlugin(std::make_unique<EHFrameRegistrationPlugin>(
- std::make_unique<jitlink::InProcessEHFrameRegistrar>()));
+ ES, std::make_unique<jitlink::InProcessEHFrameRegistrar>()));
return std::move(ObjLinkingLayer);
};
}
@@ -973,6 +975,8 @@ Error LLJITBuilderState::prepareForConstruction() {
LLJIT::~LLJIT() {
if (CompileThreads)
CompileThreads->wait();
+ if (auto Err = ES->endSession())
+ ES->reportError(std::move(Err));
}
Error LLJIT::addIRModule(JITDylib &JD, ThreadSafeModule TSM) {
@@ -982,14 +986,13 @@ Error LLJIT::addIRModule(JITDylib &JD, ThreadSafeModule TSM) {
TSM.withModuleDo([&](Module &M) { return applyDataLayout(M); }))
return Err;
- return InitHelperTransformLayer->add(JD, std::move(TSM),
- ES->allocateVModule());
+ return InitHelperTransformLayer->add(JD, std::move(TSM));
}
Error LLJIT::addObjectFile(JITDylib &JD, std::unique_ptr<MemoryBuffer> Obj) {
assert(Obj && "Can not add null object");
- return ObjTransformLayer.add(JD, std::move(Obj), ES->allocateVModule());
+ return ObjTransformLayer.add(JD, std::move(Obj));
}
Expected<JITEvaluatedSymbol> LLJIT::lookupLinkerMangled(JITDylib &JD,
@@ -1157,7 +1160,7 @@ Error LLLazyJIT::addLazyIRModule(JITDylib &JD, ThreadSafeModule TSM) {
[&](Module &M) -> Error { return applyDataLayout(M); }))
return Err;
- return CODLayer->add(JD, std::move(TSM), ES->allocateVModule());
+ return CODLayer->add(JD, std::move(TSM));
}
LLLazyJIT::LLLazyJIT(LLLazyJITBuilderState &S, Error &Err) : LLJIT(S, Err) {
diff --git a/llvm/lib/ExecutionEngine/Orc/Layer.cpp b/llvm/lib/ExecutionEngine/Orc/Layer.cpp
index 8052e7b08a5a..5e27e343d23b 100644
--- a/llvm/lib/ExecutionEngine/Orc/Layer.cpp
+++ b/llvm/lib/ExecutionEngine/Orc/Layer.cpp
@@ -22,16 +22,18 @@ namespace orc {
IRLayer::~IRLayer() {}
-Error IRLayer::add(JITDylib &JD, ThreadSafeModule TSM, VModuleKey K) {
+Error IRLayer::add(ResourceTrackerSP RT, ThreadSafeModule TSM) {
+ assert(RT && "RT can not be null");
+ auto &JD = RT->getJITDylib();
return JD.define(std::make_unique<BasicIRLayerMaterializationUnit>(
- *this, *getManglingOptions(), std::move(TSM), std::move(K)));
+ *this, *getManglingOptions(), std::move(TSM)),
+ std::move(RT));
}
IRMaterializationUnit::IRMaterializationUnit(
ExecutionSession &ES, const IRSymbolMapper::ManglingOptions &MO,
- ThreadSafeModule TSM, VModuleKey K)
- : MaterializationUnit(SymbolFlagsMap(), nullptr, std::move(K)),
- TSM(std::move(TSM)) {
+ ThreadSafeModule TSM)
+ : MaterializationUnit(SymbolFlagsMap(), nullptr), TSM(std::move(TSM)) {
assert(this->TSM && "Module must not be null");
@@ -96,10 +98,9 @@ IRMaterializationUnit::IRMaterializationUnit(
}
IRMaterializationUnit::IRMaterializationUnit(
- ThreadSafeModule TSM, VModuleKey K, SymbolFlagsMap SymbolFlags,
+ ThreadSafeModule TSM, SymbolFlagsMap SymbolFlags,
SymbolStringPtr InitSymbol, SymbolNameToDefinitionMap SymbolToDefinition)
- : MaterializationUnit(std::move(SymbolFlags), std::move(InitSymbol),
- std::move(K)),
+ : MaterializationUnit(std::move(SymbolFlags), std::move(InitSymbol)),
TSM(std::move(TSM)), SymbolToDefinition(std::move(SymbolToDefinition)) {}
StringRef IRMaterializationUnit::getName() const {
@@ -126,11 +127,9 @@ void IRMaterializationUnit::discard(const JITDylib &JD,
}
BasicIRLayerMaterializationUnit::BasicIRLayerMaterializationUnit(
- IRLayer &L, const IRSymbolMapper::ManglingOptions &MO, ThreadSafeModule TSM,
- VModuleKey K)
- : IRMaterializationUnit(L.getExecutionSession(), MO, std::move(TSM),
- std::move(K)),
- L(L), K(std::move(K)) {}
+ IRLayer &L, const IRSymbolMapper::ManglingOptions &MO, ThreadSafeModule TSM)
+ : IRMaterializationUnit(L.getExecutionSession(), MO, std::move(TSM)), L(L) {
+}
void BasicIRLayerMaterializationUnit::materialize(
std::unique_ptr<MaterializationResponsibility> R) {
@@ -160,17 +159,17 @@ ObjectLayer::ObjectLayer(ExecutionSession &ES) : ES(ES) {}
ObjectLayer::~ObjectLayer() {}
-Error ObjectLayer::add(JITDylib &JD, std::unique_ptr<MemoryBuffer> O,
- VModuleKey K) {
- auto ObjMU = BasicObjectLayerMaterializationUnit::Create(*this, std::move(K),
- std::move(O));
+Error ObjectLayer::add(ResourceTrackerSP RT, std::unique_ptr<MemoryBuffer> O) {
+ assert(RT && "RT can not be null");
+ auto ObjMU = BasicObjectLayerMaterializationUnit::Create(*this, std::move(O));
if (!ObjMU)
return ObjMU.takeError();
- return JD.define(std::move(*ObjMU));
+ auto &JD = RT->getJITDylib();
+ return JD.define(std::move(*ObjMU), std::move(RT));
}
Expected<std::unique_ptr<BasicObjectLayerMaterializationUnit>>
-BasicObjectLayerMaterializationUnit::Create(ObjectLayer &L, VModuleKey K,
+BasicObjectLayerMaterializationUnit::Create(ObjectLayer &L,
std::unique_ptr<MemoryBuffer> O) {
auto ObjSymInfo =
getObjectSymbolInfo(L.getExecutionSession(), O->getMemBufferRef());
@@ -183,15 +182,14 @@ BasicObjectLayerMaterializationUnit::Create(ObjectLayer &L, VModuleKey K,
return std::unique_ptr<BasicObjectLayerMaterializationUnit>(
new BasicObjectLayerMaterializationUnit(
- L, K, std::move(O), std::move(SymbolFlags), std::move(InitSymbol)));
+ L, std::move(O), std::move(SymbolFlags), std::move(InitSymbol)));
}
BasicObjectLayerMaterializationUnit::BasicObjectLayerMaterializationUnit(
- ObjectLayer &L, VModuleKey K, std::unique_ptr<MemoryBuffer> O,
- SymbolFlagsMap SymbolFlags, SymbolStringPtr InitSymbol)
- : MaterializationUnit(std::move(SymbolFlags), std::move(InitSymbol),
- std::move(K)),
- L(L), O(std::move(O)) {}
+ ObjectLayer &L, std::unique_ptr<MemoryBuffer> O, SymbolFlagsMap SymbolFlags,
+ SymbolStringPtr InitSymbol)
+ : MaterializationUnit(std::move(SymbolFlags), std::move(InitSymbol)), L(L),
+ O(std::move(O)) {}
StringRef BasicObjectLayerMaterializationUnit::getName() const {
if (O)
diff --git a/llvm/lib/ExecutionEngine/Orc/LazyReexports.cpp b/llvm/lib/ExecutionEngine/Orc/LazyReexports.cpp
index 695f6cc9c1cb..e1f494415e86 100644
--- a/llvm/lib/ExecutionEngine/Orc/LazyReexports.cpp
+++ b/llvm/lib/ExecutionEngine/Orc/LazyReexports.cpp
@@ -143,9 +143,8 @@ createLocalLazyCallThroughManager(const Triple &T, ExecutionSession &ES,
LazyReexportsMaterializationUnit::LazyReexportsMaterializationUnit(
LazyCallThroughManager &LCTManager, IndirectStubsManager &ISManager,
- JITDylib &SourceJD, SymbolAliasMap CallableAliases, ImplSymbolMap *SrcJDLoc,
- VModuleKey K)
- : MaterializationUnit(extractFlags(CallableAliases), nullptr, std::move(K)),
+ JITDylib &SourceJD, SymbolAliasMap CallableAliases, ImplSymbolMap *SrcJDLoc)
+ : MaterializationUnit(extractFlags(CallableAliases), nullptr),
LCTManager(LCTManager), ISManager(ISManager), SourceJD(SourceJD),
CallableAliases(std::move(CallableAliases)), AliaseeTable(SrcJDLoc) {}
@@ -166,8 +165,13 @@ void LazyReexportsMaterializationUnit::materialize(
}
if (!CallableAliases.empty())
- R->replace(lazyReexports(LCTManager, ISManager, SourceJD,
- std::move(CallableAliases), AliaseeTable));
+ if (auto Err = R->replace(lazyReexports(LCTManager, ISManager, SourceJD,
+ std::move(CallableAliases),
+ AliaseeTable))) {
+ R->getExecutionSession().reportError(std::move(Err));
+ R->failMaterialization();
+ return;
+ }
IndirectStubsManager::StubInitsMap StubInits;
for (auto &Alias : RequestedAliases) {
diff --git a/llvm/lib/ExecutionEngine/Orc/MachOPlatform.cpp b/llvm/lib/ExecutionEngine/Orc/MachOPlatform.cpp
index 6306cc8e1d01..17b9465a0541 100644
--- a/llvm/lib/ExecutionEngine/Orc/MachOPlatform.cpp
+++ b/llvm/lib/ExecutionEngine/Orc/MachOPlatform.cpp
@@ -159,7 +159,9 @@ Error MachOPlatform::setupJITDylib(JITDylib &JD) {
return ObjLinkingLayer.add(JD, std::move(ObjBuffer));
}
-Error MachOPlatform::notifyAdding(JITDylib &JD, const MaterializationUnit &MU) {
+Error MachOPlatform::notifyAdding(ResourceTracker &RT,
+ const MaterializationUnit &MU) {
+ auto &JD = RT.getJITDylib();
const auto &InitSym = MU.getInitializerSymbol();
if (!InitSym)
return Error::success();
@@ -173,7 +175,7 @@ Error MachOPlatform::notifyAdding(JITDylib &JD, const MaterializationUnit &MU) {
return Error::success();
}
-Error MachOPlatform::notifyRemoving(JITDylib &JD, VModuleKey K) {
+Error MachOPlatform::notifyRemoving(ResourceTracker &RT) {
llvm_unreachable("Not supported yet");
}
@@ -185,7 +187,7 @@ MachOPlatform::getInitializerSequence(JITDylib &JD) {
<< JD.getName() << "\n";
});
- std::vector<std::shared_ptr<JITDylib>> DFSLinkOrder;
+ std::vector<JITDylibSP> DFSLinkOrder;
while (true) {
@@ -247,7 +249,7 @@ MachOPlatform::getInitializerSequence(JITDylib &JD) {
Expected<MachOPlatform::DeinitializerSequence>
MachOPlatform::getDeinitializerSequence(JITDylib &JD) {
- std::vector<std::shared_ptr<JITDylib>> DFSLinkOrder = JD.getDFSLinkOrder();
+ std::vector<JITDylibSP> DFSLinkOrder = JD.getDFSLinkOrder();
DeinitializerSequence FullDeinitSeq;
{
diff --git a/llvm/lib/ExecutionEngine/Orc/ObjectLinkingLayer.cpp b/llvm/lib/ExecutionEngine/Orc/ObjectLinkingLayer.cpp
index 9e3245d9cc99..5dd32fbc224e 100644
--- a/llvm/lib/ExecutionEngine/Orc/ObjectLinkingLayer.cpp
+++ b/llvm/lib/ExecutionEngine/Orc/ObjectLinkingLayer.cpp
@@ -44,6 +44,8 @@ class ObjectLinkingLayerJITLinkContext final : public JITLinkContext {
}
void notifyFailed(Error Err) override {
+ for (auto &P : Layer.Plugins)
+ Err = joinErrors(std::move(Err), P->notifyFailed(*MR));
Layer.getExecutionSession().reportError(std::move(Err));
MR->failMaterialization();
}
@@ -442,15 +444,19 @@ ObjectLinkingLayer::Plugin::~Plugin() {}
ObjectLinkingLayer::ObjectLinkingLayer(ExecutionSession &ES,
JITLinkMemoryManager &MemMgr)
- : ObjectLayer(ES), MemMgr(MemMgr) {}
+ : ObjectLayer(ES), MemMgr(MemMgr) {
+ ES.registerResourceManager(*this);
+}
ObjectLinkingLayer::ObjectLinkingLayer(
ExecutionSession &ES, std::unique_ptr<JITLinkMemoryManager> MemMgr)
- : ObjectLayer(ES), MemMgr(*MemMgr), MemMgrOwnership(std::move(MemMgr)) {}
+ : ObjectLayer(ES), MemMgr(*MemMgr), MemMgrOwnership(std::move(MemMgr)) {
+ ES.registerResourceManager(*this);
+}
ObjectLinkingLayer::~ObjectLinkingLayer() {
- if (auto Err = removeAllModules())
- getExecutionSession().reportError(std::move(Err));
+ assert(Allocs.empty() && "Layer destroyed with resources still attached");
+ getExecutionSession().deregisterResourceManager(*this);
}
void ObjectLinkingLayer::emit(std::unique_ptr<MaterializationResponsibility> R,
@@ -481,63 +487,56 @@ Error ObjectLinkingLayer::notifyEmitted(MaterializationResponsibility &MR,
if (Err)
return Err;
- {
- std::lock_guard<std::mutex> Lock(LayerMutex);
- UntrackedAllocs.push_back(std::move(Alloc));
- }
-
- return Error::success();
+ return MR.withResourceKeyDo(
+ [&](ResourceKey K) { Allocs[K].push_back(std::move(Alloc)); });
}
-Error ObjectLinkingLayer::removeModule(VModuleKey K) {
+Error ObjectLinkingLayer::handleRemoveResources(ResourceKey K) {
+
Error Err = Error::success();
for (auto &P : Plugins)
- Err = joinErrors(std::move(Err), P->notifyRemovingModule(K));
-
- AllocPtr Alloc;
+ Err = joinErrors(std::move(Err), P->notifyRemovingResources(K));
+
+ std::vector<AllocPtr> AllocsToRemove;
+ getExecutionSession().runSessionLocked([&] {
+ auto I = Allocs.find(K);
+ if (I != Allocs.end()) {
+ std::swap(AllocsToRemove, I->second);
+ Allocs.erase(I);
+ }
+ });
- {
- std::lock_guard<std::mutex> Lock(LayerMutex);
- auto AllocItr = TrackedAllocs.find(K);
- Alloc = std::move(AllocItr->second);
- TrackedAllocs.erase(AllocItr);
+ while (!AllocsToRemove.empty()) {
+ Err = joinErrors(std::move(Err), AllocsToRemove.back()->deallocate());
+ AllocsToRemove.pop_back();
}
- assert(Alloc && "No allocation for key K");
-
- return joinErrors(std::move(Err), Alloc->deallocate());
+ return Err;
}
-Error ObjectLinkingLayer::removeAllModules() {
-
- Error Err = Error::success();
-
- for (auto &P : Plugins)
- Err = joinErrors(std::move(Err), P->notifyRemovingAllModules());
-
- std::vector<AllocPtr> Allocs;
- {
- std::lock_guard<std::mutex> Lock(LayerMutex);
- Allocs = std::move(UntrackedAllocs);
-
- for (auto &KV : TrackedAllocs)
- Allocs.push_back(std::move(KV.second));
-
- TrackedAllocs.clear();
+void ObjectLinkingLayer::handleTransferResources(ResourceKey DstKey,
+ ResourceKey SrcKey) {
+ auto I = Allocs.find(SrcKey);
+ if (I != Allocs.end()) {
+ auto &SrcAllocs = I->second;
+ auto &DstAllocs = Allocs[DstKey];
+ DstAllocs.reserve(DstAllocs.size() + SrcAllocs.size());
+ for (auto &Alloc : SrcAllocs)
+ DstAllocs.push_back(std::move(Alloc));
+
+ // Erase SrcKey entry using value rather than iterator I: I may have been
+ // invalidated when we looked up DstKey.
+ Allocs.erase(SrcKey);
}
- while (!Allocs.empty()) {
- Err = joinErrors(std::move(Err), Allocs.back()->deallocate());
- Allocs.pop_back();
- }
-
- return Err;
+ for (auto &P : Plugins)
+ P->notifyTransferringResources(DstKey, SrcKey);
}
EHFrameRegistrationPlugin::EHFrameRegistrationPlugin(
- std::unique_ptr<EHFrameRegistrar> Registrar)
- : Registrar(std::move(Registrar)) {}
+ ExecutionSession &ES, std::unique_ptr<EHFrameRegistrar> Registrar)
+ : ES(ES), Registrar(std::move(Registrar)) {}
void EHFrameRegistrationPlugin::modifyPassConfig(
MaterializationResponsibility &MR, const Triple &TT,
@@ -556,64 +555,70 @@ void EHFrameRegistrationPlugin::modifyPassConfig(
Error EHFrameRegistrationPlugin::notifyEmitted(
MaterializationResponsibility &MR) {
- std::lock_guard<std::mutex> Lock(EHFramePluginMutex);
-
- auto EHFrameRangeItr = InProcessLinks.find(&MR);
- if (EHFrameRangeItr == InProcessLinks.end())
- return Error::success();
-
- auto EHFrameRange = EHFrameRangeItr->second;
- assert(EHFrameRange.Addr &&
- "eh-frame addr to register can not be null");
-
- InProcessLinks.erase(EHFrameRangeItr);
- if (auto Key = MR.getVModuleKey())
- TrackedEHFrameRanges[Key] = EHFrameRange;
- else
- UntrackedEHFrameRanges.push_back(EHFrameRange);
- return Registrar->registerEHFrames(EHFrameRange.Addr, EHFrameRange.Size);
-}
-
-Error EHFrameRegistrationPlugin::notifyRemovingModule(VModuleKey K) {
- std::lock_guard<std::mutex> Lock(EHFramePluginMutex);
+ EHFrameRange EmittedRange;
+ {
+ std::lock_guard<std::mutex> Lock(EHFramePluginMutex);
- auto EHFrameRangeItr = TrackedEHFrameRanges.find(K);
- if (EHFrameRangeItr == TrackedEHFrameRanges.end())
- return Error::success();
+ auto EHFrameRangeItr = InProcessLinks.find(&MR);
+ if (EHFrameRangeItr == InProcessLinks.end())
+ return Error::success();
- auto EHFrameRange = EHFrameRangeItr->second;
- assert(EHFrameRange.Addr && "Tracked eh-frame range must not be null");
+ EmittedRange = EHFrameRangeItr->second;
+ assert(EmittedRange.Addr && "eh-frame addr to register can not be null");
+ InProcessLinks.erase(EHFrameRangeItr);
+ }
- TrackedEHFrameRanges.erase(EHFrameRangeItr);
+ if (auto Err = MR.withResourceKeyDo(
+ [&](ResourceKey K) { EHFrameRanges[K].push_back(EmittedRange); }))
+ return Err;
- return Registrar->deregisterEHFrames(EHFrameRange.Addr, EHFrameRange.Size);
+ return Registrar->registerEHFrames(EmittedRange.Addr, EmittedRange.Size);
}
-Error EHFrameRegistrationPlugin::notifyRemovingAllModules() {
+Error EHFrameRegistrationPlugin::notifyFailed(
+ MaterializationResponsibility &MR) {
std::lock_guard<std::mutex> Lock(EHFramePluginMutex);
+ InProcessLinks.erase(&MR);
+ return Error::success();
+}
- std::vector<EHFrameRange> EHFrameRanges =
- std::move(UntrackedEHFrameRanges);
- EHFrameRanges.reserve(EHFrameRanges.size() + TrackedEHFrameRanges.size());
-
- for (auto &KV : TrackedEHFrameRanges)
- EHFrameRanges.push_back(KV.second);
+Error EHFrameRegistrationPlugin::notifyRemovingResources(ResourceKey K) {
+ std::vector<EHFrameRange> RangesToRemove;
- TrackedEHFrameRanges.clear();
+ ES.runSessionLocked([&] {
+ auto I = EHFrameRanges.find(K);
+ if (I != EHFrameRanges.end()) {
+ RangesToRemove = std::move(I->second);
+ EHFrameRanges.erase(I);
+ }
+ });
Error Err = Error::success();
-
- while (!EHFrameRanges.empty()) {
- auto EHFrameRange = EHFrameRanges.back();
- assert(EHFrameRange.Addr && "Untracked eh-frame range must not be null");
- EHFrameRanges.pop_back();
- Err = joinErrors(std::move(Err), Registrar->deregisterEHFrames(
- EHFrameRange.Addr, EHFrameRange.Size));
+ while (!RangesToRemove.empty()) {
+ auto RangeToRemove = RangesToRemove.back();
+ RangesToRemove.pop_back();
+ assert(RangeToRemove.Addr && "Untracked eh-frame range must not be null");
+ Err = joinErrors(
+ std::move(Err),
+ Registrar->deregisterEHFrames(RangeToRemove.Addr, RangeToRemove.Size));
}
return Err;
}
+void EHFrameRegistrationPlugin::notifyTransferringResources(
+ ResourceKey DstKey, ResourceKey SrcKey) {
+ auto SI = EHFrameRanges.find(SrcKey);
+ if (SI != EHFrameRanges.end()) {
+ auto &SrcRanges = SI->second;
+ auto &DstRanges = EHFrameRanges[DstKey];
+ DstRanges.reserve(DstRanges.size() + SrcRanges.size());
+ for (auto &SrcRange : SrcRanges)
+ DstRanges.push_back(std::move(SrcRange));
+ EHFrameRanges.erase(SI);
+ }
+}
+
} // End namespace orc.
} // End namespace llvm.
diff --git a/llvm/lib/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.cpp b/llvm/lib/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.cpp
index b71e9d4f5f91..a6cccb90936c 100644
--- a/llvm/lib/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.cpp
+++ b/llvm/lib/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.cpp
@@ -77,16 +77,12 @@ namespace orc {
RTDyldObjectLinkingLayer::RTDyldObjectLinkingLayer(
ExecutionSession &ES, GetMemoryManagerFunction GetMemoryManager)
- : ObjectLayer(ES), GetMemoryManager(GetMemoryManager) {}
+ : ObjectLayer(ES), GetMemoryManager(GetMemoryManager) {
+ ES.registerResourceManager(*this);
+}
RTDyldObjectLinkingLayer::~RTDyldObjectLinkingLayer() {
- std::lock_guard<std::mutex> Lock(RTDyldLayerMutex);
- for (auto &MemMgr : MemMgrs) {
- for (auto *L : EventListeners)
- L->notifyFreeingObject(
- static_cast<uint64_t>(reinterpret_cast<uintptr_t>(MemMgr.get())));
- MemMgr->deregisterEHFrames();
- }
+ assert(MemMgrs.empty() && "Layer destroyed with resources still attached");
}
void RTDyldObjectLinkingLayer::emit(
@@ -141,16 +137,8 @@ void RTDyldObjectLinkingLayer::emit(
}
}
- auto K = R->getVModuleKey();
- RuntimeDyld::MemoryManager *MemMgr = nullptr;
-
- // Create a record a memory manager for this object.
- {
- auto Tmp = GetMemoryManager();
- std::lock_guard<std::mutex> Lock(RTDyldLayerMutex);
- MemMgrs.push_back(std::move(Tmp));
- MemMgr = MemMgrs.back().get();
- }
+ auto MemMgr = GetMemoryManager();
+ auto &MemMgrRef = *MemMgr;
// Switch to shared ownership of MR so that it can be captured by both
// lambdas below.
@@ -160,17 +148,20 @@ void RTDyldObjectLinkingLayer::emit(
jitLinkForORC(
object::OwningBinary<object::ObjectFile>(std::move(*Obj), std::move(O)),
- *MemMgr, Resolver, ProcessAllSections,
- [this, K, SharedR, MemMgr, InternalSymbols](
+ MemMgrRef, Resolver, ProcessAllSections,
+ [this, SharedR, &MemMgrRef, InternalSymbols](
const object::ObjectFile &Obj,
- std::unique_ptr<RuntimeDyld::LoadedObjectInfo> LoadedObjInfo,
+ RuntimeDyld::LoadedObjectInfo &LoadedObjInfo,
std::map<StringRef, JITEvaluatedSymbol> ResolvedSymbols) {
- return onObjLoad(K, *SharedR, Obj, MemMgr, std::move(LoadedObjInfo),
+ return onObjLoad(*SharedR, Obj, MemMgrRef, LoadedObjInfo,
ResolvedSymbols, *InternalSymbols);
},
- [this, K, SharedR, MemMgr](object::OwningBinary<object::ObjectFile> Obj,
- Error Err) mutable {
- onObjEmit(K, *SharedR, std::move(Obj), MemMgr, std::move(Err));
+ [this, SharedR, MemMgr = std::move(MemMgr)](
+ object::OwningBinary<object::ObjectFile> Obj,
+ std::unique_ptr<RuntimeDyld::LoadedObjectInfo> LoadedObjInfo,
+ Error Err) mutable {
+ onObjEmit(*SharedR, std::move(Obj), std::move(MemMgr),
+ std::move(LoadedObjInfo), std::move(Err));
});
}
@@ -190,9 +181,9 @@ void RTDyldObjectLinkingLayer::unregisterJITEventListener(JITEventListener &L) {
}
Error RTDyldObjectLinkingLayer::onObjLoad(
- VModuleKey K, MaterializationResponsibility &R,
- const object::ObjectFile &Obj, RuntimeDyld::MemoryManager *MemMgr,
- std::unique_ptr<RuntimeDyld::LoadedObjectInfo> LoadedObjInfo,
+ MaterializationResponsibility &R, const object::ObjectFile &Obj,
+ RuntimeDyld::MemoryManager &MemMgr,
+ RuntimeDyld::LoadedObjectInfo &LoadedObjInfo,
std::map<StringRef, JITEvaluatedSymbol> Resolved,
std::set<StringRef> &InternalSymbols) {
SymbolFlagsMap ExtraSymbolsToClaim;
@@ -273,19 +264,16 @@ Error RTDyldObjectLinkingLayer::onObjLoad(
}
if (NotifyLoaded)
- NotifyLoaded(K, Obj, *LoadedObjInfo);
-
- std::lock_guard<std::mutex> Lock(RTDyldLayerMutex);
- assert(!LoadedObjInfos.count(MemMgr) && "Duplicate loaded info for MemMgr");
- LoadedObjInfos[MemMgr] = std::move(LoadedObjInfo);
+ NotifyLoaded(R, Obj, LoadedObjInfo);
return Error::success();
}
void RTDyldObjectLinkingLayer::onObjEmit(
- VModuleKey K, MaterializationResponsibility &R,
+ MaterializationResponsibility &R,
object::OwningBinary<object::ObjectFile> O,
- RuntimeDyld::MemoryManager *MemMgr, Error Err) {
+ std::unique_ptr<RuntimeDyld::MemoryManager> MemMgr,
+ std::unique_ptr<RuntimeDyld::LoadedObjectInfo> LoadedObjInfo, Error Err) {
if (Err) {
getExecutionSession().reportError(std::move(Err));
R.failMaterialization();
@@ -305,17 +293,59 @@ void RTDyldObjectLinkingLayer::onObjEmit(
// Run EventListener notifyLoaded callbacks.
{
std::lock_guard<std::mutex> Lock(RTDyldLayerMutex);
- auto LOIItr = LoadedObjInfos.find(MemMgr);
- assert(LOIItr != LoadedObjInfos.end() && "LoadedObjInfo missing");
for (auto *L : EventListeners)
- L->notifyObjectLoaded(
- static_cast<uint64_t>(reinterpret_cast<uintptr_t>(MemMgr)), *Obj,
- *LOIItr->second);
- LoadedObjInfos.erase(MemMgr);
+ L->notifyObjectLoaded(pointerToJITTargetAddress(MemMgr.get()), *Obj,
+ *LoadedObjInfo);
}
if (NotifyEmitted)
- NotifyEmitted(K, std::move(ObjBuffer));
+ NotifyEmitted(R, std::move(ObjBuffer));
+
+ if (auto Err = R.withResourceKeyDo(
+ [&](ResourceKey K) { MemMgrs[K].push_back(std::move(MemMgr)); })) {
+ getExecutionSession().reportError(std::move(Err));
+ R.failMaterialization();
+ }
+}
+
+Error RTDyldObjectLinkingLayer::handleRemoveResources(ResourceKey K) {
+
+ std::vector<MemoryManagerUP> MemMgrsToRemove;
+
+ getExecutionSession().runSessionLocked([&] {
+ auto I = MemMgrs.find(K);
+ if (I != MemMgrs.end()) {
+ std::swap(MemMgrsToRemove, I->second);
+ MemMgrs.erase(I);
+ }
+ });
+
+ {
+ std::lock_guard<std::mutex> Lock(RTDyldLayerMutex);
+ for (auto &MemMgr : MemMgrsToRemove) {
+ for (auto *L : EventListeners)
+ L->notifyFreeingObject(pointerToJITTargetAddress(MemMgr.get()));
+ MemMgr->deregisterEHFrames();
+ }
+ }
+
+ return Error::success();
+}
+
+void RTDyldObjectLinkingLayer::handleTransferResources(ResourceKey DstKey,
+ ResourceKey SrcKey) {
+ auto I = MemMgrs.find(SrcKey);
+ if (I != MemMgrs.end()) {
+ auto &SrcMemMgrs = I->second;
+ auto &DstMemMgrs = MemMgrs[DstKey];
+ DstMemMgrs.reserve(DstMemMgrs.size() + SrcMemMgrs.size());
+ for (auto &MemMgr : SrcMemMgrs)
+ DstMemMgrs.push_back(std::move(MemMgr));
+
+ // Erase SrcKey entry using value rather than iterator I: I may have been
+ // invalidated when we looked up DstKey.
+ MemMgrs.erase(SrcKey);
+ }
}
} // End namespace orc.
diff --git a/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp b/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp
index 7e9b0690ccea..7b2d05163127 100644
--- a/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp
+++ b/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp
@@ -1200,16 +1200,19 @@ Error RuntimeDyldImpl::resolveExternalSymbols() {
void RuntimeDyldImpl::finalizeAsync(
std::unique_ptr<RuntimeDyldImpl> This,
- unique_function<void(object::OwningBinary<object::ObjectFile>, Error)>
+ unique_function<void(object::OwningBinary<object::ObjectFile>,
+ std::unique_ptr<RuntimeDyld::LoadedObjectInfo>, Error)>
OnEmitted,
- object::OwningBinary<object::ObjectFile> O) {
+ object::OwningBinary<object::ObjectFile> O,
+ std::unique_ptr<RuntimeDyld::LoadedObjectInfo> Info) {
auto SharedThis = std::shared_ptr<RuntimeDyldImpl>(std::move(This));
auto PostResolveContinuation =
- [SharedThis, OnEmitted = std::move(OnEmitted), O = std::move(O)](
+ [SharedThis, OnEmitted = std::move(OnEmitted), O = std::move(O),
+ Info = std::move(Info)](
Expected<JITSymbolResolver::LookupResult> Result) mutable {
if (!Result) {
- OnEmitted(std::move(O), Result.takeError());
+ OnEmitted(std::move(O), std::move(Info), Result.takeError());
return;
}
@@ -1223,11 +1226,11 @@ void RuntimeDyldImpl::finalizeAsync(
SharedThis->registerEHFrames();
std::string ErrMsg;
if (SharedThis->MemMgr.finalizeMemory(&ErrMsg))
- OnEmitted(std::move(O),
+ OnEmitted(std::move(O), std::move(Info),
make_error<StringError>(std::move(ErrMsg),
inconvertibleErrorCode()));
else
- OnEmitted(std::move(O), Error::success());
+ OnEmitted(std::move(O), std::move(Info), Error::success());
};
JITSymbolResolver::LookupSet Symbols;
@@ -1418,12 +1421,12 @@ void jitLinkForORC(
object::OwningBinary<object::ObjectFile> O,
RuntimeDyld::MemoryManager &MemMgr, JITSymbolResolver &Resolver,
bool ProcessAllSections,
- unique_function<
- Error(const object::ObjectFile &Obj,
- std::unique_ptr<RuntimeDyld::LoadedObjectInfo> LoadedObj,
- std::map<StringRef, JITEvaluatedSymbol>)>
+ unique_function<Error(const object::ObjectFile &Obj,
+ RuntimeDyld::LoadedObjectInfo &LoadedObj,
+ std::map<StringRef, JITEvaluatedSymbol>)>
OnLoaded,
- unique_function<void(object::OwningBinary<object::ObjectFile>, Error)>
+ unique_function<void(object::OwningBinary<object::ObjectFile>,
+ std::unique_ptr<RuntimeDyld::LoadedObjectInfo>, Error)>
OnEmitted) {
RuntimeDyld RTDyld(MemMgr, Resolver);
@@ -1432,17 +1435,17 @@ void jitLinkForORC(
auto Info = RTDyld.loadObject(*O.getBinary());
if (RTDyld.hasError()) {
- OnEmitted(std::move(O), make_error<StringError>(RTDyld.getErrorString(),
- inconvertibleErrorCode()));
+ OnEmitted(std::move(O), std::move(Info),
+ make_error<StringError>(RTDyld.getErrorString(),
+ inconvertibleErrorCode()));
return;
}
- if (auto Err =
- OnLoaded(*O.getBinary(), std::move(Info), RTDyld.getSymbolTable()))
- OnEmitted(std::move(O), std::move(Err));
+ if (auto Err = OnLoaded(*O.getBinary(), *Info, RTDyld.getSymbolTable()))
+ OnEmitted(std::move(O), std::move(Info), std::move(Err));
RuntimeDyldImpl::finalizeAsync(std::move(RTDyld.Dyld), std::move(OnEmitted),
- std::move(O));
+ std::move(O), std::move(Info));
}
} // end namespace llvm
diff --git a/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldImpl.h b/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldImpl.h
index a9346536fd09..d34fae9aaf0c 100644
--- a/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldImpl.h
+++ b/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldImpl.h
@@ -535,9 +535,12 @@ class RuntimeDyldImpl {
static void finalizeAsync(
std::unique_ptr<RuntimeDyldImpl> This,
- unique_function<void(object::OwningBinary<object::ObjectFile>, Error)>
+ unique_function<void(object::OwningBinary<object::ObjectFile>,
+ std::unique_ptr<RuntimeDyld::LoadedObjectInfo>,
+ Error)>
OnEmitted,
- object::OwningBinary<object::ObjectFile> O);
+ object::OwningBinary<object::ObjectFile> O,
+ std::unique_ptr<RuntimeDyld::LoadedObjectInfo> Info);
void reassignSectionAddress(unsigned SectionID, uint64_t Addr);
diff --git a/llvm/tools/llvm-jitlink/llvm-jitlink.cpp b/llvm/tools/llvm-jitlink/llvm-jitlink.cpp
index be3a26c18754..4c0fc210cdc3 100644
--- a/llvm/tools/llvm-jitlink/llvm-jitlink.cpp
+++ b/llvm/tools/llvm-jitlink/llvm-jitlink.cpp
@@ -482,12 +482,11 @@ LLVMJITLinkObjectLinkingLayer::LLVMJITLinkObjectLinkingLayer(
Session &S, JITLinkMemoryManager &MemMgr)
: ObjectLinkingLayer(S.ES, MemMgr), S(S) {}
-Error LLVMJITLinkObjectLinkingLayer::add(JITDylib &JD,
- std::unique_ptr<MemoryBuffer> O,
- VModuleKey K) {
+Error LLVMJITLinkObjectLinkingLayer::add(ResourceTrackerSP RT,
+ std::unique_ptr<MemoryBuffer> O) {
if (S.HarnessFiles.empty() || S.HarnessFiles.count(O->getBufferIdentifier()))
- return ObjectLinkingLayer::add(JD, std::move(O), std::move(K));
+ return ObjectLinkingLayer::add(std::move(RT), std::move(O));
// Use getObjectSymbolInfo to compute the init symbol, but ignore
// the symbols field. We'll handle that manually to include promotion.
@@ -556,9 +555,10 @@ Error LLVMJITLinkObjectLinkingLayer::add(JITDylib &JD,
}
auto MU = std::make_unique<BasicObjectLayerMaterializationUnit>(
- *this, K, std::move(O), std::move(SymbolFlags), std::move(InitSymbol));
+ *this, std::move(O), std::move(SymbolFlags), std::move(InitSymbol));
- return JD.define(std::move(MU));
+ auto &JD = RT->getJITDylib();
+ return JD.define(std::move(MU), std::move(RT));
}
class PhonyExternalsGenerator : public JITDylib::DefinitionGenerator {
@@ -586,6 +586,11 @@ Expected<std::unique_ptr<Session>> Session::Create(Triple TT) {
return std::move(S);
}
+Session::~Session() {
+ if (auto Err = ES.endSession())
+ ES.reportError(std::move(Err));
+}
+
// FIXME: Move to createJITDylib if/when we start using Platform support in
// llvm-jitlink.
Session::Session(Triple TT, uint64_t PageSize, Error &Err)
@@ -603,6 +608,15 @@ Session::Session(Triple TT, uint64_t PageSize, Error &Err)
S.modifyPassConfig(TT, PassConfig);
}
+ Error notifyFailed(MaterializationResponsibility &MR) override {
+ return Error::success();
+ }
+ Error notifyRemovingResources(ResourceKey K) override {
+ return Error::success();
+ }
+ void notifyTransferringResources(ResourceKey DstKey,
+ ResourceKey SrcKey) override {}
+
private:
Session &S;
};
@@ -618,7 +632,7 @@ Session::Session(Triple TT, uint64_t PageSize, Error &Err)
if (!NoExec && !TT.isOSWindows())
ObjLayer.addPlugin(std::make_unique<EHFrameRegistrationPlugin>(
- std::make_unique<InProcessEHFrameRegistrar>()));
+ ES, std::make_unique<InProcessEHFrameRegistrar>()));
ObjLayer.addPlugin(std::make_unique<JITLinkSessionPlugin>(*this));
diff --git a/llvm/tools/llvm-jitlink/llvm-jitlink.h b/llvm/tools/llvm-jitlink/llvm-jitlink.h
index 3b452a1c1036..f0ede14050fd 100644
--- a/llvm/tools/llvm-jitlink/llvm-jitlink.h
+++ b/llvm/tools/llvm-jitlink/llvm-jitlink.h
@@ -33,11 +33,13 @@ struct Session;
/// ObjectLinkingLayer with additional support for symbol promotion.
class LLVMJITLinkObjectLinkingLayer : public orc::ObjectLinkingLayer {
public:
+ using orc::ObjectLinkingLayer::add;
+
LLVMJITLinkObjectLinkingLayer(Session &S,
jitlink::JITLinkMemoryManager &MemMgr);
- Error add(orc::JITDylib &JD, std::unique_ptr<MemoryBuffer> O,
- orc::VModuleKey K = orc::VModuleKey()) override;
+ Error add(orc::ResourceTrackerSP RT,
+ std::unique_ptr<MemoryBuffer> O) override;
private:
Session &S;
@@ -50,6 +52,8 @@ struct Session {
LLVMJITLinkObjectLinkingLayer ObjLayer;
std::vector<orc::JITDylib *> JDSearchOrder;
+ ~Session();
+
static Expected<std::unique_ptr<Session>> Create(Triple TT);
void dumpSessionInfo(raw_ostream &OS);
void modifyPassConfig(const Triple &FTT,
diff --git a/llvm/unittests/ExecutionEngine/Orc/CMakeLists.txt b/llvm/unittests/ExecutionEngine/Orc/CMakeLists.txt
index a440d14dcd61..b382e4c4d731 100644
--- a/llvm/unittests/ExecutionEngine/Orc/CMakeLists.txt
+++ b/llvm/unittests/ExecutionEngine/Orc/CMakeLists.txt
@@ -18,6 +18,7 @@ add_llvm_unittest(OrcJITTests
LazyCallThroughAndReexportsTest.cpp
OrcTestCommon.cpp
QueueChannel.cpp
+ ResourceTrackerTest.cpp
RPCUtilsTest.cpp
RTDyldObjectLinkingLayerTest.cpp
SymbolStringPoolTest.cpp
diff --git a/llvm/unittests/ExecutionEngine/Orc/CoreAPIsTest.cpp b/llvm/unittests/ExecutionEngine/Orc/CoreAPIsTest.cpp
index bad61a4ccbfa..dbce1ede3502 100644
--- a/llvm/unittests/ExecutionEngine/Orc/CoreAPIsTest.cpp
+++ b/llvm/unittests/ExecutionEngine/Orc/CoreAPIsTest.cpp
@@ -1242,7 +1242,7 @@ TEST_F(CoreAPIsStandardTest, TestGetRequestedSymbolsAndReplace) {
BarMaterialized = true;
});
- R->replace(std::move(NewMU));
+ cantFail(R->replace(std::move(NewMU)));
cantFail(R->notifyResolved(SymbolMap({{Foo, FooSym}})));
cantFail(R->notifyEmitted());
@@ -1272,7 +1272,7 @@ TEST_F(CoreAPIsStandardTest, TestMaterializationResponsibilityDelegation) {
auto MU = std::make_unique<SimpleMaterializationUnit>(
SymbolFlagsMap({{Foo, FooSym.getFlags()}, {Bar, BarSym.getFlags()}}),
[&](std::unique_ptr<MaterializationResponsibility> R) {
- auto R2 = R->delegate({Bar});
+ auto R2 = cantFail(R->delegate({Bar}));
cantFail(R->notifyResolved({{Foo, FooSym}}));
cantFail(R->notifyEmitted());
@@ -1333,7 +1333,7 @@ TEST_F(CoreAPIsStandardTest, TestMaterializeWeakSymbol) {
cantFail(FooR->notifyEmitted());
}
-static bool linkOrdersEqual(const std::vector<std::shared_ptr<JITDylib>> &LHS,
+static bool linkOrdersEqual(const std::vector<JITDylibSP> &LHS,
ArrayRef<JITDylib *> RHS) {
if (LHS.size() != RHS.size())
return false;
@@ -1367,23 +1367,21 @@ TEST(JITDylibTest, GetDFSLinkOrderTree) {
LibB.setLinkOrder(makeJITDylibSearchOrder({&LibD, &LibE}));
LibC.setLinkOrder(makeJITDylibSearchOrder({&LibF}));
- auto DFSOrderFromB = JITDylib::getDFSLinkOrder({LibB.shared_from_this()});
+ auto DFSOrderFromB = JITDylib::getDFSLinkOrder({&LibB});
EXPECT_TRUE(linkOrdersEqual(DFSOrderFromB, {&LibB, &LibD, &LibE}))
<< "Incorrect DFS link order for LibB";
- auto DFSOrderFromA = JITDylib::getDFSLinkOrder({LibA.shared_from_this()});
+ auto DFSOrderFromA = JITDylib::getDFSLinkOrder({&LibA});
EXPECT_TRUE(linkOrdersEqual(DFSOrderFromA,
{&LibA, &LibB, &LibD, &LibE, &LibC, &LibF}))
<< "Incorrect DFS link order for libA";
- auto DFSOrderFromAB = JITDylib::getDFSLinkOrder(
- {LibA.shared_from_this(), LibB.shared_from_this()});
+ auto DFSOrderFromAB = JITDylib::getDFSLinkOrder({&LibA, &LibB});
EXPECT_TRUE(linkOrdersEqual(DFSOrderFromAB,
{&LibA, &LibB, &LibD, &LibE, &LibC, &LibF}))
<< "Incorrect DFS link order for { libA, libB }";
- auto DFSOrderFromBA = JITDylib::getDFSLinkOrder(
- {LibB.shared_from_this(), LibA.shared_from_this()});
+ auto DFSOrderFromBA = JITDylib::getDFSLinkOrder({&LibB, &LibA});
EXPECT_TRUE(linkOrdersEqual(DFSOrderFromBA,
{&LibB, &LibD, &LibE, &LibA, &LibC, &LibF}))
<< "Incorrect DFS link order for { libB, libA }";
@@ -1406,7 +1404,7 @@ TEST(JITDylibTest, GetDFSLinkOrderDiamond) {
LibB.setLinkOrder(makeJITDylibSearchOrder({&LibD}));
LibC.setLinkOrder(makeJITDylibSearchOrder({&LibD}));
- auto DFSOrderFromA = JITDylib::getDFSLinkOrder({LibA.shared_from_this()});
+ auto DFSOrderFromA = JITDylib::getDFSLinkOrder({&LibA});
EXPECT_TRUE(linkOrdersEqual(DFSOrderFromA, {&LibA, &LibB, &LibD, &LibC}))
<< "Incorrect DFS link order for libA";
}
@@ -1426,15 +1424,15 @@ TEST(JITDylibTest, GetDFSLinkOrderCycle) {
LibB.setLinkOrder(makeJITDylibSearchOrder({&LibC}));
LibC.setLinkOrder(makeJITDylibSearchOrder({&LibA}));
- auto DFSOrderFromA = JITDylib::getDFSLinkOrder({LibA.shared_from_this()});
+ auto DFSOrderFromA = JITDylib::getDFSLinkOrder({&LibA});
EXPECT_TRUE(linkOrdersEqual(DFSOrderFromA, {&LibA, &LibB, &LibC}))
<< "Incorrect DFS link order for libA";
- auto DFSOrderFromB = JITDylib::getDFSLinkOrder({LibB.shared_from_this()});
+ auto DFSOrderFromB = JITDylib::getDFSLinkOrder({&LibB});
EXPECT_TRUE(linkOrdersEqual(DFSOrderFromB, {&LibB, &LibC, &LibA}))
<< "Incorrect DFS link order for libB";
- auto DFSOrderFromC = JITDylib::getDFSLinkOrder({LibC.shared_from_this()});
+ auto DFSOrderFromC = JITDylib::getDFSLinkOrder({&LibC});
EXPECT_TRUE(linkOrdersEqual(DFSOrderFromC, {&LibC, &LibA, &LibB}))
<< "Incorrect DFS link order for libC";
}
diff --git a/llvm/unittests/ExecutionEngine/Orc/OrcTestCommon.h b/llvm/unittests/ExecutionEngine/Orc/OrcTestCommon.h
index afbc4a9ffaa5..96e89d7a8066 100644
--- a/llvm/unittests/ExecutionEngine/Orc/OrcTestCommon.h
+++ b/llvm/unittests/ExecutionEngine/Orc/OrcTestCommon.h
@@ -44,6 +44,12 @@ namespace orc {
// linkage and non-hidden visibility.
// (5) V -- A JITDylib associated with ES.
class CoreAPIsBasedStandardTest : public testing::Test {
+public:
+ ~CoreAPIsBasedStandardTest() {
+ if (auto Err = ES.endSession())
+ ES.reportError(std::move(Err));
+ }
+
protected:
std::shared_ptr<SymbolStringPool> SSP = std::make_shared<SymbolStringPool>();
ExecutionSession ES{SSP};
@@ -96,8 +102,7 @@ class SimpleMaterializationUnit : public orc::MaterializationUnit {
orc::SymbolStringPtr InitSym = nullptr,
DiscardFunction Discard = DiscardFunction(),
DestructorFunction Destructor = DestructorFunction())
- : MaterializationUnit(std::move(SymbolFlags), std::move(InitSym),
- orc::VModuleKey()),
+ : MaterializationUnit(std::move(SymbolFlags), std::move(InitSym)),
Materialize(std::move(Materialize)), Discard(std::move(Discard)),
Destructor(std::move(Destructor)) {}
@@ -157,6 +162,11 @@ class OrcExecutionTest {
}
};
+ ~OrcExecutionTest() {
+ if (auto Err = ES.endSession())
+ ES.reportError(std::move(Err));
+ }
+
protected:
orc::ExecutionSession ES;
LLVMContext Context;
diff --git a/llvm/unittests/ExecutionEngine/Orc/RTDyldObjectLinkingLayerTest.cpp b/llvm/unittests/ExecutionEngine/Orc/RTDyldObjectLinkingLayerTest.cpp
index afee081f13ab..bdbc00129eb0 100644
--- a/llvm/unittests/ExecutionEngine/Orc/RTDyldObjectLinkingLayerTest.cpp
+++ b/llvm/unittests/ExecutionEngine/Orc/RTDyldObjectLinkingLayerTest.cpp
@@ -59,11 +59,14 @@ static bool testSetProcessAllSections(std::unique_ptr<MemoryBuffer> Obj,
};
ObjLayer.setProcessAllSections(ProcessAllSections);
- cantFail(ObjLayer.add(JD, std::move(Obj), ES.allocateVModule()));
+ cantFail(ObjLayer.add(JD, std::move(Obj)));
ES.lookup(LookupKind::Static, makeJITDylibSearchOrder(&JD),
SymbolLookupSet(Foo), SymbolState::Resolved, OnResolveDoNothing,
NoDependenciesToRegister);
+ if (auto Err = ES.endSession())
+ ES.reportError(std::move(Err));
+
return DebugSectionSeen;
}
@@ -159,12 +162,15 @@ TEST(RTDyldObjectLinkingLayerTest, TestOverrideObjectFlags) {
ObjLayer.setOverrideObjectFlagsWithResponsibilityFlags(true);
- cantFail(CompileLayer.add(JD, std::move(M), ES.allocateVModule()));
+ cantFail(CompileLayer.add(JD, std::move(M)));
ES.lookup(
LookupKind::Static, makeJITDylibSearchOrder(&JD), SymbolLookupSet(Foo),
SymbolState::Resolved,
[](Expected<SymbolMap> R) { cantFail(std::move(R)); },
NoDependenciesToRegister);
+
+ if (auto Err = ES.endSession())
+ ES.reportError(std::move(Err));
}
TEST(RTDyldObjectLinkingLayerTest, TestAutoClaimResponsibilityForSymbols) {
@@ -226,12 +232,15 @@ TEST(RTDyldObjectLinkingLayerTest, TestAutoClaimResponsibilityForSymbols) {
ObjLayer.setAutoClaimResponsibilityForObjectSymbols(true);
- cantFail(CompileLayer.add(JD, std::move(M), ES.allocateVModule()));
+ cantFail(CompileLayer.add(JD, std::move(M)));
ES.lookup(
LookupKind::Static, makeJITDylibSearchOrder(&JD), SymbolLookupSet(Foo),
SymbolState::Resolved,
[](Expected<SymbolMap> R) { cantFail(std::move(R)); },
NoDependenciesToRegister);
+
+ if (auto Err = ES.endSession())
+ ES.reportError(std::move(Err));
}
} // end anonymous namespace
diff --git a/llvm/unittests/ExecutionEngine/Orc/ResourceTrackerTest.cpp b/llvm/unittests/ExecutionEngine/Orc/ResourceTrackerTest.cpp
new file mode 100644
index 000000000000..e999909a4401
--- /dev/null
+++ b/llvm/unittests/ExecutionEngine/Orc/ResourceTrackerTest.cpp
@@ -0,0 +1,457 @@
+//===------ ResourceTrackerTest.cpp - Unit tests ResourceTracker API
+//-------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "OrcTestCommon.h"
+#include "llvm/ADT/FunctionExtras.h"
+#include "llvm/Config/llvm-config.h"
+#include "llvm/ExecutionEngine/Orc/Core.h"
+#include "llvm/ExecutionEngine/Orc/OrcError.h"
+#include "llvm/Testing/Support/Error.h"
+
+using namespace llvm;
+using namespace llvm::orc;
+
+class ResourceTrackerStandardTest : public CoreAPIsBasedStandardTest {};
+
+namespace {
+
+template <typename ResourceT = unsigned>
+class SimpleResourceManager : public ResourceManager {
+public:
+ using HandleRemoveFunction = unique_function<Error(ResourceKey)>;
+
+ using HandleTransferFunction =
+ unique_function<void(ResourceKey, ResourceKey)>;
+
+ using RecordedResourcesMap = DenseMap<ResourceKey, ResourceT>;
+
+ SimpleResourceManager(
+ ExecutionSession &ES,
+ HandleRemoveFunction HandleRemove = HandleRemoveFunction(),
+ HandleTransferFunction HandleTransfer = HandleTransferFunction())
+ : ES(ES), HandleRemove(std::move(HandleRemove)),
+ HandleTransfer(std::move(HandleTransfer)) {
+
+ // If HandleRemvoe is not supplied then use the default.
+ if (!this->HandleRemove)
+ this->HandleRemove = [&](ResourceKey K) -> Error {
+ ES.runSessionLocked([&] { removeResource(K); });
+ return Error::success();
+ };
+
+ // If HandleTransfer is not supplied then use the default.
+ if (!this->HandleTransfer)
+ this->HandleTransfer = [this](ResourceKey DstKey, ResourceKey SrcKey) {
+ transferResources(DstKey, SrcKey);
+ };
+
+ ES.registerResourceManager(*this);
+ }
+
+ SimpleResourceManager(const SimpleResourceManager &) = delete;
+ SimpleResourceManager &operator=(const SimpleResourceManager &) = delete;
+ SimpleResourceManager(SimpleResourceManager &&) = delete;
+ SimpleResourceManager &operator=(SimpleResourceManager &&) = delete;
+
+ ~SimpleResourceManager() { ES.deregisterResourceManager(*this); }
+
+ /// Create an association between the given key and resource.
+ template <typename MergeOp = std::plus<ResourceT>>
+ void recordResource(ResourceKey K, ResourceT Val = ResourceT(),
+ MergeOp Merge = MergeOp()) {
+ auto Tmp = std::move(Resources[K]);
+ Resources[K] = Merge(std::move(Tmp), std::move(Val));
+ }
+
+ /// Remove the resource associated with K from the map if present.
+ void removeResource(ResourceKey K) { Resources.erase(K); }
+
+ /// Transfer resources from DstKey to SrcKey.
+ template <typename MergeOp = std::plus<ResourceT>>
+ void transferResources(ResourceKey DstKey, ResourceKey SrcKey,
+ MergeOp Merge = MergeOp()) {
+ auto &DstResourceRef = Resources[DstKey];
+ ResourceT DstResources;
+ std::swap(DstResourceRef, DstResources);
+
+ auto SI = Resources.find(SrcKey);
+ assert(SI != Resources.end() && "No resource associated with SrcKey");
+
+ DstResourceRef = Merge(std::move(DstResources), std::move(SI->second));
+ Resources.erase(SI);
+ }
+
+ /// Return a reference to the Resources map.
+ RecordedResourcesMap &getRecordedResources() { return Resources; }
+ const RecordedResourcesMap &getRecordedResources() const { return Resources; }
+
+ Error handleRemoveResources(ResourceKey K) override {
+ return HandleRemove(K);
+ }
+
+ void handleTransferResources(ResourceKey DstKey,
+ ResourceKey SrcKey) override {
+ HandleTransfer(DstKey, SrcKey);
+ }
+
+ static void transferNotAllowed(ResourceKey DstKey, ResourceKey SrcKey) {
+ llvm_unreachable("Resource transfer not allowed");
+ }
+
+private:
+ ExecutionSession &ES;
+ HandleRemoveFunction HandleRemove;
+ HandleTransferFunction HandleTransfer;
+ RecordedResourcesMap Resources;
+};
+
+TEST_F(ResourceTrackerStandardTest,
+ BasicDefineAndRemoveAllBeforeMaterializing) {
+
+ bool ResourceManagerGotRemove = false;
+ SimpleResourceManager<> SRM(ES, [&](ResourceKey K) -> Error {
+ ResourceManagerGotRemove = true;
+ EXPECT_EQ(SRM.getRecordedResources().size(), 0U)
+ << "Unexpected resources recorded";
+ SRM.removeResource(K);
+ return Error::success();
+ });
+
+ bool MaterializationUnitDestroyed = false;
+ auto MU = std::make_unique<SimpleMaterializationUnit>(
+ SymbolFlagsMap({{Foo, FooSym.getFlags()}}),
+ [&](std::unique_ptr<MaterializationResponsibility> R) {
+ llvm_unreachable("Never called");
+ },
+ nullptr, SimpleMaterializationUnit::DiscardFunction(),
+ [&]() { MaterializationUnitDestroyed = true; });
+
+ auto RT = JD.createResourceTracker();
+ cantFail(JD.define(std::move(MU), RT));
+ cantFail(RT->remove());
+ auto SymFlags = cantFail(JD.lookupFlags(
+ LookupKind::Static, JITDylibLookupFlags::MatchExportedSymbolsOnly,
+ SymbolLookupSet(Foo)));
+
+ EXPECT_EQ(SymFlags.size(), 0U)
+ << "Symbols should have been removed from the symbol table";
+ EXPECT_TRUE(ResourceManagerGotRemove)
+ << "ResourceManager did not receive handleRemoveResources";
+ EXPECT_TRUE(MaterializationUnitDestroyed)
+ << "MaterializationUnit not destroyed in response to removal";
+}
+
+TEST_F(ResourceTrackerStandardTest, BasicDefineAndRemoveAllAfterMaterializing) {
+
+ bool ResourceManagerGotRemove = false;
+ SimpleResourceManager<> SRM(ES, [&](ResourceKey K) -> Error {
+ ResourceManagerGotRemove = true;
+ EXPECT_EQ(SRM.getRecordedResources().size(), 1U)
+ << "Unexpected number of resources recorded";
+ EXPECT_EQ(SRM.getRecordedResources().count(K), 1U)
+ << "Unexpected recorded resource";
+ SRM.removeResource(K);
+ return Error::success();
+ });
+
+ auto MU = std::make_unique<SimpleMaterializationUnit>(
+ SymbolFlagsMap({{Foo, FooSym.getFlags()}}),
+ [&](std::unique_ptr<MaterializationResponsibility> R) {
+ cantFail(R->withResourceKeyDo(
+ [&](ResourceKey K) { SRM.recordResource(K); }));
+ cantFail(R->notifyResolved({{Foo, FooSym}}));
+ cantFail(R->notifyEmitted());
+ });
+
+ auto RT = JD.createResourceTracker();
+ cantFail(JD.define(std::move(MU), RT));
+ cantFail(ES.lookup({&JD}, Foo));
+ cantFail(RT->remove());
+ auto SymFlags = cantFail(JD.lookupFlags(
+ LookupKind::Static, JITDylibLookupFlags::MatchExportedSymbolsOnly,
+ SymbolLookupSet(Foo)));
+
+ EXPECT_EQ(SymFlags.size(), 0U)
+ << "Symbols should have been removed from the symbol table";
+ EXPECT_TRUE(ResourceManagerGotRemove)
+ << "ResourceManager did not receive handleRemoveResources";
+}
+
+TEST_F(ResourceTrackerStandardTest, BasicDefineAndRemoveAllWhileMaterializing) {
+
+ bool ResourceManagerGotRemove = false;
+ SimpleResourceManager<> SRM(ES, [&](ResourceKey K) -> Error {
+ ResourceManagerGotRemove = true;
+ EXPECT_EQ(SRM.getRecordedResources().size(), 0U)
+ << "Unexpected resources recorded";
+ SRM.removeResource(K);
+ return Error::success();
+ });
+
+ std::unique_ptr<MaterializationResponsibility> MR;
+ auto MU = std::make_unique<SimpleMaterializationUnit>(
+ SymbolFlagsMap({{Foo, FooSym.getFlags()}}),
+ [&](std::unique_ptr<MaterializationResponsibility> R) {
+ MR = std::move(R);
+ });
+
+ auto RT = JD.createResourceTracker();
+ cantFail(JD.define(std::move(MU), RT));
+
+ ES.lookup(
+ LookupKind::Static, makeJITDylibSearchOrder(&JD), SymbolLookupSet(Foo),
+ SymbolState::Ready,
+ [](Expected<SymbolMap> Result) {
+ EXPECT_THAT_EXPECTED(Result, Failed<FailedToMaterialize>())
+ << "Lookup failed unexpectedly";
+ },
+ NoDependenciesToRegister);
+
+ cantFail(RT->remove());
+ auto SymFlags = cantFail(JD.lookupFlags(
+ LookupKind::Static, JITDylibLookupFlags::MatchExportedSymbolsOnly,
+ SymbolLookupSet(Foo)));
+
+ EXPECT_EQ(SymFlags.size(), 0U)
+ << "Symbols should have been removed from the symbol table";
+ EXPECT_TRUE(ResourceManagerGotRemove)
+ << "ResourceManager did not receive handleRemoveResources";
+
+ EXPECT_THAT_ERROR(MR->withResourceKeyDo([](ResourceKey K) {
+ ADD_FAILURE() << "Should not reach withResourceKeyDo body for removed key";
+ }),
+ Failed<ResourceTrackerDefunct>())
+ << "withResourceKeyDo on MR with removed tracker should have failed";
+ EXPECT_THAT_ERROR(MR->notifyResolved({{Foo, FooSym}}),
+ Failed<ResourceTrackerDefunct>())
+ << "notifyResolved on MR with removed tracker should have failed";
+
+ MR->failMaterialization();
+}
+
+TEST_F(ResourceTrackerStandardTest, JITDylibClear) {
+ SimpleResourceManager<> SRM(ES);
+
+ // Add materializer for Foo.
+ cantFail(JD.define(std::make_unique<SimpleMaterializationUnit>(
+ SymbolFlagsMap({{Foo, FooSym.getFlags()}}),
+ [&](std::unique_ptr<MaterializationResponsibility> R) {
+ cantFail(R->withResourceKeyDo(
+ [&](ResourceKey K) { ++SRM.getRecordedResources()[K]; }));
+ cantFail(R->notifyResolved({{Foo, FooSym}}));
+ cantFail(R->notifyEmitted());
+ })));
+
+ // Add materializer for Bar.
+ cantFail(JD.define(std::make_unique<SimpleMaterializationUnit>(
+ SymbolFlagsMap({{Bar, BarSym.getFlags()}}),
+ [&](std::unique_ptr<MaterializationResponsibility> R) {
+ cantFail(R->withResourceKeyDo(
+ [&](ResourceKey K) { ++SRM.getRecordedResources()[K]; }));
+ cantFail(R->notifyResolved({{Bar, BarSym}}));
+ cantFail(R->notifyEmitted());
+ })));
+
+ EXPECT_TRUE(SRM.getRecordedResources().empty())
+ << "Expected no resources recorded yet.";
+
+ cantFail(
+ ES.lookup(makeJITDylibSearchOrder(&JD), SymbolLookupSet({Foo, Bar})));
+
+ auto JDResourceKey = JD.getDefaultResourceTracker()->getKeyUnsafe();
+ EXPECT_EQ(SRM.getRecordedResources().size(), 1U)
+ << "Expected exactly one entry (for JD's ResourceKey)";
+ EXPECT_EQ(SRM.getRecordedResources().count(JDResourceKey), 1U)
+ << "Expected an entry for JD's ResourceKey";
+ EXPECT_EQ(SRM.getRecordedResources()[JDResourceKey], 2U)
+ << "Expected value of 2 for JD's ResourceKey "
+ "(+1 for each of Foo and Bar)";
+
+ cantFail(JD.clear());
+
+ EXPECT_TRUE(SRM.getRecordedResources().empty())
+ << "Expected no resources recorded after clear";
+}
+
+TEST_F(ResourceTrackerStandardTest,
+ BasicDefineAndExplicitTransferBeforeMaterializing) {
+
+ bool ResourceManagerGotTransfer = false;
+ SimpleResourceManager<> SRM(
+ ES,
+ [&](ResourceKey K) -> Error {
+ SRM.removeResource(K);
+ return Error::success();
+ },
+ [&](ResourceKey DstKey, ResourceKey SrcKey) {
+ ResourceManagerGotTransfer = true;
+ auto &RR = SRM.getRecordedResources();
+ EXPECT_EQ(RR.size(), 0U) << "Expected no resources recorded yet";
+ });
+
+ auto MakeMU = [&](SymbolStringPtr Name, JITEvaluatedSymbol Sym) {
+ return std::make_unique<SimpleMaterializationUnit>(
+ SymbolFlagsMap({{Name, Sym.getFlags()}}),
+ [=, &SRM](std::unique_ptr<MaterializationResponsibility> R) {
+ cantFail(R->withResourceKeyDo(
+ [&](ResourceKey K) { SRM.recordResource(K); }));
+ cantFail(R->notifyResolved({{Name, Sym}}));
+ cantFail(R->notifyEmitted());
+ });
+ };
+
+ auto FooRT = JD.createResourceTracker();
+ cantFail(JD.define(MakeMU(Foo, FooSym), FooRT));
+
+ auto BarRT = JD.createResourceTracker();
+ cantFail(JD.define(MakeMU(Bar, BarSym), BarRT));
+
+ BarRT->transferTo(*FooRT);
+
+ EXPECT_TRUE(ResourceManagerGotTransfer)
+ << "ResourceManager did not receive transfer";
+ EXPECT_TRUE(BarRT->isDefunct()) << "BarRT should now be defunct";
+
+ cantFail(
+ ES.lookup(makeJITDylibSearchOrder({&JD}), SymbolLookupSet({Foo, Bar})));
+
+ EXPECT_EQ(SRM.getRecordedResources().size(), 1U)
+ << "Expected exactly one entry (for FooRT's Key)";
+ EXPECT_EQ(SRM.getRecordedResources().count(FooRT->getKeyUnsafe()), 1U)
+ << "Expected an entry for FooRT's ResourceKey";
+ EXPECT_EQ(SRM.getRecordedResources().count(BarRT->getKeyUnsafe()), 0U)
+ << "Expected no entry for BarRT's ResourceKey";
+
+ // We need to explicitly destroy FooRT or its resources will be implicitly
+ // transferred to the default tracker triggering a second call to our
+ // transfer function above (which expects only one call).
+ cantFail(FooRT->remove());
+}
+
+TEST_F(ResourceTrackerStandardTest,
+ BasicDefineAndExplicitTransferAfterMaterializing) {
+
+ bool ResourceManagerGotTransfer = false;
+ SimpleResourceManager<> SRM(
+ ES,
+ [&](ResourceKey K) -> Error {
+ SRM.removeResource(K);
+ return Error::success();
+ },
+ [&](ResourceKey DstKey, ResourceKey SrcKey) {
+ ResourceManagerGotTransfer = true;
+ SRM.transferResources(DstKey, SrcKey);
+ });
+
+ auto MakeMU = [&](SymbolStringPtr Name, JITEvaluatedSymbol Sym) {
+ return std::make_unique<SimpleMaterializationUnit>(
+ SymbolFlagsMap({{Name, Sym.getFlags()}}),
+ [=, &SRM](std::unique_ptr<MaterializationResponsibility> R) {
+ cantFail(R->withResourceKeyDo(
+ [&](ResourceKey K) { SRM.recordResource(K, 1); }));
+ cantFail(R->notifyResolved({{Name, Sym}}));
+ cantFail(R->notifyEmitted());
+ });
+ };
+
+ auto FooRT = JD.createResourceTracker();
+ cantFail(JD.define(MakeMU(Foo, FooSym), FooRT));
+
+ auto BarRT = JD.createResourceTracker();
+ cantFail(JD.define(MakeMU(Bar, BarSym), BarRT));
+
+ EXPECT_EQ(SRM.getRecordedResources().size(), 0U)
+ << "Expected no recorded resources yet";
+
+ cantFail(
+ ES.lookup(makeJITDylibSearchOrder({&JD}), SymbolLookupSet({Foo, Bar})));
+
+ EXPECT_EQ(SRM.getRecordedResources().size(), 2U)
+ << "Expected recorded resources for both Foo and Bar";
+
+ BarRT->transferTo(*FooRT);
+
+ EXPECT_TRUE(ResourceManagerGotTransfer)
+ << "ResourceManager did not receive transfer";
+ EXPECT_TRUE(BarRT->isDefunct()) << "BarRT should now be defunct";
+
+ EXPECT_EQ(SRM.getRecordedResources().size(), 1U)
+ << "Expected recorded resources for Foo only";
+ EXPECT_EQ(SRM.getRecordedResources().count(FooRT->getKeyUnsafe()), 1U)
+ << "Expected recorded resources for Foo";
+ EXPECT_EQ(SRM.getRecordedResources()[FooRT->getKeyUnsafe()], 2U)
+ << "Expected resources value for for Foo to be '2'";
+}
+
+TEST_F(ResourceTrackerStandardTest,
+ BasicDefineAndExplicitTransferWhileMaterializing) {
+
+ bool ResourceManagerGotTransfer = false;
+ SimpleResourceManager<> SRM(
+ ES,
+ [&](ResourceKey K) -> Error {
+ SRM.removeResource(K);
+ return Error::success();
+ },
+ [&](ResourceKey DstKey, ResourceKey SrcKey) {
+ ResourceManagerGotTransfer = true;
+ SRM.transferResources(DstKey, SrcKey);
+ });
+
+ auto FooRT = JD.createResourceTracker();
+ std::unique_ptr<MaterializationResponsibility> FooMR;
+ cantFail(JD.define(std::make_unique<SimpleMaterializationUnit>(
+ SymbolFlagsMap({{Foo, FooSym.getFlags()}}),
+ [&](std::unique_ptr<MaterializationResponsibility> R) {
+ FooMR = std::move(R);
+ }),
+ FooRT));
+
+ auto BarRT = JD.createResourceTracker();
+
+ ES.lookup(
+ LookupKind::Static, makeJITDylibSearchOrder(&JD), SymbolLookupSet(Foo),
+ SymbolState::Ready,
+ [](Expected<SymbolMap> Result) { cantFail(Result.takeError()); },
+ NoDependenciesToRegister);
+
+ cantFail(FooMR->withResourceKeyDo([&](ResourceKey K) {
+ EXPECT_EQ(FooRT->getKeyUnsafe(), K)
+ << "Expected FooRT's ResourceKey for Foo here";
+ SRM.recordResource(K, 1);
+ }));
+
+ EXPECT_EQ(SRM.getRecordedResources().size(), 1U)
+ << "Expected one recorded resource here";
+ EXPECT_EQ(SRM.getRecordedResources()[FooRT->getKeyUnsafe()], 1U)
+ << "Expected Resource value for FooRT to be '1' here";
+
+ FooRT->transferTo(*BarRT);
+
+ EXPECT_TRUE(ResourceManagerGotTransfer)
+ << "Expected resource manager to receive handleTransferResources call";
+
+ cantFail(FooMR->withResourceKeyDo([&](ResourceKey K) {
+ EXPECT_EQ(BarRT->getKeyUnsafe(), K)
+ << "Expected BarRT's ResourceKey for Foo here";
+ SRM.recordResource(K, 1);
+ }));
+
+ EXPECT_EQ(SRM.getRecordedResources().size(), 1U)
+ << "Expected one recorded resource here";
+ EXPECT_EQ(SRM.getRecordedResources().count(BarRT->getKeyUnsafe()), 1U)
+ << "Expected RecordedResources to contain an entry for BarRT";
+ EXPECT_EQ(SRM.getRecordedResources()[BarRT->getKeyUnsafe()], 2U)
+ << "Expected Resource value for BarRT to be '2' here";
+
+ cantFail(FooMR->notifyResolved({{Foo, FooSym}}));
+ cantFail(FooMR->notifyEmitted());
+}
+
+} // namespace
More information about the llvm-commits
mailing list