<div class="socmaildefaultfont" dir="ltr" style="font-family:Arial, Helvetica, sans-serif;font-size:10.5pt" ><div dir="ltr" >Awesome, thanks!</div>
<div dir="ltr" > </div>
<blockquote data-history-content-modified="1" dir="ltr" style="border-left:solid #aaaaaa 2px; margin-left:5px; padding-left:5px; direction:ltr; margin-right:0px" >----- Original message -----<br>From: Lang Hames <lhames@gmail.com><br>To: sfertile@ca.ibm.com<br>Cc: Commit Messages and Patches for LLVM <llvm-commits@lists.llvm.org>, syzaara@ca.ibm.com, aheejin@gmail.com<br>Subject: Re: [llvm] r336385 - [ORC] In CompileOnDemandLayer2, clone modules on to different contexts by<br>Date: Thu, Jul 5, 2018 10:02 PM<br>
<div dir="ltr" >Looks like Heejin Ahn fixed this in r336389. Thanks Heejin!
<div> </div>
<div>-- Lang.</div></div>
<div><div dir="ltr" >On Thu, Jul 5, 2018 at 6:57 PM Lang Hames <<a href="mailto:lhames@gmail.com" target="_blank">lhames@gmail.com</a>> wrote:</div>
<blockquote style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex" ><div dir="ltr" >It did. Sorry! Looking in to it now. I think it's just missing a dependency.
<div> </div>
<div>-- Lang.</div></div>
<div><div dir="ltr" >On Thu, Jul 5, 2018 at 2:07 PM Sean Fertile <<a href="mailto:sfertile@ca.ibm.com" target="_blank">sfertile@ca.ibm.com</a>> wrote:</div>
<blockquote style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex" ><div dir="ltr" style="font-family:Arial,Helvetica,sans-serif;font-size:10.5pt" ><div dir="ltr" >Hi Lang,</div>
<div dir="ltr" > </div>
<div dir="ltr" >I think this commit broke <a href="http://lab.llvm.org:8011/builders/clang-ppc64le-linux-multistage" target="_blank">http://lab.llvm.org:8011/builders/clang-ppc64le-linux-multistage</a></div>
<div dir="ltr" > </div>
<div dir="ltr" >Thanks</div>
<div dir="ltr" >Sean</div>
<div dir="ltr" > </div>
<div dir="ltr" > </div>
<blockquote dir="ltr" style="border-left:solid #aaaaaa 2px;margin-left:5px;padding-left:5px;direction:ltr;margin-right:0px" >----- Original message -----<br>From: Lang Hames via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>><br>Sent by: "llvm-commits" <<a href="mailto:llvm-commits-bounces@lists.llvm.org" target="_blank">llvm-commits-bounces@lists.llvm.org</a>><br>To: <a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a><br>Cc:<br>Subject: [llvm] r336385 - [ORC] In CompileOnDemandLayer2, clone modules on to different contexts by<br>Date: Thu, Jul 5, 2018 3:06 PM<br>
<div><font size="2" face="Default Monospace,Courier New,Courier,monospace" >Author: lhames<br>Date: Thu Jul 5 12:01:27 2018<br>New Revision: 336385<br><br>URL: <a href="http://llvm.org/viewvc/llvm-project?rev=336385&view=rev" target="_blank">http://llvm.org/viewvc/llvm-project?rev=336385&view=rev</a><br>Log:<br>[ORC] In CompileOnDemandLayer2, clone modules on to different contexts by<br>writing them to a buffer and re-loading them.<br><br>Also introduces a multithreaded variant of SimpleCompiler<br>(MultiThreadedSimpleCompiler) for compiling IR concurrently on multiple<br>threads.<br><br>These changes are required to JIT IR on multiple threads correctly.<br><br>No test case yet. I will be looking at how to modify LLI / LLJIT to test<br>multithreaded JIT support soon.<br><br>Modified:<br> llvm/trunk/include/llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h<br> llvm/trunk/include/llvm/ExecutionEngine/Orc/CompileUtils.h<br> llvm/trunk/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp<br><br>Modified: llvm/trunk/include/llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h<br>URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h?rev=336385&r1=336384&r2=336385&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h?rev=336385&r1=336384&r2=336385&view=diff</a><br>==============================================================================<br>--- llvm/trunk/include/llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h (original)<br>+++ llvm/trunk/include/llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h Thu Jul 5 12:01:27 2018<br>@@ -93,15 +93,8 @@ private:<br> using StubManagersMap =<br> std::map<const VSO *, std::unique_ptr<IndirectStubsManager>>;<br> <br>- using SymbolNameToDefinitionMap =<br>- IRMaterializationUnit::SymbolNameToDefinitionMap;<br>-<br> IndirectStubsManager &getStubsManager(const VSO &V);<br> <br>- std::unique_ptr<Module><br>- extractFunctions(Module &M, const SymbolNameSet &SymbolNames,<br>- const SymbolNameToDefinitionMap &SymbolToDefiniton);<br>-<br> void emitExtractedFunctionsModule(MaterializationResponsibility R,<br> std::unique_ptr<Module> M,<br> std::shared_ptr<SymbolResolver> Resolver);<br><br>Modified: llvm/trunk/include/llvm/ExecutionEngine/Orc/CompileUtils.h<br>URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ExecutionEngine/Orc/CompileUtils.h?rev=336385&r1=336384&r2=336385&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ExecutionEngine/Orc/CompileUtils.h?rev=336385&r1=336384&r2=336385&view=diff</a><br>==============================================================================<br>--- llvm/trunk/include/llvm/ExecutionEngine/Orc/CompileUtils.h (original)<br>+++ llvm/trunk/include/llvm/ExecutionEngine/Orc/CompileUtils.h Thu Jul 5 12:01:27 2018<br>@@ -16,6 +16,7 @@<br> <br> #include "llvm/ADT/SmallVector.h"<br> #include "llvm/ExecutionEngine/ObjectCache.h"<br>+#include "llvm/ExecutionEngine/Orc/ExecutionUtils.h"<br> #include "llvm/IR/LegacyPassManager.h"<br> #include "llvm/Object/Binary.h"<br> #include "llvm/Object/ObjectFile.h"<br>@@ -35,24 +36,10 @@ class Module;<br> <br> namespace orc {<br> <br>-/// Simple compile functor: Takes a single IR module and returns an<br>-/// ObjectFile.<br>+/// Simple compile functor: Takes a single IR module and returns an ObjectFile.<br>+/// This compiler supports a single compilation thread and LLVMContext only.<br>+/// For multithreaded compilation, use MultiThreadedSimpleCompiler below.<br> class SimpleCompiler {<br>-private:<br>- class SmallVectorMemoryBuffer : public MemoryBuffer {<br>- public:<br>- SmallVectorMemoryBuffer(SmallVector<char, 0> Buffer)<br>- : Buffer(std::move(Buffer)) {<br>- init(this->Buffer.data(), this->Buffer.data() + this->Buffer.size(),<br>- false);<br>- }<br>-<br>- BufferKind getBufferKind() const override { return MemoryBuffer_Malloc; }<br>-<br>- private:<br>- SmallVector<char, 0> Buffer;<br>- };<br>-<br> public:<br> using CompileResult = std::unique_ptr<MemoryBuffer>;<br> <br>@@ -114,6 +101,29 @@ private:<br> ObjectCache *ObjCache = nullptr;<br> };<br> <br>+/// A thread-safe version of SimpleCompiler.<br>+///<br>+/// This class creates a new TargetMachine and SimpleCompiler instance for each<br>+/// compile.<br>+class MultiThreadedSimpleCompiler {<br>+public:<br>+ MultiThreadedSimpleCompiler(JITTargetMachineBuilder JTMB,<br>+ ObjectCache *ObjCache = nullptr)<br>+ : JTMB(std::move(JTMB)), ObjCache(ObjCache) {}<br>+<br>+ void setObjectCache(ObjectCache *ObjCache) { this->ObjCache = ObjCache; }<br>+<br>+ std::unique_ptr<MemoryBuffer> operator()(Module &M) {<br>+ auto TM = cantFail(JTMB.createTargetMachine());<br>+ SimpleCompiler C(*TM, ObjCache);<br>+ return C(M);<br>+ }<br>+<br>+private:<br>+ JITTargetMachineBuilder JTMB;<br>+ ObjectCache *ObjCache = nullptr;<br>+};<br>+<br> } // end namespace orc<br> <br> } // end namespace llvm<br><br>Modified: llvm/trunk/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp<br>URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp?rev=336385&r1=336384&r2=336385&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp?rev=336385&r1=336384&r2=336385&view=diff</a><br>==============================================================================<br>--- llvm/trunk/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp (original)<br>+++ llvm/trunk/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp Thu Jul 5 12:01:27 2018<br>@@ -8,9 +8,12 @@<br> //===----------------------------------------------------------------------===//<br> <br> #include "llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h"<br>+#include "llvm/Bitcode/BitcodeReader.h"<br>+#include "llvm/Bitcode/BitcodeWriter.h"<br> #include "llvm/IR/Mangler.h"<br> #include "llvm/IR/Module.h"<br> #include "llvm/Support/raw_ostream.h"<br>+#include "llvm/Transforms/Utils/Cloning.h"<br> <br> using namespace llvm;<br> using namespace llvm::orc;<br>@@ -68,36 +71,58 @@ static void extractAliases(Materializati<br> R.delegate(symbolAliases(std::move(Aliases)));<br> }<br> <br>-static std::unique_ptr<Module> extractGlobals(Module &M) {<br>- // FIXME: Add alias support.<br>+static std::unique_ptr<Module><br>+extractAndClone(Module &M, LLVMContext &NewContext, StringRef Suffix,<br>+ function_ref<bool(const GlobalValue *)> ShouldCloneDefinition) {<br>+ SmallVector<char, 1> ClonedModuleBuffer;<br> <br>- auto GlobalsModule = llvm::make_unique<Module>(<br>- (M.getName() + ".globals").str(), M.getContext());<br>- GlobalsModule->setDataLayout(M.getDataLayout());<br>-<br>- ValueToValueMapTy VMap;<br>-<br>- for (auto &GV : M.globals())<br>- if (!GV.isDeclaration() && !VMap.count(&GV))<br>- cloneGlobalVariableDecl(*GlobalsModule, GV, &VMap);<br>-<br>- // Clone the module flags.<br>- cloneModuleFlagsMetadata(*GlobalsModule, M, VMap);<br>-<br>- auto Materializer = createLambdaValueMaterializer([&](Value *V) -> Value * {<br>- if (auto *F = dyn_cast<Function>(V))<br>- return cloneFunctionDecl(*GlobalsModule, *F);<br>- return nullptr;<br>- });<br>+ {<br>+ std::set<GlobalValue *> ClonedDefsInSrc;<br>+ ValueToValueMapTy VMap;<br>+ auto Tmp = CloneModule(M, VMap, [&](const GlobalValue *GV) {<br>+ if (ShouldCloneDefinition(GV)) {<br>+ ClonedDefsInSrc.insert(const_cast<GlobalValue *>(GV));<br>+ return true;<br>+ }<br>+ return false;<br>+ });<br>+<br>+ for (auto *GV : ClonedDefsInSrc) {<br>+ // Delete the definition and bump the linkage in the source module.<br>+ if (isa<Function>(GV)) {<br>+ auto &F = *cast<Function>(GV);<br>+ F.deleteBody();<br>+ F.setPersonalityFn(nullptr);<br>+ } else if (isa<GlobalVariable>(GV)) {<br>+ cast<GlobalVariable>(GV)->setInitializer(nullptr);<br>+ } else<br>+ llvm_unreachable("Unsupported global type");<br>+<br>+ GV->setLinkage(GlobalValue::ExternalLinkage);<br>+ }<br>+<br>+ BitcodeWriter BCWriter(ClonedModuleBuffer);<br> <br>- // Move the global variable initializers.<br>- for (auto &GV : M.globals()) {<br>- if (!GV.isDeclaration())<br>- moveGlobalVariableInitializer(GV, VMap, &Materializer);<br>- GV.setInitializer(nullptr);<br>+ BCWriter.writeModule(*Tmp);<br>+ BCWriter.writeSymtab();<br>+ BCWriter.writeStrtab();<br> }<br> <br>- return GlobalsModule;<br>+ MemoryBufferRef ClonedModuleBufferRef(<br>+ StringRef(ClonedModuleBuffer.data(), ClonedModuleBuffer.size()),<br>+ "cloned module buffer");<br>+<br>+ auto ClonedModule =<br>+ cantFail(parseBitcodeFile(ClonedModuleBufferRef, NewContext));<br>+ ClonedModule->setModuleIdentifier((M.getName() + Suffix).str());<br>+ return ClonedModule;<br>+}<br>+<br>+static std::unique_ptr<Module> extractGlobals(Module &M,<br>+ LLVMContext &NewContext) {<br>+ return extractAndClone(M, NewContext, ".globals", [](const GlobalValue *GV) {<br>+ return isa<GlobalVariable>(GV);<br>+ });<br> }<br> <br> namespace llvm {<br>@@ -132,12 +157,29 @@ private:<br> // original function definitions in the target VSO. All other<br> // symbols should be looked up in the backing resolver.<br> <br>- // Find the functions that have been requested.<br> auto RequestedSymbols = R.getRequestedSymbols();<br> <br>- // Extract them into a new module.<br>- auto ExtractedFunctionsModule =<br>- Parent.extractFunctions(*M, RequestedSymbols, SymbolToDefinition);<br>+ // Extract the requested functions into a new module.<br>+ std::unique_ptr<Module> ExtractedFunctionsModule;<br>+ if (!RequestedSymbols.empty()) {<br>+ std::string Suffix;<br>+ std::set<const GlobalValue *> FunctionsToClone;<br>+ for (auto &Name : RequestedSymbols) {<br>+ auto I = SymbolToDefinition.find(Name);<br>+ assert(I != SymbolToDefinition.end() && I->second != nullptr &&<br>+ "Should have a non-null definition");<br>+ FunctionsToClone.insert(I->second);<br>+ Suffix += ".";<br>+ Suffix += *Name;<br>+ }<br>+<br>+ std::lock_guard<std::mutex> Lock(SourceModuleMutex);<br>+ ExtractedFunctionsModule =<br>+ extractAndClone(*M, Parent.GetAvailableContext(), Suffix,<br>+ [&](const GlobalValue *GV) -> bool {<br>+ return FunctionsToClone.count(GV);<br>+ });<br>+ }<br> <br> // Build a new ExtractingIRMaterializationUnit to delegate the unrequested<br> // symbols to.<br>@@ -162,8 +204,9 @@ private:<br> std::move(DelegatedSymbolToDefinition), Parent, BackingResolver));<br> }<br> <br>- Parent.emitExtractedFunctionsModule(<br>- std::move(R), std::move(ExtractedFunctionsModule), BackingResolver);<br>+ if (ExtractedFunctionsModule)<br>+ Parent.emitExtractedFunctionsModule(<br>+ std::move(R), std::move(ExtractedFunctionsModule), BackingResolver);<br> }<br> <br> void discard(const VSO &V, SymbolStringPtr Name) override {<br>@@ -173,6 +216,7 @@ private:<br> "ExtractingIRMaterializationUnit");<br> }<br> <br>+ mutable std::mutex SourceModuleMutex;<br> CompileOnDemandLayer2 &Parent;<br> std::shared_ptr<SymbolResolver> BackingResolver;<br> };<br>@@ -207,7 +251,7 @@ void CompileOnDemandLayer2::emit(Materia<br> <br> extractAliases(R, *M, Mangle);<br> <br>- auto GlobalsModule = extractGlobals(*M);<br>+ auto GlobalsModule = extractGlobals(*M, GetAvailableContext());<br> <br> // Delete the bodies of any available externally functions, rename the<br> // rest, and build the compile callbacks.<br>@@ -221,6 +265,7 @@ void CompileOnDemandLayer2::emit(Materia<br> <br> if (F.hasAvailableExternallyLinkage()) {<br> F.deleteBody();<br>+ F.setPersonalityFn(nullptr);<br> continue;<br> }<br> <br>@@ -229,7 +274,10 @@ void CompileOnDemandLayer2::emit(Materia<br> F.setName(F.getName() + "$body");<br> auto StubDecl = cloneFunctionDecl(*M, F);<br> StubDecl->setName(StubUnmangledName);<br>+ StubDecl->setPersonalityFn(nullptr);<br>+ StubDecl->setLinkage(GlobalValue::ExternalLinkage);<br> F.replaceAllUsesWith(StubDecl);<br>+<br> auto StubName = Mangle(StubUnmangledName);<br> auto BodyName = Mangle(F.getName());<br> if (auto CallbackAddr = CCMgr.getCompileCallback(<br>@@ -302,52 +350,6 @@ IndirectStubsManager &CompileOnDemandLay<br> return *I->second;<br> }<br> <br>-std::unique_ptr<Module> CompileOnDemandLayer2::extractFunctions(<br>- Module &M, const SymbolNameSet &SymbolNames,<br>- const SymbolNameToDefinitionMap &SymbolToDefinition) {<br>- assert(!SymbolNames.empty() && "Can not extract an empty function set");<br>-<br>- std::string ExtractedModName;<br>- {<br>- raw_string_ostream ExtractedModNameStream(ExtractedModName);<br>- ExtractedModNameStream << M.getName();<br>- for (auto &Name : SymbolNames)<br>- ExtractedModNameStream << "." << *Name;<br>- }<br>-<br>- auto ExtractedFunctionsModule =<br>- llvm::make_unique<Module>(ExtractedModName, GetAvailableContext());<br>- ExtractedFunctionsModule->setDataLayout(M.getDataLayout());<br>-<br>- ValueToValueMapTy VMap;<br>-<br>- auto Materializer = createLambdaValueMaterializer([&](Value *V) -> Value * {<br>- GlobalValue *NewGV = nullptr;<br>- if (auto *F = dyn_cast<Function>(V))<br>- NewGV = cloneFunctionDecl(*ExtractedFunctionsModule, *F);<br>- else if (auto *GV = dyn_cast<GlobalVariable>(V))<br>- NewGV = cloneGlobalVariableDecl(*ExtractedFunctionsModule, *GV);<br>-<br>- if (NewGV)<br>- NewGV->setLinkage(GlobalValue::ExternalLinkage);<br>- return NewGV;<br>- });<br>-<br>- std::vector<std::pair<Function *, Function *>> OrigToNew;<br>- for (auto &FunctionName : SymbolNames) {<br>- assert(SymbolToDefinition.count(FunctionName) &&<br>- "No definition for symbol");<br>- auto *OrigF = cast<Function>(SymbolToDefinition.find(FunctionName)->second);<br>- auto *NewF = cloneFunctionDecl(*ExtractedFunctionsModule, *OrigF, &VMap);<br>- OrigToNew.push_back(std::make_pair(OrigF, NewF));<br>- }<br>-<br>- for (auto &KV : OrigToNew)<br>- moveFunctionBody(*KV.first, VMap, &Materializer, KV.second);<br>-<br>- return ExtractedFunctionsModule;<br>-}<br>-<br> void CompileOnDemandLayer2::emitExtractedFunctionsModule(<br> MaterializationResponsibility R, std::unique_ptr<Module> M,<br> std::shared_ptr<SymbolResolver> Resolver) {<br><br><br>_______________________________________________<br>llvm-commits mailing list<br><a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a><br><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a></font><br> </div></blockquote>
<div dir="ltr" > </div></div></blockquote></div></blockquote></div></blockquote>
<div dir="ltr" > </div></div><BR>