<div dir="ltr">Hi Manoel,<div><br></div><div>CCing the Dev List.</div><div><br></div><div>Symbol resolution in Orc is handled by supplying a RuntimeDyld::SymbolResolver instance when you add modules to the JIT. When the module you've added to the JIT is linked, the SymbolResolver will be invoked to find any symbols not contained in the module itself.</div><div><br></div><div>The Kaleidoscope code that builds the SymbolResolver instance in Kaleidoscope/initial/toy.cpp file looks like this:</div><div><br></div><font face="monospace, monospace"> auto Resolver = createLambdaResolver(<br> [&](const std::string &Name) {<br> if (auto Sym = findSymbol(Name))<br> return RuntimeDyld::SymbolInfo(Sym.getAddress(),<br> Sym.getFlags());<br> return RuntimeDyld::SymbolInfo(nullptr);<br> },<br> [](const std::string &S) { return nullptr; }<br> );<br> return CompileLayer.addModuleSet(singletonSet(std::move(M)),</font><div><span style="font-family:monospace,monospace"> make_unique<SectionMemoryManager>(),</span><div><font face="monospace, monospace"> std::move</font><span style="font-family:monospace,monospace">(Resolver));</span></div><div><div><br></div><div><br></div><div>For the purposes of this discussion we can ignore the second lambda (it's for "logical dylib lookup", which is a feature you probably don't need). The first lambda is the important one here, and it only tries to do one thing: resolve the requested symbol by looking elsewhere in the JIT (that's the call to 'findSymbol').<br></div><div><br></div><div>To enable resolution of other symbols there are two straightforward approaches.</div><div><br></div><div>(1) Construct a map of the external symbols you want to make visible to the JIT. I prefer this approach where it's practical. It minimizes the surface area between the JIT'd code and the host, which is good for security.</div><div><br></div><div>To use this approach, you just add a StringMap<TargetAddress> member to your JIT class (call it "ExternalSymbols", or something like that). In your constructor you populate the map:</div><div><br></div><div><font face="monospace, monospace">ExternalSymbols[Mangle("printd")] = (uint64_t)&printd;</font></div><div><br></div><div>Then in your addModules method you fall back to searching the map if you can't find the symbol in the JIT:</div><div><br></div><font face="monospace, monospace"> auto Resolver = createLambdaResolver(<br> [&](const std::string &Name) {<br><br> /// Try the JIT first....<br> if (auto Sym = findSymbol(Name))<br> return RuntimeDyld::SymbolInfo(Sym.getAddress(),<br> Sym.getFlags());<br><br> // Try the external symbols map...<br> if (auto Addr = ExternalSymbols.lookup(Name))</font><div><font face="monospace, monospace"> return RuntimeDyld::SymbolInfo(Addr,</font></div><div><font face="monospace, monospace"> JITSymbolFlags::Exported);<br><br> // Symbol not found.<br> return RuntimeDyld::SymbolInfo(nullptr);<br> },<br> [](const std::string &S) { return nullptr; }<br> );</font></div><div><br></div><div><br></div><div><br></div><div><br></div><div>Option (2) is to expose all the symbols from the host process. You can see an example of this in tools/lli/OrcLazyJIT.{h,cpp}. Before you construct your JIT you run:</div><br><font face="monospace, monospace">sys::DynamicLibrary::LoadLibraryPermanently(nullptr);</font><div><br></div><div>This will make the host process symbols visible to the DynamicLibrary class. Then you can use the RTDyldMemoryManager::getSymbolAddressInProcess method (which uses DynamicLibrary for lookup) to search the host process in your resolver:</div><div><br></div><font face="monospace, monospace"> auto Resolver = createLambdaResolver(<br> [&](const std::string &Name) {<br><br> if (auto Sym = findSymbol(Name))<br> return RuntimeDyld::SymbolInfo(Sym.getAddress(),</font><div><font face="monospace, monospace"> Sym.getFlags());<br><br> if (auto Addr = RTDyldMemoryManager::getSymbolAddressInProcess(Name))<br> return RuntimeDyld::SymbolInfo(Addr, JITSymbolFlags::Exported);<br><br> return RuntimeDyld::SymbolInfo(nullptr);<br> },</font></div><div><font face="monospace, monospace"> [](const std::string &S) { return nullptr; }</font></div><div><font face="monospace, monospace"> };</font><div><br></div><div>Hope this helps!</div><div><br></div><div>Cheers,</div><div>Lang.</div></div><div><br></div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Apr 2, 2015 at 4:46 AM, Manoel Teixeira <span dir="ltr"><<a href="mailto:mbsteixeira@gmail.com" target="_blank">mbsteixeira@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div>Hi, Lang.</div><div><br></div><div>I'm beginning the OrcJit. Now I need to import some extern 'C' code. <span style="font-size:12.8000001907349px">Using the old jit it was done with </span><span style="font-size:12.8000001907349px">/getEngine()->addGlobalMapping( exfn, functionsToLoad[i]->funcPtr ); </span></div><div><span style="font-size:12.8000001907349px">and with the new MCJIT it runs fine with </span><span style="font-size:12.8000001907349px;white-space:pre-wrap"> </span><span style="font-size:12.8000001907349px">sys::DynamicLibrary::AddSymbol(functionsToLoad[i]->funcName, functionsToLoad[i]->funcPtr);</span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><div>Your examples in <span style="font-size:12.8000001907349px">Kaleidoscope/Orc/ do not call the extern printd. </span></div><div><div><span style="font-size:12.8000001907349px">ready> extern printd(c)</span></div><div><span style="font-size:12.8000001907349px">printd(4);</span></div><div><span style="font-size:12.8000001907349px">ready> Expression function:</span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px">define double @__anon_expr() {</span></div><div><span style="font-size:12.8000001907349px">entry:</span></div><div><span style="font-size:12.8000001907349px"> %calltmp = call double @printd(double 4.000000e+00)</span></div><div><span style="font-size:12.8000001907349px"> ret double %calltmp</span></div><div><span style="font-size:12.8000001907349px">}</span></div></div><div><span style="font-size:12.8000001907349px">When the Orc tries to load the anon_expr, it fails. </span></div><div><span style="font-size:12.8000001907349px">Is there some extra step to do that?</span></div></div><div><span style="font-size:12.8000001907349px"><br></span></div><div>Cheers,</div><div>Manoel</div><div><div style="font-size:12.8000001907349px"><br></div></div></div><div class="gmail_extra"><br><div class="gmail_quote"><span class="">On Fri, Mar 27, 2015 at 12:15 AM, Lang Hames <span dir="ltr"><<a href="mailto:lhames@gmail.com" target="_blank">lhames@gmail.com</a>></span> wrote:<br></span><div><div class="h5"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Manoel,<div><br></div><div>CC'ing the dev list, since the answer may be relevant to others...</div><span><div><br></div>> Do you have news about Modify a module at runtime in MCJIT?<br>> We need this feature. Who develops dynamic language like lua, javascript has the same problem.<div class="gmail_extra"><br></div></span><div class="gmail_extra">MCJIT does not support modification of modules that have already been added to the JIT. There is nothing preventing you from modifying the IR, but if the module has already been compiled then your modifications will have no effect.</div><div class="gmail_extra"><br></div><div class="gmail_extra">To modify code in MCJIT you would usually delete the JIT state containing the module (often this just means deleting the whole MCJIT instance), then adding the new version of the Module to the JIT (or building a new MCJIT instance), then update any callers. How this is done is left up to the client - MCJIT provides no built-in support.</div><div class="gmail_extra"><br></div><div class="gmail_extra">You can build on MCJIT to do this, and several clients have, but you won't get much help from the infrastructure. I would suggest the new Orc JIT APIs instead, as they offer two advantages over MCJIT:</div><div class="gmail_extra"><br></div><div class="gmail_extra">(1) The Orc layers don't take ownership of the IR by default. This means you can retain ownership of the Module and modify it place.</div><div class="gmail_extra"><br></div><div class="gmail_extra">(2) The Orc layers provide a "removeModuleSet" method that enables you to delete the JIT state associated with a particular module (or set of modules) that you have added. This saves you from manually managing multiple MCJIT interfaces.</div><div class="gmail_extra"><br></div><div class="gmail_extra">Some of the functions in "include/llvm/ExecutionEngine/Orc/IndirectionUtils.h" may also be of use to you if you want to partition multi-function modules or use stubs to handle redirecting function calls.</div><div class="gmail_extra"><br></div><div class="gmail_extra">The basic Orc-JIT setup shown in examples/Kaleidoscope/Orc/initial/toy.cpp may help you get up and running with the new APIs if you would like to try them.</div><div class="gmail_extra"><br></div><div class="gmail_extra">I hope this helps!</div><div class="gmail_extra"><br></div><div class="gmail_extra">Cheers,</div><div class="gmail_extra">Lang.</div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Mar 26, 2015 at 10:58 PM, Manoel Teixeira <span dir="ltr"><<a href="mailto:mbsteixeira@gmail.com" target="_blank">mbsteixeira@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><br><div> Hello, Lang.</div><div><br></div><div> Do you have news about <span style="color:rgb(0,0,0);font-family:'Times New Roman'">Modify a module at runtime in MCJIT?</span></div><div><span style="color:rgb(0,0,0);font-family:'Times New Roman'"> We need this feature. Who develops dynamic language like lua, javascript has the same problem.</span></div><div><span style="color:rgb(0,0,0);font-family:'Times New Roman'"><br></span></div><div><span style="color:rgb(0,0,0);font-family:'Times New Roman'"> Cheers,</span></div><div><span style="color:rgb(0,0,0);font-family:'Times New Roman'"> Manoel</span></div></div>
</blockquote></div><br></div></div></div></div>
</blockquote></div></div></div><br></div>
</blockquote></div><br></div>