[llvm-dev] Linking existing functions from JITed code

[Lang Hames via llvm-dev]

Hi Andy,

I think that makes sense. I'm currently rewriting the core Kaleidoscope
tutorials - I'll look at adding support for this.

- Lang.

On Fri, Aug 14, 2015 at 7:38 AM, Andy Somogyi <andy.somogyi at gmail.com>
wrote:

> After some fiddling with it, it does in fact look like it works as you
> describe Lang.
>
> The trick was you had to call
>
> llvm::sys::DynamicLibrary::LoadLibraryPermanently(nullptr);
>
> to add the currently running process before calling
>
> llvm::RTDyldMemoryManager::getSymbolAddressInProcess(name);.
>
> Also, all of the function needs to be declared as extern C to avoid name
> mangling.
>
> I however think that adding the addGlobalMapping/getPointerToGlobalMapping
> functions to the ORC kaleidoscope examples is very useful, as for example
> one wants a set of private symbols made available to JITed code.
>
>
>
> On Aug 13, 2015, at 7:41 PM, Lang Hames <lhames at gmail.com> wrote:
>
> Hi Andy,
>
> I haven't tested this on Linux, but on MacOS the
> RuntimeDyldMemorManager::getSymbolAddressInProcess method should find
> symbol addresses in the host program, including symbols from static
> archives linked into the program. However, one gotcha is that the symbol
> has to be reachable from main, otherwise the linker may strip it from the
> final executable.
>
> Do you have a test-case that I could try to reproduce the issue with?
>
> Alternatively, if you run the code under a debugger, do you see symbols
> that the JIT failed to find? If the symbols are visible in the debugger but
> invisible to the JIT that sounds like a bug in the JIT. If the symbols are
> invisible in both that would suggest that the linker is stripping them out.
>
> - Lang.
>
>
> On Thu, Aug 13, 2015 at 1:12 PM, David Blaikie <dblaikie at gmail.com> wrote:
>
>> +"Lang Hames, Linker of Linkers"
>>
>> (Lang developed the ORC JIT you seem to be using & can provide more
>> context than I have)
>>
>> On Thu, Aug 13, 2015 at 12:43 PM, Andy Somogyi via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>> Hi
>>>
>>> I’ve previously used the ExecutionEngine::addGlobalMapping to make
>>> existing functions available to my JITed code.
>>>
>>> I’m currently using ORC, as MCJIT does not appear to be maintained any
>>> longer (the kaleidoscope examples have not worked for some time with
>>> MCJIT).
>>>
>>> I’m using just the basic ORC CompileLayer directly.
>>>
>>> So, I’ve essentially copied the ExecutionEngine::addGlobalMapping
>>> related function to my JIT context, and I create a lambda resolver as such:
>>>
>>> JITContext::addModule(…) {
>>>
>>> auto Resolver = createLambdaResolver(
>>> [&](const std::string &name) {
>>>
>>> // look up first in JIT'ed code
>>> if (auto sym = findMangledSymbol(name)) {
>>> return RuntimeDyld::SymbolInfo(sym.getAddress(),
>>> sym.getFlags());
>>> return RuntimeDyld::SymbolInfo(nullptr);
>>> }
>>>
>>> // look up in added globals
>>> if (auto addr = getPointerToGlobalMapping(name)) {
>>> return RuntimeDyld::SymbolInfo(addr, JITSymbolFlags::Exported);
>>> }
>>>
>>> // finally try to look up existing process symbols, note
>>> // this works for symbols loaded in shared libraries, but
>>> // does NOT seem to find symbols declared in the executable.
>>> if (auto Addr =
>>> RTDyldMemoryManager::getSymbolAddressInProcess(name)) {
>>> return RuntimeDyld::SymbolInfo(Addr, JITSymbolFlags::Exported);
>>> }
>>> },
>>> [](const std::string &S) { return nullptr; }
>>> );
>>> }
>>>
>>> Here the getPointerToGlobalMapping function looks in a uint64 StringMap
>>> into which values are added via the addGlobalMapping functions.
>>>
>>>
>>> This approach seems to be working, but my question is do you suppose
>>> there any are issues with such an approach?
>>>
>>> The troubling thing is why doesn’t  RTDyldMemoryManager::
>>> getSymbolAddressInProcess(name)) return an address for a symbol that is
>>> defined in either a static library, or in the executable itself.
>>>
>>> If this approach is correct, in adding the global values to the context,
>>> and looking them up the lambda resolver, in addition to looking up external
>>> symbols, and considering that the ORC kaleidoscope examples do in fact
>>> allow external function calls (which are broken currently), should they be
>>> fixed with this approach?
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org         http://llvm.cs.uiuc.edu
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150819/6b9c4b9f/attachment-0001.html>