[llvm-dev] New ORC v2 LLJIT - global variables

[chris boese via llvm-dev]

Hi Lang,

Thanks for the info. I think we have issues with all of the JIT "prep" work
we are currently doing around changing linkage for symbols before jitting.
I believe this was done to work with the OrcMCJitReplacement, but it
doesn't seem compatible with the new LLJIT. I will look into it further. On
second point, we will just turn it off for now.

Thanks,
Chris

On Mon, Aug 19, 2019 at 1:35 PM Lang Hames <lhames at gmail.com> wrote:

> Hi Chris,
>
> 1. Using the addIRModule method for adding new global variables, I'm
>> having to set them to external linkage, so that they can be found during
>> lookup. One issue with this, is I have to manually loop through all the
>> underlying globals/functions and change their linkage to external as well.
>> If I don't do that, I'll get a JIT Session Error about not being able to
>> find the underlying globals/functions. Is there an easy way to set all of
>> the underlying globals/functions to external linkage as well?
>
>
> There is no "change all linkages" convenience function, you just have to
> loop over all the global values and set them individually. If you're doing
> that though there's likely a design issue somewhere else: In general
> languages will define which symbols get external linkage and which do not,
> and the front-end should set up the linkages for you: you shouldn't need to
> modify linkages in order to add something to the JIT. E.g. If you have the
> following in C:
>
> static void foo(void) { }
> void bar(void) { foo(); }
>
> then clang will translate that to:
>
> define internal void @foo() { ... }
> define void @bar() { ... }
>
> This IR, with foo having internal linkage and bar external, can be added
> to a JITDylib as-is and will result in one visible definition: bar. (ORC
> can handle internal symbols just fine, they just don't appear in the public
> interface for the JITDylib because that's what internal linkage means:
> visible only from within the module).
>
> The two main exceptions to the "you shouldn't need to promote linkage
> yourself" rule are:
>
> (1) Debuggers / REPLs, where you want to give a user the ability to call
> directly into what would usually be an internal function. In that case you
> should only need to promote the linkage on the internal symbols that you
> want the user to be able to access. If that's everything, then promote
> everything. If that's only certain functions/variables then just promote
> those functions/variables.
>
> (2) Manually breaking up modules: If you want to perform more fine grained
> or parallel compilation you may choose to break up large modules. In that
> case you need to promote internal symbols to external-hidden symbols so
> that they can still be resolved across the new module boundaries. You will
> usually have to rename them to avoid clashes too. If you're using LLLazyJIT
> though you don't need to worry: all the symbol promoting and renaming
> required for lazy compilation is taken care of internally.
>
> So: Why is your front-end producing internal definitions for symbols that
> you want to be able to call in the JIT? Should your front-end be producing
> external definitions?
>
> 2. We've had this problem ever since we switched to OrcMCJitReplacement.
>> When jitting code on linux, we have to comment the body of
>> registerEHFrames(). If we don't, then any exceptions that are thrown in
>> jitted code will result in a segfault. Do you know what issue we might be
>> running into here? RuntimeDyIdELF::registerEHFrames() is commented out for
>> new LLJIT. OrcMCJitReplacement::registerEHFrames when using
>> OrcMCJitReplacement.
>
>
> registerEHFrames is supposed to register the __eh_frames section with
> libunwind. I know that RuntimeDyldMachO's __eh_frame support was weak, and
> would often result in registering broken frames. I don't know
> RuntimeDyldELF's __eh_frame support code well, but from a glance I suspect
> it suffers the same problems. That is almost certainly why your code is
> segfaulting when exceptions are thrown: the JIT is registering broken
> frames.
>
> The poor quality of RuntimeDyld's exception handling is why I turned it
> off by default. The new JIT linker (JTILink) has much better eh_frame
> support, but unfortunately there is no ELF JITLink implementation yet.
>
> The upshot of all this is that at the moment it's not safe in general to
> throw exceptions from/through JIT'd code on ELF. I'm hoping that JITLink
> ELF variants will solve this eventually, but there's no clear timeline for
> that work.
>
> -- Lang.
>
> On Mon, Aug 19, 2019 at 11:12 AM chris boese <chris107565 at gmail.com>
> wrote:
>
>> Hi Lang,
>>
>> Had a couple more questions I thought I'd run by you.
>>
>> 1. Using the addIRModule method for adding new global variables, I'm
>> having to set them to external linkage, so that they can be found during
>> lookup. One issue with this, is I have to manually loop through all the
>> underlying globals/functions and change their linkage to external as well.
>> If I don't do that, I'll get a JIT Session Error about not being able to
>> find the underlying globals/functions. Is there an easy way to set all of
>> the underlying globals/functions to external linkage as well?
>>
>> 2. We've had this problem ever since we switched to OrcMCJitReplacement.
>> When jitting code on linux, we have to comment the body of
>> registerEHFrames(). If we don't, then any exceptions that are thrown in
>> jitted code will result in a segfault. Do you know what issue we might be
>> running into here? RuntimeDyIdELF::registerEHFrames() is commented out for
>> new LLJIT. OrcMCJitReplacement::registerEHFrames when using
>> OrcMCJitReplacement.
>>
>> Thanks,
>> Chris
>>
>> On Thu, Aug 15, 2019 at 9:22 AM chris boese <chris107565 at gmail.com>
>> wrote:
>>
>>> Hi Lang,
>>>
>>> So that was my issue. Putting modules on the same ThreadSafeContext was
>>> causing a deadlock. Our current version of LLVM does not have r367686
>>> patch. I resolved by creating a separate ThreadSafeContext, and now I'm
>>> able to add/find globals by using the addLazyIRModule function.
>>>
>>> Thank you so much for your help. For now I believe I'm unblocked. I will
>>> reach out again if encountering any more issues.
>>>
>>> Thanks,
>>> Chris
>>>
>>> On Tue, Aug 13, 2019 at 2:49 PM Lang Hames <lhames at gmail.com> wrote:
>>>
>>>> Hi Chris,
>>>>
>>>> Thanks for clarifying that. That makes sense.
>>>>>
>>>>
>>>>
>>>> So I apologize. It was not an error I got on the AddLazyIRModule. The
>>>>> issue is actually after I call AddLazyIRModule(). Performing a lookup on
>>>>> that global variable causes a hang. I've stepped through the debugger, and
>>>>> it seems to hang inside Core.cpp, ExecutionSession::lookup().
>>>>
>>>>
>>>> Are you (a) trying to JIT concurrently and (b) loading all modules on
>>>> the same ThreadSafeContext? Notionally you should be able to do both, but
>>>> in my experience bugs in this area are the most frequent source of
>>>> deadlocks in lookup.
>>>>
>>>> Are you able to get a backtrace for each thread in the debugger while
>>>> the lookup is deadlocked? If you're using LLDB you can use "thread list" to
>>>> list the threads, "thread select N" to switch to each, and "bt" to get the
>>>> backtrace. I am not sure what the equivalent commands in GDB are. Either
>>>> way, the lookup thread is the least interesting one: I expect to see some
>>>> other thread waiting on a context lock, and the thing we *really* want to
>>>> know is who owns the lock and how did they end up blocking while holding it.
>>>>
>>>> Are you able to share any of your code / test cases?
>>>>
>>>> Cheers,
>>>> Lang.
>>>>
>>>> Prior to r367686, ORCv2 took out a lock on the context when a module
>>>> was emitted and did not relinquish it until that module had been fully
>>>> emitted. The problem with that is that if the module being emitted has
>>>> dependencies on other modules living on the same
>>>>
>>>> On Tue, Aug 13, 2019 at 2:09 PM chris boese <chris107565 at gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Lang,
>>>>>
>>>>> Thanks for clarifying that. That makes sense.
>>>>>
>>>>> So I apologize. It was not an error I got on the AddLazyIRModule. The
>>>>> issue is actually after I call AddLazyIRModule(). Performing a lookup on
>>>>> that global variable causes a hang. I've stepped through the debugger, and
>>>>> it seems to hang inside Core.cpp, ExecutionSession::lookup().
>>>>>
>>>>> Thanks,
>>>>> Chris
>>>>>
>>>>> On Mon, Aug 12, 2019 at 2:33 PM Lang Hames <lhames at gmail.com> wrote:
>>>>>
>>>>>> Hi Chris,
>>>>>>
>>>>>>
>>>>>>> Thanks so much for the quick help. I was able to find them after
>>>>>>> changing the linkage. Does this new orc JIT follow different standards than
>>>>>>> the MCJitReplacement? We used to set a lot of our functions/globals to
>>>>>>> internal linkage before doing a lookup.
>>>>>>
>>>>>>
>>>>>> They are different: ORCv2 follows the static linker's lookup rules,
>>>>>> whereas MCJITReplacement mimicked MCJIT's rules which were never clearly
>>>>>> spelled out, but in practice allowed you to search for internal symbols.
>>>>>>
>>>>>> The advantage of the new scheme is that if you are compiling/running
>>>>>> something on the command line. E.g.:
>>>>>>
>>>>>> $ llc -filetype=obj -o extra.o extra.ll
>>>>>> $ llc -filetype=obj -o prog.o prog.ll
>>>>>> $ clang -o prog prog.o extra.o
>>>>>> $ ./prog
>>>>>>
>>>>>> Then you can add the same modules to the JIT and expect the resulting
>>>>>> JIT'd code to behave the same as the statically compiled code. E.g.
>>>>>> (omitting some error handling for clarity):
>>>>>>
>>>>>> J->addIRModule(loadModule("extra.ll"));
>>>>>> J->addIRModule(loadModule("main.ll"));
>>>>>> auto Main = (int(*)())J->lookup("main").getAddress();
>>>>>> Main();
>>>>>>
>>>>>> If you are generating code specifically for the JIT it is often
>>>>>> easiest to just use external linkage and default visibility for all
>>>>>> symbols. However, for people who want to hook up an existing front-end, or
>>>>>> to use more advanced features in their JIT (duplicate symbol names with
>>>>>> internal linkage, symbols with weak linkage, hidden visibility, etc.) this
>>>>>> scheme has the nice property that "What does linker-feature X do in the
>>>>>> JIT?" is more-or-less equivalent to "What does linker-feature X do in a
>>>>>> regular build". The answer to the latter question is generally more widely
>>>>>> known, and is much easier to test from the command line.
>>>>>>
>>>>>> When adding a new module with a single global variable in it, I get
>>>>>>> an error during the addLazyIRModule.
>>>>>>
>>>>>>
>>>>>> Are you hitting an assertion, or is addLazyIRModule returning a
>>>>>> non-success value? If the latter, what is the error message?
>>>>>> (You can find this by calling 'logAllUnhandledErrors(std::move(Err),
>>>>>> errs(), "")', or, in tool code, using the ExitOnError utility:
>>>>>> http://llvm.org/docs/ProgrammersManual.html#using-exitonerror-to-simplify-tool-code
>>>>>> ).
>>>>>>
>>>>>> Right now we are treating the module as one big single module, and
>>>>>>> when we go to lookup a symbol, I believe we are jitting the entire thing.
>>>>>>> Once we make the switch to LLLazyJIT, we want to start splitting up our
>>>>>>> modules.
>>>>>>
>>>>>>
>>>>>> If you are using MCJITReplacement you are definitely JITing the whole
>>>>>> module up-front, and in that case lazy compilation may help reduce your
>>>>>> compile time. I'd love to hear how it works for you!
>>>>>>
>>>>>> Cheers,
>>>>>> Lang.
>>>>>>
>>>>>> On Mon, Aug 12, 2019 at 10:26 AM David Blaikie <dblaikie at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> +Lang Hames <lhames at gmail.com>, Admiral of the Orcish Fleet (Lang's
>>>>>>> the primary developer/creator of the ORC JIT & might have some answers to
>>>>>>> your question)
>>>>>>>
>>>>>>> On Thu, Aug 8, 2019 at 4:15 PM chris boese via llvm-dev <
>>>>>>> llvm-dev at lists.llvm.org> wrote:
>>>>>>>
>>>>>>>> We are trying to switch to the new orc v2 lljit engine at my work
>>>>>>>> with hopes of parallel jitting. We are switching from the ExecutionEngine
>>>>>>>> w/ OrcMCJitReplacement. We are having a hard time with global variables. We
>>>>>>>> have found a way to create/emit new globals during jitting by using the old
>>>>>>>> ExecutionEngine::getOrEmitGlobalVariable. Is there an easier way to do this
>>>>>>>> with the new jit engine? We were hoping to create a new ThreadSafeModule,
>>>>>>>> add a GlobalVariable, and then add the module to the JIT engine through
>>>>>>>> addLazyIRModule(), but this doesn't seem to work.
>>>>>>>>
>>>>>>>> 2nd question; Is the lookup function supposed to work with global
>>>>>>>> variables by name? Looking through the CompileOnDemandLayer it seems to
>>>>>>>> change the names of globals to "__orc_lcl.<name><id>". Does this affect the
>>>>>>>> lookup methods? I noticed that all of the globals we are looking for are
>>>>>>>> tagged with Internal linkage and end up getting emitted through
>>>>>>>> extractSubModule().
>>>>>>>>
>>>>>>>> Any help is appreciated.
>>>>>>>> _______________________________________________
>>>>>>>> LLVM Developers mailing list
>>>>>>>> llvm-dev at lists.llvm.org
>>>>>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>>>
>>>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190821/ebc8d660/attachment.html>