[llvm-dev] New ORC v2 LLJIT - global variables

[Lang Hames via llvm-dev]

Hi Chris,

Thanks for the info. I think we have issues with all of the JIT "prep" work
> we are currently doing around changing linkage for symbols before jitting.
> I believe this was done to work with the OrcMCJitReplacement, but it
> doesn't seem compatible with the new LLJIT. I will look into it further. On
> second point, we will just turn it off for now.


Sounds good to me. Hopefully JITLink will evolve ELF support in the
not-too-distant future and we will be able to re-enable exception handling.

Cheers,
Lang.

On Wed, Aug 21, 2019 at 3:58 PM chris boese <chris107565 at gmail.com> wrote:

> Hi Lang,
>
> Thanks for the info. I think we have issues with all of the JIT "prep"
> work we are currently doing around changing linkage for symbols before
> jitting. I believe this was done to work with the OrcMCJitReplacement, but
> it doesn't seem compatible with the new LLJIT. I will look into it further.
> On second point, we will just turn it off for now.
>
> Thanks,
> Chris
>
> On Mon, Aug 19, 2019 at 1:35 PM Lang Hames <lhames at gmail.com> wrote:
>
>> Hi Chris,
>>
>> 1. Using the addIRModule method for adding new global variables, I'm
>>> having to set them to external linkage, so that they can be found during
>>> lookup. One issue with this, is I have to manually loop through all the
>>> underlying globals/functions and change their linkage to external as well.
>>> If I don't do that, I'll get a JIT Session Error about not being able to
>>> find the underlying globals/functions. Is there an easy way to set all of
>>> the underlying globals/functions to external linkage as well?
>>
>>
>> There is no "change all linkages" convenience function, you just have to
>> loop over all the global values and set them individually. If you're doing
>> that though there's likely a design issue somewhere else: In general
>> languages will define which symbols get external linkage and which do not,
>> and the front-end should set up the linkages for you: you shouldn't need to
>> modify linkages in order to add something to the JIT. E.g. If you have the
>> following in C:
>>
>> static void foo(void) { }
>> void bar(void) { foo(); }
>>
>> then clang will translate that to:
>>
>> define internal void @foo() { ... }
>> define void @bar() { ... }
>>
>> This IR, with foo having internal linkage and bar external, can be added
>> to a JITDylib as-is and will result in one visible definition: bar. (ORC
>> can handle internal symbols just fine, they just don't appear in the public
>> interface for the JITDylib because that's what internal linkage means:
>> visible only from within the module).
>>
>> The two main exceptions to the "you shouldn't need to promote linkage
>> yourself" rule are:
>>
>> (1) Debuggers / REPLs, where you want to give a user the ability to call
>> directly into what would usually be an internal function. In that case you
>> should only need to promote the linkage on the internal symbols that you
>> want the user to be able to access. If that's everything, then promote
>> everything. If that's only certain functions/variables then just promote
>> those functions/variables.
>>
>> (2) Manually breaking up modules: If you want to perform more fine
>> grained or parallel compilation you may choose to break up large modules.
>> In that case you need to promote internal symbols to external-hidden
>> symbols so that they can still be resolved across the new module
>> boundaries. You will usually have to rename them to avoid clashes too. If
>> you're using LLLazyJIT though you don't need to worry: all the symbol
>> promoting and renaming required for lazy compilation is taken care of
>> internally.
>>
>> So: Why is your front-end producing internal definitions for symbols that
>> you want to be able to call in the JIT? Should your front-end be producing
>> external definitions?
>>
>> 2. We've had this problem ever since we switched to OrcMCJitReplacement.
>>> When jitting code on linux, we have to comment the body of
>>> registerEHFrames(). If we don't, then any exceptions that are thrown in
>>> jitted code will result in a segfault. Do you know what issue we might be
>>> running into here? RuntimeDyIdELF::registerEHFrames() is commented out for
>>> new LLJIT. OrcMCJitReplacement::registerEHFrames when using
>>> OrcMCJitReplacement.
>>
>>
>> registerEHFrames is supposed to register the __eh_frames section with
>> libunwind. I know that RuntimeDyldMachO's __eh_frame support was weak, and
>> would often result in registering broken frames. I don't know
>> RuntimeDyldELF's __eh_frame support code well, but from a glance I suspect
>> it suffers the same problems. That is almost certainly why your code is
>> segfaulting when exceptions are thrown: the JIT is registering broken
>> frames.
>>
>> The poor quality of RuntimeDyld's exception handling is why I turned it
>> off by default. The new JIT linker (JTILink) has much better eh_frame
>> support, but unfortunately there is no ELF JITLink implementation yet.
>>
>> The upshot of all this is that at the moment it's not safe in general to
>> throw exceptions from/through JIT'd code on ELF. I'm hoping that JITLink
>> ELF variants will solve this eventually, but there's no clear timeline for
>> that work.
>>
>> -- Lang.
>>
>> On Mon, Aug 19, 2019 at 11:12 AM chris boese <chris107565 at gmail.com>
>> wrote:
>>
>>> Hi Lang,
>>>
>>> Had a couple more questions I thought I'd run by you.
>>>
>>> 1. Using the addIRModule method for adding new global variables, I'm
>>> having to set them to external linkage, so that they can be found during
>>> lookup. One issue with this, is I have to manually loop through all the
>>> underlying globals/functions and change their linkage to external as well.
>>> If I don't do that, I'll get a JIT Session Error about not being able to
>>> find the underlying globals/functions. Is there an easy way to set all of
>>> the underlying globals/functions to external linkage as well?
>>>
>>> 2. We've had this problem ever since we switched to OrcMCJitReplacement.
>>> When jitting code on linux, we have to comment the body of
>>> registerEHFrames(). If we don't, then any exceptions that are thrown in
>>> jitted code will result in a segfault. Do you know what issue we might be
>>> running into here? RuntimeDyIdELF::registerEHFrames() is commented out for
>>> new LLJIT. OrcMCJitReplacement::registerEHFrames when using
>>> OrcMCJitReplacement.
>>>
>>> Thanks,
>>> Chris
>>>
>>> On Thu, Aug 15, 2019 at 9:22 AM chris boese <chris107565 at gmail.com>
>>> wrote:
>>>
>>>> Hi Lang,
>>>>
>>>> So that was my issue. Putting modules on the same ThreadSafeContext was
>>>> causing a deadlock. Our current version of LLVM does not have r367686
>>>> patch. I resolved by creating a separate ThreadSafeContext, and now I'm
>>>> able to add/find globals by using the addLazyIRModule function.
>>>>
>>>> Thank you so much for your help. For now I believe I'm unblocked. I
>>>> will reach out again if encountering any more issues.
>>>>
>>>> Thanks,
>>>> Chris
>>>>
>>>> On Tue, Aug 13, 2019 at 2:49 PM Lang Hames <lhames at gmail.com> wrote:
>>>>
>>>>> Hi Chris,
>>>>>
>>>>> Thanks for clarifying that. That makes sense.
>>>>>>
>>>>>
>>>>>
>>>>> So I apologize. It was not an error I got on the AddLazyIRModule. The
>>>>>> issue is actually after I call AddLazyIRModule(). Performing a lookup on
>>>>>> that global variable causes a hang. I've stepped through the debugger, and
>>>>>> it seems to hang inside Core.cpp, ExecutionSession::lookup().
>>>>>
>>>>>
>>>>> Are you (a) trying to JIT concurrently and (b) loading all modules on
>>>>> the same ThreadSafeContext? Notionally you should be able to do both, but
>>>>> in my experience bugs in this area are the most frequent source of
>>>>> deadlocks in lookup.
>>>>>
>>>>> Are you able to get a backtrace for each thread in the debugger while
>>>>> the lookup is deadlocked? If you're using LLDB you can use "thread list" to
>>>>> list the threads, "thread select N" to switch to each, and "bt" to get the
>>>>> backtrace. I am not sure what the equivalent commands in GDB are. Either
>>>>> way, the lookup thread is the least interesting one: I expect to see some
>>>>> other thread waiting on a context lock, and the thing we *really* want to
>>>>> know is who owns the lock and how did they end up blocking while holding it.
>>>>>
>>>>> Are you able to share any of your code / test cases?
>>>>>
>>>>> Cheers,
>>>>> Lang.
>>>>>
>>>>> Prior to r367686, ORCv2 took out a lock on the context when a module
>>>>> was emitted and did not relinquish it until that module had been fully
>>>>> emitted. The problem with that is that if the module being emitted has
>>>>> dependencies on other modules living on the same
>>>>>
>>>>> On Tue, Aug 13, 2019 at 2:09 PM chris boese <chris107565 at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Lang,
>>>>>>
>>>>>> Thanks for clarifying that. That makes sense.
>>>>>>
>>>>>> So I apologize. It was not an error I got on the AddLazyIRModule. The
>>>>>> issue is actually after I call AddLazyIRModule(). Performing a lookup on
>>>>>> that global variable causes a hang. I've stepped through the debugger, and
>>>>>> it seems to hang inside Core.cpp, ExecutionSession::lookup().
>>>>>>
>>>>>> Thanks,
>>>>>> Chris
>>>>>>
>>>>>> On Mon, Aug 12, 2019 at 2:33 PM Lang Hames <lhames at gmail.com> wrote:
>>>>>>
>>>>>>> Hi Chris,
>>>>>>>
>>>>>>>
>>>>>>>> Thanks so much for the quick help. I was able to find them after
>>>>>>>> changing the linkage. Does this new orc JIT follow different standards than
>>>>>>>> the MCJitReplacement? We used to set a lot of our functions/globals to
>>>>>>>> internal linkage before doing a lookup.
>>>>>>>
>>>>>>>
>>>>>>> They are different: ORCv2 follows the static linker's lookup rules,
>>>>>>> whereas MCJITReplacement mimicked MCJIT's rules which were never clearly
>>>>>>> spelled out, but in practice allowed you to search for internal symbols.
>>>>>>>
>>>>>>> The advantage of the new scheme is that if you are compiling/running
>>>>>>> something on the command line. E.g.:
>>>>>>>
>>>>>>> $ llc -filetype=obj -o extra.o extra.ll
>>>>>>> $ llc -filetype=obj -o prog.o prog.ll
>>>>>>> $ clang -o prog prog.o extra.o
>>>>>>> $ ./prog
>>>>>>>
>>>>>>> Then you can add the same modules to the JIT and expect the
>>>>>>> resulting JIT'd code to behave the same as the statically compiled code.
>>>>>>> E.g. (omitting some error handling for clarity):
>>>>>>>
>>>>>>> J->addIRModule(loadModule("extra.ll"));
>>>>>>> J->addIRModule(loadModule("main.ll"));
>>>>>>> auto Main = (int(*)())J->lookup("main").getAddress();
>>>>>>> Main();
>>>>>>>
>>>>>>> If you are generating code specifically for the JIT it is often
>>>>>>> easiest to just use external linkage and default visibility for all
>>>>>>> symbols. However, for people who want to hook up an existing front-end, or
>>>>>>> to use more advanced features in their JIT (duplicate symbol names with
>>>>>>> internal linkage, symbols with weak linkage, hidden visibility, etc.) this
>>>>>>> scheme has the nice property that "What does linker-feature X do in the
>>>>>>> JIT?" is more-or-less equivalent to "What does linker-feature X do in a
>>>>>>> regular build". The answer to the latter question is generally more widely
>>>>>>> known, and is much easier to test from the command line.
>>>>>>>
>>>>>>> When adding a new module with a single global variable in it, I get
>>>>>>>> an error during the addLazyIRModule.
>>>>>>>
>>>>>>>
>>>>>>> Are you hitting an assertion, or is addLazyIRModule returning a
>>>>>>> non-success value? If the latter, what is the error message?
>>>>>>> (You can find this by calling 'logAllUnhandledErrors(std::move(Err),
>>>>>>> errs(), "")', or, in tool code, using the ExitOnError utility:
>>>>>>> http://llvm.org/docs/ProgrammersManual.html#using-exitonerror-to-simplify-tool-code
>>>>>>> ).
>>>>>>>
>>>>>>> Right now we are treating the module as one big single module, and
>>>>>>>> when we go to lookup a symbol, I believe we are jitting the entire thing.
>>>>>>>> Once we make the switch to LLLazyJIT, we want to start splitting up our
>>>>>>>> modules.
>>>>>>>
>>>>>>>
>>>>>>> If you are using MCJITReplacement you are definitely JITing the
>>>>>>> whole module up-front, and in that case lazy compilation may help reduce
>>>>>>> your compile time. I'd love to hear how it works for you!
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Lang.
>>>>>>>
>>>>>>> On Mon, Aug 12, 2019 at 10:26 AM David Blaikie <dblaikie at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> +Lang Hames <lhames at gmail.com>, Admiral of the Orcish Fleet
>>>>>>>> (Lang's the primary developer/creator of the ORC JIT & might have some
>>>>>>>> answers to your question)
>>>>>>>>
>>>>>>>> On Thu, Aug 8, 2019 at 4:15 PM chris boese via llvm-dev <
>>>>>>>> llvm-dev at lists.llvm.org> wrote:
>>>>>>>>
>>>>>>>>> We are trying to switch to the new orc v2 lljit engine at my work
>>>>>>>>> with hopes of parallel jitting. We are switching from the ExecutionEngine
>>>>>>>>> w/ OrcMCJitReplacement. We are having a hard time with global variables. We
>>>>>>>>> have found a way to create/emit new globals during jitting by using the old
>>>>>>>>> ExecutionEngine::getOrEmitGlobalVariable. Is there an easier way to do this
>>>>>>>>> with the new jit engine? We were hoping to create a new ThreadSafeModule,
>>>>>>>>> add a GlobalVariable, and then add the module to the JIT engine through
>>>>>>>>> addLazyIRModule(), but this doesn't seem to work.
>>>>>>>>>
>>>>>>>>> 2nd question; Is the lookup function supposed to work with global
>>>>>>>>> variables by name? Looking through the CompileOnDemandLayer it seems to
>>>>>>>>> change the names of globals to "__orc_lcl.<name><id>". Does this affect the
>>>>>>>>> lookup methods? I noticed that all of the globals we are looking for are
>>>>>>>>> tagged with Internal linkage and end up getting emitted through
>>>>>>>>> extractSubModule().
>>>>>>>>>
>>>>>>>>> Any help is appreciated.
>>>>>>>>> _______________________________________________
>>>>>>>>> LLVM Developers mailing list
>>>>>>>>> llvm-dev at lists.llvm.org
>>>>>>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>>>>
>>>>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190822/41f78b7b/attachment.html>