[llvm-dev] LLJIT vs. thread-local storage

[Lang Hames via llvm-dev]

Hi Geoff,

Ok — I think I’m caught up now. You’re using LLVM 8, and thread locals are working for you on ORCv1, but not ORCv2.

> Is there any chance that ORC v2 will support thread-local variables in the foreseeable future?

I think so. I was able to reproduce an issue that looks related to yours on my Darwin machine:

In <main> resolving { ("___emutls_v.x": 0x000000012a406000 [Data]), ("___emutls_t.x": 0x000000012a406020 [Data]), ("_main": 0x000000012a405000 [Callable]) }
Assertion failed: (I != SymbolFlags.end() && "Resolving symbol outside this responsibility set"), function notifyResolved, file llvm/lib/ExecutionEngine/Orc/Core.cpp, line 448.

If I look at what was supposed to be resolved (the “responsibility set”) I see that the JIT expected a definition for _x, but instead it got an unexpected definition for __emutls_t.x. You’re seeing the reverse: the JIT linker is trying to resolve __emutls_t.x (presumably you’ve compiled some code with an extern therad_local reference), but the JIT only knows about _x.

The reason for the change in behavior has to do with concurrency support, which was introduced in ORCv2. To make concurrency safe you need to declare the symbols that any program representation will produce up front (So we can know which program representation to compile, kick off its compilation, and synchronize around it). Usually, knowing what symbols a program representation will produce is easy: You just mangle all the defined symbol names and you’re done. In this case though emulated TLS will rename some symbols during compilation. I think the solution is to teach the JIT to mangle IR thread-local names (with an _emutls_t. prefix) when emulated TLS is enabled. I will take a shot at this tomorrow.

— Lang.

Sent from my iPad

>> On Jan 7, 2020, at 2:29 AM, Geoff Levner <glevner at gmail.com> wrote:
> 
> So, picking up this thread again after the holidays (burp)... Is there any chance that ORC v2 will support thread-local variables in the foreseeable future? Is there some way of configuring it today to ensure they are handled correctly? Or are we better off sticking with ORC v1?
> 
>> On Fri, Dec 20, 2019 at 9:59 PM Jameson Nash via llvm-dev <llvm-dev at lists.llvm.org> wrote:
>> I don't think it's especially hard, but just not specifically unimplemented because nobody's had a strong need for it. There's probably some combinations of code models and machines that does happen to work (e.g. emutls+linux+large-code+large-data+no-PIC). Julia has some support for thread locals, but as a JIT in control of the language we currently try to generate better code than would otherwise happen (often bits of inline assembly that's more similar to the initial-exec model than a dynamic library would normally be able to use)—but also with more limitations on how it may be used and requiring some additional support from the runtime.
>> 
>>> On Fri, Dec 20, 2019 at 1:17 PM Praveen Velliengiri <praveenvelliengiri at gmail.com> wrote:
>>> Hi Jameson,
>>> Why thread local support is hard to support in JITs? Whether Julia supports Thread locals? If so, it would be very much helpful to know how.
>>> 
>>> Thanks
>>> 
>>>> On Fri, 20 Dec 2019 at 23:31, Jameson Nash via llvm-dev <llvm-dev at lists.llvm.org> wrote:
>>>> This had also came up at llvm-devmtg briefly at the JIT roundtable. One of the collaborators on my project had started a patch years ago to implement some of it https://reviews.llvm.org/D8815, but then we went a different direction with TLS in our frontend and it became unnecessary.
>>>> 
>>>>> On Fri, Dec 20, 2019 at 12:36 PM David Blaikie via llvm-dev <llvm-dev at lists.llvm.org> wrote:
>>>>> +Lang for visibility
>>>>> 
>>>>>> On Fri, Dec 20, 2019 at 5:09 AM Geoff Levner via llvm-dev <llvm-dev at lists.llvm.org> wrote:
>>>>>> I am in the process of porting our ORC code to ORC v2 and LLJIT. Now that I have worked around a problem getting global constructors to be called, everything seems to work unless a module declares a static thread-local variable. In that case I get a "JIT session error" saying that the symbol __emutls_v.xyz was not found (substitute the mangled variable name for "xyz").
>>>>>> 
>>>>>> Does that mean anything to anybody out there?
>>>>>> 
>>>>>> (I don't know if it's relevant, but we are using LLVM 8, and we are using Clang to compile C++ modules that are all put into a single JITDylib.)
>>>>>> 
>>>>>> Geoff
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200116/8cfa2664/attachment.html>