[llvm-dev] How to use thread local storage with ORC JIT?

Tue Mar 2 23:24:30 PST 2021

Hey Lang,

I can’t remember anymore but I think I haven’t tried with -femulated-tls.
I also forwarded the question to the Cling people because I think they got it running – but also haven’t heard back from there. I decided against emulated-tls and tried resolving the Windows symbols myself by adding another IR file to it which was compiled as the following:
extern "C"
{
    void Sleep(unsigned long dwMilliseconds);

    static constexpr int EpochStart = std::numeric_limits<int>::min();

    unsigned int _tls_index         = 0;
    int          _Init_global_epoch = EpochStart;
    int          _Init_thread_epoch = EpochStart;

    void _Init_thread_header(volatile int* ptss)
    {
        while(true)
        {
            /* Try to acquire the first initialization lock */
            int oldTss = _InterlockedCompareExchange(reinterpret_cast<volatile long*>(ptss), -1, 0);
            if(oldTss == -1)
            {
                /* Busy, wait for the other thread to do the initialization */
                Sleep(0);
                continue;
            }

            /* Either we acquired the lock and the caller will do the initializaion
            or the initialization is complete and the caller will skip it */
            break;
        }
    }

    void _Init_thread_footer(int *ptss)
    {
        *ptss = _InterlockedIncrement(reinterpret_cast<long*>(&_Init_global_epoch));
    }

    void _Init_thread_abort(volatile int* ptss)
    {
        /* Abort the initialization */
        _InterlockedAnd(reinterpret_cast<volatile long*>(ptss), 0);
    }
}

This was good enough for me until I hear a better way to do this – cause I got a feel that this is not really working? However my static variables were only initialized once no matter how many threads I threw at.
I haven’t mentioned that solution yet cause I’m not much confident about it’s reliability. Maybe I should show this to the cling people too…

Kind greetings
Björn

From: Lang Hames <lhames at gmail.com>
Sent: 02 March 2021 23:24
To: David Blaikie <dblaikie at gmail.com>
Cc: Gaier, Bjoern <Bjoern.Gaier at horiba.com>; LLVM Developers Mailing List <llvm-dev at lists.llvm.org>
Subject: Re: [llvm-dev] How to use thread local storage with ORC JIT?

Hi All,

Björn -- I know we discussed this on discord already, but something just occurred to me: Are you compiling your target executable (whatever you're JITing in to, probably the same process as your JIT) with -femulated-tls? If so, have you made sure to include at least one thread local variable that is either used or marked as used (to ensure it's not dead stripped)? JIT'd code will be looking for those symbols in your executable so you'll need to make sure they're linked in there.

I am no expert, so take anything I say with a pinch of salt, but my understanding is that Orc JIT does not support native thread-local storage, it just emulates it.

That is true for now, but I hope to have native thread locals supported (on MachO at least) by LLVM 12. The ORC runtime prototype [1] already contains support for POD thread locals, and I will be aiming to add support for thread locals with nontrivial constructors in the near future.

-- Lang.

On Tue, Feb 23, 2021 at 7:11 AM David Blaikie <dblaikie at gmail.com<mailto:dblaikie at gmail.com>> wrote:
+Lang Hames<mailto:lhames at gmail.com> (though I think he's not got much bandwidth for a few months - so responses may be delayed)

On Mon, Feb 15, 2021 at 12:07 AM Gaier, Bjoern via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
Hello everyone,

I'm using the ORC JIT to load and execute some IR files which I compiled with Clang-Cl from some source files - the host is Windows 64bit. Apparently, clang decided to generate code using the thread local storage for some of the code. When now loading and jitting those files I get the following undefined references:
__emutls_get_address
__emutls_v._Init_thread_epoch

A while ago I was told, that those references are coming from compiler-rt - so I build that project and at least I found "__emutls_get_address" in the "clang_rt.builtins-x86_64.lib" file. However, I haven't found "__emutls_v._Init_thread_epoch" anyway and don't know what to do with this symbol.

Any ideas?

Also, in case you see double - I asked this question in the LLVM Discord already with no big success...

Kind greetings
Björn
Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Takashi Nagano, Junichi Tajika, Ergin Cansiz.
_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Takashi Nagano, Junichi Tajika, Ergin Cansiz.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210303/3f0c14b2/attachment.html>