[PATCH] D109293: [JITLink][WIP] Add initial native TLS support to ELFNix platform

Fri Sep 10 17:55:06 PDT 2021

lhames added a comment.

In D109293#2985882 <https://reviews.llvm.org/D109293#2985882>, @StephenFan wrote:

> In D109293#2984338 <https://reviews.llvm.org/D109293#2984338>, @lhames wrote:
>
>> Side note: Re-using the GOT in MachO is safe, but was a bit lazy. We could probably come up with a generic TableSection<T> utility that we could re-use for GOTs, PLTs, and these new thread data structures.
>
> Emm, I don't know how to come up with a generic TableSection<T> utility. What does the generic type `T' represents?

@StephenFan -- got-pointer, tlv-pointer, plt-stub, tlv-entry-pair, ....
This would generalize the behavior in `PerGraphGOTAndPLTStubsBuilder`, which is a nice cleanup. I don't think it's important for this discussion though.

In D109293#2985444 <https://reviews.llvm.org/D109293#2985444>, @MoritzS wrote:

> In D109293#2985435 <https://reviews.llvm.org/D109293#2985435>, @MoritzS wrote:
>
>> 1. Don't allow GOTTPOFF/TPOFF relocations at all. This means that we will not be able to link many pre-compiled static libraries.
>> 2. Allow all TLS relocations. Since we can't add new TLS storage at runtime, define a global thread-local variable which will then be used by the linker to resolve GOTTPOFF/TPOFF relocations.
>> 3. Rewrite the relocated code so that it does not use the fs register. On Linux we could probably replace all references to fs by gs and then manually set the gs register? I'm not sure if this can break code if we change instructions that did not originally have a relocation.
>>
>> I implemented 2. in llvm-rtdyld. Since this is mainly used for testing, the size of the pre-allocated TLS section is only 16 B. In a "real" program you could easily allocate an entire page without noticing any runtime overhead. This is more than enough to link an entire static glibc, for example.
>
> The point I wanted to make here is that if we go for 2. (or 3.), we might as well convert all TLSGD/LD relocations to TPOFF. We would implement all the logic for the TPOFF relocations anyway and if we assume that the pre-allocated TLS storage is large enough to link all object files, this will lead to less runtime overhead to access TLS variables.

By default we should optimize for full support of TLS and ORC features, at the cost of performance. We can also offer plugins that people can opt in to to go the other way and give up features (e.g. dynamically extensible TLV) to recapture performance.

On x86-64 and arm64 I think that we can synthesize a per-TLV stub and re-write the TLV access as a call to that stub to get the address of the requested TLV. There may be higher performance schemes that still fit our constraints (full TLS and ORC feature support), but specialized-stubs at least offer a baseline solution.

================
Comment at: compiler-rt/lib/orc/elfnix_tls.x86-64.S:20
+	.globl ___orc_rt_elfnix_tls_get_addr
+___orc_rt_elfnix_tls_get_addr:
+        pushq           %rbp
----------------
StephenFan wrote:
> MoritzS wrote:
> > Do we need to save all registers here? I saw you took this from the implementation in MachO and I'm not familiar with the ABI there. But for ELFNix I don't think that is necessary. For TLSGD/TLSLD relocations the compiler already emits a regular function call to `__tls_get_addr` which means that it already takes care of saving the caller saved registers. Also, since you implemented `__orc_rt_elfnix_tls_get_addr_impl` as a regular function in C++, its generated assembly will also correctly store all callee saved registers.
> I agree with you, although I am not a ELF x86-64 expert.
We don't need to save all the registers, that is just a conservative way to get up and running. I've filed https://llvm.org/PR51820 with a rough sketch of the scheme that I would like to move to eventually for MachO (and I think ELFNix could share the same code).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D109293/new/

https://reviews.llvm.org/D109293