[llvm-dev] Automatically backing up and restoring x18 around function calls on AArch64?

[Martin Storsjö via llvm-dev]

On Tue, 30 Apr 2019, James Y Knight wrote:

> 
> 
> On Fri, Apr 26, 2019 at 8:30 AM Martin Storsjö via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>       Hi,
>
>       When using Wine to run Windows ARM64 executables on Linux,
>       there's one
>       major ABI incompatibility between the two; Windows treats the
>       x18
>       register as the reserved platform register, while it is free to
>       be
>       clobbered anywhere in code on Linux.
>
>       The Wine code sets up this register before passing control over
>       to the
>       Windows executable code, but whenever the Windows code calls a
>       function
>       implemented in Wine, this register ends up clobbered.
>
>       The initial solution is to compile Wine itself with the
>       -ffixed-x18 flag,
>       to treat the x18 register as reserved within Wine, so that no
>       compiler
>       generated code touches it. This works fine as long as Wine makes
>       no calls
>       to other libraries from the outside Linux distribution - but
>       e.g. as soon
>       as any glibc function is called, it can end up clobbered again.
>
>       The full, proper solution would of course to be rebuilding one's
>       Linux
>       distribution from scratch with -ffixed-x18 on every single
>       library. But
>       this is of course pretty much impractical.
>
>       My current makeshift workaround for the matter is to reroute
>       calls from
>       native Windows code to Wine functions via a relay wrapper, which
>       backs up
>       and restores the register. This works in practice, but hasn't
>       been
>       accepted upstream, as it is not deemed a correct/complete
>       solution. Also,
>       if the emulated Windows API functions do callbacks, the register
>       would
>       need to be restored before the callbacks call windows code.
> 
> 
> 
> You don't actually need to back up the value, since it doesn't change, just
> copy it from another per-thread location back into x18, right?

Yes, that's right. If done within Wine code, restoring the right value is 
of course the best option.

> Anyhow, this "workaround" seems like the correct solution, IMO. You have two
> slightly-incompatible ABIs, and using an adapter between the two seems
> entirely reasonable. As long as this adapter code is either written in asm,
> or compiled with -ffixed-x18, you can be sure that the x18 won't be
> overwritten by the compiler inside the adapter function. Yes, it does need
> to be invoked both on the return path back to windows code, and on callouts
> to windows functions, but that doesn't seem like it should be terribly
> tricky to arrange?

Handling things on entry/return to Wine code seems straightforward, but I 
haven't figured out if there's any mechanism for doing any special 
adaptation for callbacks (other than using compiler attributes for 
different calling conventions).

AFAIK Wine does support thunking for running Win16 things, but I haven't 
really figured out where it comes into play, and if that could be used for 
injecting code in callback calls.

> Having the compiler handle it for you might seem nice, but since this is a
> value which is not part of the Linux ABI at all, copying it yourself as
> needed really seems like the best plan. Otherwise, you'd have to teach the
> compiler where the secondary thread-local location it's stashed is, and how
> to retrieve it from there. Which doesn't seem like the compiler ought to be
> in the business of doing.

Yes, the compiler has no business doing things like that. But various 
variants of backing up and restoring the register doesn't seem all that 
out of place either; the thing that Reid suggested is pretty neat, even if 
it doesn't handle the callback case.

// Martin