[LLVMdev] x86-64 sign extension for parameters and return values

[Eli Friedman]

On Wed, Feb 8, 2012 at 6:49 PM, Meador Inge <meadori at gmail.com> wrote:
> I recently noticed a difference between the way GCC and LLVM treat
> sign extension for parameters and return values on x86-64.  I could
> not find a clear answer in the x86-64 ABI [1] concerning whether
> parameters should be sign extended by the caller or callee and
> similarly whether return values should be sign extended by the caller
> or callee.
>
> Consider a simple 'signed char' division routine:
>
> signed char sdiv(signed char x, signed char y)
> {
>  return x / y;
> }
>
> On my Fedora 16 Linux host GCC 4.6.2 produces:
>
> sdiv:
> .LFB11:
>        .cfi_startproc
>        movsbl  %dil, %eax
>        movsbl  %sil, %esi
>        movl    %eax, %edx
>        sarl    $31, %edx
>        idivl   %esi
>        ret
>
> where as clang 2.9 produces:
>
> sdiv:                                   # @sdiv
> .Leh_func_begin0:
> # BB#0:
>        movl    %edi, %eax
>        cltd
>        idivl   %esi
>        movsbl  %al, %eax
>        ret
>
> GCC assumes the parameters have *not* been sign extended and assumes
> it is the receivers job to extend the return value.  LLVM assumes the
> parameters *have* been sign extended and that the receiver expects a
> sign extended result.
>
> The LLVM model implies the the caller always has to sign extend for
> signed promotion to argument registers and that the callee always has
> to sign extend for signed promotion to return value registers.  The
> GCC model seems more reasonable and efficient to me because whoever is
> actually using the values gets to choose whether a sign extension is
> done (or not done at all).
>
> Could someone explain the LLVM implementation to me?
>
> [1] http://www.x86-64.org/documentation/abi.pdf

Sorry about the delayed response; I was sort of expecting someone else
to respond.  It looks like a rather nasty issue, and deserves some
attention.

LLVM has traditionally assumed that all integer argument and return
types narrower than int are promoted to int on all architectures.
Nobody has actually noticed any issues with this before now, as far as
I know.

If gcc has decided to assume no sign/zero-extension on x86-64, we need
to follow their lead, at least on Linux. Please file at
http://llvm.org/bugs/ ; an executable testcase to go with this would
be nice, so we can compare various compilers and different platforms.

(clang 2.9 relatively old, but I don't think anything has changed here
since it was released.)

-Eli