[llvm-dev] BUG: complete misunterstanding of the MS-ABI

[Stefan Kanthak via llvm-dev]

Objects compiled for the MS-ABI don't conform to it!

Data types beyond 64 bit MUST BE returned by the callee via the
hidden first argument allocated by the caller, NOT in XMM0!

Demo/proof: from this source

--- llvm-bug.c ---
#ifndef __clang__
typedef struct {
    unsigned __int64 low;
    unsigned __int64 high;
} __uint128_t; 
#else
__attribute__((ms_abi))
#endif
__uint128_t __udivmodti4(__uint128_t dividend, __uint128_t divisor, __uint128_t *remainder) {
    if (remainder != 0)
        *remainder = divisor;
    return dividend;
}
--- EOF ---

clang -c -O1 generates the following INCOMPATIBLE and WRONG code:

__udivmodti4 proc public
        movaps  xmm0, xmmword ptr [rcx]
        test    r8, r8
        jz      0f
        movaps  xmm1, xmmword ptr [rdx]
        movaps  xmmword ptr [r8], xmm1
0:      ret
__udivmodti4 endp


clang's misunderstanding of the MS-ABI can be clearly seen here:

- RCX holds the address of the return value, NOT the address
  of the dividend;

- RDX holds the address of the dividend, NOT the address of
  the divisor;

- R8 holds the address of the divisor, NOT the address of the
  remainder;

- R9 holds the address of the remainder;

- aggregate data types are NOT returned in XMM0, but via the
  hidden first argument addressed by RCX;

- the address of the hidden first argument is returned in RAX!

JFTR: an 128-bit integer data type is not supported by MS.
      clang is also rather confused here: why is the return
      value mapped to an XMM register, but not the arguments?


Microsoft's CL.EXE -c -Ox generates the following (of course)
CONFORMANT code:

__udivmodti4 proc public
; Line 10
        test    r9, r9
        je      SHORT $LN1 at udivmodti4
; Line 11
        mov     rax, QWORD PTR [r8]
        mov     QWORD PTR [r9], rax
        mov     rax, QWORD PTR [r8+8]
        mov     QWORD PTR [r9+8], rax
$LN1 at udivmodti4:
; Line 12
        mov     rax, QWORD PTR [rdx]
        mov     QWORD PTR [rcx], rax
        mov     rax, QWORD PTR [rdx+8]
        mov     QWORD PTR [rcx+8], rax
        mov     rax, rcx
; Line 13
        ret     0
__udivmodti4 endp


NOT AMUSED
Stefan