[PATCH] 128-bit ABI for x86_64-w64-mingw32 incorrectly generated

[Jameson Nash]

  > Again, this is still a layering violation. Move the logic to X86 target.

  I know, but hoped this would make it easier to identify the correct solution, by moving the logic closer to the invocation of the target-dependent code, X86TargetLowering::LowerCall

  > You need to extend the calling convention bits to pass 128-bit stuff by value (see the existing code which already does this for vectors, i.e. __m128)

  Is this the patch you were expecting:
  ```
  diff --git a/lib/Target/X86/X86CallingConv.td b/lib/Target/X86/X86CallingConv.td
  index a78b5c0..7bc179d 100644
  --- a/lib/Target/X86/X86CallingConv.td
  +++ b/lib/Target/X86/X86CallingConv.td
  @@ -137,6 +137,8 @@ def RetCC_X86_64_C : CallingConv<[
   def RetCC_X86_Win64_C : CallingConv<[
     // The X86-Win64 calling convention always returns __m64 values in RAX.
     CCIfType<[x86mmx], CCBitConvertToType<i64>>,
  +  // The X86-Win64 calling convention always returns __m128i values in XMM.
  +  CCIfType<[i128], CCAssignToReg<[XMM0,XMM1,XMM2,XMM3]>>,

     // Otherwise, everything is the same as 'normal' X86-64 C CC.
     CCDelegateTo<RetCC_X86_64_C>
  @@ -324,7 +326,10 @@ def CC_X86_Win64_C : CallingConv<[

     // Long doubles get stack slots whose size and alignment depends on the
     // subtarget.
  -  CCIfType<[f80], CCAssignToStack<0, 0>>
  +  CCIfType<[f80], CCAssignToStack<0, 0>>,
  +
  +  // 128 bit numbers are passed by pointer
  +  CCIfType<[i128], CCPassIndirect<i64>>
   ]>;

   def CC_X86_64_GHC : CallingConv<[
  ```

  This was how I first attempted this patch. However, it does not work because line 6998 of the original file in the diff above ensures that the target-specific code will never see a type larger than a register. It seems like either more of the LowerCallTo code should be moved to the target dependent sections, or it should provide some hook to allow target-dependent register allocation


================
Comment at: test/CodeGen/X86/mod128.ll:11
@@ +10,3 @@
+    ; WIN64-NOT: xorl       %r9d, %r9d
+    ; WIN64: callq   __modti3
+
----------------
Anton Korobeynikov wrote:
> This is incorrect. You need to make sure that the args are indeed passed where necessary, not only that the 'just' call is emitted.
each version of llvm emits the arguments in a different order. looking at the documentation for FileCheck, it looks like WIN64-DAG may be the syntax I was looking for. I'll fix this in my next diff.


http://reviews.llvm.org/D1998