[PATCH] D72110: [X86] ABI compat bugfix for MSVC vectorcall

[Reid Kleckner via Phabricator via cfe-commits]

rnk created this revision.
rnk added reviewers: erichkeane, craig.topper.
Herald added a project: clang.

Before this change, X86_32ABIInfo::classifyArgument would be called
twice on vector arguments to vectorcall functions. This function has
side effects to track GPR register usage, and this would lead to
incorrect GPR usage in some cases.  The specific case I noticed is from
running out of XMM registers with mixed FP and vector arguments and no
aggregates of any kind. Consider this prototype:

  void __vectorcall vectorcall_indirect_vec(
      double xmm0, double xmm1, double xmm2, double xmm3, double xmm4,
      __m128 xmm5,
      __m128 ecx,
      int edx,
      __m128 mem);

classifyArgument has no effects when called on a plain FP type, but when
called on a vector type, it modifies FreeRegs to model GPR consumption.
However, this should not happen during the vector call first pass.

I refactored the code to unify vectorcall HVA logic with regcall HVA
logic. The conventions pass HVAs in registers differently (expanded vs.
not expanded), but if they do not fit in registers, they both pass them
indirectly by address.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D72110

Files:
  clang/include/clang/CodeGen/CGFunctionInfo.h
  clang/lib/CodeGen/TargetInfo.cpp
  clang/test/CodeGen/vectorcall.c

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D72110.235946.patch
Type: text/x-patch
Size: 10665 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20200102/d3ea6e49/attachment-0001.bin>