[PATCH] D35259: Complex Long Double classification In RegCall calling convention

[Elizabeth Andrews via Phabricator via cfe-commits]

eandrews added a comment.

In https://reviews.llvm.org/D35259#805415, @erichkeane wrote:

> In https://reviews.llvm.org/D35259#805409, @rnk wrote:
>
> > In https://reviews.llvm.org/D35259#805284, @erichkeane wrote:
> >
> > > Oren discovered this miss to the original implementation.  I'd reviewed this internally quite a bit.
> > >
> > > The reason for the Win32ABI test is that MSVC 'long double' is actually small enough for SSE registers in this case.  I DO now (looking again, sorry Elizabeth) wonder if there is a better way to exclude extended-length LongDouble type?  Could we us the 'length' of it instead? @rnk : opinion?
> >
> >
> > Yeah, you can ask clang::TargetInfo for the format of most basic FP types. The code to do that looks like:
> >
> >   &TI.getLongDoubleFormat() == &llvm::APFloat::x87DoubleExtended()
> >   
> >
> > Any reason you can't just add that condition to isX86VectorTypeForVectorCall? I assume we don't want to pass x86_fp80s in SSE registers for vectorcall either, right? That would eliminate the need for the isRegCallReturnableHA helper and the IsWin32StructABI parameter, which is a poorly named variable.
>
>
> It actually WOULD make sense to apply this to vectorcall as well, wouldn't it?  I presumed we didnt want to change its behavior, however vectorcall is MSVC-only (other than us), so the long-double issue isn't a thing over there. at eandrews?


I can apply this condition to isX86VectorTypeForVectorCall like Reid suggested. The only reason I did not do that originally was because I wasn't sure how to differentiate between the calling conventions without changing function declaration. If I don't need to explicitly differentiate between RegCall and VectorCall and can  use the 'length' instead, I think this works better. Like Reid mentioned, it would eliminate the need for isRegCallReturnableHA and IsWin32StructABI.


https://reviews.llvm.org/D35259