[cfe-dev] Possible bug in Win64 ABI in Clang?

[Keane, Erich via cfe-dev]

That was long enough ago that I don't really remember.  At the time, I wrote tests to validate the behaviors I think (which would mean it didn't crash?), but I could buy that I did something wrong back then.  Do we have an idea what the return-type ABIArgInfo should be?  I'm sorry I cannot be more helpful here.  

-----Original Message-----
From: Hans Wennborg <hans at chromium.org> 
Sent: Monday, November 16, 2020 8:24 AM
To: Christoffer Lernö <christoffer at aegik.com>; Keane, Erich <erich.keane at intel.com>
Cc: clang developer list <cfe-dev at lists.llvm.org>
Subject: Re: [cfe-dev] Possible bug in Win64 ABI in Clang?

On Sat, Nov 14, 2020 at 12:36 PM Christoffer Lernö via cfe-dev <cfe-dev at lists.llvm.org> wrote:
>
> Perusing the Clang source, I found something rather confusing:
>
> if ((IsVectorCall || IsRegCall) &&
>       isHomogeneousAggregate(Ty, Base, NumElts)) {
>     if (IsRegCall) {
>       if (FreeSSERegs >= NumElts) {
>         FreeSSERegs -= NumElts;
>         if (IsReturnType || Ty->isBuiltinType() || Ty->isVectorType())
>           return ABIArgInfo::getDirect();
>         return ABIArgInfo::getExpand();
>       }
>       return ABIArgInfo::getIndirect(Align, /*ByVal=*/false);
>     } else if (IsVectorCall) {
>       if (FreeSSERegs >= NumElts &&
>           (IsReturnType || Ty->isBuiltinType() || Ty->isVectorType())) {
>         FreeSSERegs -= NumElts;
>         return ABIArgInfo::getDirect();
>       } else if (IsReturnType) {
>         return ABIArgInfo::getExpand();
>       } else if (!Ty->isBuiltinType() && !Ty->isVectorType()) {
>         // HVAs are delayed and reclassified in the 2nd step.
>         return ABIArgInfo::getIndirect(Align, /*ByVal=*/false);
>       }
>     }
>   }
>
>
> If we look at ”isReturnType” for IsVectorCall = true has ”ABIArgInfo::getExpand()” however, ”expand” is not a valid type of ABIArgInfo and will throw an error.
>
> So this seems to be incorrect and should crash on vectorcall with HVA. Can someone confirm?

For reference, that code is from WinX86_64ABIInfo::classify() here:
https://github.com/llvm/llvm-project/blob/bc7df035ae68648fe39304d9e77cd7618812cca8/clang/lib/CodeGen/TargetInfo.cpp#L4200

I'm not familiar with this code, but it looks like Erich wrote it in
https://reviews.llvm.org/D27529 Maybe he can comment?

Thanks,
Hans