[cfe-dev] Possible bug in Win64 ABI in Clang?

[Hans Wennborg via cfe-dev]

On Sat, Nov 14, 2020 at 12:36 PM Christoffer Lernö via cfe-dev
<cfe-dev at lists.llvm.org> wrote:
>
> Perusing the Clang source, I found something rather confusing:
>
> if ((IsVectorCall || IsRegCall) &&
>       isHomogeneousAggregate(Ty, Base, NumElts)) {
>     if (IsRegCall) {
>       if (FreeSSERegs >= NumElts) {
>         FreeSSERegs -= NumElts;
>         if (IsReturnType || Ty->isBuiltinType() || Ty->isVectorType())
>           return ABIArgInfo::getDirect();
>         return ABIArgInfo::getExpand();
>       }
>       return ABIArgInfo::getIndirect(Align, /*ByVal=*/false);
>     } else if (IsVectorCall) {
>       if (FreeSSERegs >= NumElts &&
>           (IsReturnType || Ty->isBuiltinType() || Ty->isVectorType())) {
>         FreeSSERegs -= NumElts;
>         return ABIArgInfo::getDirect();
>       } else if (IsReturnType) {
>         return ABIArgInfo::getExpand();
>       } else if (!Ty->isBuiltinType() && !Ty->isVectorType()) {
>         // HVAs are delayed and reclassified in the 2nd step.
>         return ABIArgInfo::getIndirect(Align, /*ByVal=*/false);
>       }
>     }
>   }
>
>
> If we look at ”isReturnType” for IsVectorCall = true has ”ABIArgInfo::getExpand()” however, ”expand” is not a valid type of ABIArgInfo and will throw an error.
>
> So this seems to be incorrect and should crash on vectorcall with HVA. Can someone confirm?

For reference, that code is from WinX86_64ABIInfo::classify() here:
https://github.com/llvm/llvm-project/blob/bc7df035ae68648fe39304d9e77cd7618812cca8/clang/lib/CodeGen/TargetInfo.cpp#L4200

I'm not familiar with this code, but it looks like Erich wrote it in
https://reviews.llvm.org/D27529 Maybe he can comment?

Thanks,
Hans