[clang] [clang][CodeGen][MSVC] Return vector types from methods indirectly (PR #157365)

Henry Baba-Weiss via cfe-commits cfe-commits at lists.llvm.org
Thu Sep 25 21:15:44 PDT 2025


henrybw wrote:

> I worry that the real invariant here is that instance methods must always pass `this` in the first argument register (`RCX`), and we've only covered a few of the cases that enable indirect return. Like, I wonder if we can hit this via member pointers.
> 
> Regardless, I don't want to expand scope, I'd rather approve the PR and handle it in a follow-up. Please update the test and we can land this.

To follow up on this, I dug into this some more, and found a couple of concerning discoveries:

* On x86-64, MSVC does *not* return `__m64` vectors directly from vectorcall member functions: https://godbolt.org/z/aa3E918xY. This is not consistent with how MSVC returns `__m128`, `__m256`, and `__m512` vectors from vectorcall member functions. But, it means that my patch does not adhere to the MSVC ABI for `__m64`.
* On ARM64, MSVC returns 64-bit and 128-bit vectors directly from member functions, matching Clang. However, MSVC *also* returns 256-bit and 512-bit vectors directly, whereas Clang returns them indirectly: https://godbolt.org/z/YP6M34zvq, https://godbolt.org/z/bzadn9Yre

(Note: I'm not as familiar with ARM64 as I am with x86/x86-64, so if the ABI discrepancy there is actually user error on my part, then my apologies.)

Given the discrepancy between `__m64` and `__m128` on x86-64, I wonder if this has something to do with how MSVC handles types that can map to SSE registers (and is not specific to just vector types).

Anyway, given these findings, do we still think it's a good idea to land this PR? I'm wary of causing further churn in the Clang ABI, because my patch amounts to a partial fix, and I'd be happy to back this out in favor of a more complete fix. (This is my first time contributing to LLVM, so I'm not sure what the right call here is.)

https://github.com/llvm/llvm-project/pull/157365


More information about the cfe-commits mailing list