[llvm] [LoopVectorize] Vectorize fixed-order recurrence with vscale x 1. (PR #142772)
Mel Chen via llvm-commits
llvm-commits at lists.llvm.org
Mon Jun 9 03:22:50 PDT 2025
================
@@ -8652,6 +8651,12 @@ static void addExitUsersForFirstOrderRecurrences(
for (VPIRInstruction *ExitIRI : ExitUsersToFix) {
if (ExitIRI->getOperand(0) != FOR)
continue;
+ // For VF vscale x 1, if vscale = 1, we are unable to extract the
+ // penultimate value of the recurrence. Instead, we can extract the last
+ // element directly from VPInstruction::FirstOrderRecurrenceSplice.
----------------
Mel-Chen wrote:
> Sounds somewhat similar to noted (exactly a year ago) in [#93396 (comment)](https://github.com/llvm/llvm-project/pull/93396#discussion_r1625882431) - would replacing ExtractPenultimate with a suitable ExtractLast work well for all targets?
>
Yes, it will work well for all target. We're trying to enable tail folding (specifically EVL tail folding) when a fixed-order recurrence (FOR) phi is live-out from the loop. In this case, all live-out values of the FOR phi must be obtained by extracting the last active lane from the recurrence phi.
> As noted in [#137030 (comment)](https://github.com/llvm/llvm-project/pull/137030#discussion_r2059765886), extract penultimate works for VF=1 as well - by extracting penultimate part which is hooked-up when unrolling-by-UF.
Yes, I also left a TODO comment for UF because we currently don’t have a method to clamp the VPlan for UF.
https://github.com/llvm/llvm-project/pull/142772
More information about the llvm-commits
mailing list