[llvm] [LoopVectorize] Vectorize fixed-order recurrence with vscale x 1. (PR #142772)

Mel Chen via llvm-commits llvm-commits at lists.llvm.org
Mon Jun 9 03:22:50 PDT 2025


================
@@ -8652,6 +8651,12 @@ static void addExitUsersForFirstOrderRecurrences(
     for (VPIRInstruction *ExitIRI : ExitUsersToFix) {
       if (ExitIRI->getOperand(0) != FOR)
         continue;
+      // For VF vscale x 1, if vscale = 1, we are unable to extract the
+      // penultimate value of the recurrence. Instead, we can extract the last
+      // element directly from VPInstruction::FirstOrderRecurrenceSplice.
----------------
Mel-Chen wrote:

> Sounds somewhat similar to noted (exactly a year ago) in [#93396 (comment)](https://github.com/llvm/llvm-project/pull/93396#discussion_r1625882431) - would replacing ExtractPenultimate with a suitable ExtractLast work well for all targets?
> 
Yes, it will work well for all target. We're trying to enable tail folding (specifically EVL tail folding) when a fixed-order recurrence (FOR) phi is live-out from the loop. In this case, all live-out values of the FOR phi must be obtained by extracting the last active lane from the recurrence phi.

> As noted in [#137030 (comment)](https://github.com/llvm/llvm-project/pull/137030#discussion_r2059765886), extract penultimate works for VF=1 as well - by extracting penultimate part which is hooked-up when unrolling-by-UF.
Yes, I also left a TODO comment for UF because we currently don’t have a method to clamp the VPlan for UF.

https://github.com/llvm/llvm-project/pull/142772


More information about the llvm-commits mailing list