[llvm] [LV][EVL] Support fixed-order recurrence idiom with EVL tail folding. (PR #124093)

Mel Chen via llvm-commits llvm-commits at lists.llvm.org
Mon Feb 17 06:38:09 PST 2025


================
@@ -1727,6 +1783,30 @@ static void transformRecipestoEVLRecipes(VPlan &Plan, VPValue &EVL) {
   VPTypeAnalysis TypeInfo(CanonicalIVType);
   LLVMContext &Ctx = CanonicalIVType->getContext();
   VPValue *AllOneMask = Plan.getOrAddLiveIn(ConstantInt::getTrue(Ctx));
+  VPRegionBlock *LoopRegion = Plan.getVectorLoopRegion();
+  VPBasicBlock *Header = LoopRegion->getEntryBasicBlock();
+
+  // Create a scalar phi to track the previous EVL if fixed-order recurrence is
+  // contained.
+  VPScalarPHIRecipe *PrevEVL = nullptr;
+  bool ContainsFORs =
+      any_of(Header->phis(), IsaPred<VPFirstOrderRecurrencePHIRecipe>);
+  if (ContainsFORs) {
+    // TODO: Use VPInstruction::ExplicitVectorLength to get maximum EVL.
----------------
Mel-Chen wrote:

It is correct, if we adjust `VPFirstOrderRecurrencePHIRecipe::execute()` for folding by EVL. However, since the performance impact of this change is still uncertain, I prefer to prioritize correctness for this patch as an initial patch for fixed-order recurrence.

https://github.com/llvm/llvm-project/pull/124093


More information about the llvm-commits mailing list