[llvm] [LV] Support tail-folded epilogue loops (PR #208764)

[David Sherwood via llvm-commits]

================
@@ -7554,6 +7697,23 @@ static SmallVector<Instruction *> preparePlanForEpilogueVectorLoop(
           continue;
         }
       }
+    } else if (isa<VPActiveLaneMaskPHIRecipe>(&R)) {
+      // The active-lane-mask phi's entry value was computed assuming the
+      // epilogue vector loop starts at 0. Rebuild it using the resume value
+      // instead, so the mask reflects how many elements the main vector loop
+      // already processed.
+      VPBuilder EntryBuilder(Plan.getVectorPreheader());
+      Type *CanIVTy = VectorLoop->getCanonicalIVType();
+      VPValue *ALMMultiplier = Plan.getConstantInt(CanIVTy, 1);
+      auto *EntryIncrement = EntryBuilder.createOverflowingOp(
+          VPInstruction::CanonicalIVIncrementForPart, {VPV, &Plan.getVF()}, {},
----------------
david-arm wrote:

I don't think we should be using `CanonicalIVIncrementForPart` here because there won't be any unrolling in the epilogue. I think we can just do a normal add here.

https://github.com/llvm/llvm-project/pull/208764