[llvm] [LV] Support tail-folded epilogue loops (PR #208764)

[David Sherwood via llvm-commits]

================
@@ -45,20 +45,44 @@ void VPlanTransforms::replaceWideCanonicalIVWithWideIV(
   if (!LoopRegion)
     return;
 
-  auto *WideCanIV =
-      findUserOf<VPWidenCanonicalIVRecipe>(LoopRegion->getCanonicalIV());
+  VPWidenCanonicalIVRecipe *WideCanIV = nullptr;
+  VPIRValue *StartValue = nullptr;
+  // VPWidenCanonicalIVRecipe is either a direct user of CanonicalIV or
+  // Add (CanonicalIV, resumeValue) (like the case for tail-folded epilogue).
+  for (auto *User : LoopRegion->getCanonicalIV()->users()) {
+    if (isa<VPWidenCanonicalIVRecipe>(User)) {
+      WideCanIV = cast<VPWidenCanonicalIVRecipe>(User);
+      StartValue = Plan.getZero(WideCanIV->getScalarType());
+      break;
+    }
+    if (isa<VPInstruction>(User)) {
+      auto *UserInstr = cast<VPInstruction>(User);
+      auto *It = find_if(UserInstr->users(), IsaPred<VPWidenCanonicalIVRecipe>);
+      if (It != UserInstr->user_end()) {
+        WideCanIV = cast<VPWidenCanonicalIVRecipe>(*It);
----------------
david-arm wrote:

This seems to work, but what code in the loop vectoriser guarantees that the first user of `User` is a VPWidenCanonicalIVRecipe recipe? Should we instead walk through all users of `User` to find the VPWidenCanonicalIVRecipe recipe?

https://github.com/llvm/llvm-project/pull/208764