[llvm] [VPlan] Create header phi recipes after initial scalar optimizations. (PR #200920)

[Florian Hahn via llvm-commits]

================
@@ -871,17 +884,20 @@ static bool tryToSinkOrHoistRecurrenceUsers(VPBasicBlock *HeaderVPBB,
       Previous = PrevPhi->getBackedgeValue()->getDefiningRecipe();
     }
 
-    assert(Previous && "Previous must be a recipe");
-    // Sink FOR users after Previous or hoist Previous before FOR users.
-    if (!sinkRecurrenceUsersAfterPrevious(FOR, Previous, VPDT) &&
-        !hoistPreviousBeforeFORUsers(FOR, Previous, VPDT))
-      return false;
+    VPBasicBlock *InsertBlock = FOR->getParent();
+    VPBasicBlock::iterator InsertPt = InsertBlock->getFirstNonPhi();
+    if (Previous) {
+      // Sink FOR users after Previous or hoist Previous before FOR users.
+      if (!sinkRecurrenceUsersAfterPrevious(FOR, Previous, VPDT) &&
+          !hoistPreviousBeforeFORUsers(FOR, Previous, VPDT))
+        return false;
+      InsertBlock = Previous->getParent();
+      InsertPt = isa<VPHeaderPHIRecipe>(Previous)
+                     ? InsertBlock->getFirstNonPhi()
+                     : std::next(Previous->getIterator());
----------------
fhahn wrote:

Yep, we need to adjust the insert point for the code below if Previous is not null

https://github.com/llvm/llvm-project/pull/200920