[llvm] [VPlan] Build initial VPlan 0 using HCFGBuilder for inner loops. (NFC) (PR #124432)

[Florian Hahn via llvm-commits]

================
@@ -9360,15 +9396,16 @@ LoopVectorizationPlanner::tryToBuildVPlanWithVPRecipes(VFRange &Range) {
       // in the exit block, a uniform store recipe will be created for the final
       // invariant store of the reduction.
       StoreInst *SI;
-      if ((SI = dyn_cast<StoreInst>(&I)) &&
+      if ((SI = dyn_cast<StoreInst>(Instr)) &&
           Legal->isInvariantAddressOfReduction(SI->getPointerOperand())) {
         // Only create recipe for the final invariant store of the reduction.
-        if (!Legal->isInvariantStoreOfReduction(SI))
-          continue;
-        auto *Recipe = new VPReplicateRecipe(
-            SI, make_range(Operands.begin(), Operands.end()),
-            true /* IsUniform */);
-        Recipe->insertBefore(*MiddleVPBB, MBIP);
+        if (Legal->isInvariantStoreOfReduction(SI)) {
+          auto *Recipe = new VPReplicateRecipe(
+              SI, make_range(Operands.begin(), Operands.end()),
+              true /* IsUniform */);
+          Recipe->insertBefore(*MiddleVPBB, MBIP);
+        }
+        R.eraseFromParent();
----------------
fhahn wrote:

There may be multiple stores to the invariant address of a reduction (`Legal->isInvariantAddressOfReduction(SI->getPointerOperand())`). All of those need to be removed, and only the final one (`isInvariantStoreOfReduction`) will produce the final store recipe .

There are tests covering the cases with single and multiple stores

https://github.com/llvm/llvm-project/pull/124432