[llvm] [VPlan] Dissolve replicate regions with vector live-outs. (PR #189022)

[via llvm-commits]

================
@@ -808,6 +819,73 @@ static void dissolveReplicateRegion(VPRegionBlock *Region, ElementCount VF,
   // FirstLaneExiting.
   VPBlockUtils::connectBlocks(Predecessor, FirstLaneEntry);
   VPBlockUtils::connectBlocks(FirstLaneExiting, NextLaneEntry);
+
+  // Collect per-lane predicated scalar phis and create InsertElement chains or
+  // BuildStructVector to pack them into a vector result. Phis are in the same
+  // order across all cloned exit blocks, so we can match them by position.
+  SmallVector<SmallVector<VPValue *, 4>> PhisByLane;
+  for (VPBasicBlock *Exit : ExitingBlocks) {
+    auto &Phis = PhisByLane.emplace_back();
+    for (auto &Phi : Exit->phis())
+      Phis.push_back(Phi.getVPSingleValue());
+  }
+
+  VPTypeAnalysis TypeInfo(Plan);
+  SmallVector<VPValue *> LaneValues;
+  for (unsigned I = 0; I < PhisByLane[0].size(); ++I) {
+    LaneValues.clear();
+    for (auto &LanePhis : PhisByLane)
+      LaneValues.push_back(LanePhis[I]);
----------------
ayalz wrote:

The phi's are held in a 2D matrix where one (inner, vertical) direction is "phis of a lane", i.e., distinct phi's of an exiting basic block, and the other (outer, vertical) is "phis across lanes", i.e., replication of the same phi across exiting basic blocks. Filling the matrix can be done during replication, which clones one lane after the other, so one column can be filled after another. Processing the matrix however needs to traverse one row after the other. Better fill the matrix in the order needed eventually, rather than transposing it later?
(Perhaps `PhisByLane` better be called `AllPhisByLane`, and `LaneValues` - `APhiAcrossLanes`?)

https://github.com/llvm/llvm-project/pull/189022