[llvm] [VPlan] Sink recipes from the vector loop region in licm. (PR #168031)

[Luke Lau via llvm-commits]

================
@@ -2496,14 +2496,63 @@ void VPlanTransforms::cse(VPlan &Plan) {
 
 /// Move loop-invariant recipes out of the vector loop region in \p Plan.
 static void licm(VPlan &Plan) {
-  VPBasicBlock *Preheader = Plan.getVectorPreheader();
+  VPRegionBlock *LoopRegion = Plan.getVectorLoopRegion();
+
+  // Collect loop regions in preorder.
+  SmallVector<VPRegionBlock *> WorklistForSink{LoopRegion};
+  for (VPRegionBlock *R : VPBlockUtils::blocksOnly<VPRegionBlock>(
+           vp_depth_first_deep(LoopRegion->getEntry()))) {
+    if (!R->isReplicator())
+      WorklistForSink.push_back(R);
+  }
+
+  VPDominatorTree VPDT(Plan);
+  // Sink recipes with no users inside the vector loop region into a dedicated
+  // exit block.
+  while (!WorklistForSink.empty()) {
+    VPRegionBlock *CurLoop = WorklistForSink.pop_back_val();
+    auto *SingleExit =
+        cast_or_null<VPBasicBlock>(CurLoop->getSingleSuccessor());
+    // Check whether there is a unique dedicated exit block.
+    // TODO: Should check all predecessors of the exit block.
+    if (!SingleExit || SingleExit->getSinglePredecessor() != CurLoop)
+      continue;
+
----------------
lukel97 wrote:

> Would it be acceptable to first land the UserBBs.size() == 1 case, since we currently can’t test scenarios where UserBBs.size() > 1?

Yes, that also works for me given that the only users are in vector.early.exit.

Out of curiosity are there cases in-tree today where we need to sink to multiple exit blocks + clone? 

https://github.com/llvm/llvm-project/pull/168031