[llvm] [VPlan] Sink recipes from the vector loop region in licm. (PR #168031)

[Luke Lau via llvm-commits]

================
@@ -2547,6 +2547,69 @@ static void licm(VPlan &Plan) {
       R.moveBefore(*Preheader, Preheader->end());
     }
   }
+
+  // Collect dedicated exit blocks, which have only the loop region as
+  // predecessor.
+  // TODO: Sinking to non-dedicated exits is not supported yet, as it would
+  // require splitting the edge to create a dedicated exit block. Without this,
+  // the sunk instruction would incorrectly execute on paths entering the exit
+  // block from other predecessors.
+  SmallPtrSet<VPBasicBlock *, 2> DedicatedExits;
+  for (VPBlockBase *Succ : LoopRegion->getSuccessors()) {
+    auto *ExitBB = cast<VPBasicBlock>(Succ);
+    if (ExitBB->getSinglePredecessor() == LoopRegion)
+      DedicatedExits.insert(ExitBB);
+  }
+  if (DedicatedExits.empty())
+    return;
+
+  VPDominatorTree VPDT(Plan);
+  // Sink recipes with no users inside the vector loop region into a dedicated
+  // exit block.
+  // TODO: Extend to sink recipes from inner loops.
+  for (VPBasicBlock *VPBB : VPBlockUtils::blocksOnly<VPBasicBlock>(
+           vp_post_order_shallow(LoopRegion->getEntry()))) {
+    for (VPRecipeBase &R : make_early_inc_range(reverse(*VPBB))) {
+      if (cannotHoistOrSinkRecipe(R))
+        continue;
+
+      auto *Def = cast<VPSingleDefRecipe>(&R);
----------------
lukel97 wrote:

I don't think there's anything preventing interleaved load recipes from being sunk. Probably better to not cast to VPSingleDefRecipe and just iterate over all defined values, e.g

```c++
     if (any_of(R.definedValues(), [&UserVPBBs](VPValue *V) {
            return any_of(V->users(), [&UserVPBBs](VPUser *U) {
```

https://github.com/llvm/llvm-project/pull/168031