[llvm] [VPlan] Try to hoist Previous (and operands), if sinking fails for FORs. (PR #108945)

[Florian Hahn via llvm-commits]

================
@@ -771,6 +771,92 @@ sinkRecurrenceUsersAfterPrevious(VPFirstOrderRecurrencePHIRecipe *FOR,
   return true;
 }
 
+/// Try to hoist \p Previous and its operands before all users of \p FOR.
+static bool hoistPreviousBeforeFORUsers(VPFirstOrderRecurrencePHIRecipe *FOR,
+                                        VPRecipeBase *Previous,
+                                        VPDominatorTree &VPDT) {
+  using namespace llvm::VPlanPatternMatch;
+  if (Previous->mayHaveSideEffects() || Previous->mayReadFromMemory())
+    return false;
+
+  // Collect recipes that need hoisting.
+  SmallVector<VPRecipeBase *> WorkList;
+  SmallPtrSet<VPRecipeBase *, 8> Seen;
+  VPBasicBlock *HoistBlock = FOR->getParent();
+  auto HoistPoint = HoistBlock->getFirstNonPhi();
+  auto TryToPushHoistCandidate = [&](VPRecipeBase *HoistCandidate) {
+    // If we reach FOR, it means the original Previous depends on some other
+    // recurrence that in turn depends on FOR. If that is the case, we would
+    // also need to hoist recipes involving the other FOR, which may break
+    // dependencies.
----------------
fhahn wrote:

It can occur at the moment I think.

 There can be cases where 'previous' of one FOR uses another FOR (e.g. as in https://github.com/llvm/llvm-project/blob/main/llvm/test/Transforms/LoopVectorize/first-order-recurrence-multiply-recurrences.ll#L32). They get classified as FORs but not vectorized because we cannot rearrange the users.

https://github.com/llvm/llvm-project/pull/108945