[llvm] [LV] Support EVL for partial reduction and VPExpressionRecipe. (PR #205741)

[Elvis Wang via llvm-commits]

================
@@ -3149,6 +3157,28 @@ static VPRecipeBase *optimizeMaskToEVL(VPValue *HeaderMask,
                                            Mask ? Mask : Plan->getTrue(), &EVL},
                                           IntrR->getScalarType(), {}, {}, DL);
 
+  // Transforms the VPReductionRecipe inside the VPExpressionRecipe.
+  if (auto *Expr = dyn_cast<VPExpressionRecipe>(&CurRecipe))
+    if (match(Expr->getOperand(Expr->getNumOperands() - 1),
+              m_RemoveMask(HeaderMask, Mask))) {
+      // Decompose first and construct with VPReductionEVLRecipe later.
+      SmallVector<VPSingleDefRecipe *> ExpressionRecipes = Expr->decompose();
+      VPReductionRecipe *Red =
+          cast<VPReductionRecipe>(ExpressionRecipes.pop_back_val());
+
+      // Convert to VPReductionEVLRecipe.
+      auto *NewRed = new VPReductionEVLRecipe(
+          *Red, EVL, Mask ? Mask : Plan->getTrue(), Red->getDebugLoc());
+      NewRed->insertBefore(Expr);
+      ExpressionRecipes.push_back(NewRed);
+      auto *NewExpr =
+          new VPExpressionRecipe(Expr->getExpressionType(), ExpressionRecipes);
+
+      // Replace uses and remove the old non-EVL reduction.
+      Red->replaceAllUsesWith(NewExpr);
+      Red->eraseFromParent();
----------------
ElvisWang123 wrote:

For this approach, it cannot. The user of the original ExpressionRecipe will be set to the old ReductionRecipe once it decomposed. So we need to manually set the user for the New expression recipe from the old reduction recipe.

https://github.com/llvm/llvm-project/pull/205741