[llvm] [LV] Support EVL for partial reduction and VPExpressionRecipe. (PR #205741)

Mel Chen via llvm-commits llvm-commits at lists.llvm.org
Fri Jul 3 09:23:39 PDT 2026


================
@@ -3356,7 +3356,27 @@ void VPReductionEVLRecipe::execute(VPTransformState &State) {
     Mask = Builder.CreateVectorSplat(State.VF, Builder.getTrue());
 
   Value *NewRed;
-  if (isOrdered()) {
+  if (isPartialReduction()) {
+    // For partial reduction, we need to generate the predicated select
+    // (vp.merge) since `@llvm.vector.partial.reduce()` doesn't have vector
+    // predicated version.
+    VectorType *VecTy = cast<VectorType>(VecOp->getType());
+    Value *Identity = getRecurrenceIdentity(Kind, VecTy->getElementType(),
+                                            getFastMathFlagsOrNone());
+    if (State.VF.isVector())
+      Identity =
+          State.Builder.CreateVectorSplat(VecTy->getElementCount(), Identity);
+
+    Value *NewVecOp = State.Builder.CreateIntrinsic(
+        VecTy, Intrinsic::vp_merge, {Mask, VecOp, Identity, EVL});
+    assert((Kind == RecurKind::Add || Kind == RecurKind::FAdd) &&
+           "Unexpected partial reduction kind");
+    NewRed = State.Builder.CreateIntrinsic(
+        Prev->getType(),
+        Kind == RecurKind::Add ? Intrinsic::vector_partial_reduce_add
----------------
Mel-Chen wrote:

Indeed. Since vector.partial.reduce lacks a VP intrinsic variant and does not feature a mask operand, vp.merge recipe + VPReductionRecipe seems to be the cleaner approach.
However, VPReductionRecipe currently handles the condition select directly inside its execute(). If we want to explicitly emit the vp.merge recipe in VPlan, it would be better to first represent the condition select of VPReductionRecipe as a recipe from the start, and then replace the select recipe in the expression with vp.merge.

https://github.com/llvm/llvm-project/pull/205741


More information about the llvm-commits mailing list