[llvm] [LV] Support EVL for partial reduction and VPExpressionRecipe. (PR #205741)
Elvis Wang via llvm-commits
llvm-commits at lists.llvm.org
Mon Jul 6 23:51:31 PDT 2026
================
@@ -3356,7 +3356,27 @@ void VPReductionEVLRecipe::execute(VPTransformState &State) {
Mask = Builder.CreateVectorSplat(State.VF, Builder.getTrue());
Value *NewRed;
- if (isOrdered()) {
+ if (isPartialReduction()) {
+ // For partial reduction, we need to generate the predicated select
+ // (vp.merge) since `@llvm.vector.partial.reduce()` doesn't have vector
+ // predicated version.
+ VectorType *VecTy = cast<VectorType>(VecOp->getType());
+ Value *Identity = getRecurrenceIdentity(Kind, VecTy->getElementType(),
+ getFastMathFlagsOrNone());
+ if (State.VF.isVector())
+ Identity =
+ State.Builder.CreateVectorSplat(VecTy->getElementCount(), Identity);
+
+ Value *NewVecOp = State.Builder.CreateIntrinsic(
+ VecTy, Intrinsic::vp_merge, {Mask, VecOp, Identity, EVL});
+ assert((Kind == RecurKind::Add || Kind == RecurKind::FAdd) &&
+ "Unexpected partial reduction kind");
+ NewRed = State.Builder.CreateIntrinsic(
+ Prev->getType(),
+ Kind == RecurKind::Add ? Intrinsic::vector_partial_reduce_add
----------------
ElvisWang123 wrote:
We can, but we need to handle the identity construction here and only partial EVL reduction needs predicated select.
Generate the predicated `vp.merge` match the behavior with non-EVL reduction recipe.
I think a possible approach is that we can generate the predicated select for all reduction when the reduction construction(https://github.com/llvm/llvm-project/pull/201740).
https://github.com/llvm/llvm-project/pull/205741
More information about the llvm-commits
mailing list