[llvm] [LV] Use VPReductionRecipe for partial reductions (PR #144908)
Sander de Smalen via llvm-commits
llvm-commits at lists.llvm.org
Fri Jun 20 03:59:53 PDT 2025
================
@@ -2408,23 +2414,32 @@ class VPInterleaveRecipe : public VPRecipeBase {
Instruction *getInsertPos() const { return IG->getInsertPos(); }
};
-/// A recipe to represent inloop reduction operations, performing a reduction on
-/// a vector operand into a scalar value, and adding the result to a chain.
-/// The Operands are {ChainOp, VecOp, [Condition]}.
+/// A recipe to represent inloop, ordered or partial reduction operations. It
+/// performs a reduction on a vector operand into a scalar (vector in the case
+/// of a partial reduction) value, and adds the result to a chain. The Operands
+/// are {ChainOp, VecOp, [Condition]}.
class VPReductionRecipe : public VPRecipeWithIRFlags {
/// The recurrence kind for the reduction in question.
RecurKind RdxKind;
bool IsOrdered;
/// Whether the reduction is conditional.
bool IsConditional = false;
+ /// The scaling factor, relative to the VF, that this recipe's output is
+ /// divided by.
+ /// For outer-loop reductions this is equal to 1.
----------------
sdesmalen-arm wrote:
> Aren't VPReductionRecipes for in-loop reductions only?
Partial-reductions are a form of in-loop reductions. They partially reduce (in-loop) to a smaller vector, and in the outer-loop further reduce to a scalar. The reason for making this change is so that we benefit from the code we'd otherwise have to duplicate for partial reductions. Given that in-loop/ordered and unordered reductions are represented by VPReductionRecipe, it seems like a natural extension to use this class to represent partial reductions as well.
> Should this say for unordered reductions?
Ordered reductions must be in-loop reductions although the inverse is not true, i.e. in-loop reductions are not required to be ordered reductions. It's a target's choice to implement unordered reductions in-loop. Similarly, it's a target's choice to implement unordered reductions with a partial-reduction.
> I'm also a bit confused since I thought both unordered and ordered reductions also produced a scalar result, so the VFScaleFactor would be always be 1 for them?
* In-order and in-loop reductions reductions use a scalar PHI
* partial-reductions use a vector PHI (albeit scaled down by the VFScaleFactor)
* 'regular' out-of-order reductions use a vector PHI (not scaled <=> scaled by a factor of `1`)
For ordered/in-loop reductions, the result type is a scalar, so we'd scale the result VF down to a scalar (`1`), meaning that scaling factor must be equal to the VF (`VF/VF=1`). But the way VPlan works, the VF is not necessarily known yet at this point, it may still be a range of VFs, so we instead represent that with a VFScaleFactor of `0`.
https://github.com/llvm/llvm-project/pull/144908
More information about the llvm-commits
mailing list