[llvm] [SLP]Allow min-VF vectorization of seed-level reduction groups (PR #222757)
Dmitry Sidorov via llvm-commits
llvm-commits at lists.llvm.org
Fri Sep 11 04:53:24 PDT 2026
================
@@ -30238,17 +30240,18 @@ class HorizontalReduction {
// sort them by size.
optimizeReducedVals(R, DT, DL, TTI, TLI);
// Sort the reduced values by number of same/alternate opcode and/or
- // pointer operand. For the sign-aware reductions, same-size groups are
- // ordered by their expected profitability: the first vectorized group
- // is charged the cost of the reduction operation itself, which a group
- // of loads or non-instructions rarely amortizes on its own, while it is
- // a cheap addition (a single vector operation) to an already
- // vectorized group - such groups go last. Among the rest, non-negated
- // groups go first.
+ // pointer operand. For the sign-aware and the seed-level fadd
+ // reductions, same-size groups are ordered by their expected
+ // profitability: the first vectorized group is charged the cost of the
+ // reduction operation itself, which a group of loads or
+ // non-instructions rarely amortizes on its own, while it is a cheap
+ // addition (a single vector operation) to an already vectorized group -
+ // such groups go last. Among the rest, non-negated groups go first.
stable_sort(ReducedVals, [&](ArrayRef<Value *> P1, ArrayRef<Value *> P2) {
if (P1.size() != P2.size())
return P1.size() > P2.size();
- if (NegatedReducedVals.empty())
+ if (NegatedReducedVals.empty() &&
+ !(IsSeedRoot && RdxKind == RecurKind::FAdd))
----------------
MrSidims wrote:
This seem to reorder reductions where `MinVFAllowed` can't be true, see changes of`X86/horizontal-list.ll` `extra_args_no_replace` at `-slp-threshold=-10` behaviour. Gating the ordering on the same predicate as `NoScalarLeftovers` fixes it. How about something like this?
```cpp
const bool SortSeedGroups =
IsSeedRoot && RdxKind == RecurKind::FAdd &&
all_of(ReducedVals, [this](ArrayRef<Value *> Vals) {
return Vals.size() >= ReductionLimit ||
(Vals.size() == 2 && Vals.front() != Vals.back());
});
...
if (NegatedReducedVals.empty() && !SortSeedGroups)
return false;
```
?
This check runs before the regrouping, so it only approximates `NoScalarLeftovers`. A shared helper would at least avoid the duplication.
https://github.com/llvm/llvm-project/pull/222757
More information about the llvm-commits
mailing list