[llvm] [SLP] Fix canConvertToFMA operand selection and fmul costing (PR #216425)
Dmitry Sidorov via llvm-commits
llvm-commits at lists.llvm.org
Sun Aug 16 12:56:50 PDT 2026
================
@@ -14382,18 +14382,47 @@ static InstructionCost canConvertToFMA(ArrayRef<Value *> VL,
InstructionsCompatibilityAnalysis Analysis(DT, DL, TTI, TLI);
SmallVector<BoUpSLP::ValueList> Operands = Analysis.buildOperands(S, VL);
- InstructionsState OpS = getSameOpcode(Operands.front(), TLI);
- if (!OpS.valid())
- return InstructionCost::getInvalid();
-
- if (OpS.isAltShuffle() || OpS.getOpcode() != Instruction::FMul)
- return InstructionCost::getInvalid();
- if (!CheckForContractable(Operands.front()))
+ // The fmul may sit on either side of the add/sub. Look past operand 0 only
+ // for chains that do not allow reassociation. The gate is profitability, not
+ // correctness. A reassociative chain can be vectorized into a vector fmul
+ // feeding a reduction, which is usually better than the scalar fma chain
+ // this check protects. An fsub can only fold an fmul on its left, so stop at
+ // operand 0 there as well.
+ bool AllowReassoc =
+ any_of(VL, [](Value *V) { return match(V, m_AllowReassoc(m_Value())); });
+ bool OnlyFirstOperand = AllowReassoc || S.getOpcode() == Instruction::FSub;
----------------
MrSidims wrote:
reassoc isn't what allows the fmul to sit on operand 1 as fadd is commutative regardless of fast-math flags. But it protects reduction path (at least as of right now). When the whole chain is reassociable, SLP can build a vector fmul feeding a vector.reduce.fadd, and that usually beats the scalar fma chain this check preserves. So for reassoc chains I would stop at operand 0 and let the reduction win.
I can share what changes in X86/redux-feed-buildvector.ll if AllowReassoc is not reversed to show it.
https://github.com/llvm/llvm-project/pull/216425
More information about the llvm-commits
mailing list