[llvm] [AArch64] Combine vector FNEG+FMA into `FNML[A|S]` (PR #167900)
Benjamin Maxwell via llvm-commits
llvm-commits at lists.llvm.org
Mon Dec 1 07:27:14 PST 2025
================
@@ -7730,6 +7733,45 @@ SDValue AArch64TargetLowering::LowerFMUL(SDValue Op, SelectionDAG &DAG) const {
return FCVTNT(VT, BottomBF16, Pg, TopF32);
}
+SDValue AArch64TargetLowering::LowerFMA(SDValue Op, SelectionDAG &DAG) const {
+ SDValue OpA = Op->getOperand(0);
+ SDValue OpB = Op->getOperand(1);
+ SDValue OpC = Op->getOperand(2);
+ EVT VT = Op.getValueType();
+ SDLoc DL(Op);
+
+ // Bail early if we're definitely not looking to merge FNEGs into the FMA.
+ if (!VT.isFixedLengthVector() || OpC.getOpcode() != ISD::FNEG) {
+ if (VT.isScalableVector() || VT.getScalarType() == MVT::bf16 ||
+ useSVEForFixedLengthVectorVT(VT, !Subtarget->isNeonAvailable()))
+ return LowerToPredicatedOp(Op, DAG, AArch64ISD::FMA_PRED);
+ return Op; // Fallback to NEON lowering.
+ }
+
+ // Convert FMA/FNEG nodes to SVE to enable the following patterns:
+ // fma(a, b, neg(c)) -> fnmls(a, b, c)
+ // fma(neg(a), b, neg(c)) -> fnmla(a, b, c)
+ // fma(a, neg(b), neg(c)) -> fnmla(a, b, c)
+ SDValue Pg = getPredicateForVector(DAG, DL, VT);
+ EVT ContainerVT = getContainerForFixedLengthVector(DAG, VT);
+
+ // Reuse `LowerToPredicatedOp` but drop the subsequent `extract_subvector`
+ OpA = OpA.getOpcode() == ISD::FNEG
+ ? LowerToPredicatedOp(OpA, DAG, AArch64ISD::FNEG_MERGE_PASSTHRU)
+ ->getOperand(0)
+ : convertToScalableVector(DAG, ContainerVT, OpA);
----------------
MacDue wrote:
Ignore me if this does not work, but I think this could be changed to (which avoids the `->getOperand(0)`).
The extract/insert pair should cancel out. Same below (for OpB/C).
```suggestion
if (OpA.getOpcode() == ISD::FNEG)
OpA = LowerToPredicatedOp(OpA, DAG, AArch64ISD::FNEG_MERGE_PASSTHRU);
OpA = convertToScalableVector(DAG, ContainerVT, OpA);
```
https://github.com/llvm/llvm-project/pull/167900
More information about the llvm-commits
mailing list