[llvm] [AArch64] Combine vector FNEG+FMA into `FNML[A|S]` (PR #167900)

Benjamin Maxwell via llvm-commits llvm-commits at lists.llvm.org
Mon Dec 1 07:27:14 PST 2025


================
@@ -7730,6 +7733,45 @@ SDValue AArch64TargetLowering::LowerFMUL(SDValue Op, SelectionDAG &DAG) const {
   return FCVTNT(VT, BottomBF16, Pg, TopF32);
 }
 
+SDValue AArch64TargetLowering::LowerFMA(SDValue Op, SelectionDAG &DAG) const {
+  SDValue OpA = Op->getOperand(0);
+  SDValue OpB = Op->getOperand(1);
+  SDValue OpC = Op->getOperand(2);
+  EVT VT = Op.getValueType();
+  SDLoc DL(Op);
+
+  // Bail early if we're definitely not looking to merge FNEGs into the FMA.
+  if (!VT.isFixedLengthVector() || OpC.getOpcode() != ISD::FNEG) {
+    if (VT.isScalableVector() || VT.getScalarType() == MVT::bf16 ||
+        useSVEForFixedLengthVectorVT(VT, !Subtarget->isNeonAvailable()))
+      return LowerToPredicatedOp(Op, DAG, AArch64ISD::FMA_PRED);
+    return Op; // Fallback to NEON lowering.
+  }
+
+  // Convert FMA/FNEG nodes to SVE to enable the following patterns:
+  // fma(a, b, neg(c)) -> fnmls(a, b, c)
+  // fma(neg(a), b, neg(c)) -> fnmla(a, b, c)
+  // fma(a, neg(b), neg(c)) -> fnmla(a, b, c)
+  SDValue Pg = getPredicateForVector(DAG, DL, VT);
+  EVT ContainerVT = getContainerForFixedLengthVector(DAG, VT);
+
+  // Reuse `LowerToPredicatedOp` but drop the subsequent `extract_subvector`
+  OpA = OpA.getOpcode() == ISD::FNEG
+            ? LowerToPredicatedOp(OpA, DAG, AArch64ISD::FNEG_MERGE_PASSTHRU)
+                  ->getOperand(0)
+            : convertToScalableVector(DAG, ContainerVT, OpA);
----------------
MacDue wrote:

Ignore me if this does not work, but I think this could be changed to (which avoids the `->getOperand(0)`). 
The extract/insert pair should cancel out. Same below (for OpB/C).

```suggestion
if (OpA.getOpcode() == ISD::FNEG)
  OpA = LowerToPredicatedOp(OpA, DAG, AArch64ISD::FNEG_MERGE_PASSTHRU);
OpA = convertToScalableVector(DAG, ContainerVT, OpA);
```

https://github.com/llvm/llvm-project/pull/167900


More information about the llvm-commits mailing list