[llvm] [SelectionDAG][AArch64] Add dot product lowering in NEON for PARTIAL_REDUCE_*MLA ISD nodes (PR #140075)
Gaƫtan Bossu via llvm-commits
llvm-commits at lists.llvm.org
Tue May 27 01:29:58 PDT 2025
================
@@ -29518,37 +29532,60 @@ SDValue AArch64TargetLowering::LowerVECTOR_HISTOGRAM(SDValue Op,
}
/// If a PARTIAL_REDUCE_MLA node comes in with an accumulator-input type pairing
-/// of nxv2i64/nxv16i8, we cannot directly lower it to a (u|s)dot. We can
+/// of (nx)v2i64/(nx)v16i8, we cannot directly lower it to a (u|s)dot. We can
/// however still make use of the dot product instruction by instead
-/// accumulating over two steps: nxv16i8 -> nxv4i32 -> nxv2i64.
+/// accumulating over two steps: (nx)v16i8 -> (nx)v4i32 -> (nx)v2i64.
+/// If available, make use of the (U|S)ADDW(B|T) instructions, otherwise
+/// the following pattern is emitted:
+/// add(add(Acc, ext(EXTRACT_SUBVECTOR(N, 0)), ext(EXTRACT_SUBVECTOR(N,
+/// NTy/2))))
SDValue
AArch64TargetLowering::LowerPARTIAL_REDUCE_MLA(SDValue Op,
SelectionDAG &DAG) const {
+ bool Scalable = Op.getValueType().isScalableVector();
+
+ assert((!Scalable || Subtarget->isSVEorStreamingSVEAvailable()) &&
+ "SVE or StreamingSVE must be available when using scalable vectors.");
+ assert(
+ (Scalable || (Subtarget->isNeonAvailable() || Subtarget->hasDotProd())) &&
+ "Neon or dotprod must be available when using fixed-width vectors.");
----------------
gbossu wrote:
Curious piggy-back: Does `hasDotProd()` only refer to NEON dot products? I assume it's not possible to get SVE dot products without NEON ones?
https://github.com/llvm/llvm-project/pull/140075
More information about the llvm-commits
mailing list