[llvm] [AArch64][NEON][SVE] Lower i8 to i64 partial reduction to a dot product (PR #110220)
James Chesterman via llvm-commits
llvm-commits at lists.llvm.org
Fri Sep 27 07:35:25 PDT 2024
================
@@ -21942,6 +21944,20 @@ SDValue tryLowerPartialReductionToDot(SDNode *N,
else
Opcode = AArch64ISD::UDOT;
+ // Partial reduction lowering for (nx)v16i8 to (nx)v4i64 requires an i32 dot
+ // product followed by a zero / sign extension
+ if ((ReducedType == MVT::nxv4i64 && MulSrcType == MVT::nxv16i8) ||
+ (ReducedType == MVT::v4i64 && MulSrcType == MVT::v16i8)) {
+ EVT ReducedTypeHalved = (ReducedType.isScalableVector()) ? MVT::nxv4i32 : MVT::v4i32;
+
+ auto Doti32 =
+ DAG.getNode(Opcode, DL, ReducedTypeHalved,
+ DAG.getConstant(0, DL, ReducedTypeHalved), A, B);
+ auto Extended = DAG.getSExtOrTrunc(Doti32, DL, ReducedType);
+ return DAG.getNode(ISD::ADD, DL, NarrowOp.getValueType(),
+ {NarrowOp, Extended});
----------------
JamesChesterman wrote:
Done
https://github.com/llvm/llvm-project/pull/110220
More information about the llvm-commits
mailing list