[llvm] [AArch64][SDAG][NFC] Use CNTP intrinsic for MSTORE lowering (PR #218353)

[Jack Styles via llvm-commits]

https://github.com/Stylie777 created https://github.com/llvm/llvm-project/pull/218353

Currently, when lowering MSTORE the process is to Zero Extend, VECREDUCE_ADD and then Zero Extend the result for types that are not i64. This for i8/i16 types does not work as AArch64 cannot zero extend these types to i64. Instead, use the CNTP intrinsic to combine the values before storing them.

>From cb617e2f9f6a33d7138787d374a2da7f0bffe9ba Mon Sep 17 00:00:00 2001
From: Jack Styles <jack.styles at arm.com>
Date: Mon, 24 Aug 2026 09:44:28 +0100
Subject: [PATCH] [AArch64][SDAG][NFC] Use CNTP intrinsic for MSTORE lowering

Currently, when lowering MSTORE the process is to Zero Extend,
VECREDUCE_ADD and then Zero Extend the result for types that are
not i64. This for i8/i16 types does not work as AArch64 cannot
zero extend these types to i64. Instead, use the CNTP intrinsic
to combine the values before storing them.
---
 llvm/lib/Target/AArch64/AArch64ISelLowering.cpp | 13 ++++---------
 1 file changed, 4 insertions(+), 9 deletions(-)

diff --git a/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp b/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
index 36a86241fe3b2..c1ca0f789e534 100644
--- a/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
+++ b/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
@@ -33922,16 +33922,11 @@ SDValue AArch64TargetLowering::LowerMSTORE(SDValue Op,
     return SDValue();
 
   EVT MaskVT = Store->getMask().getValueType();
-  EVT MaskExtVT = getPromotedVTForPredicate(MaskVT);
-  EVT MaskReduceVT = MaskExtVT.getScalarType();
   SDValue Zero = DAG.getConstant(0, DL, MVT::i64);
-
-  SDValue MaskExt =
-      DAG.getNode(ISD::ZERO_EXTEND, DL, MaskExtVT, Store->getMask());
-  SDValue CntActive =
-      DAG.getNode(ISD::VECREDUCE_ADD, DL, MaskReduceVT, MaskExt);
-  if (MaskReduceVT != MVT::i64)
-    CntActive = DAG.getNode(ISD::ZERO_EXTEND, DL, MVT::i64, CntActive);
+  SDValue CntActive = DAG.getNode(
+      ISD::INTRINSIC_WO_CHAIN, DL, MVT::i64,
+      DAG.getTargetConstant(Intrinsic::aarch64_sve_cntp, DL, MVT::i64),
+      Store->getMask(), Store->getMask());
 
   SDValue CompressedValue =
       DAG.getNode(ISD::VECTOR_COMPRESS, DL, VT, Store->getValue(),