[llvm] Reland [AArch64][CostModel] Consider some nxv1 operations as legal (#214471) (PR #228371)
via llvm-commits
llvm-commits at lists.llvm.org
Fri Oct 2 02:29:38 PDT 2026
llvmorg-github-actions[bot] wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-backend-aarch64
Author: Gaëtan Bossu (gbossu)
<details>
<summary>Changes</summary>
This is allowing some operations on vscale x 1 types, namely:
- load/store
- masked load/store
- arithmetic instructions like add/sub/mul
For those, there is already codegen coverage. See e.g.
- llvm/test/CodeGen/AArch64/sve-int-arith.ll
- llvm/test/CodeGen/AArch64/sve-load-store-legalisation.ll
- llvm/test/CodeGen/AArch64/sve-masked-gather.ll
- llvm/test/CodeGen/AArch64/sve-masked-scatter.ll
These types are relevant for enabling REVEC in LoopVectorizer. For AArch64, the main target there is turning NEON vector sizes into SVE vector size, i.e. using VF = "vscale x 1". For loops with mixed vector and scalar types, we'll want to allow vectorising scalar types with such a VF.
The original PR #<!-- -->214471 got reverted due to a failure in llvm-test-suite. The codegen issue was fixed in #<!-- -->226939 and llvm-test-suite passes locally now.
---
Patch is 30.42 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/228371.diff
10 Files Affected:
- (modified) llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp (+31-20)
- (modified) llvm/test/Analysis/CostModel/AArch64/masked_ldst.ll (+22-41)
- (modified) llvm/test/Analysis/CostModel/AArch64/sve-arith.ll (+3-3)
- (modified) llvm/test/Analysis/CostModel/AArch64/sve-ldst.ll (+4-4)
- (modified) llvm/test/CodeGen/AArch64/sve-int-arith.ll (+48-9)
- (modified) llvm/test/Transforms/LoopVectorize/AArch64/force-scalable-vectorization-always.ll (+1-1)
- (modified) llvm/test/Transforms/LoopVectorize/AArch64/invalid-costs.ll (+2-1)
- (modified) llvm/test/Transforms/LoopVectorize/AArch64/scalable-alloca.ll (-1)
- (modified) llvm/test/Transforms/LoopVectorize/AArch64/scalable-call.ll (-6)
- (modified) llvm/test/Transforms/LoopVectorize/AArch64/scalable-vectorization-cost-tuning.ll (+3-3)
``````````diff
diff --git a/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp b/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp
index ddea78fa4e808..c29c1e43aa90b 100644
--- a/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp
+++ b/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp
@@ -617,9 +617,12 @@ AArch64TTIImpl::getIntrinsicInstrCost(const IntrinsicCostAttributes &ICA,
// of <vscale x 1 x eltty> yet, so return an invalid cost to avoid selecting
// it. This change will be removed when code-generation for these types is
// sufficiently reliable.
+ // Only allow masked ld/st to pass through to getMemIntrinsicInstrCost().
auto *RetTy = ICA.getReturnType();
if (auto *VTy = dyn_cast<ScalableVectorType>(RetTy))
- if (VTy->getElementCount() == ElementCount::getScalable(1))
+ if (VTy->getElementCount() == ElementCount::getScalable(1) &&
+ !is_contained({Intrinsic::masked_load, Intrinsic::masked_store},
+ ICA.getID()))
return InstructionCost::getInvalid();
switch (ICA.getID()) {
@@ -4930,15 +4933,15 @@ InstructionCost AArch64TTIImpl::getArithmeticInstrCost(
// The code-generator is currently not able to handle scalable vectors
// of <vscale x 1 x eltty> yet, so return an invalid cost to avoid selecting
- // it. This change will be removed when code-generation for these types is
- // sufficiently reliable.
+ // it until all instructions are vetted.
+ int ISD = TLI->InstructionOpcodeToISD(Opcode);
if (auto *VTy = dyn_cast<ScalableVectorType>(Ty))
if (VTy->getElementCount() == ElementCount::getScalable(1))
- return InstructionCost::getInvalid();
+ if (!is_contained({ISD::ADD, ISD::SUB, ISD::MUL}, ISD))
+ return InstructionCost::getInvalid();
// Legalize the type.
std::pair<InstructionCost, MVT> LT = getTypeLegalizationCost(Ty);
- int ISD = TLI->InstructionOpcodeToISD(Opcode);
// TODO: Handle more cost kinds for floating point operations.
if (ISD == ISD::FADD || ISD == ISD::FSUB || ISD == ISD::FMUL ||
@@ -5550,12 +5553,13 @@ AArch64TTIImpl::getMaskedMemoryOpCost(const MemIntrinsicCostAttributes &MICA,
if (VT->getElementType()->isIntegerTy(1))
return InstructionCost::getInvalid();
- // The code-generator is currently not able to handle scalable vectors
- // of <vscale x 1 x eltty> yet, so return an invalid cost to avoid selecting
- // it. This change will be removed when code-generation for these types is
- // sufficiently reliable.
+ // <vscale x 1 x eltty> operations require mask adaptation.
+ // Allow it for normal masked ld/st
if (VT->getElementCount() == ElementCount::getScalable(1))
- return InstructionCost::getInvalid();
+ return is_contained({Intrinsic::masked_load, Intrinsic::masked_store},
+ MICA.getID())
+ ? LT.first + 1
+ : InstructionCost::getInvalid();
InstructionCost MemOpCost = LT.first;
if (MICA.getID() == Intrinsic::masked_expandload) {
@@ -5677,18 +5681,25 @@ InstructionCost AArch64TTIImpl::getMemoryOpCost(unsigned Opcode, Type *Ty,
if (!LT.first.isValid())
return InstructionCost::getInvalid();
- // The code-generator is currently not able to handle scalable vectors
- // of <vscale x 1 x eltty> yet, so return an invalid cost to avoid selecting
- // it. This change will be removed when code-generation for these types is
- // sufficiently reliable.
- // We also only support full register predicate loads and stores.
- if (auto *VTy = dyn_cast<ScalableVectorType>(Ty))
- if (VTy->getElementCount() == ElementCount::getScalable(1) ||
- (VTy->getElementType()->isIntegerTy(1) &&
- !VTy->getElementCount().isKnownMultipleOf(
- ElementCount::getScalable(16))))
+ if (auto *VTy = dyn_cast<ScalableVectorType>(Ty)) {
+
+ // We only support full register predicate loads and stores.
+ if (VTy->getElementType()->isIntegerTy(1) &&
+ !VTy->getElementCount().isKnownMultipleOf(
+ ElementCount::getScalable(16)))
return InstructionCost::getInvalid();
+ // <vscale x 1 x eltty> operations require crafting a new mask.
+ if (VTy->getElementCount() == ElementCount::getScalable(1)) {
+ Intrinsic::ID IID = Opcode == Instruction::Load ? Intrinsic::masked_load
+ : Intrinsic::masked_store;
+ return getMaskedMemoryOpCost(
+ MemIntrinsicCostAttributes(IID, Ty, Alignment, AddressSpace),
+ CostKind) +
+ 1;
+ }
+ }
+
// TODO: consider latency as well for TCK_SizeAndLatency.
if (CostKind == TTI::TCK_CodeSize || CostKind == TTI::TCK_SizeAndLatency)
return LT.first;
diff --git a/llvm/test/Analysis/CostModel/AArch64/masked_ldst.ll b/llvm/test/Analysis/CostModel/AArch64/masked_ldst.ll
index c1ee229ae8e2c..7aeba01cf321e 100644
--- a/llvm/test/Analysis/CostModel/AArch64/masked_ldst.ll
+++ b/llvm/test/Analysis/CostModel/AArch64/masked_ldst.ll
@@ -72,7 +72,7 @@ define void @scalable() {
; CHECK-NEXT: Cost Model: Found costs of 1 for: %nxv2f32 = call <vscale x 2 x float> @llvm.masked.load.nxv2f32.p0(ptr align 8 undef, <vscale x 2 x i1> undef, <vscale x 2 x float> undef)
; CHECK-NEXT: Cost Model: Found costs of 1 for: %nxv4f32 = call <vscale x 4 x float> @llvm.masked.load.nxv4f32.p0(ptr align 8 undef, <vscale x 4 x i1> undef, <vscale x 4 x float> undef)
; CHECK-NEXT: Cost Model: Found costs of 1 for: %nxv2f64 = call <vscale x 2 x double> @llvm.masked.load.nxv2f64.p0(ptr align 8 undef, <vscale x 2 x i1> undef, <vscale x 2 x double> undef)
-; CHECK-NEXT: Cost Model: Found costs of Invalid for: %nxv1i64 = call <vscale x 1 x i64> @llvm.masked.load.nxv1i64.p0(ptr align 8 undef, <vscale x 1 x i1> undef, <vscale x 1 x i64> undef)
+; CHECK-NEXT: Cost Model: Found costs of 2 for: %nxv1i64 = call <vscale x 1 x i64> @llvm.masked.load.nxv1i64.p0(ptr align 8 undef, <vscale x 1 x i1> undef, <vscale x 1 x i64> undef)
; CHECK-NEXT: Cost Model: Found costs of 4 for: %nxv4i64 = call <vscale x 4 x i64> @llvm.masked.load.nxv4i64.p0(ptr align 8 undef, <vscale x 4 x i1> undef, <vscale x 4 x i64> undef)
; CHECK-NEXT: Cost Model: Found costs of 8 for: %nxv32f16 = call <vscale x 32 x half> @llvm.masked.load.nxv32f16.p0(ptr align 8 undef, <vscale x 32 x i1> undef, <vscale x 32 x half> undef)
; CHECK-NEXT: Cost Model: Found costs of Invalid for: %nxv4i1 = call <vscale x 4 x i1> @llvm.masked.load.nxv4i1.p0(ptr align 16 undef, <vscale x 4 x i1> undef, <vscale x 4 x i1> undef)
@@ -250,44 +250,25 @@ define void @scalable_ext_loads() {
ret void
}
+define void @scalable_nxv1() {
+; CHECK-LABEL: 'scalable_nxv1'
+; CHECK-NEXT: Cost Model: Found costs of 2 for: %nxv1i8 = call <vscale x 1 x i8> @llvm.masked.load.nxv1i8.p0(ptr poison, <vscale x 1 x i1> poison, <vscale x 1 x i8> poison)
+; CHECK-NEXT: Cost Model: Found costs of 2 for: %nxv1i16 = call <vscale x 1 x i16> @llvm.masked.load.nxv1i16.p0(ptr poison, <vscale x 1 x i1> poison, <vscale x 1 x i16> poison)
+; CHECK-NEXT: Cost Model: Found costs of 2 for: %nxv1i32 = call <vscale x 1 x i32> @llvm.masked.load.nxv1i32.p0(ptr poison, <vscale x 1 x i1> poison, <vscale x 1 x i32> poison)
+; CHECK-NEXT: Cost Model: Found costs of 2 for: %nxv1i64 = call <vscale x 1 x i64> @llvm.masked.load.nxv1i64.p0(ptr poison, <vscale x 1 x i1> poison, <vscale x 1 x i64> poison)
+; CHECK-NEXT: Cost Model: Found costs of 2 for: %nxv1f16 = call <vscale x 1 x half> @llvm.masked.load.nxv1f16.p0(ptr poison, <vscale x 1 x i1> poison, <vscale x 1 x half> poison)
+; CHECK-NEXT: Cost Model: Found costs of 2 for: %nxv1f32 = call <vscale x 1 x float> @llvm.masked.load.nxv1f32.p0(ptr poison, <vscale x 1 x i1> poison, <vscale x 1 x float> poison)
+; CHECK-NEXT: Cost Model: Found costs of 2 for: %nxv1f64 = call <vscale x 1 x double> @llvm.masked.load.nxv1f64.p0(ptr poison, <vscale x 1 x i1> poison, <vscale x 1 x double> poison)
+; CHECK-NEXT: Cost Model: Found costs of RThru:0 CodeSize:1 Lat:1 SizeLat:1 for: ret void
+;
+entry:
+ %nxv1i8 = call <vscale x 1 x i8> @llvm.masked.load.nxv1i8.p0(ptr poison, <vscale x 1 x i1> poison, <vscale x 1 x i8> poison)
+ %nxv1i16 = call <vscale x 1 x i16> @llvm.masked.load.nxv1i16.p0(ptr poison, <vscale x 1 x i1> poison, <vscale x 1 x i16> poison)
+ %nxv1i32 = call <vscale x 1 x i32> @llvm.masked.load.nxv1i32.p0(ptr poison, <vscale x 1 x i1> poison, <vscale x 1 x i32> poison)
+ %nxv1i64 = call <vscale x 1 x i64> @llvm.masked.load.nxv1i64.p0(ptr poison, <vscale x 1 x i1> poison, <vscale x 1 x i64> poison)
+ %nxv1f16 = call <vscale x 1 x half> @llvm.masked.load.nxv1f16.p0(ptr poison, <vscale x 1 x i1> poison, <vscale x 1 x half> poison)
+ %nxv1f32 = call <vscale x 1 x float> @llvm.masked.load.nxv1f32.p0(ptr poison, <vscale x 1 x i1> poison, <vscale x 1 x float> poison)
+ %nxv1f64 = call <vscale x 1 x double> @llvm.masked.load.nxv1f64.p0(ptr poison, <vscale x 1 x i1> poison, <vscale x 1 x double> poison)
-declare <2 x i8> @llvm.masked.load.v2i8.p0(ptr, i32, <2 x i1>, <2 x i8>)
-declare <4 x i8> @llvm.masked.load.v4i8.p0(ptr, i32, <4 x i1>, <4 x i8>)
-declare <8 x i8> @llvm.masked.load.v8i8.p0(ptr, i32, <8 x i1>, <8 x i8>)
-declare <16 x i8> @llvm.masked.load.v16i8.p0(ptr, i32, <16 x i1>, <16 x i8>)
-declare <2 x i16> @llvm.masked.load.v2i16.p0(ptr, i32, <2 x i1>, <2 x i16>)
-declare <4 x i16> @llvm.masked.load.v4i16.p0(ptr, i32, <4 x i1>, <4 x i16>)
-declare <8 x i16> @llvm.masked.load.v8i16.p0(ptr, i32, <8 x i1>, <8 x i16>)
-declare <2 x i32> @llvm.masked.load.v2i32.p0(ptr, i32, <2 x i1>, <2 x i32>)
-declare <4 x i32> @llvm.masked.load.v4i32.p0(ptr, i32, <4 x i1>, <4 x i32>)
-declare <2 x i64> @llvm.masked.load.v2i64.p0(ptr, i32, <2 x i1>, <2 x i64>)
-declare <4 x i64> @llvm.masked.load.v4i64.p0(ptr, i32, <4 x i1>, <4 x i64>)
-declare <2 x half> @llvm.masked.load.v2f16.p0(ptr, i32, <2 x i1>, <2 x half>)
-declare <4 x half> @llvm.masked.load.v4f16.p0(ptr, i32, <4 x i1>, <4 x half>)
-declare <8 x half> @llvm.masked.load.v8f16.p0(ptr, i32, <8 x i1>, <8 x half>)
-declare <32 x half> @llvm.masked.load.v32f16.p0(ptr, i32, <32 x i1>, <32 x half>)
-declare <2 x float> @llvm.masked.load.v2f32.p0(ptr, i32, <2 x i1>, <2 x float>)
-declare <4 x float> @llvm.masked.load.v4f32.p0(ptr, i32, <4 x i1>, <4 x float>)
-declare <2 x double> @llvm.masked.load.v2f64.p0(ptr, i32, <2 x i1>, <2 x double>)
-declare <vscale x 4 x i1> @llvm.masked.load.nxv4i1.p0(ptr, i32, <vscale x 4 x i1>, <vscale x 4 x i1>)
-
-
-declare <vscale x 2 x i8> @llvm.masked.load.nxv2i8.p0(ptr, i32, <vscale x 2 x i1>, <vscale x 2 x i8>)
-declare <vscale x 4 x i8> @llvm.masked.load.nxv4i8.p0(ptr, i32, <vscale x 4 x i1>, <vscale x 4 x i8>)
-declare <vscale x 8 x i8> @llvm.masked.load.nxv8i8.p0(ptr, i32, <vscale x 8 x i1>, <vscale x 8 x i8>)
-declare <vscale x 16 x i8> @llvm.masked.load.nxv16i8.p0(ptr, i32, <vscale x 16 x i1>, <vscale x 16 x i8>)
-declare <vscale x 2 x i16> @llvm.masked.load.nxv2i16.p0(ptr, i32, <vscale x 2 x i1>, <vscale x 2 x i16>)
-declare <vscale x 4 x i16> @llvm.masked.load.nxv4i16.p0(ptr, i32, <vscale x 4 x i1>, <vscale x 4 x i16>)
-declare <vscale x 8 x i16> @llvm.masked.load.nxv8i16.p0(ptr, i32, <vscale x 8 x i1>, <vscale x 8 x i16>)
-declare <vscale x 2 x i32> @llvm.masked.load.nxv2i32.p0(ptr, i32, <vscale x 2 x i1>, <vscale x 2 x i32>)
-declare <vscale x 4 x i32> @llvm.masked.load.nxv4i32.p0(ptr, i32, <vscale x 4 x i1>, <vscale x 4 x i32>)
-declare <vscale x 2 x i64> @llvm.masked.load.nxv2i64.p0(ptr, i32, <vscale x 2 x i1>, <vscale x 2 x i64>)
-declare <vscale x 4 x i64> @llvm.masked.load.nxv4i64.p0(ptr, i32, <vscale x 4 x i1>, <vscale x 4 x i64>)
-declare <vscale x 1 x i64> @llvm.masked.load.nxv1i64.p0(ptr, i32, <vscale x 1 x i1>, <vscale x 1 x i64>)
-declare <vscale x 2 x half> @llvm.masked.load.nxv2f16.p0(ptr, i32, <vscale x 2 x i1>, <vscale x 2 x half>)
-declare <vscale x 4 x half> @llvm.masked.load.nxv4f16.p0(ptr, i32, <vscale x 4 x i1>, <vscale x 4 x half>)
-declare <vscale x 8 x half> @llvm.masked.load.nxv8f16.p0(ptr, i32, <vscale x 8 x i1>, <vscale x 8 x half>)
-declare <vscale x 32 x half> @llvm.masked.load.nxv32f16.p0(ptr, i32, <vscale x 32 x i1>, <vscale x 32 x half>)
-declare <vscale x 2 x float> @llvm.masked.load.nxv2f32.p0(ptr, i32, <vscale x 2 x i1>, <vscale x 2 x float>)
-declare <vscale x 4 x float> @llvm.masked.load.nxv4f32.p0(ptr, i32, <vscale x 4 x i1>, <vscale x 4 x float>)
-declare <vscale x 2 x double> @llvm.masked.load.nxv2f64.p0(ptr, i32, <vscale x 2 x i1>, <vscale x 2 x double>)
+ ret void
+}
diff --git a/llvm/test/Analysis/CostModel/AArch64/sve-arith.ll b/llvm/test/Analysis/CostModel/AArch64/sve-arith.ll
index 0a9823afdeff5..ecca4fea2e520 100644
--- a/llvm/test/Analysis/CostModel/AArch64/sve-arith.ll
+++ b/llvm/test/Analysis/CostModel/AArch64/sve-arith.ll
@@ -9,7 +9,7 @@ define void @scalable_add() #0 {
; CHECK-NEXT: Cost Model: Found costs of 1 for: %nxv8i16 = add <vscale x 8 x i16> undef, undef
; CHECK-NEXT: Cost Model: Found costs of 1 for: %nxv4i32 = add <vscale x 4 x i32> undef, undef
; CHECK-NEXT: Cost Model: Found costs of 1 for: %nxv2i64 = add <vscale x 2 x i64> undef, undef
-; CHECK-NEXT: Cost Model: Found costs of Invalid for: %nxv1i64 = add <vscale x 1 x i64> undef, undef
+; CHECK-NEXT: Cost Model: Found costs of 1 for: %nxv1i64 = add <vscale x 1 x i64> undef, undef
; CHECK-NEXT: Cost Model: Found costs of Invalid for: %nxv2i128 = add <vscale x 2 x i128> undef, undef
; CHECK-NEXT: Cost Model: Found costs of RThru:0 CodeSize:1 Lat:1 SizeLat:1 for: ret void
;
@@ -30,7 +30,7 @@ define void @scalable_sub() #0 {
; CHECK-NEXT: Cost Model: Found costs of 1 for: %nxv8i16 = sub <vscale x 8 x i16> undef, undef
; CHECK-NEXT: Cost Model: Found costs of 1 for: %nxv4i32 = sub <vscale x 4 x i32> undef, undef
; CHECK-NEXT: Cost Model: Found costs of 1 for: %nxv2i64 = sub <vscale x 2 x i64> undef, undef
-; CHECK-NEXT: Cost Model: Found costs of Invalid for: %nxv1i64 = sub <vscale x 1 x i64> undef, undef
+; CHECK-NEXT: Cost Model: Found costs of 1 for: %nxv1i64 = sub <vscale x 1 x i64> undef, undef
; CHECK-NEXT: Cost Model: Found costs of Invalid for: %nxv2i128 = sub <vscale x 2 x i128> undef, undef
; CHECK-NEXT: Cost Model: Found costs of RThru:0 CodeSize:1 Lat:1 SizeLat:1 for: ret void
;
@@ -51,7 +51,7 @@ define void @scalable_mul() #0 {
; CHECK-NEXT: Cost Model: Found costs of 1 for: %nxv8i16 = mul <vscale x 8 x i16> undef, undef
; CHECK-NEXT: Cost Model: Found costs of 1 for: %nxv4i32 = mul <vscale x 4 x i32> undef, undef
; CHECK-NEXT: Cost Model: Found costs of 1 for: %nxv2i64 = mul <vscale x 2 x i64> undef, undef
-; CHECK-NEXT: Cost Model: Found costs of Invalid for: %nxv1i64 = mul <vscale x 1 x i64> undef, undef
+; CHECK-NEXT: Cost Model: Found costs of 1 for: %nxv1i64 = mul <vscale x 1 x i64> undef, undef
; CHECK-NEXT: Cost Model: Found costs of Invalid for: %nxv2i128 = mul <vscale x 2 x i128> undef, undef
; CHECK-NEXT: Cost Model: Found costs of RThru:0 CodeSize:1 Lat:1 SizeLat:1 for: ret void
;
diff --git a/llvm/test/Analysis/CostModel/AArch64/sve-ldst.ll b/llvm/test/Analysis/CostModel/AArch64/sve-ldst.ll
index b52fddaf8e776..e6e029981156b 100644
--- a/llvm/test/Analysis/CostModel/AArch64/sve-ldst.ll
+++ b/llvm/test/Analysis/CostModel/AArch64/sve-ldst.ll
@@ -9,7 +9,7 @@ define void @scalable_loads() {
; CHECK-NEXT: Cost Model: Found costs of RThru:1 CodeSize:1 Lat:4 SizeLat:1 for: %res.nxv8i8 = load <vscale x 8 x i8>, ptr undef, align 8
; CHECK-NEXT: Cost Model: Found costs of RThru:1 CodeSize:1 Lat:4 SizeLat:1 for: %res.nxv16i8 = load <vscale x 16 x i8>, ptr undef, align 16
; CHECK-NEXT: Cost Model: Found costs of RThru:2 CodeSize:2 Lat:5 SizeLat:2 for: %res.nxv32i8 = load <vscale x 32 x i8>, ptr undef, align 32
-; CHECK-NEXT: Cost Model: Found costs of Invalid for: %res.nxv1i64 = load <vscale x 1 x i64>, ptr undef, align 8
+; CHECK-NEXT: Cost Model: Found costs of 3 for: %res.nxv1i64 = load <vscale x 1 x i64>, ptr undef, align 8
; CHECK-NEXT: Cost Model: Found costs of RThru:2 CodeSize:2 Lat:5 SizeLat:2 for: %res.nxv32i1 = load <vscale x 32 x i1>, ptr undef, align 4
; CHECK-NEXT: Cost Model: Found costs of RThru:1 CodeSize:1 Lat:4 SizeLat:1 for: %res.nxv16i1 = load <vscale x 16 x i1>, ptr undef, align 2
; CHECK-NEXT: Cost Model: Found costs of Invalid for: %res.nxv4i1 = load <vscale x 4 x i1>, ptr undef, align 1
@@ -19,7 +19,7 @@ define void @scalable_loads() {
; A510-NEXT: Cost Model: Found costs of RThru:1 CodeSize:1 Lat:3 SizeLat:1 for: %res.nxv8i8 = load <vscale x 8 x i8>, ptr undef, align 8
; A510-NEXT: Cost Model: Found costs of RThru:1 CodeSize:1 Lat:3 SizeLat:1 for: %res.nxv16i8 = load <vscale x 16 x i8>, ptr undef, align 16
; A510-NEXT: Cost Model: Found costs of RThru:2 CodeSize:2 Lat:3 SizeLat:2 for: %res.nxv32i8 = load <vscale x 32 x i8>, ptr undef, align 32
-; A510-NEXT: Cost Model: Found costs of Invalid for: %res.nxv1i64 = load <vscale x 1 x i64>, ptr undef, align 8
+; A510-NEXT: Cost Model: Found costs of 3 for: %res.nxv1i64 = load <vscale x 1 x i64>, ptr undef, align 8
; A510-NEXT: Cost Model: Found costs of RThru:2 CodeSize:2 Lat:3 SizeLat:2 for: %res.nxv32i1 = load <vscale x 32 x i1>, ptr undef, align 4
; A510-NEXT: Cost Model: Found costs of RThru:1 CodeSize:1 Lat:3 SizeLat:1 for: %res.nxv16i1 = load <vscale x 16 x i1>, ptr undef, align 2
; A510-NEXT: Cost Model: Found costs of Invalid for: %res.nxv4i1 = load <vscale x 4 x i1>, ptr undef, align 1
@@ -40,7 +40,7 @@ define void @scalable_stores() {
; CHECK-NEXT: Cost Model: Found costs of 1 for: store <vscale x 8 x i8> undef, ptr undef, align 8
; CHECK-NEXT: Cost Model: Found costs of 1 for: store <vscale x 16 x i8> undef, ptr undef, align 16
; CHECK-NEXT: Cost Model: Found costs of RThru:2 CodeSize:2 Lat:1 SizeLat:2 for: store <vscale x 32 x i8> undef, ptr undef, align 32
-; CHECK-NEXT: Cost Model: Found costs of Invalid for: store <vscale x 1 x i64> undef, ptr undef, align 8
+; CHECK-NEXT: Cost Model: Found costs of 3 for: store <vscale x 1 x i64> undef, ptr undef, align 8
; CHECK-NEXT: Cost Model: Found costs of RThru:2 CodeSize:2 Lat:1 SizeLat:2 for: store <vscale x 32 x i1> undef, ptr undef, align 4
; CHECK-NEXT: Cost Model: Found costs of 1 for: store <vscale x 16 x i1> undef, ptr undef, align 2
; CHECK-NEXT: Cost Model: Found costs of Invalid for: store <vscale x 4 x i1> undef, ptr undef, align 1
@@ -50,7 +50,7 @@ define void @scalable_stores() {
; A510-NEXT: Cost Model: Found costs of 1 for: store <vscale x 8 x i8> undef, ptr undef, align 8
; A510-NEXT: Cost Model: Found costs of 1 for: store <vscale x 16 x i8> undef, ptr undef, align 16
; A510-NEXT: Cost Model: Found costs of RThru:2 CodeSize:2 Lat:1 SizeLat:2 for: store <vscale x 32 x i8> undef, ptr undef, align 32
-; A510-NEXT: Cost Model: Found costs of Invalid for: store <vscale x 1 x i64> undef, ptr undef, align 8
+; A510-NEXT: Cost Model: Found costs of 3 for: store <vscale x 1 x i64> undef, ptr undef, align 8
; A510-NEXT: Cost Model: Found costs of RThru:2 CodeSize:2 Lat:1 SizeLat:2 for: store <vscale x 32 x i1> undef, ptr undef, align 4
; A510-NEXT: Cost Model: Found costs of 1 for: store <vscale x 16 x i1> undef, ptr undef, align 2
; A510-NEXT: Cost Model: Found costs of Invalid for: store <vscale x 4 x i1> undef, ptr undef, align 1
diff --git a/llvm/test/CodeGen/AArch64/sve-int-arith.ll b/llvm/test/CodeGen/AArch64/sve-int-arith.ll
index c59b1d430ff4f..6c6c8aea46ec7 100644
--- a/llvm/test/CodeGen/AArch64/sve-int-arith.ll
+++ b/llvm/test/CodeGen/AArch64/sve-int-arith.ll
@@ -45,14 +45,28 @@ define <vscale x 16 x i8> @add_i8_zero(<vscale x 16 x i8> %a) {
ret <vscale x 16 x i8> %res
}
-define <vscale x 1 x i32> @add_nxv1i32(<vscale x 1 x i32> %a, <vscale x 1 x i32> %b) {
+define <vscale x 4 x i32> @add_nxv1i32(<vscale x 4 x i32> %a, <vscale x 4 x i32> %b) {
; CHECK-LABEL: add_nxv1i32:
-; CHECK: // %bb.0: // %entry
+; CHECK: // %bb.0:
; CHECK-NEXT: add z0.s, z0.s, z1.s
; CHECK-NEXT: ret
-entry:
- %c = add <vscale x 1 x i32> %a, %b
- ret <vscale x 1 x i32> %c
+ %a.nxv1 = call <vscale x 1 x i32> @llvm.vector.extract.nxv1i32.nxv4i32(<vscale x 4 x i32> %a, i64 0)
+ %b.nxv1 = call <vscale x 1 x i32> @llvm.vector.extract.nxv1i32.nxv4i32(<vscale x 4 x i32> %b, i64 0)
+ %res.nxv1 = add <vscale x 1 x i32> %a.nxv1, %b.nxv1
+ %res = call <vscale x 4 x i32> @llvm.vector.insert.nxv4i32.nxv1i32(<vscale x 4 x i32> poison, <v...
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/228371
More information about the llvm-commits
mailing list