[llvm] [AArch64] Adjust cost for min/max intrinsics using v2i8 and v2i16 (PR #212801)
Jack Styles via llvm-commits
llvm-commits at lists.llvm.org
Wed Jul 29 08:47:33 PDT 2026
https://github.com/Stylie777 updated https://github.com/llvm/llvm-project/pull/212801
>From ae7bff7ef08c7d23d977eb340887e60a7bc57055 Mon Sep 17 00:00:00 2001
From: Jack Styles <jack.styles at arm.com>
Date: Mon, 27 Jul 2026 09:46:28 +0100
Subject: [PATCH 1/2] [AArch64] Adjust cost for min/max intrinsics using v2i8
and v2i16
When vectorizing min/max intrinsics with these types, using the
changes in #195823, the Type Promotion cost outweighs that of
leaving the loop as Scalar. The costs need to be adjusted to
account for the promotion costs to ensure loops of this
kind will continue to be scalar.
---
.../AArch64/AArch64TargetTransformInfo.cpp | 6 ++++++
llvm/test/Analysis/CostModel/AArch64/min-max.ll | 16 ++++++++--------
.../LoopVectorize/AArch64/intrinsiccost.ll | 2 +-
3 files changed, 15 insertions(+), 9 deletions(-)
diff --git a/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp b/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp
index 341ac1008d7e7..cfdf07c84d159 100644
--- a/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp
+++ b/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp
@@ -711,6 +711,12 @@ AArch64TTIImpl::getIntrinsicInstrCost(const IntrinsicCostAttributes &ICA,
MVT::nxv16i8, MVT::nxv8i16, MVT::nxv4i32,
MVT::nxv2i64};
auto LT = getTypeLegalizationCost(RetTy);
+ // Type promotion for v2i8 and v2i16 types have a heavy cost when vectorising.
+ // Account for this cost to avoid vectorising unprofitable examples when vectorising
+ // loops with low trip counts.
+ MVT VT = MVT::getVT(RetTy);
+ if (VT == MVT::v2i8 || VT == MVT::v2i16)
+ return LT.first * 6;
// v2i64 types get converted to cmp+bif hence the cost of 2
if (LT.second == MVT::v2i64)
return LT.first * 2;
diff --git a/llvm/test/Analysis/CostModel/AArch64/min-max.ll b/llvm/test/Analysis/CostModel/AArch64/min-max.ll
index 5e5026ccfa4df..e04898634ae6f 100644
--- a/llvm/test/Analysis/CostModel/AArch64/min-max.ll
+++ b/llvm/test/Analysis/CostModel/AArch64/min-max.ll
@@ -84,7 +84,7 @@ define void @umin() {
; CHECK-BASE-NEXT: Cost Model: Found costs of 1 for: %V16i8 = call <16 x i8> @llvm.umin.v16i8(<16 x i8> undef, <16 x i8> undef)
; CHECK-BASE-NEXT: Cost Model: Found costs of 2 for: %V32i8 = call <32 x i8> @llvm.umin.v32i8(<32 x i8> undef, <32 x i8> undef)
; CHECK-BASE-NEXT: Cost Model: Found costs of 4 for: %V64i8 = call <64 x i8> @llvm.umin.v64i8(<64 x i8> undef, <64 x i8> undef)
-; CHECK-BASE-NEXT: Cost Model: Found costs of 1 for: %V2i16 = call <2 x i16> @llvm.umin.v2i16(<2 x i16> undef, <2 x i16> undef)
+; CHECK-BASE-NEXT: Cost Model: Found costs of 6 for: %V2i16 = call <2 x i16> @llvm.umin.v2i16(<2 x i16> undef, <2 x i16> undef)
; CHECK-BASE-NEXT: Cost Model: Found costs of 1 for: %V4i16 = call <4 x i16> @llvm.umin.v4i16(<4 x i16> undef, <4 x i16> undef)
; CHECK-BASE-NEXT: Cost Model: Found costs of 1 for: %V8i16 = call <8 x i16> @llvm.umin.v8i16(<8 x i16> undef, <8 x i16> undef)
; CHECK-BASE-NEXT: Cost Model: Found costs of 2 for: %V16i16 = call <16 x i16> @llvm.umin.v16i16(<16 x i16> undef, <16 x i16> undef)
@@ -104,7 +104,7 @@ define void @umin() {
; CHECK-CSSC-NEXT: Cost Model: Found costs of 1 for: %V16i8 = call <16 x i8> @llvm.umin.v16i8(<16 x i8> undef, <16 x i8> undef)
; CHECK-CSSC-NEXT: Cost Model: Found costs of 2 for: %V32i8 = call <32 x i8> @llvm.umin.v32i8(<32 x i8> undef, <32 x i8> undef)
; CHECK-CSSC-NEXT: Cost Model: Found costs of 4 for: %V64i8 = call <64 x i8> @llvm.umin.v64i8(<64 x i8> undef, <64 x i8> undef)
-; CHECK-CSSC-NEXT: Cost Model: Found costs of 1 for: %V2i16 = call <2 x i16> @llvm.umin.v2i16(<2 x i16> undef, <2 x i16> undef)
+; CHECK-CSSC-NEXT: Cost Model: Found costs of 6 for: %V2i16 = call <2 x i16> @llvm.umin.v2i16(<2 x i16> undef, <2 x i16> undef)
; CHECK-CSSC-NEXT: Cost Model: Found costs of 1 for: %V4i16 = call <4 x i16> @llvm.umin.v4i16(<4 x i16> undef, <4 x i16> undef)
; CHECK-CSSC-NEXT: Cost Model: Found costs of 1 for: %V8i16 = call <8 x i16> @llvm.umin.v8i16(<8 x i16> undef, <8 x i16> undef)
; CHECK-CSSC-NEXT: Cost Model: Found costs of 2 for: %V16i16 = call <16 x i16> @llvm.umin.v16i16(<16 x i16> undef, <16 x i16> undef)
@@ -145,7 +145,7 @@ define void @umax() {
; CHECK-BASE-NEXT: Cost Model: Found costs of 1 for: %V16i8 = call <16 x i8> @llvm.umax.v16i8(<16 x i8> undef, <16 x i8> undef)
; CHECK-BASE-NEXT: Cost Model: Found costs of 2 for: %V32i8 = call <32 x i8> @llvm.umax.v32i8(<32 x i8> undef, <32 x i8> undef)
; CHECK-BASE-NEXT: Cost Model: Found costs of 4 for: %V64i8 = call <64 x i8> @llvm.umax.v64i8(<64 x i8> undef, <64 x i8> undef)
-; CHECK-BASE-NEXT: Cost Model: Found costs of 1 for: %V2i16 = call <2 x i16> @llvm.umax.v2i16(<2 x i16> undef, <2 x i16> undef)
+; CHECK-BASE-NEXT: Cost Model: Found costs of 6 for: %V2i16 = call <2 x i16> @llvm.umax.v2i16(<2 x i16> undef, <2 x i16> undef)
; CHECK-BASE-NEXT: Cost Model: Found costs of 1 for: %V4i16 = call <4 x i16> @llvm.umax.v4i16(<4 x i16> undef, <4 x i16> undef)
; CHECK-BASE-NEXT: Cost Model: Found costs of 1 for: %V8i16 = call <8 x i16> @llvm.umax.v8i16(<8 x i16> undef, <8 x i16> undef)
; CHECK-BASE-NEXT: Cost Model: Found costs of 2 for: %V16i16 = call <16 x i16> @llvm.umax.v16i16(<16 x i16> undef, <16 x i16> undef)
@@ -165,7 +165,7 @@ define void @umax() {
; CHECK-CSSC-NEXT: Cost Model: Found costs of 1 for: %V16i8 = call <16 x i8> @llvm.umax.v16i8(<16 x i8> undef, <16 x i8> undef)
; CHECK-CSSC-NEXT: Cost Model: Found costs of 2 for: %V32i8 = call <32 x i8> @llvm.umax.v32i8(<32 x i8> undef, <32 x i8> undef)
; CHECK-CSSC-NEXT: Cost Model: Found costs of 4 for: %V64i8 = call <64 x i8> @llvm.umax.v64i8(<64 x i8> undef, <64 x i8> undef)
-; CHECK-CSSC-NEXT: Cost Model: Found costs of 1 for: %V2i16 = call <2 x i16> @llvm.umax.v2i16(<2 x i16> undef, <2 x i16> undef)
+; CHECK-CSSC-NEXT: Cost Model: Found costs of 6 for: %V2i16 = call <2 x i16> @llvm.umax.v2i16(<2 x i16> undef, <2 x i16> undef)
; CHECK-CSSC-NEXT: Cost Model: Found costs of 1 for: %V4i16 = call <4 x i16> @llvm.umax.v4i16(<4 x i16> undef, <4 x i16> undef)
; CHECK-CSSC-NEXT: Cost Model: Found costs of 1 for: %V8i16 = call <8 x i16> @llvm.umax.v8i16(<8 x i16> undef, <8 x i16> undef)
; CHECK-CSSC-NEXT: Cost Model: Found costs of 2 for: %V16i16 = call <16 x i16> @llvm.umax.v16i16(<16 x i16> undef, <16 x i16> undef)
@@ -206,7 +206,7 @@ define void @smin() {
; CHECK-BASE-NEXT: Cost Model: Found costs of 1 for: %V16i8 = call <16 x i8> @llvm.smin.v16i8(<16 x i8> undef, <16 x i8> undef)
; CHECK-BASE-NEXT: Cost Model: Found costs of 2 for: %V32i8 = call <32 x i8> @llvm.smin.v32i8(<32 x i8> undef, <32 x i8> undef)
; CHECK-BASE-NEXT: Cost Model: Found costs of 4 for: %V64i8 = call <64 x i8> @llvm.smin.v64i8(<64 x i8> undef, <64 x i8> undef)
-; CHECK-BASE-NEXT: Cost Model: Found costs of 1 for: %V2i16 = call <2 x i16> @llvm.smin.v2i16(<2 x i16> undef, <2 x i16> undef)
+; CHECK-BASE-NEXT: Cost Model: Found costs of 6 for: %V2i16 = call <2 x i16> @llvm.smin.v2i16(<2 x i16> undef, <2 x i16> undef)
; CHECK-BASE-NEXT: Cost Model: Found costs of 1 for: %V4i16 = call <4 x i16> @llvm.smin.v4i16(<4 x i16> undef, <4 x i16> undef)
; CHECK-BASE-NEXT: Cost Model: Found costs of 1 for: %V8i16 = call <8 x i16> @llvm.smin.v8i16(<8 x i16> undef, <8 x i16> undef)
; CHECK-BASE-NEXT: Cost Model: Found costs of 2 for: %V16i16 = call <16 x i16> @llvm.smin.v16i16(<16 x i16> undef, <16 x i16> undef)
@@ -226,7 +226,7 @@ define void @smin() {
; CHECK-CSSC-NEXT: Cost Model: Found costs of 1 for: %V16i8 = call <16 x i8> @llvm.smin.v16i8(<16 x i8> undef, <16 x i8> undef)
; CHECK-CSSC-NEXT: Cost Model: Found costs of 2 for: %V32i8 = call <32 x i8> @llvm.smin.v32i8(<32 x i8> undef, <32 x i8> undef)
; CHECK-CSSC-NEXT: Cost Model: Found costs of 4 for: %V64i8 = call <64 x i8> @llvm.smin.v64i8(<64 x i8> undef, <64 x i8> undef)
-; CHECK-CSSC-NEXT: Cost Model: Found costs of 1 for: %V2i16 = call <2 x i16> @llvm.smin.v2i16(<2 x i16> undef, <2 x i16> undef)
+; CHECK-CSSC-NEXT: Cost Model: Found costs of 6 for: %V2i16 = call <2 x i16> @llvm.smin.v2i16(<2 x i16> undef, <2 x i16> undef)
; CHECK-CSSC-NEXT: Cost Model: Found costs of 1 for: %V4i16 = call <4 x i16> @llvm.smin.v4i16(<4 x i16> undef, <4 x i16> undef)
; CHECK-CSSC-NEXT: Cost Model: Found costs of 1 for: %V8i16 = call <8 x i16> @llvm.smin.v8i16(<8 x i16> undef, <8 x i16> undef)
; CHECK-CSSC-NEXT: Cost Model: Found costs of 2 for: %V16i16 = call <16 x i16> @llvm.smin.v16i16(<16 x i16> undef, <16 x i16> undef)
@@ -267,7 +267,7 @@ define void @smax() {
; CHECK-BASE-NEXT: Cost Model: Found costs of 1 for: %V16i8 = call <16 x i8> @llvm.smax.v16i8(<16 x i8> undef, <16 x i8> undef)
; CHECK-BASE-NEXT: Cost Model: Found costs of 2 for: %V32i8 = call <32 x i8> @llvm.smax.v32i8(<32 x i8> undef, <32 x i8> undef)
; CHECK-BASE-NEXT: Cost Model: Found costs of 4 for: %V64i8 = call <64 x i8> @llvm.smax.v64i8(<64 x i8> undef, <64 x i8> undef)
-; CHECK-BASE-NEXT: Cost Model: Found costs of 1 for: %V2i16 = call <2 x i16> @llvm.smax.v2i16(<2 x i16> undef, <2 x i16> undef)
+; CHECK-BASE-NEXT: Cost Model: Found costs of 6 for: %V2i16 = call <2 x i16> @llvm.smax.v2i16(<2 x i16> undef, <2 x i16> undef)
; CHECK-BASE-NEXT: Cost Model: Found costs of 1 for: %V4i16 = call <4 x i16> @llvm.smax.v4i16(<4 x i16> undef, <4 x i16> undef)
; CHECK-BASE-NEXT: Cost Model: Found costs of 1 for: %V8i16 = call <8 x i16> @llvm.smax.v8i16(<8 x i16> undef, <8 x i16> undef)
; CHECK-BASE-NEXT: Cost Model: Found costs of 2 for: %V16i16 = call <16 x i16> @llvm.smax.v16i16(<16 x i16> undef, <16 x i16> undef)
@@ -287,7 +287,7 @@ define void @smax() {
; CHECK-CSSC-NEXT: Cost Model: Found costs of 1 for: %V16i8 = call <16 x i8> @llvm.smax.v16i8(<16 x i8> undef, <16 x i8> undef)
; CHECK-CSSC-NEXT: Cost Model: Found costs of 2 for: %V32i8 = call <32 x i8> @llvm.smax.v32i8(<32 x i8> undef, <32 x i8> undef)
; CHECK-CSSC-NEXT: Cost Model: Found costs of 4 for: %V64i8 = call <64 x i8> @llvm.smax.v64i8(<64 x i8> undef, <64 x i8> undef)
-; CHECK-CSSC-NEXT: Cost Model: Found costs of 1 for: %V2i16 = call <2 x i16> @llvm.smax.v2i16(<2 x i16> undef, <2 x i16> undef)
+; CHECK-CSSC-NEXT: Cost Model: Found costs of 6 for: %V2i16 = call <2 x i16> @llvm.smax.v2i16(<2 x i16> undef, <2 x i16> undef)
; CHECK-CSSC-NEXT: Cost Model: Found costs of 1 for: %V4i16 = call <4 x i16> @llvm.smax.v4i16(<4 x i16> undef, <4 x i16> undef)
; CHECK-CSSC-NEXT: Cost Model: Found costs of 1 for: %V8i16 = call <8 x i16> @llvm.smax.v8i16(<8 x i16> undef, <8 x i16> undef)
; CHECK-CSSC-NEXT: Cost Model: Found costs of 2 for: %V16i16 = call <16 x i16> @llvm.smax.v16i16(<16 x i16> undef, <16 x i16> undef)
diff --git a/llvm/test/Transforms/LoopVectorize/AArch64/intrinsiccost.ll b/llvm/test/Transforms/LoopVectorize/AArch64/intrinsiccost.ll
index c8555a8629ec5..c2f5ca4e0538f 100644
--- a/llvm/test/Transforms/LoopVectorize/AArch64/intrinsiccost.ll
+++ b/llvm/test/Transforms/LoopVectorize/AArch64/intrinsiccost.ll
@@ -128,7 +128,7 @@ while.end:
; CHECK-COST-LABEL: umin
; CHECK-COST: Found an estimated cost of 2 for VF 1 For instruction: %1 = tail call i8 @llvm.umin.i8(i8 %0, i8 %offset)
-; CHECK-COST: Cost of 1 for VF 2: WIDEN-INTRINSIC ir<%1> = call llvm.umin(ir<%0>, ir<%offset>)
+; CHECK-COST: Cost of 6 for VF 2: WIDEN-INTRINSIC ir<%1> = call llvm.umin(ir<%0>, ir<%offset>)
; CHECK-COST: Cost of 1 for VF 4: WIDEN-INTRINSIC ir<%1> = call llvm.umin(ir<%0>, ir<%offset>)
; CHECK-COST: Cost of 1 for VF 8: WIDEN-INTRINSIC ir<%1> = call llvm.umin(ir<%0>, ir<%offset>)
; CHECK-COST: Cost of 1 for VF 16: WIDEN-INTRINSIC ir<%1> = call llvm.umin(ir<%0>, ir<%offset>)
>From 13e921b20da19ea1c5ec53c0c0af9532f6cb3a1d Mon Sep 17 00:00:00 2001
From: Jack Styles <jack.styles at arm.com>
Date: Wed, 29 Jul 2026 16:47:15 +0100
Subject: [PATCH 2/2] format
---
llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp b/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp
index cfdf07c84d159..d2c1eb2aa2867 100644
--- a/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp
+++ b/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp
@@ -711,9 +711,9 @@ AArch64TTIImpl::getIntrinsicInstrCost(const IntrinsicCostAttributes &ICA,
MVT::nxv16i8, MVT::nxv8i16, MVT::nxv4i32,
MVT::nxv2i64};
auto LT = getTypeLegalizationCost(RetTy);
- // Type promotion for v2i8 and v2i16 types have a heavy cost when vectorising.
- // Account for this cost to avoid vectorising unprofitable examples when vectorising
- // loops with low trip counts.
+ // Type promotion for v2i8 and v2i16 types have a heavy cost when
+ // vectorising. Account for this cost to avoid vectorising unprofitable
+ // examples when vectorising loops with low trip counts.
MVT VT = MVT::getVT(RetTy);
if (VT == MVT::v2i8 || VT == MVT::v2i16)
return LT.first * 6;
More information about the llvm-commits
mailing list