[llvm] [CostModel][X86] Update AVX1/AVX2 uitofp v2i64 -> v2f32 costs based off worst case llvm-mca numbers (PR #230098)

Tim Besard via llvm-commits llvm-commits at lists.llvm.org
Thu Oct 8 05:50:08 PDT 2026


https://github.com/maleadt updated https://github.com/llvm/llvm-project/pull/230098

>From c2462a0fc8960c63983071adf9befc2849203316 Mon Sep 17 00:00:00 2001
From: Tim Besard <tim.besard at gmail.com>
Date: Thu, 8 Oct 2026 13:21:41 +0200
Subject: [PATCH 1/2] [SLP][X86] Add fdiv-seeded uitofp v2i64 -> v2f32 test
 (NFC)

---
 .../Transforms/SLPVectorizer/X86/uitofp.ll    | 25 +++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/llvm/test/Transforms/SLPVectorizer/X86/uitofp.ll b/llvm/test/Transforms/SLPVectorizer/X86/uitofp.ll
index 9b4123c0218d8..ddfc0345007cc 100644
--- a/llvm/test/Transforms/SLPVectorizer/X86/uitofp.ll
+++ b/llvm/test/Transforms/SLPVectorizer/X86/uitofp.ll
@@ -515,6 +515,31 @@ define void @uitofp_2i64_2f32() #0 {
   ret void
 }
 
+define [2 x float] @uitofp_2i64_2f32_fdiv(ptr %p, float %y) #0 {
+; CHECK-LABEL: @uitofp_2i64_2f32_fdiv(
+; CHECK-NEXT:    [[TMP1:%.*]] = load <2 x i64>, ptr [[P:%.*]], align 8
+; CHECK-NEXT:    [[TMP2:%.*]] = uitofp <2 x i64> [[TMP1]] to <2 x float>
+; CHECK-NEXT:    [[TMP3:%.*]] = insertelement <2 x float> poison, float [[Y:%.*]], i64 0
+; CHECK-NEXT:    [[TMP4:%.*]] = shufflevector <2 x float> [[TMP3]], <2 x float> poison, <2 x i32> zeroinitializer
+; CHECK-NEXT:    [[TMP5:%.*]] = fdiv <2 x float> [[TMP2]], [[TMP4]]
+; CHECK-NEXT:    [[TMP6:%.*]] = extractelement <2 x float> [[TMP5]], i64 0
+; CHECK-NEXT:    [[R0:%.*]] = insertvalue [2 x float] poison, float [[TMP6]], 0
+; CHECK-NEXT:    [[TMP7:%.*]] = extractelement <2 x float> [[TMP5]], i64 1
+; CHECK-NEXT:    [[R1:%.*]] = insertvalue [2 x float] [[R0]], float [[TMP7]], 1
+; CHECK-NEXT:    ret [2 x float] [[R1]]
+;
+  %ld0 = load i64, ptr %p, align 8
+  %p1 = getelementptr inbounds i8, ptr %p, i64 8
+  %ld1 = load i64, ptr %p1, align 8
+  %cvt0 = uitofp i64 %ld0 to float
+  %cvt1 = uitofp i64 %ld1 to float
+  %div0 = fdiv float %cvt0, %y
+  %div1 = fdiv float %cvt1, %y
+  %r0 = insertvalue [2 x float] poison, float %div0, 0
+  %r1 = insertvalue [2 x float] %r0, float %div1, 1
+  ret [2 x float] %r1
+}
+
 define void @uitofp_4i64_4f32() #0 {
 ; CHECK-LABEL: @uitofp_4i64_4f32(
 ; CHECK-NEXT:    [[TMP1:%.*]] = load <4 x i64>, ptr @src64, align 64

>From f682896d28555fd4bb060e5bf99b0ab9cc1a03bf Mon Sep 17 00:00:00 2001
From: Tim Besard <tim.besard at gmail.com>
Date: Thu, 8 Oct 2026 13:26:13 +0200
Subject: [PATCH 2/2] [CostModel][X86] Update AVX1/AVX2 uitofp v2i64 -> v2f32
 costs based off worst case llvm-mca numbers

Since #206518, SLP uses fdiv as a vectorization seed. A pair of
uitofp i64 -> float feeding fdivs by a common divisor (e.g. a complex
number with unsigned 64-bit parts divided by a float, as in Julia's
`Complex{UInt64} / Float32`) is now fully vectorized on AVX1/AVX2
targets. Without AVX512DQ, uitofp v2i64 -> v2f32 has no native
instruction: it halves large elements, converts each lane through a GPR
with cvtsi2ss and fixes the result up with blends (14 instructions).
This made the code slower than before #206518.

SLP picked that sequence because the AVX table costs it at 10, below
what it measures. Use the worst case llvm-mca throughput from
check_cost_tables.py across btver2, sandybridge, haswell, broadwell,
skylake, alderlake and znver1-3, which is 11 (haswell/broadwell). With
that, SLP keeps the conversions scalar and only vectorizes the fdiv, as
it already does on SSE4.1 targets.

Assisted-by: Claude Code, Codex
---
 .../lib/Target/X86/X86TargetTransformInfo.cpp |  2 +-
 llvm/test/Analysis/CostModel/X86/uitofp.ll    |  2 +-
 .../Transforms/SLPVectorizer/X86/uitofp.ll    | 63 +++++++++++++++----
 3 files changed, 54 insertions(+), 13 deletions(-)

diff --git a/llvm/lib/Target/X86/X86TargetTransformInfo.cpp b/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
index 0001c93fb42b6..b1bac7fe4b25a 100644
--- a/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
+++ b/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
@@ -3416,7 +3416,7 @@ InstructionCost X86TTIImpl::getCastInstrCost(unsigned Opcode, Type *Dst,
     { ISD::UINT_TO_FP,  MVT::v4f64,  MVT::v4i32,  { 6, 1, 1, 1 } },
     { ISD::UINT_TO_FP,  MVT::v8f32,  MVT::v8i32,  { 8, 1, 1, 1 } },
     { ISD::UINT_TO_FP,  MVT::v8f64,  MVT::v8i32,  {10, 1, 1, 1 } },
-    { ISD::UINT_TO_FP,  MVT::v2f32,  MVT::v2i64,  {10, 1, 1, 1 } },
+    { ISD::UINT_TO_FP,  MVT::v2f32,  MVT::v2i64,  {11, 1, 1, 1 } },
     { ISD::UINT_TO_FP,  MVT::v4f32,  MVT::v4i64,  {18, 1, 1, 1 } },
     { ISD::UINT_TO_FP,  MVT::v2f64,  MVT::v2i64,  { 5, 1, 1, 1 } },
     { ISD::UINT_TO_FP,  MVT::v4f64,  MVT::v4i64,  {10, 1, 1, 1 } },
diff --git a/llvm/test/Analysis/CostModel/X86/uitofp.ll b/llvm/test/Analysis/CostModel/X86/uitofp.ll
index 1b2a1a46fb299..28478e7fee3ce 100644
--- a/llvm/test/Analysis/CostModel/X86/uitofp.ll
+++ b/llvm/test/Analysis/CostModel/X86/uitofp.ll
@@ -348,7 +348,7 @@ define i32 @uitofp_i64_float() {
 ;
 ; AVX-LABEL: 'uitofp_i64_float'
 ; AVX-NEXT:  Cost Model: Found costs of RThru:4 CodeSize:1 Lat:1 SizeLat:1 for: %cvt_i64_f32 = uitofp i64 undef to float
-; AVX-NEXT:  Cost Model: Found costs of RThru:10 CodeSize:1 Lat:1 SizeLat:1 for: %cvt_v2i64_v2f32 = uitofp <2 x i64> undef to <2 x float>
+; AVX-NEXT:  Cost Model: Found costs of RThru:11 CodeSize:1 Lat:1 SizeLat:1 for: %cvt_v2i64_v2f32 = uitofp <2 x i64> undef to <2 x float>
 ; AVX-NEXT:  Cost Model: Found costs of RThru:18 CodeSize:1 Lat:1 SizeLat:1 for: %cvt_v4i64_v4f32 = uitofp <4 x i64> undef to <4 x float>
 ; AVX-NEXT:  Cost Model: Found costs of RThru:37 CodeSize:1 Lat:1 SizeLat:1 for: %cvt_v8i64_v8f32 = uitofp <8 x i64> undef to <8 x float>
 ; AVX-NEXT:  Cost Model: Found costs of RThru:74 CodeSize:1 Lat:1 SizeLat:1 for: %cvt_v16i64_v16f32 = uitofp <16 x i64> undef to <16 x float>
diff --git a/llvm/test/Transforms/SLPVectorizer/X86/uitofp.ll b/llvm/test/Transforms/SLPVectorizer/X86/uitofp.ll
index ddfc0345007cc..3cc9f6b08b20f 100644
--- a/llvm/test/Transforms/SLPVectorizer/X86/uitofp.ll
+++ b/llvm/test/Transforms/SLPVectorizer/X86/uitofp.ll
@@ -516,17 +516,58 @@ define void @uitofp_2i64_2f32() #0 {
 }
 
 define [2 x float] @uitofp_2i64_2f32_fdiv(ptr %p, float %y) #0 {
-; CHECK-LABEL: @uitofp_2i64_2f32_fdiv(
-; CHECK-NEXT:    [[TMP1:%.*]] = load <2 x i64>, ptr [[P:%.*]], align 8
-; CHECK-NEXT:    [[TMP2:%.*]] = uitofp <2 x i64> [[TMP1]] to <2 x float>
-; CHECK-NEXT:    [[TMP3:%.*]] = insertelement <2 x float> poison, float [[Y:%.*]], i64 0
-; CHECK-NEXT:    [[TMP4:%.*]] = shufflevector <2 x float> [[TMP3]], <2 x float> poison, <2 x i32> zeroinitializer
-; CHECK-NEXT:    [[TMP5:%.*]] = fdiv <2 x float> [[TMP2]], [[TMP4]]
-; CHECK-NEXT:    [[TMP6:%.*]] = extractelement <2 x float> [[TMP5]], i64 0
-; CHECK-NEXT:    [[R0:%.*]] = insertvalue [2 x float] poison, float [[TMP6]], 0
-; CHECK-NEXT:    [[TMP7:%.*]] = extractelement <2 x float> [[TMP5]], i64 1
-; CHECK-NEXT:    [[R1:%.*]] = insertvalue [2 x float] [[R0]], float [[TMP7]], 1
-; CHECK-NEXT:    ret [2 x float] [[R1]]
+; SSE-LABEL: @uitofp_2i64_2f32_fdiv(
+; SSE-NEXT:    [[TMP1:%.*]] = load <2 x i64>, ptr [[P:%.*]], align 8
+; SSE-NEXT:    [[TMP2:%.*]] = uitofp <2 x i64> [[TMP1]] to <2 x float>
+; SSE-NEXT:    [[TMP3:%.*]] = insertelement <2 x float> poison, float [[Y:%.*]], i64 0
+; SSE-NEXT:    [[TMP4:%.*]] = shufflevector <2 x float> [[TMP3]], <2 x float> poison, <2 x i32> zeroinitializer
+; SSE-NEXT:    [[TMP5:%.*]] = fdiv <2 x float> [[TMP2]], [[TMP4]]
+; SSE-NEXT:    [[TMP6:%.*]] = extractelement <2 x float> [[TMP5]], i64 0
+; SSE-NEXT:    [[R0:%.*]] = insertvalue [2 x float] poison, float [[TMP6]], 0
+; SSE-NEXT:    [[TMP7:%.*]] = extractelement <2 x float> [[TMP5]], i64 1
+; SSE-NEXT:    [[R1:%.*]] = insertvalue [2 x float] [[R0]], float [[TMP7]], 1
+; SSE-NEXT:    ret [2 x float] [[R1]]
+;
+; AVX256NODQ-LABEL: @uitofp_2i64_2f32_fdiv(
+; AVX256NODQ-NEXT:    [[P1:%.*]] = getelementptr inbounds i8, ptr [[P:%.*]], i64 8
+; AVX256NODQ-NEXT:    [[LD0:%.*]] = load i64, ptr [[P]], align 8
+; AVX256NODQ-NEXT:    [[LD1:%.*]] = load i64, ptr [[P1]], align 8
+; AVX256NODQ-NEXT:    [[CVT0:%.*]] = uitofp i64 [[LD0]] to float
+; AVX256NODQ-NEXT:    [[CVT1:%.*]] = uitofp i64 [[LD1]] to float
+; AVX256NODQ-NEXT:    [[TMP1:%.*]] = insertelement <2 x float> poison, float [[CVT0]], i64 0
+; AVX256NODQ-NEXT:    [[TMP2:%.*]] = insertelement <2 x float> [[TMP1]], float [[CVT1]], i64 1
+; AVX256NODQ-NEXT:    [[TMP3:%.*]] = insertelement <2 x float> poison, float [[Y:%.*]], i64 0
+; AVX256NODQ-NEXT:    [[TMP4:%.*]] = shufflevector <2 x float> [[TMP3]], <2 x float> poison, <2 x i32> zeroinitializer
+; AVX256NODQ-NEXT:    [[TMP5:%.*]] = fdiv <2 x float> [[TMP2]], [[TMP4]]
+; AVX256NODQ-NEXT:    [[TMP6:%.*]] = extractelement <2 x float> [[TMP5]], i64 0
+; AVX256NODQ-NEXT:    [[R0:%.*]] = insertvalue [2 x float] poison, float [[TMP6]], 0
+; AVX256NODQ-NEXT:    [[TMP7:%.*]] = extractelement <2 x float> [[TMP5]], i64 1
+; AVX256NODQ-NEXT:    [[R1:%.*]] = insertvalue [2 x float] [[R0]], float [[TMP7]], 1
+; AVX256NODQ-NEXT:    ret [2 x float] [[R1]]
+;
+; AVX256DQ-LABEL: @uitofp_2i64_2f32_fdiv(
+; AVX256DQ-NEXT:    [[TMP1:%.*]] = load <2 x i64>, ptr [[P:%.*]], align 8
+; AVX256DQ-NEXT:    [[TMP2:%.*]] = uitofp <2 x i64> [[TMP1]] to <2 x float>
+; AVX256DQ-NEXT:    [[TMP3:%.*]] = insertelement <2 x float> poison, float [[Y:%.*]], i64 0
+; AVX256DQ-NEXT:    [[TMP4:%.*]] = shufflevector <2 x float> [[TMP3]], <2 x float> poison, <2 x i32> zeroinitializer
+; AVX256DQ-NEXT:    [[TMP5:%.*]] = fdiv <2 x float> [[TMP2]], [[TMP4]]
+; AVX256DQ-NEXT:    [[TMP6:%.*]] = extractelement <2 x float> [[TMP5]], i64 0
+; AVX256DQ-NEXT:    [[R0:%.*]] = insertvalue [2 x float] poison, float [[TMP6]], 0
+; AVX256DQ-NEXT:    [[TMP7:%.*]] = extractelement <2 x float> [[TMP5]], i64 1
+; AVX256DQ-NEXT:    [[R1:%.*]] = insertvalue [2 x float] [[R0]], float [[TMP7]], 1
+; AVX256DQ-NEXT:    ret [2 x float] [[R1]]
+;
+; AVX512-LABEL: @uitofp_2i64_2f32_fdiv(
+; AVX512-NEXT:    [[TMP1:%.*]] = load <2 x i64>, ptr [[P:%.*]], align 8
+; AVX512-NEXT:    [[TMP2:%.*]] = uitofp <2 x i64> [[TMP1]] to <2 x float>
+; AVX512-NEXT:    [[TMP3:%.*]] = insertelement <2 x float> poison, float [[Y:%.*]], i64 0
+; AVX512-NEXT:    [[TMP4:%.*]] = shufflevector <2 x float> [[TMP3]], <2 x float> poison, <2 x i32> zeroinitializer
+; AVX512-NEXT:    [[TMP5:%.*]] = fdiv <2 x float> [[TMP2]], [[TMP4]]
+; AVX512-NEXT:    [[TMP6:%.*]] = extractelement <2 x float> [[TMP5]], i64 0
+; AVX512-NEXT:    [[R0:%.*]] = insertvalue [2 x float] poison, float [[TMP6]], 0
+; AVX512-NEXT:    [[TMP7:%.*]] = extractelement <2 x float> [[TMP5]], i64 1
+; AVX512-NEXT:    [[R1:%.*]] = insertvalue [2 x float] [[R0]], float [[TMP7]], 1
+; AVX512-NEXT:    ret [2 x float] [[R1]]
 ;
   %ld0 = load i64, ptr %p, align 8
   %p1 = getelementptr inbounds i8, ptr %p, i64 8



More information about the llvm-commits mailing list