[llvm-branch-commits] [llvm] [AMDGPU] Treat a bitcast between integer and FP types as free (PR #238412)

via llvm-branch-commits llvm-branch-commits at lists.llvm.org
Tue Nov 3 07:46:39 PST 2026


https://github.com/michaelselehov created https://github.com/llvm/llvm-project/pull/238412

The generic cost model treats a bitcast as free only when both types are integer types or both are not. AMDGPU keeps integer and floating-point values in the same registers, so a bitcast between them is a no-op when the sizes match. A scalar bitcast from i32 to float cost 1, but the vector bitcast from <2 x i32> to <2 x float> cost 0. The SLP vectorizer counted this difference as a saving. In a rocFFT kernel this saving made it vectorize trees that gave no real gain and added register pressure.

Bitcasts that involve pointers or i1 keep their costs.

Assisted-by: Claude Opus



>From f2eb1c8a00d70c38fa932ab4832a9f0bcd9114c3 Mon Sep 17 00:00:00 2001
From: Michael Selehov <michael.selehov at amd.com>
Date: Mon, 2 Nov 2026 05:48:30 -0600
Subject: [PATCH] [AMDGPU] Treat a bitcast between integer and FP types as free

The generic cost model treats a bitcast as free only when both types
are integer types or both are not. AMDGPU keeps integer and
floating-point values in the same registers, so a bitcast between them
is a no-op when the sizes match. A scalar bitcast from i32 to float
cost 1, but the vector bitcast from <2 x i32> to <2 x float> cost 0.
The SLP vectorizer counted this difference as a saving. In a rocFFT
kernel this saving made it vectorize trees that gave no real gain and
added register pressure.

Bitcasts that involve pointers or i1 keep their costs.

Assisted-by: Claude Opus
---
 .../AMDGPU/AMDGPUTargetTransformInfo.cpp      |  6 ++
 .../CostModel/AMDGPU/bitcast-int-fp.ll        | 56 +++++++++----------
 .../SLPVectorizer/AMDGPU/bitcast-int-fp.ll    |  7 ++-
 3 files changed, 38 insertions(+), 31 deletions(-)

diff --git a/llvm/lib/Target/AMDGPU/AMDGPUTargetTransformInfo.cpp b/llvm/lib/Target/AMDGPU/AMDGPUTargetTransformInfo.cpp
index db9b5c6c1ef7f7..c6291cb281d857 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPUTargetTransformInfo.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPUTargetTransformInfo.cpp
@@ -1066,6 +1066,12 @@ InstructionCost GCNTTIImpl::getCastInstrCost(unsigned Opcode, Type *Dst,
         Elts && Src->isIntegerTy(*Elts))
       return InstructionCost(MaskUnpackCostPerElt) * *Elts *
              getFullRateInstrCost();
+    // Integer and floating-point values share the same registers.
+    if (Src->getPrimitiveSizeInBits() == Dst->getPrimitiveSizeInBits() &&
+        !Src->isPtrOrPtrVectorTy() && !Dst->isPtrOrPtrVectorTy() &&
+        !Src->getScalarType()->isIntegerTy(1) &&
+        !Dst->getScalarType()->isIntegerTy(1))
+      return 0;
   }
 
   // Each f32 lane is rounded and the halves are packed in pairs. A packed
diff --git a/llvm/test/Analysis/CostModel/AMDGPU/bitcast-int-fp.ll b/llvm/test/Analysis/CostModel/AMDGPU/bitcast-int-fp.ll
index 2f1dd4fc9ee49e..2bcf0727ef5f4d 100644
--- a/llvm/test/Analysis/CostModel/AMDGPU/bitcast-int-fp.ll
+++ b/llvm/test/Analysis/CostModel/AMDGPU/bitcast-int-fp.ll
@@ -11,25 +11,25 @@
 
 define void @scalar() {
 ; ALL-LABEL: 'scalar'
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %I16toF16 = bitcast i16 poison to half
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %F16toI16 = bitcast half poison to i16
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %I16toBF16 = bitcast i16 poison to bfloat
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %BF16toI16 = bitcast bfloat poison to i16
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %I32toF32 = bitcast i32 poison to float
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %F32toI32 = bitcast float poison to i32
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %I64toF64 = bitcast i64 poison to double
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %F64toI64 = bitcast double poison to i64
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %I16toF16 = bitcast i16 poison to half
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %F16toI16 = bitcast half poison to i16
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %I16toBF16 = bitcast i16 poison to bfloat
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %BF16toI16 = bitcast bfloat poison to i16
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %I32toF32 = bitcast i32 poison to float
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %F32toI32 = bitcast float poison to i32
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %I64toF64 = bitcast i64 poison to double
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %F64toI64 = bitcast double poison to i64
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: ret void
 ;
 ; ALL-SIZE-LABEL: 'scalar'
-; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %I16toF16 = bitcast i16 poison to half
-; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %F16toI16 = bitcast half poison to i16
-; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %I16toBF16 = bitcast i16 poison to bfloat
-; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %BF16toI16 = bitcast bfloat poison to i16
-; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %I32toF32 = bitcast i32 poison to float
-; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %F32toI32 = bitcast float poison to i32
-; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %I64toF64 = bitcast i64 poison to double
-; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %F64toI64 = bitcast double poison to i64
+; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %I16toF16 = bitcast i16 poison to half
+; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %F16toI16 = bitcast half poison to i16
+; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %I16toBF16 = bitcast i16 poison to bfloat
+; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %BF16toI16 = bitcast bfloat poison to i16
+; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %I32toF32 = bitcast i32 poison to float
+; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %F32toI32 = bitcast float poison to i32
+; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %I64toF64 = bitcast i64 poison to double
+; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %F64toI64 = bitcast double poison to i64
 ; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
   %I16toF16 = bitcast i16 poison to half
@@ -45,27 +45,27 @@ define void @scalar() {
 
 define void @scalar_vector() {
 ; ALL-LABEL: 'scalar_vector'
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %I32toV2I16 = bitcast i32 poison to <2 x i16>
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %I32toV2F16 = bitcast i32 poison to <2 x half>
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %V2F16toI32 = bitcast <2 x half> poison to i32
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %I32toV2I16 = bitcast i32 poison to <2 x i16>
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %I32toV2F16 = bitcast i32 poison to <2 x half>
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %V2F16toI32 = bitcast <2 x half> poison to i32
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %V4I8toI32 = bitcast <4 x i8> poison to i32
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %F32toV2I16 = bitcast float poison to <2 x i16>
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %I64toV2F32 = bitcast i64 poison to <2 x float>
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %V2F32toI64 = bitcast <2 x float> poison to i64
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %I64toV2I32 = bitcast i64 poison to <2 x i32>
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %I64toV2F32 = bitcast i64 poison to <2 x float>
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %V2F32toI64 = bitcast <2 x float> poison to i64
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %I64toV2I32 = bitcast i64 poison to <2 x i32>
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %F64toV2F32 = bitcast double poison to <2 x float>
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %I128toV4F32 = bitcast i128 poison to <4 x float>
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: ret void
 ;
 ; ALL-SIZE-LABEL: 'scalar_vector'
-; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %I32toV2I16 = bitcast i32 poison to <2 x i16>
-; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %I32toV2F16 = bitcast i32 poison to <2 x half>
-; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %V2F16toI32 = bitcast <2 x half> poison to i32
+; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %I32toV2I16 = bitcast i32 poison to <2 x i16>
+; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %I32toV2F16 = bitcast i32 poison to <2 x half>
+; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %V2F16toI32 = bitcast <2 x half> poison to i32
 ; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %V4I8toI32 = bitcast <4 x i8> poison to i32
 ; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %F32toV2I16 = bitcast float poison to <2 x i16>
-; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %I64toV2F32 = bitcast i64 poison to <2 x float>
-; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %V2F32toI64 = bitcast <2 x float> poison to i64
-; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %I64toV2I32 = bitcast i64 poison to <2 x i32>
+; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %I64toV2F32 = bitcast i64 poison to <2 x float>
+; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %V2F32toI64 = bitcast <2 x float> poison to i64
+; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %I64toV2I32 = bitcast i64 poison to <2 x i32>
 ; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %F64toV2F32 = bitcast double poison to <2 x float>
 ; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %I128toV4F32 = bitcast i128 poison to <4 x float>
 ; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
diff --git a/llvm/test/Transforms/SLPVectorizer/AMDGPU/bitcast-int-fp.ll b/llvm/test/Transforms/SLPVectorizer/AMDGPU/bitcast-int-fp.ll
index 2799b1d0b9529e..1d6457b9ef59bc 100644
--- a/llvm/test/Transforms/SLPVectorizer/AMDGPU/bitcast-int-fp.ll
+++ b/llvm/test/Transforms/SLPVectorizer/AMDGPU/bitcast-int-fp.ll
@@ -20,9 +20,10 @@ define void @bitcast_only(i32 %a, i32 %b, ptr addrspace(3) %out) {
 ;
 ; PACKED-LABEL: define void @bitcast_only(
 ; PACKED-SAME: i32 [[A:%.*]], i32 [[B:%.*]], ptr addrspace(3) [[OUT:%.*]]) #[[ATTR0:[0-9]+]] {
-; PACKED-NEXT:    [[TMP1:%.*]] = insertelement <2 x i32> poison, i32 [[A]], i64 0
-; PACKED-NEXT:    [[TMP2:%.*]] = insertelement <2 x i32> [[TMP1]], i32 [[B]], i64 1
-; PACKED-NEXT:    [[TMP3:%.*]] = bitcast <2 x i32> [[TMP2]] to <2 x float>
+; PACKED-NEXT:    [[AF:%.*]] = bitcast i32 [[A]] to float
+; PACKED-NEXT:    [[BF:%.*]] = bitcast i32 [[B]] to float
+; PACKED-NEXT:    [[R0:%.*]] = insertelement <2 x float> poison, float [[AF]], i64 0
+; PACKED-NEXT:    [[TMP3:%.*]] = insertelement <2 x float> [[R0]], float [[BF]], i64 1
 ; PACKED-NEXT:    store <2 x float> [[TMP3]], ptr addrspace(3) [[OUT]], align 8
 ; PACKED-NEXT:    ret void
 ;



More information about the llvm-branch-commits mailing list