[llvm-branch-commits] [llvm] [AMDGPU] Price misaligned flat and LDS accesses with the LDS misaligned bug (PR #238843)
Dmitry Sidorov via llvm-branch-commits
llvm-branch-commits at lists.llvm.org
Sun Nov 8 12:35:25 PST 2026
https://github.com/MrSidims updated https://github.com/llvm/llvm-project/pull/238843
>From 5878ac47f4ababd791825e9fa59dc482bb30aade Mon Sep 17 00:00:00 2001
From: Dmitry Sidorov <Dmitry.Sidorov at amd.com>
Date: Wed, 4 Nov 2026 14:55:19 +0100
Subject: [PATCH] [AMDGPU] Price misaligned flat and LDS accesses with the LDS
misaligned bug
In WGP mode on subtargets with the LDS misaligned bug, a flat access is
split into pieces no wider than its alignment and no narrower than 4
bytes, and an LDS access aligned below 4 bytes uses 8-byte pieces as one
aligned to 4 does.
---
.../AMDGPU/AMDGPUTargetTransformInfo.cpp | 5 +-
llvm/test/Analysis/CostModel/AMDGPU/load.ll | 181 ++++++++++++------
llvm/test/Analysis/CostModel/AMDGPU/store.ll | 181 ++++++++++++------
3 files changed, 246 insertions(+), 121 deletions(-)
diff --git a/llvm/lib/Target/AMDGPU/AMDGPUTargetTransformInfo.cpp b/llvm/lib/Target/AMDGPU/AMDGPUTargetTransformInfo.cpp
index d32bda2cd88785..94c0023691a414 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPUTargetTransformInfo.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPUTargetTransformInfo.cpp
@@ -2329,6 +2329,9 @@ InstructionCost GCNTTIImpl::getMemoryOpCost(unsigned Opcode, Type *Src,
if (AddressSpace == AMDGPUAS::FLAT_ADDRESS &&
!ST->hasMultiDwordFlatScratchAddressing())
MaxBytes = getLoadStoreVecRegBitWidth(AMDGPUAS::PRIVATE_ADDRESS) / 8;
+ if (AddressSpace == AMDGPUAS::FLAT_ADDRESS &&
+ ST->hasLDSMisalignedBugInWGPMode())
+ MaxBytes = std::clamp<uint64_t>(Alignment.value(), 4, MaxBytes);
bool IsDS = AddressSpace == AMDGPUAS::LOCAL_ADDRESS ||
AddressSpace == AMDGPUAS::REGION_ADDRESS;
bool HasDwordx3 = ST->hasDwordx3LoadStores();
@@ -2338,7 +2341,7 @@ InstructionCost GCNTTIImpl::getMemoryOpCost(unsigned Opcode, Type *Src,
HasUnalignedAccess = ST->hasUnalignedScratchAccessEnabled();
else if (IsDS) {
HasUnalignedAccess = ST->hasUnalignedDSAccessEnabled();
- if (Alignment >= 4 && Alignment < 8)
+ if ((Alignment >= 4 || ST->hasLDSMisalignedBugInWGPMode()) && Alignment < 8)
MaxBytes = 8;
HasDwordx3 = HasDwordx3 &&
(Alignment >= 16 || (Alignment < 4 && HasUnalignedAccess));
diff --git a/llvm/test/Analysis/CostModel/AMDGPU/load.ll b/llvm/test/Analysis/CostModel/AMDGPU/load.ll
index 711732c38cf0ec..7b13623c98e37b 100644
--- a/llvm/test/Analysis/CostModel/AMDGPU/load.ll
+++ b/llvm/test/Analysis/CostModel/AMDGPU/load.ll
@@ -2,8 +2,8 @@
; RUN: opt -passes="print<cost-model>" -disable-output -mtriple=amdgpu8.03-unknown-amdhsa < %s 2>&1 | FileCheck %s --check-prefixes=CHECK,GFX8
; RUN: opt -passes="print<cost-model>" -disable-output -mtriple=amdgpu9.0a-unknown-amdhsa < %s 2>&1 | FileCheck %s --check-prefixes=CHECK,GFX90A,GFX90A-UNALIGNED
; RUN: opt -passes="print<cost-model>" -disable-output -mtriple=amdgpu9.0a-unknown-amdhsa -mattr=-unaligned-access-mode < %s 2>&1 | FileCheck %s --check-prefixes=CHECK,GFX90A,GFX90A-NOUNALIGNED
-; RUN: opt -passes="print<cost-model>" -disable-output -mtriple=amdgpu10.10-unknown-amdhsa < %s 2>&1 | FileCheck %s --check-prefixes=CHECK,GFX10
-; RUN: opt -passes="print<cost-model>" -disable-output -mtriple=amdgpu10.10-unknown-amdhsa -mattr=+cumode < %s 2>&1 | FileCheck %s --check-prefixes=CHECK,GFX10
+; RUN: opt -passes="print<cost-model>" -disable-output -mtriple=amdgpu10.10-unknown-amdhsa < %s 2>&1 | FileCheck %s --check-prefixes=CHECK,GFX10,GFX10-WGP
+; RUN: opt -passes="print<cost-model>" -disable-output -mtriple=amdgpu10.10-unknown-amdhsa -mattr=+cumode < %s 2>&1 | FileCheck %s --check-prefixes=CHECK,GFX10,GFX10-CU
; RUN: opt -passes="print<cost-model>" -disable-output -mtriple=amdgpu12.01-unknown-amdhsa < %s 2>&1 | FileCheck %s --check-prefixes=CHECK,GFX12
; RUN: opt -passes="print<cost-model>" -disable-output -mtriple=amdgpu12.50-unknown-amdhsa < %s 2>&1 | FileCheck %s --check-prefixes=CHECK,GFX12
@@ -178,24 +178,43 @@ define void @loads_i16(i32 %arg) {
; GFX90A-NOUNALIGNED-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %16 = load <4 x i16>, ptr poison, align 8
; GFX90A-NOUNALIGNED-NEXT: Cost Model: Found an estimated cost of 10 for instruction: ret void
;
-; GFX10-LABEL: 'loads_i16'
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %1 = load i16, ptr poison, align 2
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %2 = load <2 x i16>, ptr poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %3 = load <3 x i16>, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %4 = load <4 x i16>, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %5 = load i16, ptr poison, align 1
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %6 = load <2 x i16>, ptr poison, align 1
-; GFX10-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %7 = load <3 x i16>, ptr poison, align 1
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %8 = load <4 x i16>, ptr poison, align 1
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %9 = load i16, ptr poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %10 = load <2 x i16>, ptr poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %11 = load <3 x i16>, ptr poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %12 = load <4 x i16>, ptr poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %13 = load i16, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %14 = load <2 x i16>, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %15 = load <3 x i16>, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %16 = load <4 x i16>, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 10 for instruction: ret void
+; GFX10-WGP-LABEL: 'loads_i16'
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %1 = load i16, ptr poison, align 2
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %2 = load <2 x i16>, ptr poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %3 = load <3 x i16>, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %4 = load <4 x i16>, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %5 = load i16, ptr poison, align 1
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %6 = load <2 x i16>, ptr poison, align 1
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %7 = load <3 x i16>, ptr poison, align 1
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %8 = load <4 x i16>, ptr poison, align 1
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %9 = load i16, ptr poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %10 = load <2 x i16>, ptr poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %11 = load <3 x i16>, ptr poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %12 = load <4 x i16>, ptr poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %13 = load i16, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %14 = load <2 x i16>, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %15 = load <3 x i16>, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %16 = load <4 x i16>, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 10 for instruction: ret void
+;
+; GFX10-CU-LABEL: 'loads_i16'
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %1 = load i16, ptr poison, align 2
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %2 = load <2 x i16>, ptr poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %3 = load <3 x i16>, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %4 = load <4 x i16>, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %5 = load i16, ptr poison, align 1
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %6 = load <2 x i16>, ptr poison, align 1
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %7 = load <3 x i16>, ptr poison, align 1
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %8 = load <4 x i16>, ptr poison, align 1
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %9 = load i16, ptr poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %10 = load <2 x i16>, ptr poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %11 = load <3 x i16>, ptr poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %12 = load <4 x i16>, ptr poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %13 = load i16, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %14 = load <2 x i16>, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %15 = load <3 x i16>, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %16 = load <4 x i16>, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 10 for instruction: ret void
;
; GFX12-LABEL: 'loads_i16'
; GFX12-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %1 = load i16, ptr poison, align 2
@@ -309,28 +328,51 @@ define void @loads_i32(i32 %arg) {
; GFX90A-NOUNALIGNED-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %20 = load <4 x i32>, ptr poison, align 16
; GFX90A-NOUNALIGNED-NEXT: Cost Model: Found an estimated cost of 10 for instruction: ret void
;
-; GFX10-LABEL: 'loads_i32'
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %1 = load i32, ptr poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %2 = load <2 x i32>, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %3 = load <3 x i32>, ptr poison, align 16
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %4 = load <4 x i32>, ptr poison, align 16
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %5 = load i32, ptr poison, align 1
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %6 = load <2 x i32>, ptr poison, align 1
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %7 = load <3 x i32>, ptr poison, align 1
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %8 = load <4 x i32>, ptr poison, align 1
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %9 = load i32, ptr poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %10 = load <2 x i32>, ptr poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %11 = load <3 x i32>, ptr poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %12 = load <4 x i32>, ptr poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %13 = load i32, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %14 = load <2 x i32>, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %15 = load <3 x i32>, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %16 = load <4 x i32>, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %17 = load i32, ptr poison, align 16
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %18 = load <2 x i32>, ptr poison, align 16
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %19 = load <3 x i32>, ptr poison, align 16
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %20 = load <4 x i32>, ptr poison, align 16
-; GFX10-NEXT: Cost Model: Found an estimated cost of 10 for instruction: ret void
+; GFX10-WGP-LABEL: 'loads_i32'
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %1 = load i32, ptr poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %2 = load <2 x i32>, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %3 = load <3 x i32>, ptr poison, align 16
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %4 = load <4 x i32>, ptr poison, align 16
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %5 = load i32, ptr poison, align 1
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %6 = load <2 x i32>, ptr poison, align 1
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %7 = load <3 x i32>, ptr poison, align 1
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %8 = load <4 x i32>, ptr poison, align 1
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %9 = load i32, ptr poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %10 = load <2 x i32>, ptr poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %11 = load <3 x i32>, ptr poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %12 = load <4 x i32>, ptr poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %13 = load i32, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %14 = load <2 x i32>, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %15 = load <3 x i32>, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %16 = load <4 x i32>, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %17 = load i32, ptr poison, align 16
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %18 = load <2 x i32>, ptr poison, align 16
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %19 = load <3 x i32>, ptr poison, align 16
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %20 = load <4 x i32>, ptr poison, align 16
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 10 for instruction: ret void
+;
+; GFX10-CU-LABEL: 'loads_i32'
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %1 = load i32, ptr poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %2 = load <2 x i32>, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %3 = load <3 x i32>, ptr poison, align 16
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %4 = load <4 x i32>, ptr poison, align 16
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %5 = load i32, ptr poison, align 1
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %6 = load <2 x i32>, ptr poison, align 1
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %7 = load <3 x i32>, ptr poison, align 1
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %8 = load <4 x i32>, ptr poison, align 1
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %9 = load i32, ptr poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %10 = load <2 x i32>, ptr poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %11 = load <3 x i32>, ptr poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %12 = load <4 x i32>, ptr poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %13 = load i32, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %14 = load <2 x i32>, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %15 = load <3 x i32>, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %16 = load <4 x i32>, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %17 = load i32, ptr poison, align 16
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %18 = load <2 x i32>, ptr poison, align 16
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %19 = load <3 x i32>, ptr poison, align 16
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %20 = load <4 x i32>, ptr poison, align 16
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 10 for instruction: ret void
;
; GFX12-LABEL: 'loads_i32'
; GFX12-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %1 = load i32, ptr poison, align 4
@@ -936,24 +978,43 @@ define void @loads_underaligned(i32 %arg) {
; GFX90A-NOUNALIGNED-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %16 = load <4 x i32>, ptr poison, align 8
; GFX90A-NOUNALIGNED-NEXT: Cost Model: Found an estimated cost of 10 for instruction: ret void
;
-; GFX10-LABEL: 'loads_underaligned'
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %1 = load <3 x i32>, ptr addrspace(3) poison, align 1
-; GFX10-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %2 = load <3 x i32>, ptr addrspace(3) poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %3 = load <3 x i32>, ptr addrspace(3) poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %4 = load <4 x i32>, ptr addrspace(3) poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %5 = load <4 x i32>, ptr addrspace(5) poison, align 1
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %6 = load i32, ptr addrspace(1) poison, align 1
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %7 = load i32, ptr addrspace(1) poison, align 2
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %8 = load <4 x i32>, ptr addrspace(1) poison, align 1
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %9 = load <4 x i32>, ptr addrspace(1) poison, align 2
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %10 = load i32, ptr poison, align 2
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %11 = load <4 x i32>, ptr poison, align 2
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %12 = load <2 x i64>, ptr addrspace(3) poison, align 2
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %13 = load i64, ptr poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %14 = load <3 x i32>, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %15 = load <4 x i32>, ptr poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %16 = load <4 x i32>, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 10 for instruction: ret void
+; GFX10-WGP-LABEL: 'loads_underaligned'
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %1 = load <3 x i32>, ptr addrspace(3) poison, align 1
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %2 = load <3 x i32>, ptr addrspace(3) poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %3 = load <3 x i32>, ptr addrspace(3) poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %4 = load <4 x i32>, ptr addrspace(3) poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %5 = load <4 x i32>, ptr addrspace(5) poison, align 1
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %6 = load i32, ptr addrspace(1) poison, align 1
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %7 = load i32, ptr addrspace(1) poison, align 2
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %8 = load <4 x i32>, ptr addrspace(1) poison, align 1
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %9 = load <4 x i32>, ptr addrspace(1) poison, align 2
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %10 = load i32, ptr poison, align 2
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %11 = load <4 x i32>, ptr poison, align 2
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %12 = load <2 x i64>, ptr addrspace(3) poison, align 2
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %13 = load i64, ptr poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %14 = load <3 x i32>, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %15 = load <4 x i32>, ptr poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %16 = load <4 x i32>, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 10 for instruction: ret void
+;
+; GFX10-CU-LABEL: 'loads_underaligned'
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %1 = load <3 x i32>, ptr addrspace(3) poison, align 1
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %2 = load <3 x i32>, ptr addrspace(3) poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %3 = load <3 x i32>, ptr addrspace(3) poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %4 = load <4 x i32>, ptr addrspace(3) poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %5 = load <4 x i32>, ptr addrspace(5) poison, align 1
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %6 = load i32, ptr addrspace(1) poison, align 1
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %7 = load i32, ptr addrspace(1) poison, align 2
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %8 = load <4 x i32>, ptr addrspace(1) poison, align 1
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %9 = load <4 x i32>, ptr addrspace(1) poison, align 2
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %10 = load i32, ptr poison, align 2
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %11 = load <4 x i32>, ptr poison, align 2
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %12 = load <2 x i64>, ptr addrspace(3) poison, align 2
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %13 = load i64, ptr poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %14 = load <3 x i32>, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %15 = load <4 x i32>, ptr poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %16 = load <4 x i32>, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 10 for instruction: ret void
;
; GFX12-LABEL: 'loads_underaligned'
; GFX12-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %1 = load <3 x i32>, ptr addrspace(3) poison, align 1
diff --git a/llvm/test/Analysis/CostModel/AMDGPU/store.ll b/llvm/test/Analysis/CostModel/AMDGPU/store.ll
index eb992b307ee576..e567b751c17982 100644
--- a/llvm/test/Analysis/CostModel/AMDGPU/store.ll
+++ b/llvm/test/Analysis/CostModel/AMDGPU/store.ll
@@ -2,8 +2,8 @@
; RUN: opt -passes="print<cost-model>" -disable-output -mtriple=amdgpu8.03-unknown-amdhsa < %s 2>&1 | FileCheck %s --check-prefixes=CHECK,GFX8
; RUN: opt -passes="print<cost-model>" -disable-output -mtriple=amdgpu9.0a-unknown-amdhsa < %s 2>&1 | FileCheck %s --check-prefixes=CHECK,GFX90A,GFX90A-UNALIGNED
; RUN: opt -passes="print<cost-model>" -disable-output -mtriple=amdgpu9.0a-unknown-amdhsa -mattr=-unaligned-access-mode < %s 2>&1 | FileCheck %s --check-prefixes=CHECK,GFX90A,GFX90A-NOUNALIGNED
-; RUN: opt -passes="print<cost-model>" -disable-output -mtriple=amdgpu10.10-unknown-amdhsa < %s 2>&1 | FileCheck %s --check-prefixes=CHECK,GFX10
-; RUN: opt -passes="print<cost-model>" -disable-output -mtriple=amdgpu10.10-unknown-amdhsa -mattr=+cumode < %s 2>&1 | FileCheck %s --check-prefixes=CHECK,GFX10
+; RUN: opt -passes="print<cost-model>" -disable-output -mtriple=amdgpu10.10-unknown-amdhsa < %s 2>&1 | FileCheck %s --check-prefixes=CHECK,GFX10,GFX10-WGP
+; RUN: opt -passes="print<cost-model>" -disable-output -mtriple=amdgpu10.10-unknown-amdhsa -mattr=+cumode < %s 2>&1 | FileCheck %s --check-prefixes=CHECK,GFX10,GFX10-CU
; RUN: opt -passes="print<cost-model>" -disable-output -mtriple=amdgpu12.01-unknown-amdhsa < %s 2>&1 | FileCheck %s --check-prefixes=CHECK,GFX12
; RUN: opt -passes="print<cost-model>" -disable-output -mtriple=amdgpu12.50-unknown-amdhsa < %s 2>&1 | FileCheck %s --check-prefixes=CHECK,GFX12
@@ -175,24 +175,43 @@ define void @stores_i16(i32 %arg) {
; GFX90A-NOUNALIGNED-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i16> poison, ptr poison, align 8
; GFX90A-NOUNALIGNED-NEXT: Cost Model: Found an estimated cost of 10 for instruction: ret void
;
-; GFX10-LABEL: 'stores_i16'
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i16 poison, ptr poison, align 2
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i16> poison, ptr poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <3 x i16> poison, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i16> poison, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i16 poison, ptr poison, align 1
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i16> poison, ptr poison, align 1
-; GFX10-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <3 x i16> poison, ptr poison, align 1
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i16> poison, ptr poison, align 1
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i16 poison, ptr poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i16> poison, ptr poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <3 x i16> poison, ptr poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i16> poison, ptr poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i16 poison, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i16> poison, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <3 x i16> poison, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i16> poison, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 10 for instruction: ret void
+; GFX10-WGP-LABEL: 'stores_i16'
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i16 poison, ptr poison, align 2
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i16> poison, ptr poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <3 x i16> poison, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i16> poison, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i16 poison, ptr poison, align 1
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i16> poison, ptr poison, align 1
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <3 x i16> poison, ptr poison, align 1
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <4 x i16> poison, ptr poison, align 1
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i16 poison, ptr poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i16> poison, ptr poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <3 x i16> poison, ptr poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <4 x i16> poison, ptr poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i16 poison, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i16> poison, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <3 x i16> poison, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i16> poison, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 10 for instruction: ret void
+;
+; GFX10-CU-LABEL: 'stores_i16'
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i16 poison, ptr poison, align 2
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i16> poison, ptr poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <3 x i16> poison, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i16> poison, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i16 poison, ptr poison, align 1
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i16> poison, ptr poison, align 1
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <3 x i16> poison, ptr poison, align 1
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i16> poison, ptr poison, align 1
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i16 poison, ptr poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i16> poison, ptr poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <3 x i16> poison, ptr poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i16> poison, ptr poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i16 poison, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i16> poison, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <3 x i16> poison, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i16> poison, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 10 for instruction: ret void
;
; GFX12-LABEL: 'stores_i16'
; GFX12-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i16 poison, ptr poison, align 2
@@ -306,28 +325,51 @@ define void @stores_i32(i32 %arg) {
; GFX90A-NOUNALIGNED-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr poison, align 16
; GFX90A-NOUNALIGNED-NEXT: Cost Model: Found an estimated cost of 10 for instruction: ret void
;
-; GFX10-LABEL: 'stores_i32'
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i32 poison, ptr poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i32> poison, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <3 x i32> poison, ptr poison, align 16
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr poison, align 16
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i32 poison, ptr poison, align 1
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i32> poison, ptr poison, align 1
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <3 x i32> poison, ptr poison, align 1
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr poison, align 1
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i32 poison, ptr poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i32> poison, ptr poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <3 x i32> poison, ptr poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i32 poison, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i32> poison, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <3 x i32> poison, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i32 poison, ptr poison, align 16
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i32> poison, ptr poison, align 16
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <3 x i32> poison, ptr poison, align 16
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr poison, align 16
-; GFX10-NEXT: Cost Model: Found an estimated cost of 10 for instruction: ret void
+; GFX10-WGP-LABEL: 'stores_i32'
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i32 poison, ptr poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i32> poison, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <3 x i32> poison, ptr poison, align 16
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr poison, align 16
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i32 poison, ptr poison, align 1
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <2 x i32> poison, ptr poison, align 1
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 3 for instruction: store <3 x i32> poison, ptr poison, align 1
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 4 for instruction: store <4 x i32> poison, ptr poison, align 1
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i32 poison, ptr poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <2 x i32> poison, ptr poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 3 for instruction: store <3 x i32> poison, ptr poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 4 for instruction: store <4 x i32> poison, ptr poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i32 poison, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i32> poison, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <3 x i32> poison, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <4 x i32> poison, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i32 poison, ptr poison, align 16
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i32> poison, ptr poison, align 16
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <3 x i32> poison, ptr poison, align 16
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr poison, align 16
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 10 for instruction: ret void
+;
+; GFX10-CU-LABEL: 'stores_i32'
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i32 poison, ptr poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i32> poison, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <3 x i32> poison, ptr poison, align 16
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr poison, align 16
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i32 poison, ptr poison, align 1
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i32> poison, ptr poison, align 1
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <3 x i32> poison, ptr poison, align 1
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr poison, align 1
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i32 poison, ptr poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i32> poison, ptr poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <3 x i32> poison, ptr poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i32 poison, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i32> poison, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <3 x i32> poison, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i32 poison, ptr poison, align 16
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i32> poison, ptr poison, align 16
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <3 x i32> poison, ptr poison, align 16
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr poison, align 16
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 10 for instruction: ret void
;
; GFX12-LABEL: 'stores_i32'
; GFX12-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i32 poison, ptr poison, align 4
@@ -936,24 +978,43 @@ define void @stores_underaligned(i32 %arg) {
; GFX90A-NOUNALIGNED-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr poison, align 8
; GFX90A-NOUNALIGNED-NEXT: Cost Model: Found an estimated cost of 10 for instruction: ret void
;
-; GFX10-LABEL: 'stores_underaligned'
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <3 x i32> poison, ptr addrspace(3) poison, align 1
-; GFX10-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <3 x i32> poison, ptr addrspace(3) poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <3 x i32> poison, ptr addrspace(3) poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <4 x i32> poison, ptr addrspace(3) poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 4 for instruction: store <4 x i32> poison, ptr addrspace(5) poison, align 1
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i32 poison, ptr addrspace(1) poison, align 1
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i32 poison, ptr addrspace(1) poison, align 2
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr addrspace(1) poison, align 1
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr addrspace(1) poison, align 2
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i32 poison, ptr poison, align 2
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr poison, align 2
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i64> poison, ptr addrspace(3) poison, align 2
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i64 poison, ptr poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <3 x i32> poison, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr poison, align 4
-; GFX10-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr poison, align 8
-; GFX10-NEXT: Cost Model: Found an estimated cost of 10 for instruction: ret void
+; GFX10-WGP-LABEL: 'stores_underaligned'
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <3 x i32> poison, ptr addrspace(3) poison, align 1
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <3 x i32> poison, ptr addrspace(3) poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <3 x i32> poison, ptr addrspace(3) poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <4 x i32> poison, ptr addrspace(3) poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 4 for instruction: store <4 x i32> poison, ptr addrspace(5) poison, align 1
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i32 poison, ptr addrspace(1) poison, align 1
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i32 poison, ptr addrspace(1) poison, align 2
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr addrspace(1) poison, align 1
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr addrspace(1) poison, align 2
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i32 poison, ptr poison, align 2
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 4 for instruction: store <4 x i32> poison, ptr poison, align 2
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <2 x i64> poison, ptr addrspace(3) poison, align 2
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store i64 poison, ptr poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <3 x i32> poison, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 4 for instruction: store <4 x i32> poison, ptr poison, align 4
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <4 x i32> poison, ptr poison, align 8
+; GFX10-WGP-NEXT: Cost Model: Found an estimated cost of 10 for instruction: ret void
+;
+; GFX10-CU-LABEL: 'stores_underaligned'
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <3 x i32> poison, ptr addrspace(3) poison, align 1
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <3 x i32> poison, ptr addrspace(3) poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <3 x i32> poison, ptr addrspace(3) poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 2 for instruction: store <4 x i32> poison, ptr addrspace(3) poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 4 for instruction: store <4 x i32> poison, ptr addrspace(5) poison, align 1
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i32 poison, ptr addrspace(1) poison, align 1
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i32 poison, ptr addrspace(1) poison, align 2
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr addrspace(1) poison, align 1
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr addrspace(1) poison, align 2
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i32 poison, ptr poison, align 2
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr poison, align 2
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <2 x i64> poison, ptr addrspace(3) poison, align 2
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store i64 poison, ptr poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <3 x i32> poison, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr poison, align 4
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <4 x i32> poison, ptr poison, align 8
+; GFX10-CU-NEXT: Cost Model: Found an estimated cost of 10 for instruction: ret void
;
; GFX12-LABEL: 'stores_underaligned'
; GFX12-NEXT: Cost Model: Found an estimated cost of 1 for instruction: store <3 x i32> poison, ptr addrspace(3) poison, align 1
More information about the llvm-branch-commits
mailing list