[llvm] [AMDGPU] Use instrLatency for memory HWUI cycle accounting (PR #205418)
Jeffrey Byrnes via llvm-commits
llvm-commits at lists.llvm.org
Tue Jun 23 12:55:08 PDT 2026
https://github.com/jrbyrnes created https://github.com/llvm/llvm-project/pull/205418
Conceptually, the HWUI cycle tracking is meant to model the total usage for a given HardwareUnit for the current scheduling region. A single instruction's contribution to this usage is the number of cycles in which another instruction of the same type can not use this HardwareUnit. So, for example, a standard V_ADD_F32 contributes one cycle, as another V_ADD_F32 can use this resource in the next cycle. However, on MI450, the latency of these instructions from the SchedModel is 5. This suggests that the property we want for modelling ALU instructions it the functional unit occupancy of the instruction, which is captured by ReleaseAtCycle. Thus, we use ReleaseAtCycle for HWUI cycles for ALU instructions.
However, for memory operations, this accounting is not so simple. There are many conditions that can occur during runtime that we can not model in the compiler, but we still need to model the HardwareUnit consumption for the scheduling region. We model this by saying we can execute M instructions in N cycles. Where M instructions is defined by the buffer size (e.g. https://github.com/llvm/llvm-project/pull/192323 ) and N cycles is based on latency. This PR starts tracking memory instruction HardwareUnit consumption by latency.
>From c29679aaaf12b351e33a55c2c45e2eed134609fd Mon Sep 17 00:00:00 2001
From: Jeffrey Byrnes <Jeffrey.Byrnes at amd.com>
Date: Tue, 23 Jun 2026 12:51:19 -0700
Subject: [PATCH 1/2] Precommit test change
Change-Id: Id3546e1492a335f48cf70d9b0e93afe22b31ff7a
---
.../AMDGPU/coexec-sched-flavor-classification.mir | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/llvm/test/CodeGen/AMDGPU/coexec-sched-flavor-classification.mir b/llvm/test/CodeGen/AMDGPU/coexec-sched-flavor-classification.mir
index 3728b4012a799..c68c2e4a645f8 100644
--- a/llvm/test/CodeGen/AMDGPU/coexec-sched-flavor-classification.mir
+++ b/llvm/test/CodeGen/AMDGPU/coexec-sched-flavor-classification.mir
@@ -2,11 +2,11 @@
# RUN: llc -mtriple=amdgcn -mcpu=gfx1250 -run-pass=machine-scheduler -amdgpu-sched-strategy=coexec -debug-only=machine-scheduler %s -filetype=null 2>&1 | FileCheck %s
# CHECK: HWUI Resource Pressure:
-# CHECK-DAG: VMEM: {{[0-9]+}} cycles, 10 instrs
-# CHECK-DAG: DS: {{[0-9]+}} cycles, 4 instrs
-# CHECK-DAG: DMA: {{[0-9]+}} cycles, 2 instrs
-# CHECK-DAG: VALU(1c): {{[0-9]+}} cycles, 4 instrs
-# CHECK-DAG: TRANS: {{[0-9]+}} cycles, 2 instrs
+# CHECK-DAG: VALU(1c): 4 cycles, 4 instrs
+# CHECK-DAG: TRANS: 2 cycles, 2 instrs
+# CHECK-DAG: VMEM: 10 cycles, 10 instrs
+# CHECK-DAG: DS: 4 cycles, 4 instrs
+# CHECK-DAG: DMA: 4 cycles, 2 instrs
--- |
define void @test_flavor_classification() #0 { ret void }
>From f5d317573a4c13552c66869374fa000ff54844f9 Mon Sep 17 00:00:00 2001
From: Jeffrey Byrnes <Jeffrey.Byrnes at amd.com>
Date: Tue, 23 Jun 2026 12:03:21 -0700
Subject: [PATCH 2/2] [AMDGPU] Use instrLatency for memory HWUI cycle
accounting
Change-Id: I7a9e2deb5db7638d6f735570e65d6ca988a7477f
---
llvm/lib/Target/AMDGPU/AMDGPUCoExecSchedStrategy.cpp | 5 +++++
.../CodeGen/AMDGPU/coexec-sched-flavor-classification.mir | 6 +++---
llvm/test/CodeGen/AMDGPU/coexec-scheduler.ll | 4 ++--
llvm/test/CodeGen/AMDGPU/ldsdmacnt_sched.mir | 4 ++--
4 files changed, 12 insertions(+), 7 deletions(-)
diff --git a/llvm/lib/Target/AMDGPU/AMDGPUCoExecSchedStrategy.cpp b/llvm/lib/Target/AMDGPU/AMDGPUCoExecSchedStrategy.cpp
index 504c3aae6ca57..d29ee5084cc6c 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPUCoExecSchedStrategy.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPUCoExecSchedStrategy.cpp
@@ -180,6 +180,11 @@ CandidateHeuristics::getHWUIFromFlavor(InstructionFlavor Flavor) {
unsigned CandidateHeuristics::getHWUICyclesForInst(SUnit *SU) {
assert(SchedModel && SchedModel->hasInstrSchedModel());
+
+ MachineInstr *MI = SU->getInstr();
+ if (MI->mayLoadOrStore())
+ return SchedModel->computeInstrLatency(MI);
+
unsigned ReleaseAtCycle = 0;
const MCSchedClassDesc *SC = DAG->getSchedClass(SU);
for (TargetSchedModel::ProcResIter PI = SchedModel->getWriteProcResBegin(SC),
diff --git a/llvm/test/CodeGen/AMDGPU/coexec-sched-flavor-classification.mir b/llvm/test/CodeGen/AMDGPU/coexec-sched-flavor-classification.mir
index c68c2e4a645f8..82dd1d8748675 100644
--- a/llvm/test/CodeGen/AMDGPU/coexec-sched-flavor-classification.mir
+++ b/llvm/test/CodeGen/AMDGPU/coexec-sched-flavor-classification.mir
@@ -4,9 +4,9 @@
# CHECK: HWUI Resource Pressure:
# CHECK-DAG: VALU(1c): 4 cycles, 4 instrs
# CHECK-DAG: TRANS: 2 cycles, 2 instrs
-# CHECK-DAG: VMEM: 10 cycles, 10 instrs
-# CHECK-DAG: DS: 4 cycles, 4 instrs
-# CHECK-DAG: DMA: 4 cycles, 2 instrs
+# CHECK-DAG: VMEM: 3200 cycles, 10 instrs
+# CHECK-DAG: DS: 80 cycles, 4 instrs
+# CHECK-DAG: DMA: 640 cycles, 2 instrs
--- |
define void @test_flavor_classification() #0 { ret void }
diff --git a/llvm/test/CodeGen/AMDGPU/coexec-scheduler.ll b/llvm/test/CodeGen/AMDGPU/coexec-scheduler.ll
index ac121195de432..b225d8f18805a 100644
--- a/llvm/test/CodeGen/AMDGPU/coexec-scheduler.ll
+++ b/llvm/test/CodeGen/AMDGPU/coexec-scheduler.ll
@@ -73,9 +73,9 @@ define amdgpu_kernel void @ds_wmma(ptr addrspace(3) %base, ptr addrspace(1) %out
; COEXEC-NEXT: v_wmma_f32_16x16x32_f16 v[0:7], v[88:95], v[80:87], v[0:7]
; COEXEC-NEXT: s_cbranch_vccnz .LBB0_1
; COEXEC-NEXT: ; %bb.2: ; %end
+; COEXEC-NEXT: s_load_b64 s[0:1], s[4:5], 0x8 nv
; COEXEC-NEXT: v_nop
; COEXEC-NEXT: v_mov_b32_e32 v32, 0
-; COEXEC-NEXT: s_load_b64 s[0:1], s[4:5], 0x8 nv
; COEXEC-NEXT: s_wait_kmcnt 0x0
; COEXEC-NEXT: s_clause 0x7
; COEXEC-NEXT: global_store_b128 v32, v[28:31], s[0:1] offset:16
@@ -346,8 +346,8 @@ define amdgpu_kernel void @ds_wmma_permute(ptr addrspace(3) %base, ptr addrspace
; COEXEC-NEXT: v_wmma_f32_16x16x32_f16 v[0:7], v[120:127], v[152:159], v[0:7]
; COEXEC-NEXT: s_cbranch_vccnz .LBB1_1
; COEXEC-NEXT: ; %bb.2: ; %end
-; COEXEC-NEXT: v_mov_b32_e32 v32, 0
; COEXEC-NEXT: s_load_b64 s[0:1], s[4:5], 0x8 nv
+; COEXEC-NEXT: v_mov_b32_e32 v32, 0
; COEXEC-NEXT: s_wait_kmcnt 0x0
; COEXEC-NEXT: s_clause 0x7
; COEXEC-NEXT: global_store_b128 v32, v[28:31], s[0:1] offset:16
diff --git a/llvm/test/CodeGen/AMDGPU/ldsdmacnt_sched.mir b/llvm/test/CodeGen/AMDGPU/ldsdmacnt_sched.mir
index dea8bbed4869d..605960ca1d960 100644
--- a/llvm/test/CodeGen/AMDGPU/ldsdmacnt_sched.mir
+++ b/llvm/test/CodeGen/AMDGPU/ldsdmacnt_sched.mir
@@ -329,11 +329,11 @@ body: |
; DEFAULT-NEXT: [[DEF3:%[0-9]+]]:vgpr_32 = IMPLICIT_DEF
; DEFAULT-NEXT: [[DEF4:%[0-9]+]]:vgpr_32 = IMPLICIT_DEF
; DEFAULT-NEXT: GLOBAL_LOAD_ASYNC_TO_LDS_B64_SADDR [[DEF]], [[DEF1]], [[DEF4]], 0, 0, implicit-def dead $asynccnt, implicit $exec, implicit $asynccnt
+ ; DEFAULT-NEXT: GLOBAL_LOAD_ASYNC_TO_LDS_B64_SADDR [[DEF]], [[DEF1]], [[DEF4]], 0, 0, implicit-def dead $asynccnt, implicit $exec, implicit $asynccnt
; DEFAULT-NEXT: dead [[V_ADD_U32_e32_:%[0-9]+]]:vgpr_32 = V_ADD_U32_e32 [[DEF2]], [[DEF3]], implicit $exec
; DEFAULT-NEXT: dead [[V_ADD_U32_e32_1:%[0-9]+]]:vgpr_32 = V_ADD_U32_e32 [[DEF2]], [[DEF3]], implicit $exec
; DEFAULT-NEXT: dead [[V_ADD_U32_e32_2:%[0-9]+]]:vgpr_32 = V_ADD_U32_e32 [[DEF2]], [[DEF3]], implicit $exec
; DEFAULT-NEXT: dead [[V_ADD_U32_e32_3:%[0-9]+]]:vgpr_32 = V_ADD_U32_e32 [[DEF2]], [[DEF3]], implicit $exec
- ; DEFAULT-NEXT: GLOBAL_LOAD_ASYNC_TO_LDS_B64_SADDR [[DEF]], [[DEF1]], [[DEF4]], 0, 0, implicit-def dead $asynccnt, implicit $exec, implicit $asynccnt
; DEFAULT-NEXT: dead [[V_ADD_U32_e32_4:%[0-9]+]]:vgpr_32 = V_ADD_U32_e32 [[DEF2]], [[DEF3]], implicit $exec
; DEFAULT-NEXT: dead [[V_ADD_U32_e32_5:%[0-9]+]]:vgpr_32 = V_ADD_U32_e32 [[DEF2]], [[DEF3]], implicit $exec
; DEFAULT-NEXT: SCHED_GROUP_BARRIER 32, 1, 0
@@ -606,11 +606,11 @@ body: |
; DEFAULT-NEXT: [[DEF3:%[0-9]+]]:vgpr_32 = IMPLICIT_DEF
; DEFAULT-NEXT: [[DEF4:%[0-9]+]]:vgpr_32 = IMPLICIT_DEF
; DEFAULT-NEXT: GLOBAL_LOAD_ASYNC_TO_LDS_B64_SADDR [[DEF]], [[DEF1]], [[DEF4]], 0, 0, implicit-def dead $asynccnt, implicit $exec, implicit $asynccnt
+ ; DEFAULT-NEXT: GLOBAL_LOAD_ASYNC_TO_LDS_B64_SADDR [[DEF]], [[DEF1]], [[DEF4]], 0, 0, implicit-def dead $asynccnt, implicit $exec, implicit $asynccnt
; DEFAULT-NEXT: dead [[V_ADD_U32_e32_:%[0-9]+]]:vgpr_32 = V_ADD_U32_e32 [[DEF2]], [[DEF3]], implicit $exec
; DEFAULT-NEXT: dead [[V_ADD_U32_e32_1:%[0-9]+]]:vgpr_32 = V_ADD_U32_e32 [[DEF2]], [[DEF3]], implicit $exec
; DEFAULT-NEXT: dead [[V_ADD_U32_e32_2:%[0-9]+]]:vgpr_32 = V_ADD_U32_e32 [[DEF2]], [[DEF3]], implicit $exec
; DEFAULT-NEXT: dead [[V_ADD_U32_e32_3:%[0-9]+]]:vgpr_32 = V_ADD_U32_e32 [[DEF2]], [[DEF3]], implicit $exec
- ; DEFAULT-NEXT: GLOBAL_LOAD_ASYNC_TO_LDS_B64_SADDR [[DEF]], [[DEF1]], [[DEF4]], 0, 0, implicit-def dead $asynccnt, implicit $exec, implicit $asynccnt
; DEFAULT-NEXT: dead [[V_ADD_U32_e32_4:%[0-9]+]]:vgpr_32 = V_ADD_U32_e32 [[DEF2]], [[DEF3]], implicit $exec
; DEFAULT-NEXT: dead [[V_ADD_U32_e32_5:%[0-9]+]]:vgpr_32 = V_ADD_U32_e32 [[DEF2]], [[DEF3]], implicit $exec
; DEFAULT-NEXT: SCHED_GROUP_BARRIER 32, 1, 0
More information about the llvm-commits
mailing list