[llvm] [AMDGPU] Account for gfx950 permlane hazards during scheduling (PR #229517)
Alan Li via llvm-commits
llvm-commits at lists.llvm.org
Tue Oct 6 11:54:37 PDT 2026
https://github.com/lialan created https://github.com/llvm/llvm-project/pull/229517
The [CDNA4 ISA Reference Guide, ยง4.5, Table 11, p. 21](https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/instruction-set-architectures/amd-instinct-cdna4-instruction-set-architecture.pdf#page=29) requires 2 wait states between a VALU writing a VGPR and a permlane reading it, and 4 between a `VCMPX` writing EXEC and a permlane. As far as I know, this is GFX950 specific.
Right now the gfx950 scheduler does not query this existing hazard check, so final hazard processing can insert avoidable `nop`s despite available independent instructions.
This patch expose the existing permlane hazard check in `getHazardType()`.
Also add MIR coverage for source dependencies, unrelated writes, available fillers, and VCMPX scheduling boundaries.
>From d9616633de4b90e08a3d102bce2cf1e76b9e16fb Mon Sep 17 00:00:00 2001
From: Alan Li <me at alanli.org>
Date: Mon, 5 Oct 2026 15:29:58 -0500
Subject: [PATCH] [AMDGPU] Account for gfx950 permlane hazards during
scheduling
The final hazard pass already enforces gfx950 permlane spacing after VALU
results and VCMPX updates to EXEC. Expose the existing hazard check through
getHazardType so the scheduler can fill required wait states with
independent instructions. The final pass inserts remaining NOPs when
insufficient independent instructions are available.
Add MIR coverage for both swap sources, unrelated writes, zero, one and
two independent fillers, and a VCMPX scheduling boundary.
---
.../lib/Target/AMDGPU/GCNHazardRecognizer.cpp | 3 +
.../CodeGen/AMDGPU/postra-permlane-hazard.mir | 127 ++++++++++++++++++
2 files changed, 130 insertions(+)
create mode 100644 llvm/test/CodeGen/AMDGPU/postra-permlane-hazard.mir
diff --git a/llvm/lib/Target/AMDGPU/GCNHazardRecognizer.cpp b/llvm/lib/Target/AMDGPU/GCNHazardRecognizer.cpp
index 26bebb388654d2c..f3067114e749165 100644
--- a/llvm/lib/Target/AMDGPU/GCNHazardRecognizer.cpp
+++ b/llvm/lib/Target/AMDGPU/GCNHazardRecognizer.cpp
@@ -632,6 +632,9 @@ GCNHazardRecognizer::getHazardType(SUnit *SU, int Stalls) {
checkMAILdStHazards(MI) > 0)
return HazardType;
+ if (ST.hasGFX950Insts() && isPermlane(*MI) && checkPermlaneHazards(MI) > 0)
+ return HazardType;
+
if (MI->isInlineAsm() && checkInlineAsmHazards(MI) > 0)
return HazardType;
diff --git a/llvm/test/CodeGen/AMDGPU/postra-permlane-hazard.mir b/llvm/test/CodeGen/AMDGPU/postra-permlane-hazard.mir
new file mode 100644
index 000000000000000..7228bd773b58075
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/postra-permlane-hazard.mir
@@ -0,0 +1,127 @@
+# RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx950 -run-pass=postmisched -verify-machineinstrs -verify-misched %s -o - | FileCheck %s
+# RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx950 -run-pass=postmisched,post-RA-hazard-rec -verify-machineinstrs -verify-misched %s -o - | FileCheck %s --check-prefixes=CHECK,FINAL
+
+# A permlane reading a VALU result needs two intervening wait states on gfx950.
+# The scheduler should use available independent instructions for these slots,
+# rather than leave them after the permlane for the hazard pass to insert NOPs.
+
+---
+name: valu_vdst0_two_fillers
+tracksRegLiveness: true
+body: |
+ ; CHECK-LABEL: name: valu_vdst0_two_fillers
+ ; CHECK: $vgpr0 = V_MOV_B32_e32 0, implicit $exec
+ ; CHECK-NEXT: $sgpr0 = S_MOV_B32 0
+ ; CHECK-NEXT: $sgpr1 = S_MOV_B32 1
+ ; CHECK-NEXT: renamable $vgpr0, renamable $vgpr1 = V_PERMLANE16_SWAP_B32_e32 killed $vgpr0, killed $vgpr1, implicit $exec
+ ; CHECK-NEXT: S_ENDPGM 0
+ bb.0:
+ liveins: $vgpr1
+
+ $vgpr0 = V_MOV_B32_e32 0, implicit $exec
+ renamable $vgpr0, renamable $vgpr1 = V_PERMLANE16_SWAP_B32_e32 killed $vgpr0, killed $vgpr1, implicit $exec
+ $sgpr0 = S_MOV_B32 0
+ $sgpr1 = S_MOV_B32 1
+ S_ENDPGM 0
+...
+
+# Check the other explicit source and a VOP3 permlane32 swap.
+
+---
+name: valu_vdst1_two_fillers
+tracksRegLiveness: true
+body: |
+ ; CHECK-LABEL: name: valu_vdst1_two_fillers
+ ; CHECK: $vgpr1 = V_MOV_B32_e32 0, implicit $exec
+ ; CHECK-NEXT: $sgpr0 = S_MOV_B32 0
+ ; CHECK-NEXT: $sgpr1 = S_MOV_B32 1
+ ; CHECK-NEXT: renamable $vgpr0, renamable $vgpr1 = V_PERMLANE32_SWAP_B32_e64 killed $vgpr0, killed $vgpr1, -1, 1, implicit $exec
+ ; CHECK-NEXT: S_ENDPGM 0
+ bb.0:
+ liveins: $vgpr0
+
+ $vgpr1 = V_MOV_B32_e32 0, implicit $exec
+ renamable $vgpr0, renamable $vgpr1 = V_PERMLANE32_SWAP_B32_e64 killed $vgpr0, killed $vgpr1, -1, 1, implicit $exec
+ $sgpr0 = S_MOV_B32 0
+ $sgpr1 = S_MOV_B32 1
+ S_ENDPGM 0
+...
+
+# A write to a register that the permlane does not read needs no spacing.
+
+---
+name: unrelated_valu_write
+tracksRegLiveness: true
+body: |
+ ; CHECK-LABEL: name: unrelated_valu_write
+ ; CHECK: $vgpr2 = V_MOV_B32_e32 0, implicit $exec
+ ; CHECK-NEXT: renamable $vgpr0, renamable $vgpr1 = V_PERMLANE16_SWAP_B32_e32 killed $vgpr0, killed $vgpr1, implicit $exec
+ ; CHECK-NEXT: S_ENDPGM 0
+ bb.0:
+ liveins: $vgpr0, $vgpr1
+
+ $vgpr2 = V_MOV_B32_e32 0, implicit $exec
+ renamable $vgpr0, renamable $vgpr1 = V_PERMLANE16_SWAP_B32_e32 killed $vgpr0, killed $vgpr1, implicit $exec
+ S_ENDPGM 0
+...
+
+# Scheduler stalls do not emit instructions. With no fillers, the hazard pass
+# must still insert the required NOPs in the final instruction stream.
+
+---
+name: valu_without_fillers
+tracksRegLiveness: true
+body: |
+ ; CHECK-LABEL: name: valu_without_fillers
+ ; CHECK: $vgpr0 = V_MOV_B32_e32 0, implicit $exec
+ ; FINAL-NEXT: S_NOP 1
+ ; CHECK-NEXT: renamable $vgpr0, renamable $vgpr1 = V_PERMLANE16_SWAP_B32_e32 killed $vgpr0, killed $vgpr1, implicit $exec
+ ; CHECK-NEXT: S_ENDPGM 0
+ bb.0:
+ liveins: $vgpr1
+
+ $vgpr0 = V_MOV_B32_e32 0, implicit $exec
+ renamable $vgpr0, renamable $vgpr1 = V_PERMLANE16_SWAP_B32_e32 killed $vgpr0, killed $vgpr1, implicit $exec
+ S_ENDPGM 0
+...
+
+# One independent instruction fills one wait state; one NOP remains necessary.
+
+---
+name: valu_with_one_filler
+tracksRegLiveness: true
+body: |
+ ; CHECK-LABEL: name: valu_with_one_filler
+ ; CHECK: $vgpr0 = V_MOV_B32_e32 0, implicit $exec
+ ; CHECK-NEXT: $sgpr0 = S_MOV_B32 0
+ ; FINAL-NEXT: S_NOP 0
+ ; CHECK-NEXT: renamable $vgpr0, renamable $vgpr1 = V_PERMLANE16_SWAP_B32_e32 killed $vgpr0, killed $vgpr1, implicit $exec
+ ; CHECK-NEXT: S_ENDPGM 0
+ bb.0:
+ liveins: $vgpr1
+
+ $vgpr0 = V_MOV_B32_e32 0, implicit $exec
+ renamable $vgpr0, renamable $vgpr1 = V_PERMLANE16_SWAP_B32_e32 killed $vgpr0, killed $vgpr1, implicit $exec
+ $sgpr0 = S_MOV_B32 0
+ S_ENDPGM 0
+...
+
+# VCMPX writes EXEC and forms a scheduling boundary. Its four-wait-state hazard
+# must still be handled by the final hazard pass across that boundary.
+
+---
+name: vcmpx_without_fillers
+tracksRegLiveness: true
+body: |
+ ; CHECK-LABEL: name: vcmpx_without_fillers
+ ; CHECK: V_CMPX_EQ_I32_e32 $vgpr0, $vgpr1, implicit-def $exec, implicit-def $vcc, implicit $exec
+ ; FINAL-NEXT: S_NOP 3
+ ; CHECK-NEXT: renamable $vgpr0, renamable $vgpr1 = V_PERMLANE16_SWAP_B32_e32 killed $vgpr0, killed $vgpr1, implicit $exec
+ ; CHECK-NEXT: S_ENDPGM 0
+ bb.0:
+ liveins: $vgpr0, $vgpr1
+
+ V_CMPX_EQ_I32_e32 $vgpr0, $vgpr1, implicit-def $exec, implicit-def $vcc, implicit $exec
+ renamable $vgpr0, renamable $vgpr1 = V_PERMLANE16_SWAP_B32_e32 killed $vgpr0, killed $vgpr1, implicit $exec
+ S_ENDPGM 0
+...
More information about the llvm-commits
mailing list