[llvm] [AMDGPU] Fix missed WMMA C-operand co-exec hazard (PR #226503)
Petar Jovanovic via llvm-commits
llvm-commits at lists.llvm.org
Fri Sep 25 07:04:46 PDT 2026
https://github.com/petar-jovanovic created https://github.com/llvm/llvm-project/pull/226503
The gfx1250 WMMA co-execution hazard check treats only A, B and the SWMMAC index as registers the in-flight MMA still reads. C (src2 of a non-SWMMAC WMMA) is missing, so a VALU scheduled into the MMA's shadow can clobber C and the MMA consumes the new value.
This is latent while C is tied to vdst, since the existing D check then covers it. It miscompiles where the tie does not hold: for v_wmma_bf16f32_16x16x32_bf16, whose D is narrower than C, and for the _threeaddr form of any WMMA.
Add src2 to the checked set for non-SWMMAC WMMAs.
>From 6258c4699d2912c28d5d6df625cecf966e1620dc Mon Sep 17 00:00:00 2001
From: Petar Jovanovic <petar.jovanovic at amd.com>
Date: Fri, 25 Sep 2026 16:12:24 +0200
Subject: [PATCH] [AMDGPU] Fix missed WMMA C-operand co-exec hazard
The gfx1250 WMMA co-execution hazard check treats only A, B and the
SWMMAC index as registers the in-flight MMA still reads. C (src2 of a
non-SWMMAC WMMA) is missing, so a VALU scheduled into the MMA's shadow
can clobber C and the MMA consumes the new value.
This is latent while C is tied to vdst, since the existing D check then
covers it. It miscompiles where the tie does not hold: for
v_wmma_bf16f32_16x16x32_bf16, whose D is narrower than C, and for the
_threeaddr form of any WMMA.
Add src2 to the checked set for non-SWMMAC WMMAs.
---
.../lib/Target/AMDGPU/GCNHazardRecognizer.cpp | 5 +
.../AMDGPU/wmma-coexecution-valu-hazards.mir | 173 +++++++++++++++++-
2 files changed, 176 insertions(+), 2 deletions(-)
diff --git a/llvm/lib/Target/AMDGPU/GCNHazardRecognizer.cpp b/llvm/lib/Target/AMDGPU/GCNHazardRecognizer.cpp
index 08c3d7dd9b098d..12cb60fa598b7b 100644
--- a/llvm/lib/Target/AMDGPU/GCNHazardRecognizer.cpp
+++ b/llvm/lib/Target/AMDGPU/GCNHazardRecognizer.cpp
@@ -2689,6 +2689,11 @@ bool GCNHazardRecognizer::hasWMMAToVALURegOverlap(
if (SIInstrInfo::isSWMMAC(WMMA)) {
Register Idx0 = TII.getNamedOperand(WMMA, AMDGPU::OpName::src2)->getReg();
WMMARegs.push_back(Idx0);
+ } else {
+ // src2 is C, which is not always tied to vdst and may be an immediate.
+ const MachineOperand *C0 = TII.getNamedOperand(WMMA, AMDGPU::OpName::src2);
+ if (C0->isReg())
+ WMMARegs.push_back(C0->getReg());
}
for (const MachineOperand &ValuDef : MI.defs()) {
diff --git a/llvm/test/CodeGen/AMDGPU/wmma-coexecution-valu-hazards.mir b/llvm/test/CodeGen/AMDGPU/wmma-coexecution-valu-hazards.mir
index d8bb52b86309ec..0164635d2ffd7b 100644
--- a/llvm/test/CodeGen/AMDGPU/wmma-coexecution-valu-hazards.mir
+++ b/llvm/test/CodeGen/AMDGPU/wmma-coexecution-valu-hazards.mir
@@ -5,12 +5,13 @@
# Same hazards as gfx1251
# RUN: llc -mtriple=amdgpu12.5 -run-pass post-RA-hazard-rec %s -o - | FileCheck -check-prefix=GFX1251 %s
-# WMMA writes: D0, WMMA reads: A0/B0/Index0
+# WMMA writes: D0, WMMA reads: A0/B0/C0/Index0
# VALU writes: D1, VALU reads: Use1
# Hards could be:
# RAW: D0 overlaps Use1
# WAW: D0 overlaps D1
-# WAR: A0/B0/Index0 overlaps D1
+# WAR: A0/B0/C0/Index0 overlaps D1
+# C0 is usually tied to D0, and is then covered by the D0 cases.
---
name: test_wmma_f32_16x16x32_bf16_D0_overlaps_Use1
@@ -284,6 +285,174 @@ body: |
$vgpr8 = V_ADD_F32_e32 $vgpr24, $vgpr25, implicit $mode, implicit $exec
...
+# V_WMMA_BF16F32_16X16X32_BF16 has D narrower than C, so C is never tied.
+# A = v[0:7], B = v[8:15], C = v[16:23], D = v[24:27].
+
+---
+name: test_wmma_bf16f32_16x16x32_bf16_D0_overlaps_Use1
+body: |
+ bb.0:
+ ; GFX1250-LABEL: name: test_wmma_bf16f32_16x16x32_bf16_D0_overlaps_Use1
+ ; GFX1250: early-clobber $vgpr24_vgpr25_vgpr26_vgpr27 = V_WMMA_BF16F32_16X16X32_BF16_w32_twoaddr killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, killed $vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, 8, killed $vgpr16_vgpr17_vgpr18_vgpr19_vgpr20_vgpr21_vgpr22_vgpr23, 0, 0, 0, 0, implicit $exec
+ ; GFX1250-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1250-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1250-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1250-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1250-NEXT: $vgpr31 = V_ADD_F32_e32 $vgpr30, $vgpr24, implicit $mode, implicit $exec
+ ;
+ ; GFX1251-LABEL: name: test_wmma_bf16f32_16x16x32_bf16_D0_overlaps_Use1
+ ; GFX1251: early-clobber $vgpr24_vgpr25_vgpr26_vgpr27 = V_WMMA_BF16F32_16X16X32_BF16_w32_twoaddr killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, killed $vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, 8, killed $vgpr16_vgpr17_vgpr18_vgpr19_vgpr20_vgpr21_vgpr22_vgpr23, 0, 0, 0, 0, implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: $vgpr31 = V_ADD_F32_e32 $vgpr30, $vgpr24, implicit $mode, implicit $exec
+ ;
+ ; GFX1310-LABEL: name: test_wmma_bf16f32_16x16x32_bf16_D0_overlaps_Use1
+ ; GFX1310: early-clobber $vgpr24_vgpr25_vgpr26_vgpr27 = V_WMMA_BF16F32_16X16X32_BF16_w32_twoaddr killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, killed $vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, 8, killed $vgpr16_vgpr17_vgpr18_vgpr19_vgpr20_vgpr21_vgpr22_vgpr23, 0, 0, 0, 0, implicit $exec
+ ; GFX1310-NEXT: $vgpr31 = V_ADD_F32_e32 $vgpr30, $vgpr24, implicit $mode, implicit $exec
+ $vgpr24_vgpr25_vgpr26_vgpr27 = V_WMMA_BF16F32_16X16X32_BF16_w32_twoaddr killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, killed $vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, 8, killed $vgpr16_vgpr17_vgpr18_vgpr19_vgpr20_vgpr21_vgpr22_vgpr23, 0, 0, 0, 0, implicit $exec
+ $vgpr31 = V_ADD_F32_e32 $vgpr30, $vgpr24, implicit $mode, implicit $exec
+...
+
+---
+name: test_wmma_bf16f32_16x16x32_bf16_C0_overlaps_D1
+body: |
+ bb.0:
+ ; GFX1250-LABEL: name: test_wmma_bf16f32_16x16x32_bf16_C0_overlaps_D1
+ ; GFX1250: early-clobber $vgpr24_vgpr25_vgpr26_vgpr27 = V_WMMA_BF16F32_16X16X32_BF16_w32_twoaddr killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, killed $vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, 8, killed $vgpr16_vgpr17_vgpr18_vgpr19_vgpr20_vgpr21_vgpr22_vgpr23, 0, 0, 0, 0, implicit $exec
+ ; GFX1250-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1250-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1250-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1250-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1250-NEXT: $vgpr16 = V_ADD_F32_e32 $vgpr30, $vgpr31, implicit $mode, implicit $exec
+ ;
+ ; GFX1251-LABEL: name: test_wmma_bf16f32_16x16x32_bf16_C0_overlaps_D1
+ ; GFX1251: early-clobber $vgpr24_vgpr25_vgpr26_vgpr27 = V_WMMA_BF16F32_16X16X32_BF16_w32_twoaddr killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, killed $vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, 8, killed $vgpr16_vgpr17_vgpr18_vgpr19_vgpr20_vgpr21_vgpr22_vgpr23, 0, 0, 0, 0, implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: $vgpr16 = V_ADD_F32_e32 $vgpr30, $vgpr31, implicit $mode, implicit $exec
+ ;
+ ; GFX1310-LABEL: name: test_wmma_bf16f32_16x16x32_bf16_C0_overlaps_D1
+ ; GFX1310: early-clobber $vgpr24_vgpr25_vgpr26_vgpr27 = V_WMMA_BF16F32_16X16X32_BF16_w32_twoaddr killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, killed $vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, 8, killed $vgpr16_vgpr17_vgpr18_vgpr19_vgpr20_vgpr21_vgpr22_vgpr23, 0, 0, 0, 0, implicit $exec
+ ; GFX1310-NEXT: $vgpr16 = V_ADD_F32_e32 $vgpr30, $vgpr31, implicit $mode, implicit $exec
+ $vgpr24_vgpr25_vgpr26_vgpr27 = V_WMMA_BF16F32_16X16X32_BF16_w32_twoaddr killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, killed $vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, 8, killed $vgpr16_vgpr17_vgpr18_vgpr19_vgpr20_vgpr21_vgpr22_vgpr23, 0, 0, 0, 0, implicit $exec
+ $vgpr16 = V_ADD_F32_e32 $vgpr30, $vgpr31, implicit $mode, implicit $exec
+...
+
+---
+name: test_wmma_bf16f32_16x16x32_bf16_C0_overlaps_D1_with_4_valus_in_between
+body: |
+ bb.0:
+ ; GFX1250-LABEL: name: test_wmma_bf16f32_16x16x32_bf16_C0_overlaps_D1_with_4_valus_in_between
+ ; GFX1250: early-clobber $vgpr24_vgpr25_vgpr26_vgpr27 = V_WMMA_BF16F32_16X16X32_BF16_w32_twoaddr killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, killed $vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, 8, killed $vgpr16_vgpr17_vgpr18_vgpr19_vgpr20_vgpr21_vgpr22_vgpr23, 0, 0, 0, 0, implicit $exec
+ ; GFX1250-NEXT: $vgpr32 = V_MOV_B32_e32 32, implicit $exec
+ ; GFX1250-NEXT: $vgpr33 = V_MOV_B32_e32 33, implicit $exec
+ ; GFX1250-NEXT: $vgpr34 = V_MOV_B32_e32 34, implicit $exec
+ ; GFX1250-NEXT: $vgpr35 = V_MOV_B32_e32 35, implicit $exec
+ ; GFX1250-NEXT: $vgpr16 = V_ADD_F32_e32 $vgpr30, $vgpr31, implicit $mode, implicit $exec
+ ;
+ ; GFX1251-LABEL: name: test_wmma_bf16f32_16x16x32_bf16_C0_overlaps_D1_with_4_valus_in_between
+ ; GFX1251: early-clobber $vgpr24_vgpr25_vgpr26_vgpr27 = V_WMMA_BF16F32_16X16X32_BF16_w32_twoaddr killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, killed $vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, 8, killed $vgpr16_vgpr17_vgpr18_vgpr19_vgpr20_vgpr21_vgpr22_vgpr23, 0, 0, 0, 0, implicit $exec
+ ; GFX1251-NEXT: $vgpr32 = V_MOV_B32_e32 32, implicit $exec
+ ; GFX1251-NEXT: $vgpr33 = V_MOV_B32_e32 33, implicit $exec
+ ; GFX1251-NEXT: $vgpr34 = V_MOV_B32_e32 34, implicit $exec
+ ; GFX1251-NEXT: $vgpr35 = V_MOV_B32_e32 35, implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: $vgpr16 = V_ADD_F32_e32 $vgpr30, $vgpr31, implicit $mode, implicit $exec
+ ;
+ ; GFX1310-LABEL: name: test_wmma_bf16f32_16x16x32_bf16_C0_overlaps_D1_with_4_valus_in_between
+ ; GFX1310: early-clobber $vgpr24_vgpr25_vgpr26_vgpr27 = V_WMMA_BF16F32_16X16X32_BF16_w32_twoaddr killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, killed $vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, 8, killed $vgpr16_vgpr17_vgpr18_vgpr19_vgpr20_vgpr21_vgpr22_vgpr23, 0, 0, 0, 0, implicit $exec
+ ; GFX1310-NEXT: $vgpr32 = V_MOV_B32_e32 32, implicit $exec
+ ; GFX1310-NEXT: $vgpr33 = V_MOV_B32_e32 33, implicit $exec
+ ; GFX1310-NEXT: $vgpr34 = V_MOV_B32_e32 34, implicit $exec
+ ; GFX1310-NEXT: $vgpr35 = V_MOV_B32_e32 35, implicit $exec
+ ; GFX1310-NEXT: $vgpr16 = V_ADD_F32_e32 $vgpr30, $vgpr31, implicit $mode, implicit $exec
+ $vgpr24_vgpr25_vgpr26_vgpr27 = V_WMMA_BF16F32_16X16X32_BF16_w32_twoaddr killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, killed $vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, 8, killed $vgpr16_vgpr17_vgpr18_vgpr19_vgpr20_vgpr21_vgpr22_vgpr23, 0, 0, 0, 0, implicit $exec
+ $vgpr32 = V_MOV_B32_e32 32, implicit $exec
+ $vgpr33 = V_MOV_B32_e32 33, implicit $exec
+ $vgpr34 = V_MOV_B32_e32 34, implicit $exec
+ $vgpr35 = V_MOV_B32_e32 35, implicit $exec
+ $vgpr16 = V_ADD_F32_e32 $vgpr30, $vgpr31, implicit $mode, implicit $exec
+...
+
+---
+name: test_wmma_bf16f32_16x16x32_bf16_A0_overlaps_D1
+body: |
+ bb.0:
+ ; GFX1250-LABEL: name: test_wmma_bf16f32_16x16x32_bf16_A0_overlaps_D1
+ ; GFX1250: early-clobber $vgpr24_vgpr25_vgpr26_vgpr27 = V_WMMA_BF16F32_16X16X32_BF16_w32_twoaddr killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, killed $vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, 8, killed $vgpr16_vgpr17_vgpr18_vgpr19_vgpr20_vgpr21_vgpr22_vgpr23, 0, 0, 0, 0, implicit $exec
+ ; GFX1250-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1250-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1250-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1250-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1250-NEXT: $vgpr0 = V_ADD_F32_e32 $vgpr30, $vgpr31, implicit $mode, implicit $exec
+ ;
+ ; GFX1251-LABEL: name: test_wmma_bf16f32_16x16x32_bf16_A0_overlaps_D1
+ ; GFX1251: early-clobber $vgpr24_vgpr25_vgpr26_vgpr27 = V_WMMA_BF16F32_16X16X32_BF16_w32_twoaddr killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, killed $vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, 8, killed $vgpr16_vgpr17_vgpr18_vgpr19_vgpr20_vgpr21_vgpr22_vgpr23, 0, 0, 0, 0, implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: $vgpr0 = V_ADD_F32_e32 $vgpr30, $vgpr31, implicit $mode, implicit $exec
+ ;
+ ; GFX1310-LABEL: name: test_wmma_bf16f32_16x16x32_bf16_A0_overlaps_D1
+ ; GFX1310: early-clobber $vgpr24_vgpr25_vgpr26_vgpr27 = V_WMMA_BF16F32_16X16X32_BF16_w32_twoaddr killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, killed $vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, 8, killed $vgpr16_vgpr17_vgpr18_vgpr19_vgpr20_vgpr21_vgpr22_vgpr23, 0, 0, 0, 0, implicit $exec
+ ; GFX1310-NEXT: $vgpr0 = V_ADD_F32_e32 $vgpr30, $vgpr31, implicit $mode, implicit $exec
+ $vgpr24_vgpr25_vgpr26_vgpr27 = V_WMMA_BF16F32_16X16X32_BF16_w32_twoaddr killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, killed $vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, 8, killed $vgpr16_vgpr17_vgpr18_vgpr19_vgpr20_vgpr21_vgpr22_vgpr23, 0, 0, 0, 0, implicit $exec
+ $vgpr0 = V_ADD_F32_e32 $vgpr30, $vgpr31, implicit $mode, implicit $exec
+...
+
+# The _threeaddr form never ties C to vdst, for any WMMA.
+
+---
+name: test_wmma_f32_16x16x32_bf16_threeaddr_C0_overlaps_D1
+body: |
+ bb.0:
+ ; GFX1250-LABEL: name: test_wmma_f32_16x16x32_bf16_threeaddr_C0_overlaps_D1
+ ; GFX1250: early-clobber $vgpr24_vgpr25_vgpr26_vgpr27_vgpr28_vgpr29_vgpr30_vgpr31 = V_WMMA_F32_16X16X32_BF16_w32_threeaddr killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, killed $vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, 8, $vgpr16_vgpr17_vgpr18_vgpr19_vgpr20_vgpr21_vgpr22_vgpr23, 0, 0, 0, 0, implicit $exec
+ ; GFX1250-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1250-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1250-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1250-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1250-NEXT: $vgpr16 = V_ADD_F32_e32 $vgpr32, $vgpr33, implicit $mode, implicit $exec
+ ;
+ ; GFX1251-LABEL: name: test_wmma_f32_16x16x32_bf16_threeaddr_C0_overlaps_D1
+ ; GFX1251: early-clobber $vgpr24_vgpr25_vgpr26_vgpr27_vgpr28_vgpr29_vgpr30_vgpr31 = V_WMMA_F32_16X16X32_BF16_w32_threeaddr killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, killed $vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, 8, $vgpr16_vgpr17_vgpr18_vgpr19_vgpr20_vgpr21_vgpr22_vgpr23, 0, 0, 0, 0, implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: V_NOP_e32 implicit $exec
+ ; GFX1251-NEXT: $vgpr16 = V_ADD_F32_e32 $vgpr32, $vgpr33, implicit $mode, implicit $exec
+ ;
+ ; GFX1310-LABEL: name: test_wmma_f32_16x16x32_bf16_threeaddr_C0_overlaps_D1
+ ; GFX1310: early-clobber $vgpr24_vgpr25_vgpr26_vgpr27_vgpr28_vgpr29_vgpr30_vgpr31 = V_WMMA_F32_16X16X32_BF16_w32_threeaddr killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, killed $vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, 8, $vgpr16_vgpr17_vgpr18_vgpr19_vgpr20_vgpr21_vgpr22_vgpr23, 0, 0, 0, 0, implicit $exec
+ ; GFX1310-NEXT: $vgpr16 = V_ADD_F32_e32 $vgpr32, $vgpr33, implicit $mode, implicit $exec
+ $vgpr24_vgpr25_vgpr26_vgpr27_vgpr28_vgpr29_vgpr30_vgpr31 = V_WMMA_F32_16X16X32_BF16_w32_threeaddr killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, killed $vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, 8, $vgpr16_vgpr17_vgpr18_vgpr19_vgpr20_vgpr21_vgpr22_vgpr23, 0, 0, 0, 0, implicit $exec
+ $vgpr16 = V_ADD_F32_e32 $vgpr32, $vgpr33, implicit $mode, implicit $exec
+...
+
---
name: test_wmma_f16_16x16x64_fp8_fp8_D0_overlaps_Use1
body: |
More information about the llvm-commits
mailing list