[llvm] [AMDGPU] Add RewriteMFMAForm scheduling stage tests (PR #196008)

Matt Arsenault via llvm-commits llvm-commits at lists.llvm.org
Wed May 13 06:04:11 PDT 2026


================
@@ -0,0 +1,332 @@
+; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx950 \
+; RUN:     -amdgpu-disable-rewrite-mfma-form-sched-stage=false \
+; RUN:     < %s | FileCheck %s
+;
+; Test: RewriteMFMAFormStage — v7_slice pattern (gfx950, mfma.f32.16x16x32.f16)
+;
+; Distilled from v7_slice.llir (Triton matmul kernel, gfx950).
+;
+; Key structural features preserved from v7_slice:
+;   (1) Loop body (%loop) accumulates via loop-carried SCALAR float phis,
+;       NOT <4 x float> phis. Each MFMA acc is built by insertelement from
+;       4 individual float phis then used as src2/dst in MFMAs within the loop.
+;   (2) Epilogue (%epilogue) ALSO contains MFMAs: the final K-tile iteration
+;       is peeled. Epilogue MFMAs take scalar float phis from %loop as acc,
+;       then produce results consumed by fptrunc+store — Case 2.
+;   (3) Scalar float phis initialized to 0.0 in entry (non-MAI def) — Case 3.
+;
+; CFG: entry -> loop (back-edge) -> epilogue -> ret
+;
+; Pressure design (gfx950, 256 ArchVGPR limit):
+;   16 loop-carried scalar float phis (4 chains x 4 elems) = 16 VGPRs.
+;   4 MFMA chains of 2, each acc = 4 float phis = 16 VGPRs in loop.
+;   4 x <8 x half> operands (a0, a1, b0, b1) = 16 VGPRs.
+;   8 x <32 x float> loop-carried vec carriers (vc0-vc7) = 256 VGPRs.
+;   Peak ArchVGPR = 256 + 16 + 16 + 16 = 304 > 256 -> RegionsWithExcessArchVGPR.
+;   After rewrite: scalar phis=16 VGPR, <8 x half>=16 VGPR, vec=256 VGPR, MFMA acc->AGPR.
+;   getRewriteCost() < 0 -> rewrite() fires.
+;
+; Expected:
+;   Case 3: v_accvgpr_write inserted after insertelement defs of MFMA acc
+;           (entry zeroinitializer -> scalar float phi -> insertelement = non-MAI)
+;   Case 2: v_accvgpr_read before fptrunc in epilogue block
+;   MFMA opcodes: v_mfma_f32_16x16x32_f16 vgprcd -> agprcd form
+
----------------
arsenm wrote:

Same comment, this looks like overly verbose AI description 

https://github.com/llvm/llvm-project/pull/196008


More information about the llvm-commits mailing list