[llvm] [AMDGPU] Add safe-guard exclusion for MFMA form rewrite (PR #207672)

[via llvm-commits]

================
@@ -0,0 +1,635 @@
+# RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx90a \
+# RUN:     -run-pass=machine-scheduler \
+# RUN:     -amdgpu-disable-rewrite-mfma-form-sched-stage=false \
+# RUN:     %s -o - | FileCheck %s
+
+--- |
+  define void @test_src2_backedge_bypass_excluded() #0 {
+  entry:
+    unreachable
+  }
+  define void @test_subreg_overwrite_cross_bb() #0 {
+  entry:
+    unreachable
+  }
+  define void @test_src2_avmov_agprform_rewrite() #0 {
+  entry:
+    unreachable
+  }
+  define void @test_src2_buffer_load_early_safe_rewrite() #0 {
+  entry:
+    unreachable
+  }
+  define void @test_src2_selfref_avmov_init() #0 {
+  entry:
+    unreachable
+  }
+  define void @test_dst_subreg_loop_copy_consumer() #0 {
+  entry:
+    br label %loop
+  loop:
+    br i1 undef, label %loop, label %exit
+  exit:
+    ret void
+  }
+  define void @test_dst_subreg_copy_valu_consumer() #0 {
+  entry:
+    unreachable
+  }
+  define void @test_partial_exclude_avmov_src2() #0 {
+  entry:
+    unreachable
+  }
+  attributes #0 = { "amdgpu-waves-per-eu"="1,1" "amdgpu-flat-work-group-size"="64,64" }
+...
+
+---
+#
+# Loop with bypass path triggers hasSrc2BridgeConflict.
+#
+# CFG:
+#   bb.0 ──► bb.1 (preheader) ──► bb.2 (loop) ──► bb.3 (bridge) ──► (ret)
+#   (entry)                              ↑___________|
+#     │                              (back-edge)
+#     └────────────────────────────────────────────► bb.3   (bypass)
+#
+# %acc reaching defs for MFMA_A src2: {AV_MOV×4 (bb.1, non-MAI), MFMA_C (bb.2, MAI)}.
+# hasSrc2BridgeConflict: bb.1 does not dominate bb.3 (bypass skips bb.1) → excluded.
+# MFMA_C excluded by backward propagation; MFMA_B by forward propagation.
+# All three loop MFMAs stay in vreg form.
+#
+# Without exclusion: %acc reclassified to AGPR; the AV_MOV subreg inits in bb.1
+# have no bridge-copy reader and are DCE'd to dead undef.  On the first iteration
+# (bb.0→bb.1→bb.2) MFMA_A reads the IMPLICIT_DEF garbage from bb.0 instead of
+# the intended initial accumulator → silent wrong result on iteration 1.
+#
+# CHECK-LABEL: name: test_src2_backedge_bypass_excluded
+# CHECK: bb.2:
+# CHECK:   %r1:vreg_128_align2 = nofpexcept V_MFMA_F32_16X16X4F32_vgprcd_e64
+# CHECK:   %r2:vreg_128_align2 = nofpexcept V_MFMA_F32_16X16X4F32_vgprcd_e64
+# CHECK:   %acc:vreg_128_align2 = nofpexcept V_MFMA_F32_16X16X4F32_vgprcd_e64
+# CHECK-NOT: V_MFMA_F32_16X16X4F32_e64
+name:            test_src2_backedge_bypass_excluded
----------------
xgxanq wrote:

> > Without exclusion: %acc reclassified to AGPR; the AV_MOV subreg inits in bb.1
> > have no bridge-copy reader and are DCE'd to dead undef. On the first iteration
> > (bb.0→bb.1→bb.2) MFMA_A reads the IMPLICIT_DEF garbage from bb.0 instead of
> > the intended initial accumulator → silent wrong result on iteration 1.
> 
> Right, that's what happens without the exclusion. But is there a fundamental correctness limitation that prevents reclassification here, or is it just a case we chose not to handle? It seems like we could still do the reclassification with proper copy insertion, as I showed above.

For this entire class of edge cases, we are opting not to modify the core rewrite mechanism at this stage. Patching each individual scenario yields limited practical value: while some problematic cases are fixable with minor tweaks, resolving others would require a large-scale overhaul of the rewrite logic, which is not justified by the corresponding benefits.

https://github.com/llvm/llvm-project/pull/207672