[all-commits] [llvm/llvm-project] 62b7cf: [AMDGPU] Widen MUBUF/MTBUF source-vgpr WAR hazard ...

[Sanket Pandit via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 62b7cf9623fc310525f39ed69aaecc318a909731
      https://github.com/llvm/llvm-project/commit/62b7cf9623fc310525f39ed69aaecc318a909731
  Author: Sanket Pandit <sanket.p25 at gmail.com>
  Date:   2026-06-01 (Mon, 01 Jun 2026)

  Changed paths:
    M llvm/lib/Target/AMDGPU/GCNHazardRecognizer.cpp
    A llvm/test/CodeGen/AMDGPU/buffer-store-dwordx4-vpk-mul-war-hazard-gfx942.mir

  Log Message:
  -----------
  [AMDGPU] Widen MUBUF/MTBUF source-vgpr WAR hazard on gfx940-family to SGPR soffset (#197267)

createsVALUHazard previously gated the MUBUF/MTBUF source-vgpr WAR
hazard to fire only when SOFFSET was a literal or absent. On
gfx940-family subtargets that gate is too narrow: the hazard also fires
when SOFFSET is sourced from an SGPR.

Concretely, on gfx950 a sequence of the form

```
  buffer_store_dwordx4 v[X:X+3], voff, descr, sN offen
  v_pk_mul_f32 v[X:X+1], <src>, <src>           # next VALU cycle
```

deterministically commits the post-pk_mul value of v[X+1] to memory for
the second dword of the store; the other three dwords store correctly.

The wait-state window depends on the SOFFSET shape:

  - sgpr-sourced SOFFSET: 1 wait state  (S_NOP 0)
  - literal/absent SOFFSET: 2 wait states (S_NOP 1)
  - pre-gfx940 literal SOFFSET (unchanged): 1 wait state
  - pre-gfx940 sgpr SOFFSET (unchanged): no hazard

Implementation widens createsVALUHazard's trigger on gfx940-family and
splits checkVALUHazardsHelper's single 2-wait-state scan into a per-MI
window: a 1-cycle scan and a 2-cycle scan, taking the max. FLAT-wide
stores keep the existing 2-wait-state behavior on gfx940 family.

Empirically reproduced and characterised on gfx950 by Triton's
fused-attention backward kernel; the threshold sweep in
repro/minimal_hazard pins the sgpr-SOFFSET window at exactly 1 cycle (a
single S_NOP 0 clears the corruption). gfx942 (MI300) was not validated
on hardware; the same gate is applied there because the existing
recognizer rule treats the gfx940 family uniformly.

New MIR test covers literal and SGPR soffset on gfx900 (older), gfx942
and gfx950, plus negative cases (dwordx2 store, non-overlapping write).

Assisted-by: Cursor <cursoragent at cursor.com>



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications