[llvm-branch-commits] [llvm] [AMDGPU] Model ordered XDL writes in expert scheduling (PR #218326)

[Austin Kerbow via llvm-branch-commits]

================
@@ -884,14 +903,26 @@ void WaitcntBrackets::updateByEvent(HWEvents E, MachineInstr &Inst) {
     Increment = 2;
   }
   unsigned CurrScore = UB + Increment;
-  if (CurrScore == 0)
+  // Increment can be two, so unsigned overflow may wrap to one instead of zero.
+  if (CurrScore <= UB)
     report_fatal_error("InsertWaitcnt score wraparound");
   // PendingEvents and ScoreUB need to be update regardless if this event
   // changes the score of a register or not.
   // Examples including vm_cnt when buffer-store or lgkm_cnt when send-message.
   PendingEvents |= E;
   setScoreUB(T, CurrScore);
 
+  if (E == HWEvents::VGPR_XDL_WRITE && Context->TII.isVAVDSTOrderedXDL(Inst)) {
+    // While an earlier write is pending, every later ordered XDL write must
+    // also be pending. Count those writes as a per-dependency wait bound.
+    for (auto &Entry : VMem) {
+      auto &Info = Entry.second;
+      if (Info.Scores[AMDGPU::VA_VDST_WR] > getScoreLB(AMDGPU::VA_VDST_WR) &&
+          Info.NumOrderedXDLsAfterWrite < getLimit(AMDGPU::VA_VDST_WR) - 1)
----------------
kerbowa wrote:

I added a test for this. The largest usable bound is 14, i.e. 15-1.

https://github.com/llvm/llvm-project/pull/218326