[llvm] [AMDGPU] Per-chain MFMA->AGPR conversion (PR #217328)

[Romanov Vlad via llvm-commits]

================
@@ -1399,20 +1517,57 @@ bool RewriteMFMAFormStage::initGCNSchedStage() {
   TII = ST.getInstrInfo();
   SRI = ST.getRegisterInfo();
 
-  std::vector<std::pair<MachineInstr *, unsigned>> RewriteCands;
-  DenseMap<MachineBasicBlock *, std::set<Register>> CopyForUse;
-  SmallPtrSet<MachineInstr *, 8> CopyForDef;
+  // Collect all convertible MFMAs.
+  SmallVector<MachineInstr *, 32> AllCands;
+  for (MachineBasicBlock &MBB : MF) {
+    for (MachineInstr &MI : MBB) {
+      if (isRewriteCandidate(&MI))
+        AllCands.push_back(&MI);
+    }
+  }
 
-  if (!initHeuristics(RewriteCands, CopyForUse, CopyForDef))
+  if (AllCands.empty())
     return false;
 
-  int64_t Cost = getRewriteCost(RewriteCands, CopyForUse, CopyForDef);
+  // Identify accumulator chains and sort by size descending. We operate at
+  // chain granularity rather than individual MFMAs because:
+  // 1. It avoids converting MFMAs from different chains that would each need
+  //    separate src2 and dst bridge copies (up to 4 copies for 2 MFMAs from
+  //    2 chains) while only reducing VGPR pressure for 2 instructions.
+  // 2. It reduces the search space from O(N_mfmas) to O(N_chains).
+  //
+  // Longer chains benefit more: each chain requires at most one src2 bridge
+  // copy and one dst bridge copy regardless of length, but reduces VGPR
+  // pressure proportionally to the number of chain members.
+  SmallVector<SmallVector<unsigned, 8>, 16> Chains =
+      identifyAccChains(AllCands, TII);
+  llvm::sort(Chains,
+             [](const auto &A, const auto &B) { return A.size() > B.size(); });
+
+  int BestN = findBestChainCount(Chains, AllCands);
----------------
romanovvlad wrote:

Could you please elaborate? What should be the behavior when we opt-out?
In the original logic we were trying to convert all the chains and check if it's profitable, and if it is not we bail out.
With the patch in addition we try to find best profitable subset.

https://github.com/llvm/llvm-project/pull/217328