[llvm] [AMDGPU] Model waterfall loop EXEC update as a terminator (PR #219519)

[Matt Arsenault via llvm-commits]

================
@@ -604,6 +609,92 @@ bool SIOptimizeExecMasking::optimizeExecSequence() {
   return Changed;
 }
 
+bool SIOptimizeExecMasking::blocksAndN2Sink(const MachineInstr &MI,
+                                            Register Dst) const {
+  for (const MachineOperand &MO : MI.operands()) {
+    if (!MO.isReg())
+      continue;
+    if (TRI->regsOverlap(MO.getReg(), Dst))
+      return true;
+    if (MO.isUse() ? TRI->regsOverlap(MO.getReg(), AMDGPU::SCC)
+                   : TRI->regsOverlap(MO.getReg(), LMC.ExecReg))
+      return true;
+  }
+  return false;
+}
+
+// Fold
+//
+//     sdst = S_ANDN2_B32 ssrc, exec
+//     exec = COPY sdst
+// =>
+//     sdst = S_ANDN2_WREXEC_B32 ssrc
+//
+// The waterfall loop emits the two operations separately so that spill code
+// for sdst can be inserted before exec is narrowed.
+bool SIOptimizeExecMasking::optimizeAndN2WrExecSequence(
+    MachineInstr &CopyToExecInst, Register Dst) const {
+  if (!ST->hasNoSdstCMPX() || TII->pseudoToMCOpcode(LMC.AndN2WrExecOpc) == -1)
+    return false;
+
+  MachineBasicBlock &MBB = *CopyToExecInst.getParent();
+
+  // Keep the fused instruction ahead of any trailing meta instructions (e.g.
+  // DBG_VALUEs of Dst), which emit no code, so that the sequence is unchanged
+  // in the common case where the two are already adjacent.
+  MachineBasicBlock::iterator InsertPt = CopyToExecInst.getIterator();
+  while (InsertPt != MBB.begin() && std::prev(InsertPt)->isMetaInstruction())
----------------
arsenm wrote:

The meta instruction case runs the risk of breaking something if the meta instruction references one of the instructions. Do you just need skipDebugInstructionsForward?

https://github.com/llvm/llvm-project/pull/219519