[llvm] [AMDGPU] Add waterfall intrinsics (PR #192409)

[Jay Foad via llvm-commits]

================
@@ -2073,11 +2081,46 @@ bool AMDGPUCodeGenPrepareImpl::visitIntrinsicInst(IntrinsicInst &I) {
   case Intrinsic::uadd_sat:
   case Intrinsic::sadd_sat:
     return visitSaturatingAdd(I);
+  case Intrinsic::amdgcn_waterfall_begin:
+    CurrentWaterfall = nullptr;
+    return false;
+  case Intrinsic::amdgcn_waterfall_end:
+    CurrentWaterfall = &I;
+    return false;
   default:
     return false;
   }
 }
 
+bool AMDGPUCodeGenPrepareImpl::visitGetElementPtrInst(GetElementPtrInst &I) {
+  if (!CurrentWaterfall || UA.isUniformAtDef(&I))
+    return false;
+  if (I.getParent() != CurrentWaterfall->getParent())
+    return false;
+
+  // Divergent GEP within a waterfall region will introduce a nested waterfall.
----------------
jayfoad wrote:

I would prefer to split this part into a separate PR, even if we do end up merging it as-is.

https://github.com/llvm/llvm-project/pull/192409