[llvm] [AMDGPU] Add waterfall intrinsics (PR #192409)

[Sebastian Neubauer via llvm-commits]

================
@@ -1776,6 +1776,59 @@ The AMDGPU backend implements the following LLVM IR intrinsics.
                                                    * :ref:`Synchronization Scope<amdgpu-intrinsics-syncscope-metadata-operand>`.
                                                      Note that the scope used must ensure that the L2 cache will be hit.
 
+  llvm.amdgcn.waterfall.begin                      Marks the beginning of a waterfall region of code.
+
+                                                   The compiler generates a waterfall loop around the region.
+                                                   A waterfall loop handles the case where an operation that requires
+                                                   a uniform operand (e.g., held in an SGPR) is applied to a divergent operand
+                                                   (held in a VGPR, with values varying per lane).
+                                                   Each iteration of the waterfall loop activates a subset of lanes that
+                                                   share the same value of the VGPR (the value in the first active lane).
+                                                   The operation is then executed using that value as the uniform operand.
+                                                   If the VGPR is already uniform, the waterfall loop executes only once.
+                                                   The worst case for a waterfall loop is one iteration per lane (all lanes
+                                                   have different values of the VGPR), but it is not common in practice.
+
+                                                   The intrinsic takes a previous token
+                                                   (``i32``; use a null/zero value if this is the first ``waterfall.begin`` in
----------------
Flakebi wrote:

As this is becoming official LLVM intrinsics, I think we can use LLVM’s `token` type here instead of `i32`

https://github.com/llvm/llvm-project/pull/192409