[llvm] [AMDGPU] Add waterfall intrinsics (PR #192409)

[Nicolai Hähnle via llvm-commits]

https://github.com/nhaehnle commented:

Please add some IR verifier checks (and corresponding tests) that ensure waterfall groups are well-formed.

This style of IR is fragile? What is the long-term direction here? If we think through alternative representations, the options that come to mind are:

1. Do something based on `callbr`. That would allow the waterfall loop body to be in its own basic block. (Risk: It's unclear how well we'd be able to hoist invariant loads across waterfall loops in that representation, and that's important in many cases.)
2. Go the full length towards MLIR-style regions. That'd obviously be a massive change to LLVM IR.
3. Something in between?

https://github.com/llvm/llvm-project/pull/192409