[llvm] [AMDGPU] Sink async DMA out of s_cbranch_execz then-blocks (PR #196374)
Juan Manuel Martinez CaamaƱo via llvm-commits
llvm-commits at lists.llvm.org
Fri Jul 31 01:08:01 PDT 2026
================
@@ -0,0 +1,253 @@
+//===-- SISinkAsyncDMA.cpp - Sink async DMA out of execz then-blocks ------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+/// \file
+/// LLVM lowers a divergent branch around global_load_async_to_lds /
+/// global_store_async_from_lds with an S_CBRANCH_EXECZ. Fully-masked waves
+/// therefore skip the DMA entirely, which makes the ASYNCcnt observed at the
+/// join point depend on whether the wave took the branch. Software-pipelined
+/// kernels must then use conservative async waitcnts.
+///
+/// This pass sinks each async DMA out of such a then-block into the join
+/// block, immediately before the EXEC-mask restore (the S_OR that ends the
+/// control-flow region). EXEC at the sunk slot is the same masked value that
+/// guarded the then-block, so per-lane behavior is unchanged, but every wave
+/// now issues the DMA and ASYNCcnt becomes deterministic.
+///
+/// This runs right after SILowerControlFlow so the EXEC restore anchor exists,
+/// and before waitcnt insertion so the improved counts can be used.
+///
+/// This is determinism-only: it never relaxes or rewrites a wait (an
+/// s_wait_asynccnt 0 stays 0). Correctness never depends on the transform
+/// firing; with the pass disabled the code is still correct, just conservative.
----------------
jmmartinez wrote:
Since this part seems important for the final goal. How are you planning to deal with this?
https://github.com/llvm/llvm-project/pull/196374
More information about the llvm-commits
mailing list