[llvm] [AMDGPU] Improve the description of asyncmark semantics (PR #202579)
via llvm-commits
llvm-commits at lists.llvm.org
Tue Jun 9 04:08:25 PDT 2026
llvmorg-github-actions[bot] wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-backend-amdgpu
Author: Sameer Sahasrabuddhe (ssahasra)
<details>
<summary>Changes</summary>
- The semantics of asyncmarks is now definded purely in terms of sequences, without referring to the implementation.
- The examples incorrectly used (post)dominance. Fixed that with wording in terms of asyncmark sequences.
---
Full diff: https://github.com/llvm/llvm-project/pull/202579.diff
1 Files Affected:
- (modified) llvm/docs/AMDGPUAsyncOperations.rst (+113-51)
``````````diff
diff --git a/llvm/docs/AMDGPUAsyncOperations.rst b/llvm/docs/AMDGPUAsyncOperations.rst
index 0b8ea0ae77174..90aac42088be2 100644
--- a/llvm/docs/AMDGPUAsyncOperations.rst
+++ b/llvm/docs/AMDGPUAsyncOperations.rst
@@ -13,9 +13,7 @@ Introduction
Asynchronous operations are memory transfers (usually between the global memory
and LDS) that are completed independently at an unspecified scope. A thread that
requests one or more asynchronous transfers can use *asyncmarks* to track
-their completion. The thread waits for each asyncmark to be *completed*, which
-indicates that requests initiated in *program-order* before this asyncmark have also
-completed.
+their completion.
Operations
==========
@@ -57,49 +55,53 @@ memory and LDS memory.
void @llvm.amdgcn.tensor.load.to.lds(...)
void @llvm.amdgcn.tensor.store.from.lds(...)
-Asyncmark Operations
----------------------
+Asyncmarks
+----------
An *asyncmark* in the abstract machine tracks all the async operations that
-are *program-ordered* before that asyncmark. An asyncmark M is said to be *completed*
-only when all async operations *program-ordered* before M are reported by the
-implementation as having finished, and it is said to be *outstanding* otherwise.
-
-Thus we have the following sufficient condition:
-
- An async operation X is *completed* at a program point P if there exists an
- asyncmark M such that X is *program-ordered* before M, M is *program-ordered* before
- P, and M is completed. X is said to be *outstanding* at P otherwise.
+are *program-ordered* before that asyncmark.
The abstract machine maintains a sequence of asyncmarks during the
execution of a function body, which excludes any asyncmarks produced by calls to
other functions encountered in the currently executing function.
+This sequence is called the *current sequence* of that function body.
``@llvm.amdgcn.asyncmark()``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-When executed, inserts an asyncmark in the sequence associated with the
-currently executing function body.
+Appends an asyncmark to the current sequence.
``@llvm.amdgcn.wait.asyncmark(i16 %N)``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-Waits until there are at most N outstanding asyncmarks in the sequence associated
-with the currently executing function body.
+Ensures that the length of the current sequence is at most ``N`` by removing
+asyncmarks from the start of the sequence if it is more than ``N``.
Memory Consistency Model
========================
-Each asynchronous operation consists of a non-atomic read on the source and a
-non-atomic write on the destination. Async "LDS DMA" intrinsics result in async
-accesses that guarantee visibility relative to other memory operations as
-follows:
+An ``asyncmark()`` operation ``X`` that inserts an asyncmark ``M`` is
+*completed-at* a ``wait.asyncmark()`` operation ``Y`` in the same function body
+if:
+
+- ``X`` is *program-ordered* before ``Y``, and
+- ``M`` is not in the current sequence after ``Y`` returns.
+
+An async operation ``A`` is *completed-at* a ``wait.asyncmark()`` operation
+``Y`` if there exists an ``asyncmark()`` operation ``X`` such that:
+
+- ``A`` is *program-ordered* before ``X``, and
+- ``X`` is *completed-at* ``Y``.
- An asynchronous operation `A` program ordered before an overlapping memory
- operation `X` happens-before `X` only if `A` is completed before `X`.
+An asynchronous operation ``A`` *happens-before* an overlapping memory operation
+``B`` only if there exists a ``wait.asyncmark()`` operation ``Y`` such that:
- A memory operation `X` program ordered before an overlapping asynchronous
- operation `A` happens-before `A`.
+- ``A`` is *program-ordered* before ``Y``, and
+- ``Y`` is *program-ordered* before ``B``, and
+- ``A`` is *completed-at* ``Y``.
+
+A memory operation ``B`` *happens-before* an overlapping asynchronous
+operation ``A`` if ``B`` is *program-ordered* before ``A``.
.. note::
@@ -192,8 +194,11 @@ Ordinary function call
// third block
asyncmark();
- wait.asyncmark(1); // wait for the second block
- wait.asyncmark(0); // will wait for third block, including bar()
+ // wait for the second block
+ wait.asyncmark(1);
+
+ // wait for the third block, including bar()
+ wait.asyncmark(0);
}
Implementation notes
@@ -201,17 +206,8 @@ Implementation notes
[This section is informational.]
-Optimization
-------------
-
-The implementation may eliminate asyncmark/wait intrinsics in the following cases:
-
-1. An ``asyncmark`` operation which is not included in the wait count of a later
- wait operation in the current function. In particular, an ``asyncmark`` which
- is not post-dominated by any ``wait.asyncmark``.
-2. A ``wait.asyncmark`` whose wait count is more than the outstanding async
- asyncmarks at that point. In particular, a ``wait.asyncmark`` that is not
- dominated by any ``asyncmark``.
+Function Calls
+--------------
In general, at a function call, if the caller uses sufficient waits to track
its own async operations, the actions performed by the callee cannot affect
@@ -220,30 +216,96 @@ correctness. But inlining such a call may result in redundant waits.
.. code-block:: c++
void foo() {
- asyncmark(); // A
+ ...
+ asyncmark(); // X
+ ... // no wait.asyncmark()
}
void bar() {
- asyncmark(); // B
- asyncmark(); // C
+ asyncmark(); // B
+ asyncmark(); // C
foo();
- wait.asyncmark(1);
+ wait.asyncmark(1); // D
+ }
+
+Before inlining, it is unspecified whether ``X`` is *completed-at* ``D``, while
+``C`` is **not** *completed-at* ``D``. The programmer can only rely on ``B``
+being *completed-at* ``D``.
+
+.. code-block:: c++
+
+ void bar() {
+ asyncmark(); // B
+ asyncmark(); // C
+ asyncmark(); // X
+ wait.asyncmark(1); // D
}
-Before inlining, the ``wait.asyncmark`` waits for asyncmark B to be completed.
+After inlining, ``C`` is also *completed-at* ``D`` and ``X`` is **not**
+*completed-at* ``D``.
+
+Conversely, a ``wait.asyncmark`` call inside a callee cannot be used to track
+asyncmarks inserted by the caller, since this ``wait.asyncmark`` can only
+observe the current sequence of the callee.
.. code-block:: c++
void foo() {
+ ... // no asyncmark()
+ wait.asyncmark(0); // Y
+ ...
}
void bar() {
- asyncmark(); // B
- asyncmark(); // C
- asyncmark(); // A from call to foo()
- wait.asyncmark(1);
+ asyncmark(); // B
+ asyncmark(); // C
+ foo();
+ wait.asyncmark(1); // D
+ }
+
+In the above example, it is unspecified whether ``B`` and ``C`` in ``bar()`` are
+*completed-at* ``Y``, because they are not included in the sequence that can be
+examined at ``Y``.
+
+.. code-block:: c++
+
+ void bar() {
+ asyncmark(); // B
+ asyncmark(); // C
+ wait.asyncmark(0); // Y
+ wait.asyncmark(1); // D
}
-After inlining, the ``wait.asyncmark`` now waits for asyncmark C to complete, which is
-longer than necessary. Ideally, the optimizer should have eliminated asyncmark A in
-the body of foo() itself.
+After inlining, both ``B`` and ``C`` are *completed-at* ``Y``.
+
+Optimization
+------------
+
+The implementation may eliminate asyncmark/wait intrinsics in the following
+cases. These are just examples and not meant to be an exhaustive list.
+
+1. An ``asyncmark`` operation which remains in the current sequence along every
+ path that reaches the function exit.
+
+ .. code-block:: c++
+
+ void foo() {
+ ...
+ asyncmark(); // X
+ ... // no wait.asyncmark()
+ }
+
+ Here, ``X`` can be eliminated.
+
+2. A ``wait.asyncmark`` which sees an empty sequence of asyncmarks along every
+ path that reaches it.
+
+ .. code-block:: c++
+
+ void foo() {
+ ... // no asyncmark()
+ wait.asyncmark(0); // Y
+ ...
+ }
+
+ Here, ``Y`` can be eliminated.
``````````
</details>
https://github.com/llvm/llvm-project/pull/202579
More information about the llvm-commits
mailing list