[llvm] [Docs][AMDGPU] availability/visibility in addrspace(3) (PR #212707)
Sameer Sahasrabuddhe via llvm-commits
llvm-commits at lists.llvm.org
Wed Jul 29 04:28:15 PDT 2026
https://github.com/ssahasra updated https://github.com/llvm/llvm-project/pull/212707
>From c6ea16dcf1e84166791afd116dfe37f6e2b3e371 Mon Sep 17 00:00:00 2001
From: Sameer Sahasrabuddhe <sameer.sahasrabuddhe at amd.com>
Date: Wed, 29 Jul 2026 13:47:36 +0530
Subject: [PATCH 1/4] [Docs][AMDGPU] availability/visibility in addrspace(3)
addrspace(3) on AMDGPU corresponds to LDS, which is a fast memory directly
accessed by all threads in a workgroup (there is no intervening cache). Thus,
any accesses to this addrspace have built-in availability and visibility at
"workgroup" scope.
---
llvm/docs/AMDGPUMemoryModel.md | 67 +++++++++++++++++++++++++---------
1 file changed, 50 insertions(+), 17 deletions(-)
diff --git a/llvm/docs/AMDGPUMemoryModel.md b/llvm/docs/AMDGPUMemoryModel.md
index c057909b0539c..226dfe2983963 100644
--- a/llvm/docs/AMDGPUMemoryModel.md
+++ b/llvm/docs/AMDGPUMemoryModel.md
@@ -126,47 +126,80 @@ these necessary conditions, and hence they can be explained using the rules from
either memory model. But the new intrinsics and metadata *opt out* of the LLVM
memory model, and can only be explained using the AMDGPU memory model.
+### Store-Available and Load-Visible
+
(amdgpu-store-available)=
-### store-available
+#### store-available
```llvm
- at llvm.amdgcn.av.global.store.b128(ptr, value, scope)
-store atomic [syncscope("<target-scope>")]
-atomicrmw [syncscope("<target-scope>")]
-cmpxchg [syncscope("<target-scope>")]
+ at llvm.amdgcn.av.store.b128.p1(
+ ptr addrspace(1) %addr,
+ <4 x i32> %value,
+ metadata !scope)
```
The `@llvm.amdgcn.av.global.store.b128` intrinsic performs a non-atomic
-*store-available* operation on `ptr` with scope `scope`.
+*store-available* operation on `%addr` with scope `!scope`.
+
+```llvm
+store atomic ..., [syncscope("<target-scope>")]
+atomicrmw ..., [syncscope("<target-scope>")]
+cmpxchg ..., [syncscope("<target-scope>")]
+```
An atomic operation that results in a store operation is a *store-available*
operation with scope `syncscope`.
+```llvm
+store ..., ptr addrspace(N) %addr
+```
+
+A `store` operation to `addrspace(3)` is always performed as a *store-available*
+operation with "workgroup" scope.
+
+A `store` operation to `addrspace(0)` is performed as a *store-available*
+operation with "workgroup" scope if the flat address resolves to `addrspace(3)`.
+
(amdgpu-load-visible)=
-### load-visible
+#### load-visible
```llvm
- at llvm.amdgcn.av.global.load.b128(ptr, scope)
-load atomic [syncscope("<target-scope>")]
-atomicrmw [syncscope("<target-scope>")]
-cmpxchg [syncscope("<target-scope>")]
+%value = @llvm.amdgcn.av.global.load.b128(
+ ptr addrspace(1) %addr,
+ metadata !scope)
```
The `@llvm.amdgcn.av.global.load.b128` intrinsic performs a non-atomic
-*load-visible* operation on `ptr` with scope `scope`.
+*load-visible* operation on `%addr` with scope `!scope`.
+
+```llvm
+%value = load atomic ..., [syncscope("<target-scope>")]
+%value = atomicrmw ..., [syncscope("<target-scope>")]
+%value = cmpxchg ..., [syncscope("<target-scope>")]
+```
An atomic operation that results in a read operation is a *load-visible*
operation with scope `syncscope`.
+```llvm
+%value = load ..., ptr addrspace(N) %addr
+```
+
+A `load` operation to `addrspace(3)` is always performed as a *load-visible*
+operation with "workgroup" scope.
+
+A `load` operation to `addrspace(0)` is performed as a *load-visible* operation
+with "workgroup" if the flat address resolves to `addrspace(3)`.
+
:::{note}
Metadata cannot be used to model this using ordinary load/store operations,
because the scope is necessary for correctness. In a hypothetical operation
like this:
```llvm
-store ptr, data, !mmra !{!"amdgcn-av", !"workgroup"}
+store ptr addrspace(1) %ptr, <4 x i32> %value, !mmra !{!"amdgcn-av", !"workgroup"}
```
If the metadata is dropped or ignored, there is no guarantee that the store
@@ -367,7 +400,7 @@ particular, the following instructions are equivalent.
```{list-table}
:header-rows: 1
-:widths: 20 20 60
+:widths: 25 20 55
* - LLVM
- SPIRV
@@ -387,19 +420,19 @@ particular, the following instructions are equivalent.
* - `load atomic`
- `OpAtomicLoad`
- `MakePointerVisible`. Also `MakeVisible` when order is at least `acquire`.
- * - `load atomic !{!"amdgcn-av", !"none"}`
+ * - `load atomic`<br>`!{!"amdgcn-av", !"none"}`
- `OpAtomicLoad`
- `MakePointerVisible`
* - `store atomic`
- `OpAtomicStore`
- `MakePointerAvailable`. Also `MakeAvailable` when order is at least `release`.
- * - `store atomic !{!"amdgcn-av", !"none"}`
+ * - `store atomic`<br>`!{!"amdgcn-av", !"none"}`
- `OpAtomicStore`
- `MakePointerAvailable`
* - `fence`
- `OpMemoryBarrier`
- `MakeAvailable` when order is at least `release`, and `MakeVisible` when order is at least `acquire`.
- * - `fence !{!"amdgcn-av", !"none"}`
+ * - `fence`<br>`!{!"amdgcn-av", !"none"}`
- `OpMemoryBarrier`
- \-
```
>From 1a0baa2162e2054cc83b6ff4ce60823d7037770a Mon Sep 17 00:00:00 2001
From: Sameer Sahasrabuddhe <sameer.sahasrabuddhe at amd.com>
Date: Wed, 29 Jul 2026 13:56:42 +0530
Subject: [PATCH 2/4] fix copy-pasta
---
llvm/docs/AMDGPUMemoryModel.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/llvm/docs/AMDGPUMemoryModel.md b/llvm/docs/AMDGPUMemoryModel.md
index 226dfe2983963..e444081b4fd7b 100644
--- a/llvm/docs/AMDGPUMemoryModel.md
+++ b/llvm/docs/AMDGPUMemoryModel.md
@@ -133,7 +133,7 @@ memory model, and can only be explained using the AMDGPU memory model.
#### store-available
```llvm
- at llvm.amdgcn.av.store.b128.p1(
+ at llvm.amdgcn.av.global.store.b128(
ptr addrspace(1) %addr,
<4 x i32> %value,
metadata !scope)
>From c90f91af67782cc25d77acdcdaa60d196b6df085 Mon Sep 17 00:00:00 2001
From: Sameer Sahasrabuddhe <sameer.sahasrabuddhe at amd.com>
Date: Wed, 29 Jul 2026 14:21:59 +0530
Subject: [PATCH 3/4] add addrspace names, also update intrinsics to match
AMDGPU Usage Manual
---
llvm/docs/AMDGPUMemoryModel.md | 37 +++++++++++++++++-----------------
llvm/docs/AMDGPUUsage.rst | 2 ++
2 files changed, 20 insertions(+), 19 deletions(-)
diff --git a/llvm/docs/AMDGPUMemoryModel.md b/llvm/docs/AMDGPUMemoryModel.md
index e444081b4fd7b..fba3aadb32958 100644
--- a/llvm/docs/AMDGPUMemoryModel.md
+++ b/llvm/docs/AMDGPUMemoryModel.md
@@ -133,14 +133,12 @@ memory model, and can only be explained using the AMDGPU memory model.
#### store-available
```llvm
- at llvm.amdgcn.av.global.store.b128(
- ptr addrspace(1) %addr,
- <4 x i32> %value,
- metadata !scope)
+ at llvm.amdgcn.av.store.b128(%addr, %value, metadata !scope)
```
-The `@llvm.amdgcn.av.global.store.b128` intrinsic performs a non-atomic
-*store-available* operation on `%addr` with scope `!scope`.
+The `@llvm.amdgcn.av.store.b128` intrinsic performs a non-atomic
+*store-available* operation on `%addr` with scope `!scope`. See
+{ref}`amdgpu-av-load-store` for details.
```llvm
store atomic ..., [syncscope("<target-scope>")]
@@ -155,24 +153,24 @@ operation with scope `syncscope`.
store ..., ptr addrspace(N) %addr
```
-A `store` operation to `addrspace(3)` is always performed as a *store-available*
-operation with "workgroup" scope.
+A `store` operation to `addrspace(3)` (local or LDS) is always performed as a
+*store-available* operation with "workgroup" scope.
-A `store` operation to `addrspace(0)` is performed as a *store-available*
-operation with "workgroup" scope if the flat address resolves to `addrspace(3)`.
+A `store` operation to `addrspace(0)` (generic or flat) is performed as a
+*store-available* operation with "workgroup" scope if the flat address resolves
+to `addrspace(3)`.
(amdgpu-load-visible)=
#### load-visible
```llvm
-%value = @llvm.amdgcn.av.global.load.b128(
- ptr addrspace(1) %addr,
- metadata !scope)
+%value = @llvm.amdgcn.av.load.b128(%addr, metadata !scope)
```
-The `@llvm.amdgcn.av.global.load.b128` intrinsic performs a non-atomic
-*load-visible* operation on `%addr` with scope `!scope`.
+The `@llvm.amdgcn.av.load.b128` intrinsic performs a non-atomic *load-visible*
+operation on `%addr` with scope `!scope`. See {ref}`amdgpu-av-load-store` for
+details.
```llvm
%value = load atomic ..., [syncscope("<target-scope>")]
@@ -187,11 +185,12 @@ operation with scope `syncscope`.
%value = load ..., ptr addrspace(N) %addr
```
-A `load` operation to `addrspace(3)` is always performed as a *load-visible*
-operation with "workgroup" scope.
+A `load` operation to `addrspace(3)` (local or LDS) is always performed as a
+*load-visible* operation with "workgroup" scope.
-A `load` operation to `addrspace(0)` is performed as a *load-visible* operation
-with "workgroup" if the flat address resolves to `addrspace(3)`.
+A `load` operation to `addrspace(0)` (generic or flat) is performed as a
+*load-visible* operation with "workgroup" if the flat address resolves to
+`addrspace(3)`.
:::{note}
Metadata cannot be used to model this using ordinary load/store operations,
diff --git a/llvm/docs/AMDGPUUsage.rst b/llvm/docs/AMDGPUUsage.rst
index 2a4f290effc6d..9b829dcf9e818 100644
--- a/llvm/docs/AMDGPUUsage.rst
+++ b/llvm/docs/AMDGPUUsage.rst
@@ -2114,6 +2114,8 @@ The AMDGPU backend implements the following LLVM IR intrinsics.
List AMDGPU intrinsics.
+.. _amdgpu-av-load-store:
+
'``llvm.amdgcn.av``' Intrinsics
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>From 36b1df626ec35cf83c7fb8286fcee1146624d5de Mon Sep 17 00:00:00 2001
From: Sameer Sahasrabuddhe <sameer.sahasrabuddhe at amd.com>
Date: Wed, 29 Jul 2026 16:56:53 +0530
Subject: [PATCH 4/4] explain store-available and load-visible, make names
consistent
---
llvm/docs/AMDGPUMemoryModel.md | 55 ++++++++++++++++++++--------------
1 file changed, 33 insertions(+), 22 deletions(-)
diff --git a/llvm/docs/AMDGPUMemoryModel.md b/llvm/docs/AMDGPUMemoryModel.md
index fba3aadb32958..c8d06e11d6b77 100644
--- a/llvm/docs/AMDGPUMemoryModel.md
+++ b/llvm/docs/AMDGPUMemoryModel.md
@@ -126,11 +126,13 @@ these necessary conditions, and hence they can be explained using the rules from
either memory model. But the new intrinsics and metadata *opt out* of the LLVM
memory model, and can only be explained using the AMDGPU memory model.
-### Store-Available and Load-Visible
-
(amdgpu-store-available)=
-#### store-available
+### store-available
+
+A *store-available* operation `X` writes data to the specified location, and
+also propagates side-effects up to its scope instance `S` and also all
+the subscope instances of `S` that contain `X`.
```llvm
@llvm.amdgcn.av.store.b128(%addr, %value, metadata !scope)
@@ -153,16 +155,20 @@ operation with scope `syncscope`.
store ..., ptr addrspace(N) %addr
```
-A `store` operation to `addrspace(3)` (local or LDS) is always performed as a
-*store-available* operation with "workgroup" scope.
+A `store` operation to `addrspace(3)` (local or LDS) is a *store-available*
+operation with "workgroup" scope.
-A `store` operation to `addrspace(0)` (generic or flat) is performed as a
-*store-available* operation with "workgroup" scope if the flat address resolves
-to `addrspace(3)`.
+A `store` operation to `addrspace(0)` (generic or flat) is a *store-available*
+operation with "workgroup" scope if the flat address resolves to `addrspace(3)`.
(amdgpu-load-visible)=
-#### load-visible
+### load-visible
+
+A *load-visible* operation `Y` retrieves the state of the specified location
+from its scope instance `S`, and then loads the value at that location. In
+addition, it makes the retrieved state visible to all subscope instances of `S`
+that contain `Y`.
```llvm
%value = @llvm.amdgcn.av.load.b128(%addr, metadata !scope)
@@ -193,9 +199,9 @@ A `load` operation to `addrspace(0)` (generic or flat) is performed as a
`addrspace(3)`.
:::{note}
-Metadata cannot be used to model this using ordinary load/store operations,
-because the scope is necessary for correctness. In a hypothetical operation
-like this:
+Metadata cannot be used to express the availability and visibility of ordinary
+load/store operations, because the scope is necessary for correctness. In a
+hypothetical operation like this:
```llvm
store ptr addrspace(1) %ptr, <4 x i32> %value, !mmra !{!"amdgcn-av", !"workgroup"}
@@ -210,7 +216,7 @@ cache will fail to observe this store.
(amdgpu-av-metadata)=
-### AV Metadata
+### !"amdgcn-av" metadata
```llvm
!mmra !{!"amdgcn-av", !"none"}
@@ -225,7 +231,12 @@ release` or `load atomic acquire`), the metadata does not affect the
availability or the visibility of the access performed by the operation itself.
It only affects the synchronization of other memory accesses.
-### MakeAvailable and MakeVisible
+### make-available and make-visible
+
+*make-available* and *make-visible* operations propagate certain side-effects
+from or to the specified scope respectively. They are similar to the
+*store-available* and *load-visible* operations, except that their effect is not
+limited to specific locations.
```llvm
store atomic [syncscope("<target-scope>")] <ordering> [, !mmra !{!"amdgcn-av", !"none"}]
@@ -236,24 +247,24 @@ fence [syncscope("<target-scope>")] <ordering> [, !mmra !{!"amdgcn-av", !
```
A synchronizing operation with at least `release` ordering is a
-`MakeAvailable` operation with scope `syncscope`, if it is not marked as
+*make-available* operation with scope `syncscope`, if it is not marked as
`!{!"amdgcn-av", !"none"}`.
A synchronizing operation with at least `acquire` ordering is a
-`MakeVisible` operation with scope `syncscope`, if it is not marked as
+*make-visible* operation with scope `syncscope`, if it is not marked as
`!{!"amdgcn-av", !"none"}`.
```llvm
; This includes the following operations:
; - The atomic store at "agent" scope,
; - A store-available operation at "agent" scope on `ptr`,
-; - A `MakeAvailable` operation at "agent" scope that affects previous memory accesses.
+; - A *make-available* operation at "agent" scope that affects previous memory accesses.
store atomic syncscope("agent") release ptr
; This includes the following operations:
; - The atomic store at "agent" scope,
; - A store-available operation at "agent" scope on `ptr`.
-; Notably, it does not include a `MakeAvailable` operation on other memory accesses.
+; Notably, it does not include a *make-available* operation on other memory accesses.
store atomic syncscope("agent") release ptr, !mmra !{!"amdgcn-av", !"none"}
```
@@ -272,10 +283,10 @@ following holds:
- `X` is `W` itself, and `W` is a *store-available* operation, or,
-- `X` is a `MakeAvailable` operation that follows `W` in program order,
+- `X` is a *make-available* operation that follows `W` in program order,
or,
-- `X` is a `MakeAvailable` operation whose scope instance includes `W`,
+- `X` is a *make-available* operation whose scope instance includes `W`,
and there is an availability operation `Z` on `W` such that:
- `Z` happens-before `X`, and,
@@ -287,7 +298,7 @@ subscope instance of `S` that also includes `W`.
### Visibility Operation
An operation `Y` is a *visibility operation* on a write `W` if `Y` is a
-*load-visible* operation to the same address, or a `MakeVisible` operation,
+*load-visible* operation to the same address, or a *make-visible* operation,
and one of the following holds:
- There exists an *availability* operation `X` on write `W` such that:
@@ -372,7 +383,7 @@ The following properties follow from the definitions above:
outwards" into progressively larger scopes.
3. **Visibility is bounded by availability.** When a write is available in a
scope instance, it can be made visible in that scope instance by a visibility
- operation with the corresponding scope. Subsequent `MakeVisible` operations
+ operation with the corresponding scope. Subsequent *make-visible* operations
make that write visible into narrower scope instances towards the observer.
4. **A write can be made visible in a scope instance that does not contain it.**
The definition of a *visibility operation* anchors scope instances to the
More information about the llvm-commits
mailing list