[llvm] [Doc][AMDGPU] Add barrier execution & memory model (PR #170447)

[Pierre van Houtryve via llvm-commits]

================
@@ -6678,6 +6967,30 @@ following sections:
 * :ref:`amdgpu-amdhsa-memory-model-gfx12`
 * :ref:`amdgpu-amdhsa-memory-model-gfx125x`
 
+.. _amdgpu-amdhsa-execution-barriers-memory-model:
+
+Execution Barriers
+++++++++++++++++++
+
+.. note::
+  See :ref:`amdgpu-amdhsa-execution-barriers` for definitions of the terminology used
+  in this section.
+
+* A barrier *signal* operation ``S`` can pair with a release fence program-ordered before it
+  to form a ``barrier-signal-release`` ``BR``. The synchronization scope and the set of address
+  spaces affected are determined by the release fence.
+* A barrier *wait* operation ``W`` can pair with an acquire fence program-ordered after it to
+  form a ``barrier-wait-acquire`` ``BA``. The synchronization scope and the set of address
+  spaces affected are determined by the acquire fence.
+
+A ``BR`` *synchronizes-with* ``BA`` in an address space *AS* if and only if:
+
+* ``S`` *barrier-executes-before* ``W``.
+* *BA* and *BR*'s :ref:`synchronization scope<amdgpu-memory-scopes>` overlap.
+* *BA* and *BR*'s :ref:`synchronization scope<amdgpu-memory-scopes>`
+  allow cross address space synchronization (they cannot have ``one-as``).
----------------
Pierre-vh wrote:

> What is this last rule for?

The theoretical reason is that barrier objects are stored in an unspecified memory region which may or may not overlap with any known address space, so we demand cross-address-space synchronization to allow pairing with that special barrier AS.

The practical reason is that the waits we insert for `one-as` are softer because we can take advantage of the properties of each AS in that mode. For example we don't need to wait on LDS for a workgroup-one-as release fence, but for a fence to pair with a barrier, we really need the "cross-address-space" synchronization mode where we always require the ops to complete so they can be observed from another AS.

> (I've always felt that the one-as scopes are a bit weird and we should probably supersede them with something else, so this is more out of curiosity.)

Agreed

https://github.com/llvm/llvm-project/pull/170447