[llvm] [AA] Make getSyncEffects sync-scope aware for pointer arguments (PR #211486)

[via llvm-commits]

michaelselehov wrote:

## Summary

`getSyncEffects` (used by `getModRefInfo` for fences and stronger-than-monotonic
atomics) reports `NoModRef` for a location whose underlying object does not
escape the function. For a non-byval pointer argument this is wrong under a
cross-thread sync scope: the object is caller-owned and may be shared with peer
threads, so a cross-thread fence/atomic can order accesses to it from other
threads. `noalias`/nocapture is a single-thread aliasing property and says
nothing about other threads; cross-thread ordering is carried by the sync scope.

Symptom (motivating bug): under LTO, GVN/LICM forward a load of a
workgroup-shared buffer across a `fence syncscope("workgroup") acq_rel`,
producing wrong results on GPU targets.

The new test `llvm/test/CodeGen/AMDGPU/gvn-fence-sync-private.ll` demonstrates
the wrong behavior directly: before this change GVN forwards the reload of a
workgroup-shared (`addrspace(3)`) `noalias` buffer across the fence; with the
fix the reload is preserved, while a genuinely thread-private (`addrspace(5)`)
buffer is still forwarded.

This PR makes the exemption sync-scope aware and adds a target hook to recover
precision where it is provably sound.

## Background: what went wrong in #196923

The exemption comes from #196923 ("Reapply [AA] No synchronization effects for
never-escaping identified local"), which introduced `getSyncEffects`: a fence /
stronger-than-monotonic atomic no longer clobbers a location whose underlying
object does not escape the function. That is correct for a genuinely
thread-private object created inside the function (an `alloca`, or a `noalias`
call result such as `malloc`).

The issue is that the same capture/escape reasoning also exempts a `noalias`
pointer *argument*. Escape/capture is a single-thread aliasing property; it says
nothing about whether other threads already hold the address. A `noalias`
argument points to caller-owned memory that peer threads may be concurrently
accessing, which is exactly what a cross-thread fence is there to order. The
exemption only "worked by accident" before because fences were modeled as
clobbering all memory.

This is not hypothetical: it miscompiled real GPU code where one wave produces
values into a workgroup-shared (LDS) buffer and a separate wave consumes them,
with a workgroup-scoped fence ordering the hand-off. The buffer is passed as a
`noalias` pointer, so after the exemption the consumer wave's reload was
forwarded across the fence and read stale data instead of the produced values.

The fix here keeps #196923's win for real thread-private locals and only removes
the exemption where it is unsound: a non-byval pointer argument under a
non-single-thread scope. This also matches the direction #196923 itself hoped
for ("it may be viable to respect potential synchronization inside non-nosync
function calls"): the sync scope, not escape analysis, is what determines
cross-thread ordering.

## The change

1. Generic (`getSyncEffects`, `AliasAnalysis.cpp`): the helper already takes the
   op's `SyncScope::ID`. Under any non-`SingleThread` scope it no longer exempts
   a non-byval pointer argument (it returns the `getModRefInfoMask` result).
   `byval` args and single-thread-scope ops keep the exemption. No address space
   is special-cased.
2. Target hook (`getModRefInfoForSyncOp`): a new virtual on the AA
   `Concept`/`Model` with a `ModRef` default in `AAResultBase`.
   `AAResults::getModRefInfoForSyncOp(Loc, AAQI, SSID)` chains all registered
   AAs (each may narrow to `NoModRef`) and intersects with the generic
   `getSyncEffects` result. Fences and all stronger-than-monotonic atomics route
   through it.
3. AMDGPU (`AMDGPUAliasAnalysis`): overrides the hook to return `NoModRef` when
   every underlying object of the location is `addrspace(5)` (private/scratch).
   Private memory is per-workitem, so no peer thread can reach it under any
   scope. It keys off the underlying objects (not `Loc.Ptr`'s type) so an
   addrspacecast from another space is not mistaken for private.

## Why the generic rule is sound on every target

The generic change only ever *widens* a non-byval pointer argument to the
`getModRefInfoMask` result; it never narrows. So no target can be miscompiled by
it: targets without a refining AA simply become (correctly) conservative
(CHERI purecap, NVPTX, flat-address-space targets all just stay conservative).
The only narrowing path is the opt-in AMDGPU hook, whose `NoModRef` rests on the
invariant that `addrspace(5)` scratch is per-lane and peer-unreachable under all
scopes.

## Why a new AA `Concept` virtual rather than a TTI hook

`getSyncEffects` runs inside `AAResults`, which has no access to
`TargetTransformInfo` (AA does not depend on TTI, and threading TTI through every
AA query would be a large, invasive change). The AA result chain is the
established mechanism for target- and analysis-specific mod/ref refinement, the
same place `ScopedNoAliasAA`, `GlobalsAA` and `TypeBasedAA` already refine
`getModRefInfo`. Adding one virtual there (mirroring the existing
`getModRefInfo(FenceInst*)` hook, which is likewise pure-virtual in the
`Concept`) keeps the refinement in the layer that already owns it.

Routing atomics through the chain is a no-op for every non-AMDGPU AA: they all
inherit the `AAResultBase` default returning `ModRef`, which the intersection
leaves unchanged, so there is no behavior change or compile-time cost for other
targets/AAs. The per-AA hook takes only `(Loc, AAQI)`: a target's answer ("this
address space is peer-unreachable") is scope-independent, and the generic
scope/`SingleThread` logic stays centralized in `getSyncEffects`.

## Tests

- `llvm/test/Transforms/GVN/fence-noalias-syncscope.ll` (new): target-independent
  behavior, cross-thread vs single-thread scope, pointer arg vs `byval`, generic
  vs `addrspace(3)`; all conservative in generic AA.
- `llvm/test/CodeGen/AMDGPU/sync-noalias-private-aa.ll` (new): aa-eval for the
  AMDGPU hook, private `NoModRef`; local(AS3)/global(AS1) `ModRef`;
  addrspacecast-from-generic, select/phi of private, loaded private pointer, and
  `byval` private.
- `llvm/test/CodeGen/AMDGPU/gvn-fence-sync-private.ll` (new): end-to-end GVN on
  amdgcn, AS3 workgroup-shared reload is preserved (the fixed miscompile); AS5
  private load is forwarded (precision preserved by the hook).
- `llvm/test/CodeGen/AMDGPU/rewrite-out-arguments-2.ll` (updated): now includes
  the AMDGPU AA passes; `@fence_seq_cst_after_store` is now rewritten, a sound
  win because the AS5 store's ordering vs the fence is unobservable to peers.
- `llvm/test/Analysis/BasicAA/atomics.ll` (updated): `@noalias_no_escape`
  `%a <-> <sync op>` responses change `NoModRef` to `Both ModRef`; these encoded
  the previous unsound behavior for a cross-thread (default `System`) scope arg.

## Relationship to ScopedNoAliasAA fence metadata (orthogonal)

`ScopedNoAliasAA::getModRefInfo(FenceInst*)` already refines fence mod/ref from
`!alias.scope`/`!noalias` metadata attached to the fence. That is an orthogonal
refinement path (it depends on a pass stamping scoped-noalias metadata onto the
fence). This PR is self-contained and works from the op's inherent sync scope; it
does not depend on, conflict with, or replace the metadata path.


https://github.com/llvm/llvm-project/pull/211486