[llvm] [AA] Use the cmpxchg merged ordering in getModRefInfo (PR #210545)
Justin Lebar via llvm-commits
llvm-commits at lists.llvm.org
Sat Jul 18 13:37:38 PDT 2026
https://github.com/jlebar created https://github.com/llvm/llvm-project/pull/210545
When a cmpxchg's address is NoAlias with the queried location,
getModRefInfo still reports sync effects if the cmpxchg is stronger than
monotonic, since an ordered operation constrains code motion around
locations it never accesses. But it tested only the success ordering,
and the failure ordering can be stronger (cmpxchg ... monotonic acquire
is valid IR), so such a cmpxchg looked freely reorderable to every AA
user.
The LoadStoreVectorizer turns that into a miscompile:
define <2 x i32> @f(ptr %p, ptr noalias %flag) {
%p.1 = getelementptr i8, ptr %p, i64 4
%v0 = load i32, ptr %p, align 8
%old = cmpxchg ptr %flag, i32 0, i32 0 monotonic acquire, align 4
%v1 = load i32, ptr %p.1, align 4
...
$ opt -passes=load-store-vectorizer
%1 = load <2 x i32>, ptr %p, align 8 ; both words above the cmpxchg
%old = cmpxchg ptr %flag, i32 0, i32 0 monotonic acquire, align 4
A failed cmpxchg is an acquire read of %flag here, so a consumer that
observes a producer's release-store of *%flag == 1 is guaranteed to see
its earlier store to %p.1 — but the rewritten code loads %p.1 without
waiting for the acquire. On x86 (where the compiler did the reordering,
not the hardware) a pthread litmus test of this idiom sees a stale %p.1
in ~2.5M of 3M rounds; with this fix, zero.
Check the merged ordering, i.e. the stronger of the two.
>From a4815e07fcde7f18c34a3c974580eb12250e8037 Mon Sep 17 00:00:00 2001
From: Justin Lebar <justin.lebar at gmail.com>
Date: Sat, 18 Jul 2026 20:29:08 +0000
Subject: [PATCH] [AA] Use the cmpxchg merged ordering in getModRefInfo
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
When a cmpxchg's address is NoAlias with the queried location,
getModRefInfo still reports sync effects if the cmpxchg is stronger than
monotonic, since an ordered operation constrains code motion around
locations it never accesses. But it tested only the success ordering,
and the failure ordering can be stronger (cmpxchg ... monotonic acquire
is valid IR), so such a cmpxchg looked freely reorderable to every AA
user.
The LoadStoreVectorizer turns that into a miscompile:
define <2 x i32> @f(ptr %p, ptr noalias %flag) {
%p.1 = getelementptr i8, ptr %p, i64 4
%v0 = load i32, ptr %p, align 8
%old = cmpxchg ptr %flag, i32 0, i32 0 monotonic acquire, align 4
%v1 = load i32, ptr %p.1, align 4
...
$ opt -passes=load-store-vectorizer
%1 = load <2 x i32>, ptr %p, align 8 ; both words above the cmpxchg
%old = cmpxchg ptr %flag, i32 0, i32 0 monotonic acquire, align 4
A failed cmpxchg is an acquire read of %flag here, so a consumer that
observes a producer's release-store of *%flag == 1 is guaranteed to see
its earlier store to %p.1 — but the rewritten code loads %p.1 without
waiting for the acquire. On x86 (where the compiler did the reordering,
not the hardware) a pthread litmus test of this idiom sees a stale %p.1
in ~2.5M of 3M rounds; with this fix, zero.
Check the merged ordering, i.e. the stronger of the two.
---
llvm/lib/Analysis/AliasAnalysis.cpp | 6 +++--
llvm/test/Analysis/BasicAA/atomics.ll | 8 ++++--
.../NVPTX/merge-across-side-effects.ll | 26 +++++++++++++++++++
3 files changed, 36 insertions(+), 4 deletions(-)
diff --git a/llvm/lib/Analysis/AliasAnalysis.cpp b/llvm/lib/Analysis/AliasAnalysis.cpp
index c7984c35e87ae..e77d726f285dc 100644
--- a/llvm/lib/Analysis/AliasAnalysis.cpp
+++ b/llvm/lib/Analysis/AliasAnalysis.cpp
@@ -609,8 +609,10 @@ ModRefInfo AAResults::getModRefInfo(const AtomicCmpXchgInst *CX,
// If the cmpxchg address does not alias the location, it does not access
// it.
if (AR == AliasResult::NoAlias) {
- // Synchronization effects may affect locations that do not alias.
- if (isStrongerThanMonotonic(CX->getSuccessOrdering()))
+ // Synchronization effects may affect locations that do not alias. The
+ // failure ordering counts too: it can be stronger than the success
+ // ordering (e.g. cmpxchg ... monotonic acquire).
+ if (isStrongerThanMonotonic(CX->getMergedOrdering()))
return getSyncEffects(this, Loc, AAQI);
return ModRefInfo::NoModRef;
}
diff --git a/llvm/test/Analysis/BasicAA/atomics.ll b/llvm/test/Analysis/BasicAA/atomics.ll
index ab0556ddf62c2..b3001be0f4063 100644
--- a/llvm/test/Analysis/BasicAA/atomics.ll
+++ b/llvm/test/Analysis/BasicAA/atomics.ll
@@ -64,8 +64,10 @@ define void @alloca_no_escape(ptr %x) {
; CHECK: Both ModRef: Ptr: i32* %x <-> %1 = atomicrmw add ptr %x, i32 1 acq_rel, align 4
; CHECK: Both ModRef: Ptr: i32* %a <-> %2 = cmpxchg ptr %x, i32 0, i32 1 acq_rel monotonic, align 4
; CHECK: Both ModRef: Ptr: i32* %x <-> %2 = cmpxchg ptr %x, i32 0, i32 1 acq_rel monotonic, align 4
-; CHECK: Both ModRef: Ptr: i32* %a <-> %3 = load atomic i32, ptr %x acquire, align 4
-; CHECK: Both ModRef: Ptr: i32* %x <-> %3 = load atomic i32, ptr %x acquire, align 4
+; CHECK: Both ModRef: Ptr: i32* %a <-> %3 = cmpxchg ptr %x, i32 0, i32 1 monotonic acquire, align 4
+; CHECK: Both ModRef: Ptr: i32* %x <-> %3 = cmpxchg ptr %x, i32 0, i32 1 monotonic acquire, align 4
+; CHECK: Both ModRef: Ptr: i32* %a <-> %4 = load atomic i32, ptr %x acquire, align 4
+; CHECK: Both ModRef: Ptr: i32* %x <-> %4 = load atomic i32, ptr %x acquire, align 4
; CHECK: Both ModRef: Ptr: i32* %a <-> store atomic i32 0, ptr %x release, align 4
; CHECK: Both ModRef: Ptr: i32* %x <-> store atomic i32 0, ptr %x release, align 4
define void @alloca_escape_after(ptr %x) {
@@ -75,6 +77,8 @@ define void @alloca_escape_after(ptr %x) {
fence release
atomicrmw add ptr %x, i32 1 acq_rel
cmpxchg ptr %x, i32 0, i32 1 acq_rel monotonic
+ ; The failure ordering counts too: monotonic/acquire is an acquire operation.
+ cmpxchg ptr %x, i32 0, i32 1 monotonic acquire
load atomic i32, ptr %x acquire, align 4
store atomic i32 0, ptr %x release, align 4
diff --git a/llvm/test/Transforms/LoadStoreVectorizer/NVPTX/merge-across-side-effects.ll b/llvm/test/Transforms/LoadStoreVectorizer/NVPTX/merge-across-side-effects.ll
index 782dba25f063d..c4c0db2607223 100644
--- a/llvm/test/Transforms/LoadStoreVectorizer/NVPTX/merge-across-side-effects.ll
+++ b/llvm/test/Transforms/LoadStoreVectorizer/NVPTX/merge-across-side-effects.ll
@@ -207,3 +207,29 @@ attributes #3 = { writeonly }
attributes #4 = { readonly }
; readnone implies nounwind, so no need to test separately
attributes #5 = { nounwind willreturn readnone }
+
+; A cmpxchg with monotonic success ordering but acquire failure ordering is
+; still an acquire operation on failure: hoisting the load of %p.1 above it
+; would break release/acquire message passing, noalias or not. This relies
+; on getModRefInfo consulting the cmpxchg's failure ordering, not just its
+; success ordering.
+
+define <2 x i32> @load_monotonic_acquire_cmpxchg(ptr %p, ptr noalias %flag) {
+; CHECK-LABEL: define <2 x i32> @load_monotonic_acquire_cmpxchg(
+; CHECK-SAME: ptr [[P:%.*]], ptr noalias [[FLAG:%.*]]) {
+; CHECK-NEXT: [[P_1:%.*]] = getelementptr i8, ptr [[P]], i64 4
+; CHECK-NEXT: [[V0:%.*]] = load i32, ptr [[P]], align 8
+; CHECK-NEXT: [[OLD:%.*]] = cmpxchg ptr [[FLAG]], i32 0, i32 1 monotonic acquire, align 4
+; CHECK-NEXT: [[V1:%.*]] = load i32, ptr [[P_1]], align 4
+; CHECK-NEXT: [[R0:%.*]] = insertelement <2 x i32> poison, i32 [[V0]], i32 0
+; CHECK-NEXT: [[R1:%.*]] = insertelement <2 x i32> [[R0]], i32 [[V1]], i32 1
+; CHECK-NEXT: ret <2 x i32> [[R1]]
+;
+ %p.1 = getelementptr i8, ptr %p, i64 4
+ %v0 = load i32, ptr %p, align 8
+ %old = cmpxchg ptr %flag, i32 0, i32 1 monotonic acquire, align 4
+ %v1 = load i32, ptr %p.1, align 4
+ %r0 = insertelement <2 x i32> poison, i32 %v0, i32 0
+ %r1 = insertelement <2 x i32> %r0, i32 %v1, i32 1
+ ret <2 x i32> %r1
+}
More information about the llvm-commits
mailing list