[llvm] [LAA][NFC] Factor out MemoryDepChecker::isStoreLoadForwardingConflict (PR #212085)
via llvm-commits
llvm-commits at lists.llvm.org
Fri Aug 7 05:21:05 PDT 2026
https://github.com/mbhade-amd updated https://github.com/llvm/llvm-project/pull/212085
>From c3b42e1993e029706597c6fca2ada62684be12ac Mon Sep 17 00:00:00 2001
From: mbhade <mbhade at amd.com>
Date: Sun, 26 Jul 2026 09:34:05 +0530
Subject: [PATCH] [LAA][NFC] Factor out
MemoryDepChecker::isStoreLoadForwardingConflict
Extract the store-to-load forwarding conflict predicate used inside
MemoryDepChecker::couldPreventStoreLoadForward into a static helper so it
can be shared with other consumers (e.g. the SLP vectorizer's STLF cost
model). No functional change.
The helper asserts VectorStoreSize is non-zero rather than guarding against
it in the return expression: all callers pass a non-zero value (the loop
here starts VF at 2 * TypeByteSize), and a zero divisor would be UB.
Co-authored-by: Cursor <cursoragent at cursor.com>
---
.../include/llvm/Analysis/LoopAccessAnalysis.h | 18 ++++++++++++++++++
llvm/lib/Analysis/LoopAccessAnalysis.cpp | 10 +++-------
2 files changed, 21 insertions(+), 7 deletions(-)
diff --git a/llvm/include/llvm/Analysis/LoopAccessAnalysis.h b/llvm/include/llvm/Analysis/LoopAccessAnalysis.h
index 392321448c895..773e0136bd27b 100644
--- a/llvm/include/llvm/Analysis/LoopAccessAnalysis.h
+++ b/llvm/include/llvm/Analysis/LoopAccessAnalysis.h
@@ -235,6 +235,24 @@ class MemoryDepChecker {
std::numeric_limits<uint64_t>::max();
}
+ /// Returns true if a memory dependence at byte distance \p Distance between
+ /// a store and load (both with element size \p TypeByteSize bytes) would
+ /// prevent store-to-load forwarding when the store is widened to
+ /// \p VectorStoreSize bytes.
+ ///
+ /// The predicate fires when (a) the load is misaligned w.r.t. the widened
+ /// store window (\c Distance is not a multiple of \p VectorStoreSize), and
+ /// (b) the conflicting store is still likely to be in the store buffer
+ /// (\c Distance / VectorStoreSize is below 8 * TypeByteSize iterations).
+ static bool isStoreLoadForwardingConflict(uint64_t Distance,
+ uint64_t VectorStoreSize,
+ uint64_t TypeByteSize) {
+ assert(VectorStoreSize != 0 && "Expected non-zero vector store size");
+ const uint64_t NumItersForStoreLoadThroughMemory = 8 * TypeByteSize;
+ return Distance % VectorStoreSize != 0 &&
+ Distance / VectorStoreSize < NumItersForStoreLoadThroughMemory;
+ }
+
/// Return safe power-of-2 number of elements, which do not prevent store-load
/// forwarding, multiplied by the size of the elements in bits.
uint64_t getStoreLoadForwardSafeDistanceInBits() const {
diff --git a/llvm/lib/Analysis/LoopAccessAnalysis.cpp b/llvm/lib/Analysis/LoopAccessAnalysis.cpp
index e248b22de7d43..4210eb7cb466a 100644
--- a/llvm/lib/Analysis/LoopAccessAnalysis.cpp
+++ b/llvm/lib/Analysis/LoopAccessAnalysis.cpp
@@ -1934,20 +1934,16 @@ bool MemoryDepChecker::couldPreventStoreLoadForward(uint64_t Distance,
// place. Vectorizing in such cases does not make sense.
// Store-load forwarding distance.
- // After this many iterations store-to-load forwarding conflicts should not
- // cause any slowdowns.
- const uint64_t NumItersForStoreLoadThroughMemory = 8 * TypeByteSize;
// Maximum vector factor.
uint64_t MaxVFWithoutSLForwardIssuesPowerOf2 =
std::min(VectorizerParams::MaxVectorWidth * TypeByteSize,
MaxStoreLoadForwardSafeDistanceInBits);
- // Compute the smallest VF at which the store and load would be misaligned.
+ // Compute the smallest VF at which the store and load would be misaligned
+ // and recent enough to still be in the store buffer.
for (uint64_t VF = 2 * TypeByteSize;
VF <= MaxVFWithoutSLForwardIssuesPowerOf2; VF *= 2) {
- // If the number of vector iteration between the store and the load are
- // small we could incur conflicts.
- if (Distance % VF && Distance / VF < NumItersForStoreLoadThroughMemory) {
+ if (isStoreLoadForwardingConflict(Distance, VF, TypeByteSize)) {
MaxVFWithoutSLForwardIssuesPowerOf2 = (VF >> 1);
break;
}
More information about the llvm-commits
mailing list