[llvm] 01859da - [AliasAnalysis] Introduce getModRefInfoMask() as a generalization of pointsToConstantMemory().
Patrick Walton via llvm-commits
llvm-commits at lists.llvm.org
Mon Oct 31 13:04:08 PDT 2022
Author: Patrick Walton
Date: 2022-10-31T13:03:41-07:00
New Revision: 01859da84bad95fd51d6a03b08b60c660e642a4f
URL: https://github.com/llvm/llvm-project/commit/01859da84bad95fd51d6a03b08b60c660e642a4f
DIFF: https://github.com/llvm/llvm-project/commit/01859da84bad95fd51d6a03b08b60c660e642a4f.diff
LOG: [AliasAnalysis] Introduce getModRefInfoMask() as a generalization of pointsToConstantMemory().
The pointsToConstantMemory() method returns true only if the memory pointed to
by the memory location is globally invariant. However, the LLVM memory model
also has the semantic notion of *locally-invariant*: memory that is known to be
invariant for the life of the SSA value representing that pointer. The most
common example of this is a pointer argument that is marked readonly noalias,
which the Rust compiler frequently emits.
It'd be desirable for LLVM to treat locally-invariant memory the same way as
globally-invariant memory when it's safe to do so. This patch implements that,
by introducing the concept of a *ModRefInfo mask*. A ModRefInfo mask is a bound
on the Mod/Ref behavior of an instruction that writes to a memory location,
based on the knowledge that the memory is globally-constant memory (in which
case the mask is NoModRef) or locally-constant memory (in which case the mask
is Ref). ModRefInfo values for an instruction can be combined with the
ModRefInfo mask by simply using the & operator. Where appropriate, this patch
has modified uses of pointsToConstantMemory() to instead examine the mask.
The most notable optimization change I noticed with this patch is that now
redundant loads from readonly noalias pointers can be eliminated across calls,
even when the pointer is captured. Internally, before this patch,
AliasAnalysis was assigning Ref to reads from constant memory; now AA can
assign NoModRef, which is a tighter bound.
Differential Revision: https://reviews.llvm.org/D136659
Added:
Modified:
llvm/docs/AliasAnalysis.rst
llvm/include/llvm/Analysis/AliasAnalysis.h
llvm/include/llvm/Analysis/BasicAliasAnalysis.h
llvm/include/llvm/Analysis/ObjCARCAliasAnalysis.h
llvm/include/llvm/Analysis/TypeBasedAliasAnalysis.h
llvm/lib/Analysis/AliasAnalysis.cpp
llvm/lib/Analysis/BasicAliasAnalysis.cpp
llvm/lib/Analysis/MemoryDependenceAnalysis.cpp
llvm/lib/Analysis/MemorySSA.cpp
llvm/lib/Analysis/ObjCARCAliasAnalysis.cpp
llvm/lib/Analysis/TypeBasedAliasAnalysis.cpp
llvm/lib/Target/AMDGPU/AMDGPUAliasAnalysis.cpp
llvm/lib/Target/AMDGPU/AMDGPUAliasAnalysis.h
llvm/lib/Transforms/IPO/FunctionAttrs.cpp
llvm/lib/Transforms/InstCombine/InstCombineCalls.cpp
llvm/lib/Transforms/InstCombine/InstCombineLoadStoreAlloca.cpp
llvm/lib/Transforms/Scalar/LICM.cpp
llvm/lib/Transforms/Scalar/LoopPredication.cpp
llvm/test/Analysis/BasicAA/constant-memory.ll
llvm/test/Transforms/InstCombine/memcpy-from-global.ll
llvm/test/Transforms/InstCombine/store.ll
Removed:
################################################################################
diff --git a/llvm/docs/AliasAnalysis.rst b/llvm/docs/AliasAnalysis.rst
index b9a8a3a4eb529..046dd24d7332e 100644
--- a/llvm/docs/AliasAnalysis.rst
+++ b/llvm/docs/AliasAnalysis.rst
@@ -161,14 +161,24 @@ Other useful ``AliasAnalysis`` methods
Several other tidbits of information are often collected by various alias
analysis implementations and can be put to good use by various clients.
-The ``pointsToConstantMemory`` method
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-The ``pointsToConstantMemory`` method returns true if and only if the analysis
-can prove that the pointer only points to unchanging memory locations
-(functions, constant global variables, and the null pointer). This information
-can be used to refine mod/ref information: it is impossible for an unchanging
-memory location to be modified.
+The ``getModRefInfoMask`` method
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The ``getModRefInfoMask`` method returns a bound on Mod/Ref information for
+the supplied pointer, based on knowledge about whether the pointer points to
+globally-constant memory (for which it returns ``NoModRef``) or
+locally-invariant memory (for which it returns ``Ref``). Globally-constant
+memory includes functions, constant global variables, and the null pointer.
+Locally-invariant memory is memory that we know is invariant for the lifetime
+of its SSA value, but not necessarily for the life of the program: for example,
+the memory pointed to by ``readonly`` ``noalias`` parameters is known-invariant
+for the duration of the corresponding function call. Given Mod/Ref information
+``MRI`` for a memory location ``Loc``, ``MRI`` can be refined with a statement
+like ``MRI &= AA.getModRefInfoMask(Loc);``. Another useful idiom is
+``isModSet(AA.getModRefInfoMask(Loc))``; this checks to see if the given
+location can be modified at all. For convenience, there is also a method
+``pointsToConstantMemory(Loc)``; this is synonymous with
+``isNoModRef(AA.getModRefInfoMask(Loc))``.
.. _never access memory or only read memory:
diff --git a/llvm/include/llvm/Analysis/AliasAnalysis.h b/llvm/include/llvm/Analysis/AliasAnalysis.h
index fb0a087554db6..366e6fe626c83 100644
--- a/llvm/include/llvm/Analysis/AliasAnalysis.h
+++ b/llvm/include/llvm/Analysis/AliasAnalysis.h
@@ -359,7 +359,9 @@ class AAResults {
/// Checks whether the given location points to constant memory, or if
/// \p OrLocal is true whether it points to a local alloca.
- bool pointsToConstantMemory(const MemoryLocation &Loc, bool OrLocal = false);
+ bool pointsToConstantMemory(const MemoryLocation &Loc, bool OrLocal = false) {
+ return isNoModRef(getModRefInfoMask(Loc, OrLocal));
+ }
/// A convenience wrapper around the primary \c pointsToConstantMemory
/// interface.
@@ -372,6 +374,22 @@ class AAResults {
/// \name Simple mod/ref information
/// @{
+ /// Returns a bitmask that should be unconditionally applied to the ModRef
+ /// info of a memory location. This allows us to eliminate Mod and/or Ref
+ /// from the ModRef info based on the knowledge that the memory location
+ /// points to constant and/or locally-invariant memory.
+ ///
+ /// If IgnoreLocals is true, then this method returns NoModRef for memory
+ /// that points to a local alloca.
+ ModRefInfo getModRefInfoMask(const MemoryLocation &Loc,
+ bool IgnoreLocals = false);
+
+ /// A convenience wrapper around the primary \c getModRefInfoMask
+ /// interface.
+ ModRefInfo getModRefInfoMask(const Value *P, bool IgnoreLocals = false) {
+ return getModRefInfoMask(MemoryLocation::getBeforeOrAfter(P), IgnoreLocals);
+ }
+
/// Get the ModRef info associated with a pointer argument of a call. The
/// result's bits are set to indicate the allowed aliasing ModRef kinds. Note
/// that these bits do not necessarily account for the overall behavior of
@@ -614,6 +632,8 @@ class AAResults {
AAQueryInfo &AAQI);
bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
bool OrLocal = false);
+ ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
+ bool IgnoreLocals = false);
ModRefInfo getModRefInfo(Instruction *I, const CallBase *Call2,
AAQueryInfo &AAQIP);
ModRefInfo getModRefInfo(const CallBase *Call, const MemoryLocation &Loc,
@@ -681,6 +701,10 @@ class BatchAAResults {
bool pointsToConstantMemory(const MemoryLocation &Loc, bool OrLocal = false) {
return AA.pointsToConstantMemory(Loc, AAQI, OrLocal);
}
+ ModRefInfo getModRefInfoMask(const MemoryLocation &Loc,
+ bool IgnoreLocals = false) {
+ return AA.getModRefInfoMask(Loc, AAQI, IgnoreLocals);
+ }
ModRefInfo getModRefInfo(const CallBase *Call, const MemoryLocation &Loc) {
return AA.getModRefInfo(Call, Loc, AAQI);
}
@@ -743,16 +767,19 @@ class AAResults::Concept {
virtual AliasResult alias(const MemoryLocation &LocA,
const MemoryLocation &LocB, AAQueryInfo &AAQI) = 0;
- /// Checks whether the given location points to constant memory, or if
- /// \p OrLocal is true whether it points to a local alloca.
- virtual bool pointsToConstantMemory(const MemoryLocation &Loc,
- AAQueryInfo &AAQI, bool OrLocal) = 0;
-
/// @}
//===--------------------------------------------------------------------===//
/// \name Simple mod/ref information
/// @{
+ /// Returns a bitmask that should be unconditionally applied to the ModRef
+ /// info of a memory location. This allows us to eliminate Mod and/or Ref from
+ /// the ModRef info based on the knowledge that the memory location points to
+ /// constant and/or locally-invariant memory.
+ virtual ModRefInfo getModRefInfoMask(const MemoryLocation &Loc,
+ AAQueryInfo &AAQI,
+ bool IgnoreLocals) = 0;
+
/// Get the ModRef info associated with a pointer argument of a callsite. The
/// result's bits are set to indicate the allowed aliasing ModRef kinds. Note
/// that these bits do not necessarily account for the overall behavior of
@@ -801,9 +828,9 @@ template <typename AAResultT> class AAResults::Model final : public Concept {
return Result.alias(LocA, LocB, AAQI);
}
- bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
- bool OrLocal) override {
- return Result.pointsToConstantMemory(Loc, AAQI, OrLocal);
+ ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
+ bool IgnoreLocals) override {
+ return Result.getModRefInfoMask(Loc, AAQI, IgnoreLocals);
}
ModRefInfo getArgModRefInfo(const CallBase *Call, unsigned ArgIdx) override {
@@ -856,9 +883,9 @@ class AAResultBase {
return AliasResult::MayAlias;
}
- bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
- bool OrLocal) {
- return false;
+ ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
+ bool IgnoreLocals) {
+ return ModRefInfo::ModRef;
}
ModRefInfo getArgModRefInfo(const CallBase *Call, unsigned ArgIdx) {
diff --git a/llvm/include/llvm/Analysis/BasicAliasAnalysis.h b/llvm/include/llvm/Analysis/BasicAliasAnalysis.h
index 8f4a8d4a8d8b2..a2909f46e3ba2 100644
--- a/llvm/include/llvm/Analysis/BasicAliasAnalysis.h
+++ b/llvm/include/llvm/Analysis/BasicAliasAnalysis.h
@@ -75,9 +75,15 @@ class BasicAAResult : public AAResultBase {
ModRefInfo getModRefInfo(const CallBase *Call1, const CallBase *Call2,
AAQueryInfo &AAQI);
- /// Chases pointers until we find a (constant global) or not.
- bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
- bool OrLocal);
+ /// Returns a bitmask that should be unconditionally applied to the ModRef
+ /// info of a memory location. This allows us to eliminate Mod and/or Ref
+ /// from the ModRef info based on the knowledge that the memory location
+ /// points to constant and/or locally-invariant memory.
+ ///
+ /// If IgnoreLocals is true, then this method returns NoModRef for memory
+ /// that points to a local alloca.
+ ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
+ bool IgnoreLocals = false);
/// Get the location associated with a pointer argument of a callsite.
ModRefInfo getArgModRefInfo(const CallBase *Call, unsigned ArgIdx);
diff --git a/llvm/include/llvm/Analysis/ObjCARCAliasAnalysis.h b/llvm/include/llvm/Analysis/ObjCARCAliasAnalysis.h
index ba8b28a5adda3..790fbaf07e7f3 100644
--- a/llvm/include/llvm/Analysis/ObjCARCAliasAnalysis.h
+++ b/llvm/include/llvm/Analysis/ObjCARCAliasAnalysis.h
@@ -52,8 +52,8 @@ class ObjCARCAAResult : public AAResultBase {
AliasResult alias(const MemoryLocation &LocA, const MemoryLocation &LocB,
AAQueryInfo &AAQI);
- bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
- bool OrLocal);
+ ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
+ bool IgnoreLocals);
using AAResultBase::getMemoryEffects;
MemoryEffects getMemoryEffects(const Function *F);
diff --git a/llvm/include/llvm/Analysis/TypeBasedAliasAnalysis.h b/llvm/include/llvm/Analysis/TypeBasedAliasAnalysis.h
index 2e7af99ed4955..791ec65990e2b 100644
--- a/llvm/include/llvm/Analysis/TypeBasedAliasAnalysis.h
+++ b/llvm/include/llvm/Analysis/TypeBasedAliasAnalysis.h
@@ -40,8 +40,8 @@ class TypeBasedAAResult : public AAResultBase {
AliasResult alias(const MemoryLocation &LocA, const MemoryLocation &LocB,
AAQueryInfo &AAQI);
- bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
- bool OrLocal);
+ ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
+ bool IgnoreLocals);
MemoryEffects getMemoryEffects(const CallBase *Call, AAQueryInfo &AAQI);
MemoryEffects getMemoryEffects(const Function *F);
ModRefInfo getModRefInfo(const CallBase *Call, const MemoryLocation &Loc,
diff --git a/llvm/lib/Analysis/AliasAnalysis.cpp b/llvm/lib/Analysis/AliasAnalysis.cpp
index e686ff75a3028..d5026d13f941b 100644
--- a/llvm/lib/Analysis/AliasAnalysis.cpp
+++ b/llvm/lib/Analysis/AliasAnalysis.cpp
@@ -147,19 +147,25 @@ AliasResult AAResults::alias(const MemoryLocation &LocA,
return Result;
}
-bool AAResults::pointsToConstantMemory(const MemoryLocation &Loc,
- bool OrLocal) {
+ModRefInfo AAResults::getModRefInfoMask(const MemoryLocation &Loc,
+ bool IgnoreLocals) {
SimpleAAQueryInfo AAQIP(*this);
- return pointsToConstantMemory(Loc, AAQIP, OrLocal);
+ return getModRefInfoMask(Loc, AAQIP, IgnoreLocals);
}
-bool AAResults::pointsToConstantMemory(const MemoryLocation &Loc,
- AAQueryInfo &AAQI, bool OrLocal) {
- for (const auto &AA : AAs)
- if (AA->pointsToConstantMemory(Loc, AAQI, OrLocal))
- return true;
+ModRefInfo AAResults::getModRefInfoMask(const MemoryLocation &Loc,
+ AAQueryInfo &AAQI, bool IgnoreLocals) {
+ ModRefInfo Result = ModRefInfo::ModRef;
- return false;
+ for (const auto &AA : AAs) {
+ Result &= AA->getModRefInfoMask(Loc, AAQI, IgnoreLocals);
+
+ // Early-exit the moment we reach the bottom of the lattice.
+ if (isNoModRef(Result))
+ return ModRefInfo::NoModRef;
+ }
+
+ return Result;
}
ModRefInfo AAResults::getArgModRefInfo(const CallBase *Call, unsigned ArgIdx) {
@@ -253,10 +259,11 @@ ModRefInfo AAResults::getModRefInfo(const CallBase *Call,
Result &= ArgMR | OtherMR;
- // If Loc is a constant memory location, the call definitely could not
+ // Apply the ModRef mask. This ensures that if Loc is a constant memory
+ // location, we take into account the fact that the call definitely could not
// modify the memory location.
- if (isModSet(Result) && pointsToConstantMemory(Loc, AAQI, /*OrLocal*/ false))
- Result &= ModRefInfo::Ref;
+ if (!isNoModRef(Result))
+ Result &= getModRefInfoMask(Loc);
return Result;
}
@@ -510,9 +517,11 @@ ModRefInfo AAResults::getModRefInfo(const StoreInst *S,
if (AR == AliasResult::NoAlias)
return ModRefInfo::NoModRef;
- // If the pointer is a pointer to constant memory, then it could not have
- // been modified by this store.
- if (pointsToConstantMemory(Loc, AAQI))
+ // Examine the ModRef mask. If Mod isn't present, then return NoModRef.
+ // This ensures that if Loc is a constant memory location, we take into
+ // account the fact that the store definitely could not modify the memory
+ // location.
+ if (!isModSet(getModRefInfoMask(Loc)))
return ModRefInfo::NoModRef;
}
@@ -529,10 +538,11 @@ ModRefInfo AAResults::getModRefInfo(const FenceInst *S,
ModRefInfo AAResults::getModRefInfo(const FenceInst *S,
const MemoryLocation &Loc,
AAQueryInfo &AAQI) {
- // If we know that the location is a constant memory location, the fence
- // cannot modify this location.
- if (Loc.Ptr && pointsToConstantMemory(Loc, AAQI))
- return ModRefInfo::Ref;
+ // All we know about a fence instruction is what we get from the ModRef
+ // mask: if Loc is a constant memory location, the fence definitely could
+ // not modify it.
+ if (Loc.Ptr)
+ return getModRefInfoMask(Loc);
return ModRefInfo::ModRef;
}
@@ -552,10 +562,9 @@ ModRefInfo AAResults::getModRefInfo(const VAArgInst *V,
if (AR == AliasResult::NoAlias)
return ModRefInfo::NoModRef;
- // If the pointer is a pointer to constant memory, then it could not have
+ // If the pointer is a pointer to invariant memory, then it could not have
// been modified by this va_arg.
- if (pointsToConstantMemory(Loc, AAQI))
- return ModRefInfo::NoModRef;
+ return getModRefInfoMask(Loc, AAQI);
}
// Otherwise, a va_arg reads and writes.
@@ -572,10 +581,9 @@ ModRefInfo AAResults::getModRefInfo(const CatchPadInst *CatchPad,
const MemoryLocation &Loc,
AAQueryInfo &AAQI) {
if (Loc.Ptr) {
- // If the pointer is a pointer to constant memory,
+ // If the pointer is a pointer to invariant memory,
// then it could not have been modified by this catchpad.
- if (pointsToConstantMemory(Loc, AAQI))
- return ModRefInfo::NoModRef;
+ return getModRefInfoMask(Loc, AAQI);
}
// Otherwise, a catchpad reads and writes.
@@ -592,10 +600,9 @@ ModRefInfo AAResults::getModRefInfo(const CatchReturnInst *CatchRet,
const MemoryLocation &Loc,
AAQueryInfo &AAQI) {
if (Loc.Ptr) {
- // If the pointer is a pointer to constant memory,
+ // If the pointer is a pointer to invariant memory,
// then it could not have been modified by this catchpad.
- if (pointsToConstantMemory(Loc, AAQI))
- return ModRefInfo::NoModRef;
+ return getModRefInfoMask(Loc, AAQI);
}
// Otherwise, a catchret reads and writes.
diff --git a/llvm/lib/Analysis/BasicAliasAnalysis.cpp b/llvm/lib/Analysis/BasicAliasAnalysis.cpp
index fa03a82f78b29..2bbc2486fec00 100644
--- a/llvm/lib/Analysis/BasicAliasAnalysis.cpp
+++ b/llvm/lib/Analysis/BasicAliasAnalysis.cpp
@@ -676,33 +676,46 @@ BasicAAResult::DecomposeGEPExpression(const Value *V, const DataLayout &DL,
return Decomposed;
}
-/// Returns whether the given pointer value points to memory that is local to
-/// the function, with global constants being considered local to all
-/// functions.
-bool BasicAAResult::pointsToConstantMemory(const MemoryLocation &Loc,
- AAQueryInfo &AAQI, bool OrLocal) {
+ModRefInfo BasicAAResult::getModRefInfoMask(const MemoryLocation &Loc,
+ AAQueryInfo &AAQI,
+ bool IgnoreLocals) {
assert(Visited.empty() && "Visited must be cleared after use!");
- auto _ = make_scope_exit([&]{ Visited.clear(); });
+ auto _ = make_scope_exit([&] { Visited.clear(); });
unsigned MaxLookup = 8;
SmallVector<const Value *, 16> Worklist;
Worklist.push_back(Loc.Ptr);
+ ModRefInfo Result = ModRefInfo::NoModRef;
+
do {
const Value *V = getUnderlyingObject(Worklist.pop_back_val());
if (!Visited.insert(V).second)
continue;
- // An alloca instruction defines local memory.
- if (OrLocal && isa<AllocaInst>(V))
+ // Ignore allocas if we were instructed to do so.
+ if (IgnoreLocals && isa<AllocaInst>(V))
continue;
- // A global constant counts as local memory for our purposes.
+ // If the location points to memory that is known to be invariant for
+ // the life of the underlying SSA value, then we can exclude Mod from
+ // the set of valid memory effects.
+ //
+ // An argument that is marked readonly and noalias is known to be
+ // invariant while that function is executing.
+ if (const Argument *Arg = dyn_cast<Argument>(V)) {
+ if (Arg->hasNoAliasAttr() && Arg->onlyReadsMemory()) {
+ Result |= ModRefInfo::Ref;
+ continue;
+ }
+ }
+
+ // A global constant can't be mutated.
if (const GlobalVariable *GV = dyn_cast<GlobalVariable>(V)) {
// Note: this doesn't require GV to be "ODR" because it isn't legal for a
// global to be marked constant in some modules and non-constant in
// others. GV may even be a declaration, not a definition.
if (!GV->isConstant())
- return AAResultBase::pointsToConstantMemory(Loc, AAQI, OrLocal);
+ return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
continue;
}
@@ -718,16 +731,20 @@ bool BasicAAResult::pointsToConstantMemory(const MemoryLocation &Loc,
if (const PHINode *PN = dyn_cast<PHINode>(V)) {
// Don't bother inspecting phi nodes with many operands.
if (PN->getNumIncomingValues() > MaxLookup)
- return AAResultBase::pointsToConstantMemory(Loc, AAQI, OrLocal);
+ return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
append_range(Worklist, PN->incoming_values());
continue;
}
// Otherwise be conservative.
- return AAResultBase::pointsToConstantMemory(Loc, AAQI, OrLocal);
+ return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
} while (!Worklist.empty() && --MaxLookup);
- return Worklist.empty();
+ // If we hit the maximum number of instructions to examine, be conservative.
+ if (!Worklist.empty())
+ return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
+
+ return Result;
}
static bool isIntrinsicCall(const CallBase *Call, Intrinsic::ID IID) {
diff --git a/llvm/lib/Analysis/MemoryDependenceAnalysis.cpp b/llvm/lib/Analysis/MemoryDependenceAnalysis.cpp
index d9bae54b05be1..f0446b21e2888 100644
--- a/llvm/lib/Analysis/MemoryDependenceAnalysis.cpp
+++ b/llvm/lib/Analysis/MemoryDependenceAnalysis.cpp
@@ -525,7 +525,7 @@ MemDepResult MemoryDependenceResults::getSimplePointerDependencyFrom(
}
// Stores don't alias loads from read-only memory.
- if (BatchAA.pointsToConstantMemory(LoadLoc))
+ if (!isModSet(BatchAA.getModRefInfoMask(LoadLoc)))
continue;
// Stores depend on may/must aliased loads.
diff --git a/llvm/lib/Analysis/MemorySSA.cpp b/llvm/lib/Analysis/MemorySSA.cpp
index 461886650261b..1660321990bf2 100644
--- a/llvm/lib/Analysis/MemorySSA.cpp
+++ b/llvm/lib/Analysis/MemorySSA.cpp
@@ -378,9 +378,10 @@ static bool isUseTriviallyOptimizableToLiveOnEntry(AliasAnalysisType &AA,
const Instruction *I) {
// If the memory can't be changed, then loads of the memory can't be
// clobbered.
- if (auto *LI = dyn_cast<LoadInst>(I))
+ if (auto *LI = dyn_cast<LoadInst>(I)) {
return I->hasMetadata(LLVMContext::MD_invariant_load) ||
- AA.pointsToConstantMemory(MemoryLocation::get(LI));
+ !isModSet(AA.getModRefInfoMask(MemoryLocation::get(LI)));
+ }
return false;
}
diff --git a/llvm/lib/Analysis/ObjCARCAliasAnalysis.cpp b/llvm/lib/Analysis/ObjCARCAliasAnalysis.cpp
index 76f6a74384b46..93d56931d01c7 100644
--- a/llvm/lib/Analysis/ObjCARCAliasAnalysis.cpp
+++ b/llvm/lib/Analysis/ObjCARCAliasAnalysis.cpp
@@ -68,28 +68,29 @@ AliasResult ObjCARCAAResult::alias(const MemoryLocation &LocA,
return AliasResult::MayAlias;
}
-bool ObjCARCAAResult::pointsToConstantMemory(const MemoryLocation &Loc,
- AAQueryInfo &AAQI, bool OrLocal) {
+ModRefInfo ObjCARCAAResult::getModRefInfoMask(const MemoryLocation &Loc,
+ AAQueryInfo &AAQI,
+ bool IgnoreLocals) {
if (!EnableARCOpts)
- return AAResultBase::pointsToConstantMemory(Loc, AAQI, OrLocal);
+ return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
// First, strip off no-ops, including ObjC-specific no-ops, and try making
// a precise alias query.
const Value *S = GetRCIdentityRoot(Loc.Ptr);
- if (AAResultBase::pointsToConstantMemory(
- MemoryLocation(S, Loc.Size, Loc.AATags), AAQI, OrLocal))
- return true;
+ if (isNoModRef(AAResultBase::getModRefInfoMask(
+ MemoryLocation(S, Loc.Size, Loc.AATags), AAQI, IgnoreLocals)))
+ return ModRefInfo::NoModRef;
// If that failed, climb to the underlying object, including climbing through
// ObjC-specific no-ops, and try making an imprecise alias query.
const Value *U = GetUnderlyingObjCPtr(S);
if (U != S)
- return AAResultBase::pointsToConstantMemory(
- MemoryLocation::getBeforeOrAfter(U), AAQI, OrLocal);
+ return AAResultBase::getModRefInfoMask(MemoryLocation::getBeforeOrAfter(U),
+ AAQI, IgnoreLocals);
// If that failed, fail. We don't need to chain here, since that's covered
// by the earlier precise query.
- return false;
+ return ModRefInfo::ModRef;
}
MemoryEffects ObjCARCAAResult::getMemoryEffects(const Function *F) {
diff --git a/llvm/lib/Analysis/TypeBasedAliasAnalysis.cpp b/llvm/lib/Analysis/TypeBasedAliasAnalysis.cpp
index 8360246915e46..b7121ea8326b8 100644
--- a/llvm/lib/Analysis/TypeBasedAliasAnalysis.cpp
+++ b/llvm/lib/Analysis/TypeBasedAliasAnalysis.cpp
@@ -385,23 +385,23 @@ AliasResult TypeBasedAAResult::alias(const MemoryLocation &LocA,
return AliasResult::NoAlias;
}
-bool TypeBasedAAResult::pointsToConstantMemory(const MemoryLocation &Loc,
- AAQueryInfo &AAQI,
- bool OrLocal) {
+ModRefInfo TypeBasedAAResult::getModRefInfoMask(const MemoryLocation &Loc,
+ AAQueryInfo &AAQI,
+ bool IgnoreLocals) {
if (!EnableTBAA)
- return AAResultBase::pointsToConstantMemory(Loc, AAQI, OrLocal);
+ return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
const MDNode *M = Loc.AATags.TBAA;
if (!M)
- return AAResultBase::pointsToConstantMemory(Loc, AAQI, OrLocal);
+ return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
// If this is an "immutable" type, we can assume the pointer is pointing
// to constant memory.
if ((!isStructPathTBAA(M) && TBAANode(M).isTypeImmutable()) ||
(isStructPathTBAA(M) && TBAAStructTagNode(M).isTypeImmutable()))
- return true;
+ return ModRefInfo::NoModRef;
- return AAResultBase::pointsToConstantMemory(Loc, AAQI, OrLocal);
+ return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
}
MemoryEffects TypeBasedAAResult::getMemoryEffects(const CallBase *Call,
diff --git a/llvm/lib/Target/AMDGPU/AMDGPUAliasAnalysis.cpp b/llvm/lib/Target/AMDGPU/AMDGPUAliasAnalysis.cpp
index dd3eb3849eac1..e97c993f4b25a 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPUAliasAnalysis.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPUAliasAnalysis.cpp
@@ -124,54 +124,19 @@ AliasResult AMDGPUAAResult::alias(const MemoryLocation &LocA,
return AAResultBase::alias(LocA, LocB, AAQI);
}
-bool AMDGPUAAResult::pointsToConstantMemory(const MemoryLocation &Loc,
- AAQueryInfo &AAQI, bool OrLocal) {
+ModRefInfo AMDGPUAAResult::getModRefInfoMask(const MemoryLocation &Loc,
+ AAQueryInfo &AAQI,
+ bool IgnoreLocals) {
unsigned AS = Loc.Ptr->getType()->getPointerAddressSpace();
if (AS == AMDGPUAS::CONSTANT_ADDRESS ||
AS == AMDGPUAS::CONSTANT_ADDRESS_32BIT)
- return true;
+ return ModRefInfo::NoModRef;
const Value *Base = getUnderlyingObject(Loc.Ptr);
AS = Base->getType()->getPointerAddressSpace();
if (AS == AMDGPUAS::CONSTANT_ADDRESS ||
AS == AMDGPUAS::CONSTANT_ADDRESS_32BIT)
- return true;
-
- if (const GlobalVariable *GV = dyn_cast<GlobalVariable>(Base)) {
- if (GV->isConstant())
- return true;
- } else if (const Argument *Arg = dyn_cast<Argument>(Base)) {
- const Function *F = Arg->getParent();
-
- // Only assume constant memory for arguments on kernels.
- switch (F->getCallingConv()) {
- default:
- return AAResultBase::pointsToConstantMemory(Loc, AAQI, OrLocal);
- case CallingConv::AMDGPU_LS:
- case CallingConv::AMDGPU_HS:
- case CallingConv::AMDGPU_ES:
- case CallingConv::AMDGPU_GS:
- case CallingConv::AMDGPU_VS:
- case CallingConv::AMDGPU_PS:
- case CallingConv::AMDGPU_CS:
- case CallingConv::AMDGPU_KERNEL:
- case CallingConv::SPIR_KERNEL:
- break;
- }
+ return ModRefInfo::NoModRef;
- unsigned ArgNo = Arg->getArgNo();
- /* On an argument, ReadOnly attribute indicates that the function does
- not write through this pointer argument, even though it may write
- to the memory that the pointer points to.
- On an argument, ReadNone attribute indicates that the function does
- not dereference that pointer argument, even though it may read or write
- the memory that the pointer points to if accessed through other pointers.
- */
- if (F->hasParamAttribute(ArgNo, Attribute::NoAlias) &&
- (F->hasParamAttribute(ArgNo, Attribute::ReadNone) ||
- F->hasParamAttribute(ArgNo, Attribute::ReadOnly))) {
- return true;
- }
- }
- return AAResultBase::pointsToConstantMemory(Loc, AAQI, OrLocal);
+ return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
}
diff --git a/llvm/lib/Target/AMDGPU/AMDGPUAliasAnalysis.h b/llvm/lib/Target/AMDGPU/AMDGPUAliasAnalysis.h
index db2372d911b1a..acd437e8987ce 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPUAliasAnalysis.h
+++ b/llvm/lib/Target/AMDGPU/AMDGPUAliasAnalysis.h
@@ -38,8 +38,8 @@ class AMDGPUAAResult : public AAResultBase {
AliasResult alias(const MemoryLocation &LocA, const MemoryLocation &LocB,
AAQueryInfo &AAQI);
- bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
- bool OrLocal);
+ ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
+ bool IgnoreLocals);
};
/// Analysis pass providing a never-invalidated alias analysis result.
diff --git a/llvm/lib/Transforms/IPO/FunctionAttrs.cpp b/llvm/lib/Transforms/IPO/FunctionAttrs.cpp
index 932feea044b39..a187cb1e4790e 100644
--- a/llvm/lib/Transforms/IPO/FunctionAttrs.cpp
+++ b/llvm/lib/Transforms/IPO/FunctionAttrs.cpp
@@ -140,13 +140,14 @@ static MemoryEffects checkFunctionMemoryAccess(Function &F, bool ThisBody,
ME |= MemoryEffects::argMemOnly(ModRefInfo::ModRef);
auto AddLocAccess = [&](const MemoryLocation &Loc, ModRefInfo MR) {
- // Ignore accesses to local memory.
- if (AAR.pointsToConstantMemory(Loc, /*OrLocal=*/true))
+ // Ignore accesses to known-invariant or local memory.
+ MR &= AAR.getModRefInfoMask(Loc, /*IgnoreLocal=*/true);
+ if (isNoModRef(MR))
return;
const Value *UO = getUnderlyingObject(Loc.Ptr);
assert(!isa<AllocaInst>(UO) &&
- "Should have been handled by pointsToConstantMemory()");
+ "Should have been handled by getModRefInfoMask()");
if (isa<Argument>(UO)) {
ME |= MemoryEffects::argMemOnly(MR);
return;
diff --git a/llvm/lib/Transforms/InstCombine/InstCombineCalls.cpp b/llvm/lib/Transforms/InstCombine/InstCombineCalls.cpp
index 5adb898f26075..3df5af6ac5171 100644
--- a/llvm/lib/Transforms/InstCombine/InstCombineCalls.cpp
+++ b/llvm/lib/Transforms/InstCombine/InstCombineCalls.cpp
@@ -135,7 +135,7 @@ Instruction *InstCombinerImpl::SimplifyAnyMemTransfer(AnyMemTransferInst *MI) {
// If we have a store to a location which is known constant, we can conclude
// that the store must be storing the constant value (else the memory
// wouldn't be constant), and this must be a noop.
- if (AA->pointsToConstantMemory(MI->getDest())) {
+ if (!isModSet(AA->getModRefInfoMask(MI->getDest()))) {
// Set the size of the copy to 0, it will be deleted on the next iteration.
MI->setLength(Constant::getNullValue(MI->getLength()->getType()));
return MI;
@@ -252,7 +252,7 @@ Instruction *InstCombinerImpl::SimplifyAnyMemSet(AnyMemSetInst *MI) {
// If we have a store to a location which is known constant, we can conclude
// that the store must be storing the constant value (else the memory
// wouldn't be constant), and this must be a noop.
- if (AA->pointsToConstantMemory(MI->getDest())) {
+ if (!isModSet(AA->getModRefInfoMask(MI->getDest()))) {
// Set the size of the copy to 0, it will be deleted on the next iteration.
MI->setLength(Constant::getNullValue(MI->getLength()->getType()));
return MI;
diff --git a/llvm/lib/Transforms/InstCombine/InstCombineLoadStoreAlloca.cpp b/llvm/lib/Transforms/InstCombine/InstCombineLoadStoreAlloca.cpp
index 9acfa612bc272..c4efd03017119 100644
--- a/llvm/lib/Transforms/InstCombine/InstCombineLoadStoreAlloca.cpp
+++ b/llvm/lib/Transforms/InstCombine/InstCombineLoadStoreAlloca.cpp
@@ -31,20 +31,20 @@ using namespace PatternMatch;
STATISTIC(NumDeadStore, "Number of dead stores eliminated");
STATISTIC(NumGlobalCopies, "Number of allocas copied from constant global");
-/// isOnlyCopiedFromConstantGlobal - Recursively walk the uses of a (derived)
+/// isOnlyCopiedFromConstantMemory - Recursively walk the uses of a (derived)
/// pointer to an alloca. Ignore any reads of the pointer, return false if we
/// see any stores or other unknown uses. If we see pointer arithmetic, keep
/// track of whether it moves the pointer (with IsOffset) but otherwise traverse
/// the uses. If we see a memcpy/memmove that targets an unoffseted pointer to
-/// the alloca, and if the source pointer is a pointer to a constant global, we
-/// can optimize this.
+/// the alloca, and if the source pointer is a pointer to a constant memory
+/// location, we can optimize this.
static bool
isOnlyCopiedFromConstantMemory(AAResults *AA, AllocaInst *V,
MemTransferInst *&TheCopy,
SmallVectorImpl<Instruction *> &ToDelete) {
// We track lifetime intrinsics as we encounter them. If we decide to go
- // ahead and replace the value with the global, this lets the caller quickly
- // eliminate the markers.
+ // ahead and replace the value with the memory location, this lets the caller
+ // quickly eliminate the markers.
SmallVector<std::pair<Value *, bool>, 35> ValuesToInspect;
ValuesToInspect.emplace_back(V, false);
@@ -131,8 +131,8 @@ isOnlyCopiedFromConstantMemory(AAResults *AA, AllocaInst *V,
// If the memintrinsic isn't using the alloca as the dest, reject it.
if (U.getOperandNo() != 0) return false;
- // If the source of the memcpy/move is not a constant global, reject it.
- if (!AA->pointsToConstantMemory(MI->getSource()))
+ // If the source of the memcpy/move is not constant, reject it.
+ if (isModSet(AA->getModRefInfoMask(MI->getSource())))
return false;
// Otherwise, the transform is safe. Remember the copy instruction.
@@ -142,9 +142,10 @@ isOnlyCopiedFromConstantMemory(AAResults *AA, AllocaInst *V,
return true;
}
-/// isOnlyCopiedFromConstantGlobal - Return true if the specified alloca is only
-/// modified by a copy from a constant global. If we can prove this, we can
-/// replace any uses of the alloca with uses of the global directly.
+/// isOnlyCopiedFromConstantMemory - Return true if the specified alloca is only
+/// modified by a copy from a constant memory location. If we can prove this, we
+/// can replace any uses of the alloca with uses of the memory location
+/// directly.
static MemTransferInst *
isOnlyCopiedFromConstantMemory(AAResults *AA,
AllocaInst *AI,
@@ -398,11 +399,11 @@ Instruction *InstCombinerImpl::visitAllocaInst(AllocaInst &AI) {
}
// Check to see if this allocation is only modified by a memcpy/memmove from
- // a constant whose alignment is equal to or exceeds that of the allocation.
- // If this is the case, we can change all users to use the constant global
- // instead. This is commonly produced by the CFE by constructs like "void
- // foo() { int A[] = {1,2,3,4,5,6,7,8,9...}; }" if 'A' is only subsequently
- // read.
+ // a memory location whose alignment is equal to or exceeds that of the
+ // allocation. If this is the case, we can change all users to use the
+ // constant memory location instead. This is commonly produced by the CFE by
+ // constructs like "void foo() { int A[] = {1,2,3,4,5,6,7,8,9...}; }" if 'A'
+ // is only subsequently read.
SmallVector<Instruction *, 4> ToDelete;
if (MemTransferInst *Copy = isOnlyCopiedFromConstantMemory(AA, &AI, ToDelete)) {
Value *TheSrc = Copy->getSource();
@@ -1378,7 +1379,7 @@ Instruction *InstCombinerImpl::visitStoreInst(StoreInst &SI) {
// If we have a store to a location which is known constant, we can conclude
// that the store must be storing the constant value (else the memory
// wouldn't be constant), and this must be a noop.
- if (AA->pointsToConstantMemory(Ptr))
+ if (!isModSet(AA->getModRefInfoMask(Ptr)))
return eraseInstFromFunction(SI);
// Do really simple DSE, to catch cases where there are several consecutive
diff --git a/llvm/lib/Transforms/Scalar/LICM.cpp b/llvm/lib/Transforms/Scalar/LICM.cpp
index cddee7d3eb07b..001937cc1670f 100644
--- a/llvm/lib/Transforms/Scalar/LICM.cpp
+++ b/llvm/lib/Transforms/Scalar/LICM.cpp
@@ -1161,7 +1161,7 @@ bool llvm::canSinkOrHoistInst(Instruction &I, AAResults *AA, DominatorTree *DT,
// Loads from constant memory are always safe to move, even if they end up
// in the same alias set as something that ends up being modified.
- if (AA->pointsToConstantMemory(LI->getOperand(0)))
+ if (!isModSet(AA->getModRefInfoMask(LI->getOperand(0))))
return true;
if (LI->hasMetadata(LLVMContext::MD_invariant_load))
return true;
diff --git a/llvm/lib/Transforms/Scalar/LoopPredication.cpp b/llvm/lib/Transforms/Scalar/LoopPredication.cpp
index 65abc8fb86645..fbd4a39c7949e 100644
--- a/llvm/lib/Transforms/Scalar/LoopPredication.cpp
+++ b/llvm/lib/Transforms/Scalar/LoopPredication.cpp
@@ -569,7 +569,7 @@ bool LoopPredication::isLoopInvariantValue(const SCEV* S) {
if (const SCEVUnknown *U = dyn_cast<SCEVUnknown>(S))
if (const auto *LI = dyn_cast<LoadInst>(U->getValue()))
if (LI->isUnordered() && L->hasLoopInvariantOperands(LI))
- if (AA->pointsToConstantMemory(LI->getOperand(0)) ||
+ if (!isModSet(AA->getModRefInfoMask(LI->getOperand(0))) ||
LI->hasMetadata(LLVMContext::MD_invariant_load))
return true;
return false;
diff --git a/llvm/test/Analysis/BasicAA/constant-memory.ll b/llvm/test/Analysis/BasicAA/constant-memory.ll
index 6ef875d9358ed..2b197d6dbc71a 100644
--- a/llvm/test/Analysis/BasicAA/constant-memory.ll
+++ b/llvm/test/Analysis/BasicAA/constant-memory.ll
@@ -6,18 +6,16 @@ declare void @dummy()
declare void @foo(ptr)
-; FIXME: This could be NoModRef
; CHECK-LABEL: Function: basic
-; CHECK: Just Ref: Ptr: i32* @c <-> call void @dummy()
+; CHECK: NoModRef: Ptr: i32* @c <-> call void @dummy()
define void @basic(ptr %p) {
call void @dummy()
load i32, ptr @c
ret void
}
-; FIXME: This could be NoModRef
; CHECK-LABEL: Function: recphi
-; CHECK: Just Ref: Ptr: i32* %p <-> call void @dummy()
+; CHECK: NoModRef: Ptr: i32* %p <-> call void @dummy()
define void @recphi() {
entry:
br label %loop
@@ -34,20 +32,20 @@ exit:
ret void
}
-; Tests that readonly noalias doesn't imply !Mod yet.
+; Tests that readonly noalias implies !Mod.
;
; CHECK-LABEL: Function: readonly_noalias
-; CHECK: Both ModRef: Ptr: i32* %p <-> call void @foo(ptr %p)
+; CHECK: Just Ref: Ptr: i32* %p <-> call void @foo(ptr %p)
define void @readonly_noalias(ptr readonly noalias %p) {
call void @foo(ptr %p)
load i32, ptr %p
ret void
}
-; Tests that readnone noalias doesn't imply !Mod yet.
+; Tests that readnone noalias implies !Mod.
;
; CHECK-LABEL: Function: readnone_noalias
-; CHECK: Both ModRef: Ptr: i32* %p <-> call void @foo(ptr %p)
+; CHECK: Just Ref: Ptr: i32* %p <-> call void @foo(ptr %p)
define void @readnone_noalias(ptr readnone noalias %p) {
call void @foo(ptr %p)
load i32, ptr %p
diff --git a/llvm/test/Transforms/InstCombine/memcpy-from-global.ll b/llvm/test/Transforms/InstCombine/memcpy-from-global.ll
index cd6dfc4561b97..b230c1488f75e 100644
--- a/llvm/test/Transforms/InstCombine/memcpy-from-global.ll
+++ b/llvm/test/Transforms/InstCombine/memcpy-from-global.ll
@@ -338,12 +338,10 @@ entry:
ret float %r
}
-; Tests that we can't eliminate allocas copied from readonly noalias pointers yet.
+; Tests that we can eliminate allocas copied from readonly noalias pointers.
define void @memcpy_from_readonly_noalias(ptr readonly noalias align 8 dereferenceable(124) %arg) {
; CHECK-LABEL: @memcpy_from_readonly_noalias(
-; CHECK-NEXT: [[ALLOCA:%.*]] = alloca [[T:%.*]], align 8
-; CHECK-NEXT: call void @llvm.memcpy.p0.p0.i64(ptr noundef nonnull align 8 dereferenceable(124) [[ALLOCA]], ptr noundef nonnull align 8 dereferenceable(124) [[ARG:%.*]], i64 124, i1 false)
-; CHECK-NEXT: call void @bar(ptr nonnull [[ALLOCA]]) #[[ATTR3]]
+; CHECK-NEXT: call void @bar(ptr nonnull [[ARG:%.*]]) #[[ATTR3]]
; CHECK-NEXT: ret void
;
%alloca = alloca %T, align 8
diff --git a/llvm/test/Transforms/InstCombine/store.ll b/llvm/test/Transforms/InstCombine/store.ll
index 7cbe91b44beb8..95ba64c9e6401 100644
--- a/llvm/test/Transforms/InstCombine/store.ll
+++ b/llvm/test/Transforms/InstCombine/store.ll
@@ -336,12 +336,10 @@ define void @store_to_constant() {
ret void
}
-; We can't delete stores to readonly noalias pointers yet.
+; Delete stores to readonly noalias pointers.
define void @store_to_readonly_noalias(ptr readonly noalias %0) {
; CHECK-LABEL: @store_to_readonly_noalias(
-; CHECK-NEXT: store i32 3, ptr [[TMP0:%.*]], align 4
; CHECK-NEXT: ret void
-;
store i32 3, ptr %0, align 4
ret void
}
More information about the llvm-commits
mailing list