[llvm] d14d84a - [NewPM] Only invalidate modified functions' analyses in CGSCC passes
Arthur Eubanks via llvm-commits
llvm-commits at lists.llvm.org
Mon May 3 17:21:54 PDT 2021
Author: Arthur Eubanks
Date: 2021-05-03T17:21:44-07:00
New Revision: d14d84af2f5ebb8ae2188ce6884a29a586dc0a40
URL: https://github.com/llvm/llvm-project/commit/d14d84af2f5ebb8ae2188ce6884a29a586dc0a40
DIFF: https://github.com/llvm/llvm-project/commit/d14d84af2f5ebb8ae2188ce6884a29a586dc0a40.diff
LOG: [NewPM] Only invalidate modified functions' analyses in CGSCC passes
Previously, any change in any function in an SCC would cause all
analyses for all functions in the SCC to be invalidated. With this
change, we now manually invalidate analyses for functions we modify,
then let the pass manager know that all function analyses should be
preserved.
So far this only touches the inliner, argpromotion, funcattrs, and
updateCGAndAnalysisManager(), since they are the most used.
Slight compile time improvements:
http://llvm-compile-time-tracker.com/compare.php?from=326da4adcb8def2abdd530299d87ce951c0edec9&to=8942c7669f330082ef159f3c6c57c3c28484f4be&stat=instructions
Reviewed By: mtrofin
Differential Revision: https://reviews.llvm.org/D100917
Added:
llvm/test/Transforms/Inline/analysis-invalidation.ll
Modified:
clang/test/CodeGen/thinlto-distributed-newpm.ll
llvm/lib/Analysis/CGSCCPassManager.cpp
llvm/lib/Transforms/IPO/ArgumentPromotion.cpp
llvm/lib/Transforms/IPO/FunctionAttrs.cpp
llvm/lib/Transforms/IPO/Inliner.cpp
llvm/test/Other/opt-O3-pipeline.ll
llvm/test/Transforms/Inline/cgscc-incremental-invalidate.ll
Removed:
################################################################################
diff --git a/clang/test/CodeGen/thinlto-distributed-newpm.ll b/clang/test/CodeGen/thinlto-distributed-newpm.ll
index 398b65116bbd7..bd72ceaaff7d5 100644
--- a/clang/test/CodeGen/thinlto-distributed-newpm.ll
+++ b/clang/test/CodeGen/thinlto-distributed-newpm.ll
@@ -70,19 +70,11 @@
; CHECK-O: Starting CGSCC pass manager run.
; CHECK-O: Running pass: InlinerPass on (main)
; CHECK-O: Running pass: PostOrderFunctionAttrsPass on (main)
-; CHECK-O: Invalidating analysis: DominatorTreeAnalysis on main
-; CHECK-O: Invalidating analysis: BasicAA on main
-; CHECK-O: Invalidating analysis: AAManager on main
; CHECK-O3: Running pass: ArgumentPromotionPass on (main)
; CHECK-O: Starting {{.*}}Function pass manager run.
; CHECK-O: Running pass: SROA on main
-; These next two can appear in any order since they are accessed as parameters
-; on the same call to SROA::runImpl
-; CHECK-O-DAG: Running analysis: DominatorTreeAnalysis on main
; CHECK-O: Running pass: EarlyCSEPass on main
; CHECK-O: Running analysis: MemorySSAAnalysis on main
-; CHECK-O: Running analysis: AAManager on main
-; CHECK-O: Running analysis: BasicAA on main
; CHECK-O: Running pass: SpeculativeExecutionPass on main
; CHECK-O: Running pass: JumpThreadingPass on main
; CHECK-O: Running analysis: LazyValueAnalysis on main
@@ -128,16 +120,6 @@
; CHECK-O: Running pass: InstCombinePass on main
; CHECK-O: Finished {{.*}}Function pass manager run.
; CHECK-O: Finished CGSCC pass manager run.
-; CHECK-O: Invalidating analysis: DominatorTreeAnalysis on main
-; CHECK-O: Invalidating analysis: BasicAA on main
-; CHECK-O: Invalidating analysis: AAManager on main
-; CHECK-O: Invalidating analysis: MemorySSAAnalysis on main
-; CHECK-O: Invalidating analysis: LoopAnalysis on main
-; CHECK-O: Invalidating analysis: PhiValuesAnalysis on main
-; CHECK-O: Invalidating analysis: MemoryDependenceAnalysis on main
-; CHECK-O: Invalidating analysis: DemandedBitsAnalysis on main
-; CHECK-O: Invalidating analysis: PostDominatorTreeAnalysis on main
-; CHECK-O: Invalidating analysis: CallGraphAnalysis
; CHECK-O: Running pass: GlobalOptPass
; CHECK-O: Running pass: GlobalDCEPass
; CHECK-O: Running pass: EliminateAvailableExternallyPass
@@ -149,22 +131,14 @@
; CHECK-O: Running pass: LowerConstantIntrinsicsPass on main
; CHECK-O: Starting {{.*}}Function pass manager run.
; CHECK-O: Running pass: LoopSimplifyPass on main
-; CHECK-O: Running analysis: LoopAnalysis on main
; CHECK-O: Running pass: LCSSAPass on main
; CHECK-O: Finished {{.*}}Function pass manager run.
-; CHECK-O: Running analysis: MemorySSAAnalysis on main
-; CHECK-O: Running analysis: AAManager on main
-; CHECK-O: Running analysis: BasicAA on main
-; CHECK-O: Running analysis: ScalarEvolutionAnalysis on main
-; CHECK-O: Running analysis: InnerAnalysisManagerProxy
; CHECK-O: Running pass: LoopRotatePass on Loop at depth 1 containing: %b
; CHECK-O: Running pass: LoopDistributePass on main
; CHECK-O: Running pass: InjectTLIMappings on main
; CHECK-O: Running pass: LoopVectorizePass on main
; CHECK-O: Running analysis: BlockFrequencyAnalysis on main
; CHECK-O: Running analysis: BranchProbabilityAnalysis on main
-; CHECK-O: Running analysis: PostDominatorTreeAnalysis on main
-; CHECK-O: Running analysis: DemandedBitsAnalysis on main
; CHECK-O: Running pass: LoopLoadEliminationPass on main
; CHECK-O: Running pass: InstCombinePass on main
; CHECK-O: Running pass: SimplifyCFGPass on main
diff --git a/llvm/lib/Analysis/CGSCCPassManager.cpp b/llvm/lib/Analysis/CGSCCPassManager.cpp
index eaaa3d09a7f28..99876630e8134 100644
--- a/llvm/lib/Analysis/CGSCCPassManager.cpp
+++ b/llvm/lib/Analysis/CGSCCPassManager.cpp
@@ -864,7 +864,7 @@ incorporateNewSCCRange(const SCCRangeT &NewSCCRange, LazyCallGraph &G,
// split-off SCCs.
// We know however that this will preserve any FAM proxy so go ahead and mark
// that.
- PreservedAnalyses PA;
+ auto PA = PreservedAnalyses::allInSet<AllAnalysesOn<Function>>();
PA.preserve<FunctionAnalysisManagerCGSCCProxy>();
AM.invalidate(*OldC, PA);
diff --git a/llvm/lib/Transforms/IPO/ArgumentPromotion.cpp b/llvm/lib/Transforms/IPO/ArgumentPromotion.cpp
index dd72ac4136133..fadcb7d561d34 100644
--- a/llvm/lib/Transforms/IPO/ArgumentPromotion.cpp
+++ b/llvm/lib/Transforms/IPO/ArgumentPromotion.cpp
@@ -1021,11 +1021,12 @@ PreservedAnalyses ArgumentPromotionPass::run(LazyCallGraph::SCC &C,
do {
LocalChange = false;
+ FunctionAnalysisManager &FAM =
+ AM.getResult<FunctionAnalysisManagerCGSCCProxy>(C, CG).getManager();
+
for (LazyCallGraph::Node &N : C) {
Function &OldF = N.getFunction();
- FunctionAnalysisManager &FAM =
- AM.getResult<FunctionAnalysisManagerCGSCCProxy>(C, CG).getManager();
// FIXME: This lambda must only be used with this function. We should
// skip the lambda and just get the AA results directly.
auto AARGetter = [&](Function &F) -> AAResults & {
@@ -1048,6 +1049,13 @@ PreservedAnalyses ArgumentPromotionPass::run(LazyCallGraph::SCC &C,
C.getOuterRefSCC().replaceNodeFunction(N, *NewF);
FAM.clear(OldF, OldF.getName());
OldF.eraseFromParent();
+
+ PreservedAnalyses FuncPA;
+ FuncPA.preserveSet<CFGAnalyses>();
+ for (auto *U : NewF->users()) {
+ auto *UserF = cast<CallBase>(U)->getParent()->getParent();
+ FAM.invalidate(*UserF, FuncPA);
+ }
}
Changed |= LocalChange;
@@ -1056,7 +1064,10 @@ PreservedAnalyses ArgumentPromotionPass::run(LazyCallGraph::SCC &C,
if (!Changed)
return PreservedAnalyses::all();
- return PreservedAnalyses::none();
+ PreservedAnalyses PA;
+ PA.preserve<FunctionAnalysisManagerCGSCCProxy>();
+ PA.preserveSet<AllAnalysesOn<Function>>();
+ return PA;
}
namespace {
diff --git a/llvm/lib/Transforms/IPO/FunctionAttrs.cpp b/llvm/lib/Transforms/IPO/FunctionAttrs.cpp
index ca8660a98ded8..25b37c0a18163 100644
--- a/llvm/lib/Transforms/IPO/FunctionAttrs.cpp
+++ b/llvm/lib/Transforms/IPO/FunctionAttrs.cpp
@@ -240,7 +240,8 @@ MemoryAccessKind llvm::computeFunctionBodyMemoryAccess(Function &F,
/// Deduce readonly/readnone attributes for the SCC.
template <typename AARGetterT>
-static bool addReadAttrs(const SCCNodeSet &SCCNodes, AARGetterT &&AARGetter) {
+static void addReadAttrs(const SCCNodeSet &SCCNodes, AARGetterT &&AARGetter,
+ SmallSetVector<Function *, 8> &Changed) {
// Check if any of the functions in the SCC read or write memory. If they
// write memory then they can't be marked readnone or readonly.
bool ReadsMemory = false;
@@ -255,7 +256,7 @@ static bool addReadAttrs(const SCCNodeSet &SCCNodes, AARGetterT &&AARGetter) {
switch (checkFunctionMemoryAccess(*F, F->hasExactDefinition(),
AAR, SCCNodes)) {
case MAK_MayWrite:
- return false;
+ return;
case MAK_ReadOnly:
ReadsMemory = true;
break;
@@ -271,11 +272,10 @@ static bool addReadAttrs(const SCCNodeSet &SCCNodes, AARGetterT &&AARGetter) {
// If the SCC contains both functions that read and functions that write, then
// we cannot add readonly attributes.
if (ReadsMemory && WritesMemory)
- return false;
+ return;
// Success! Functions in this SCC do not access memory, or only read memory.
// Give them the appropriate attribute.
- bool MadeChange = false;
for (Function *F : SCCNodes) {
if (F->doesNotAccessMemory())
@@ -289,7 +289,7 @@ static bool addReadAttrs(const SCCNodeSet &SCCNodes, AARGetterT &&AARGetter) {
if (F->doesNotReadMemory() && WritesMemory)
continue;
- MadeChange = true;
+ Changed.insert(F);
// Clear out any existing attributes.
AttrBuilder AttrsToRemove;
@@ -318,8 +318,6 @@ static bool addReadAttrs(const SCCNodeSet &SCCNodes, AARGetterT &&AARGetter) {
else
++NumReadNone;
}
-
- return MadeChange;
}
namespace {
@@ -581,9 +579,8 @@ determinePointerReadAttrs(Argument *A,
}
/// Deduce returned attributes for the SCC.
-static bool addArgumentReturnedAttrs(const SCCNodeSet &SCCNodes) {
- bool Changed = false;
-
+static void addArgumentReturnedAttrs(const SCCNodeSet &SCCNodes,
+ SmallSetVector<Function *, 8> &Changed) {
// Check each function in turn, determining if an argument is always returned.
for (Function *F : SCCNodes) {
// We can infer and propagate function attributes only when we know that the
@@ -623,11 +620,9 @@ static bool addArgumentReturnedAttrs(const SCCNodeSet &SCCNodes) {
auto *A = cast<Argument>(RetArg);
A->addAttr(Attribute::Returned);
++NumReturned;
- Changed = true;
+ Changed.insert(F);
}
}
-
- return Changed;
}
/// If a callsite has arguments that are also arguments to the parent function,
@@ -693,9 +688,8 @@ static bool addReadAttr(Argument *A, Attribute::AttrKind R) {
}
/// Deduce nocapture attributes for the SCC.
-static bool addArgumentAttrs(const SCCNodeSet &SCCNodes) {
- bool Changed = false;
-
+static void addArgumentAttrs(const SCCNodeSet &SCCNodes,
+ SmallSetVector<Function *, 8> &Changed) {
ArgumentGraph AG;
// Check each function in turn, determining which pointer arguments are not
@@ -707,7 +701,8 @@ static bool addArgumentAttrs(const SCCNodeSet &SCCNodes) {
if (!F->hasExactDefinition())
continue;
- Changed |= addArgumentAttrsFromCallsites(*F);
+ if (addArgumentAttrsFromCallsites(*F))
+ Changed.insert(F);
// Functions that are readonly (or readnone) and nounwind and don't return
// a value can't capture arguments. Don't analyze them.
@@ -718,7 +713,7 @@ static bool addArgumentAttrs(const SCCNodeSet &SCCNodes) {
if (A->getType()->isPointerTy() && !A->hasNoCaptureAttr()) {
A->addAttr(Attribute::NoCapture);
++NumNoCapture;
- Changed = true;
+ Changed.insert(F);
}
}
continue;
@@ -737,7 +732,7 @@ static bool addArgumentAttrs(const SCCNodeSet &SCCNodes) {
// If it's trivially not captured, mark it nocapture now.
A->addAttr(Attribute::NoCapture);
++NumNoCapture;
- Changed = true;
+ Changed.insert(F);
} else {
// If it's not trivially captured and not trivially not captured,
// then it must be calling into another function in our SCC. Save
@@ -761,7 +756,8 @@ static bool addArgumentAttrs(const SCCNodeSet &SCCNodes) {
Self.insert(&*A);
Attribute::AttrKind R = determinePointerReadAttrs(&*A, Self);
if (R != Attribute::None)
- Changed = addReadAttr(A, R);
+ if (addReadAttr(A, R))
+ Changed.insert(F);
}
}
}
@@ -785,7 +781,7 @@ static bool addArgumentAttrs(const SCCNodeSet &SCCNodes) {
Argument *A = ArgumentSCC[0]->Definition;
A->addAttr(Attribute::NoCapture);
++NumNoCapture;
- Changed = true;
+ Changed.insert(A->getParent());
}
continue;
}
@@ -827,7 +823,7 @@ static bool addArgumentAttrs(const SCCNodeSet &SCCNodes) {
Argument *A = ArgumentSCC[i]->Definition;
A->addAttr(Attribute::NoCapture);
++NumNoCapture;
- Changed = true;
+ Changed.insert(A->getParent());
}
// We also want to compute readonly/readnone. With a small number of false
@@ -858,12 +854,11 @@ static bool addArgumentAttrs(const SCCNodeSet &SCCNodes) {
if (ReadAttr != Attribute::None) {
for (unsigned i = 0, e = ArgumentSCC.size(); i != e; ++i) {
Argument *A = ArgumentSCC[i]->Definition;
- Changed = addReadAttr(A, ReadAttr);
+ if (addReadAttr(A, ReadAttr))
+ Changed.insert(A->getParent());
}
}
}
-
- return Changed;
}
/// Tests whether a function is "malloc-like".
@@ -934,7 +929,8 @@ static bool isFunctionMallocLike(Function *F, const SCCNodeSet &SCCNodes) {
}
/// Deduce noalias attributes for the SCC.
-static bool addNoAliasAttrs(const SCCNodeSet &SCCNodes) {
+static void addNoAliasAttrs(const SCCNodeSet &SCCNodes,
+ SmallSetVector<Function *, 8> &Changed) {
// Check each function in turn, determining which functions return noalias
// pointers.
for (Function *F : SCCNodes) {
@@ -946,7 +942,7 @@ static bool addNoAliasAttrs(const SCCNodeSet &SCCNodes) {
// definition we'll get at link time is *exactly* the definition we see now.
// For more details, see GlobalValue::mayBeDerefined.
if (!F->hasExactDefinition())
- return false;
+ return;
// We annotate noalias return values, which are only applicable to
// pointer types.
@@ -954,10 +950,9 @@ static bool addNoAliasAttrs(const SCCNodeSet &SCCNodes) {
continue;
if (!isFunctionMallocLike(F, SCCNodes))
- return false;
+ return;
}
- bool MadeChange = false;
for (Function *F : SCCNodes) {
if (F->returnDoesNotAlias() ||
!F->getReturnType()->isPointerTy())
@@ -965,10 +960,8 @@ static bool addNoAliasAttrs(const SCCNodeSet &SCCNodes) {
F->setReturnDoesNotAlias();
++NumNoAlias;
- MadeChange = true;
+ Changed.insert(F);
}
-
- return MadeChange;
}
/// Tests whether this function is known to not return null.
@@ -1044,13 +1037,12 @@ static bool isReturnNonNull(Function *F, const SCCNodeSet &SCCNodes,
}
/// Deduce nonnull attributes for the SCC.
-static bool addNonNullAttrs(const SCCNodeSet &SCCNodes) {
+static void addNonNullAttrs(const SCCNodeSet &SCCNodes,
+ SmallSetVector<Function *, 8> &Changed) {
// Speculative that all functions in the SCC return only nonnull
// pointers. We may refute this as we analyze functions.
bool SCCReturnsNonNull = true;
- bool MadeChange = false;
-
// Check each function in turn, determining which functions return nonnull
// pointers.
for (Function *F : SCCNodes) {
@@ -1063,7 +1055,7 @@ static bool addNonNullAttrs(const SCCNodeSet &SCCNodes) {
// definition we'll get at link time is *exactly* the definition we see now.
// For more details, see GlobalValue::mayBeDerefined.
if (!F->hasExactDefinition())
- return false;
+ return;
// We annotate nonnull return values, which are only applicable to
// pointer types.
@@ -1079,7 +1071,7 @@ static bool addNonNullAttrs(const SCCNodeSet &SCCNodes) {
<< " as nonnull\n");
F->addAttribute(AttributeList::ReturnIndex, Attribute::NonNull);
++NumNonNullReturn;
- MadeChange = true;
+ Changed.insert(F);
}
continue;
}
@@ -1098,11 +1090,9 @@ static bool addNonNullAttrs(const SCCNodeSet &SCCNodes) {
LLVM_DEBUG(dbgs() << "SCC marking " << F->getName() << " as nonnull\n");
F->addAttribute(AttributeList::ReturnIndex, Attribute::NonNull);
++NumNonNullReturn;
- MadeChange = true;
+ Changed.insert(F);
}
}
-
- return MadeChange;
}
namespace {
@@ -1155,12 +1145,13 @@ class AttributeInferer {
InferenceDescriptors.push_back(AttrInference);
}
- bool run(const SCCNodeSet &SCCNodes);
+ void run(const SCCNodeSet &SCCNodes, SmallSetVector<Function *, 8> &Changed);
};
/// Perform all the requested attribute inference actions according to the
/// attribute predicates stored before.
-bool AttributeInferer::run(const SCCNodeSet &SCCNodes) {
+void AttributeInferer::run(const SCCNodeSet &SCCNodes,
+ SmallSetVector<Function *, 8> &Changed) {
SmallVector<InferenceDescriptor, 4> InferInSCC = InferenceDescriptors;
// Go through all the functions in SCC and check corresponding attribute
// assumptions for each of them. Attributes that are invalid for this SCC
@@ -1169,7 +1160,7 @@ bool AttributeInferer::run(const SCCNodeSet &SCCNodes) {
// No attributes whose assumptions are still valid - done.
if (InferInSCC.empty())
- return false;
+ return;
// Check if our attributes ever need scanning/can be scanned.
llvm::erase_if(InferInSCC, [F](const InferenceDescriptor &ID) {
@@ -1212,9 +1203,8 @@ bool AttributeInferer::run(const SCCNodeSet &SCCNodes) {
}
if (InferInSCC.empty())
- return false;
+ return;
- bool Changed = false;
for (Function *F : SCCNodes)
// At this point InferInSCC contains only functions that were either:
// - explicitly skipped from scan/inference, or
@@ -1223,10 +1213,9 @@ bool AttributeInferer::run(const SCCNodeSet &SCCNodes) {
for (auto &ID : InferInSCC) {
if (ID.SkipFunction(*F))
continue;
- Changed = true;
+ Changed.insert(F);
ID.SetAttribute(*F);
}
- return Changed;
}
struct SCCNodesResult {
@@ -1282,7 +1271,8 @@ static bool InstrBreaksNoFree(Instruction &I, const SCCNodeSet &SCCNodes) {
/// Attempt to remove convergent function attribute when possible.
///
/// Returns true if any changes to function attributes were made.
-static bool inferConvergent(const SCCNodeSet &SCCNodes) {
+static void inferConvergent(const SCCNodeSet &SCCNodes,
+ SmallSetVector<Function *, 8> &Changed) {
AttributeInferer AI;
// Request to remove the convergent attribute from all functions in the SCC
@@ -1305,7 +1295,7 @@ static bool inferConvergent(const SCCNodeSet &SCCNodes) {
},
/* RequiresExactDefinition= */ false});
// Perform all the requested attribute inference actions.
- return AI.run(SCCNodes);
+ AI.run(SCCNodes, Changed);
}
/// Infer attributes from all functions in the SCC by scanning every
@@ -1314,7 +1304,9 @@ static bool inferConvergent(const SCCNodeSet &SCCNodes) {
/// - addition of NoUnwind attribute
///
/// Returns true if any changes to function attributes were made.
-static bool inferAttrsFromFunctionBodies(const SCCNodeSet &SCCNodes) {
+static void
+inferAttrsFromFunctionBodies(const SCCNodeSet &SCCNodes,
+ SmallSetVector<Function *, 8> &Changed) {
AttributeInferer AI;
if (!DisableNoUnwindInference)
@@ -1363,19 +1355,20 @@ static bool inferAttrsFromFunctionBodies(const SCCNodeSet &SCCNodes) {
/* RequiresExactDefinition= */ true});
// Perform all the requested attribute inference actions.
- return AI.run(SCCNodes);
+ AI.run(SCCNodes, Changed);
}
-static bool addNoRecurseAttrs(const SCCNodeSet &SCCNodes) {
+static void addNoRecurseAttrs(const SCCNodeSet &SCCNodes,
+ SmallSetVector<Function *, 8> &Changed) {
// Try and identify functions that do not recurse.
// If the SCC contains multiple nodes we know for sure there is recursion.
if (SCCNodes.size() != 1)
- return false;
+ return;
Function *F = *SCCNodes.begin();
if (!F || !F->hasExactDefinition() || F->doesNotRecurse())
- return false;
+ return;
// If all of the calls in F are identifiable and are to norecurse functions, F
// is norecurse. This check also detects self-recursion as F is not currently
@@ -1386,7 +1379,7 @@ static bool addNoRecurseAttrs(const SCCNodeSet &SCCNodes) {
Function *Callee = CB->getCalledFunction();
if (!Callee || Callee == F || !Callee->doesNotRecurse())
// Function calls a potentially recursive function.
- return false;
+ return;
}
// Every call was to a non-recursive function other than this function, and
@@ -1394,7 +1387,7 @@ static bool addNoRecurseAttrs(const SCCNodeSet &SCCNodes) {
// recurse.
F->setDoesNotRecurse();
++NumNoRecurse;
- return true;
+ Changed.insert(F);
}
static bool instructionDoesNotReturn(Instruction &I) {
@@ -1412,9 +1405,8 @@ static bool basicBlockCanReturn(BasicBlock &BB) {
}
// Set the noreturn function attribute if possible.
-static bool addNoReturnAttrs(const SCCNodeSet &SCCNodes) {
- bool Changed = false;
-
+static void addNoReturnAttrs(const SCCNodeSet &SCCNodes,
+ SmallSetVector<Function *, 8> &Changed) {
for (Function *F : SCCNodes) {
if (!F || !F->hasExactDefinition() || F->hasFnAttribute(Attribute::Naked) ||
F->doesNotReturn())
@@ -1424,11 +1416,9 @@ static bool addNoReturnAttrs(const SCCNodeSet &SCCNodes) {
// FIXME: this doesn't handle recursion or unreachable blocks.
if (none_of(*F, basicBlockCanReturn)) {
F->setDoesNotReturn();
- Changed = true;
+ Changed.insert(F);
}
}
-
- return Changed;
}
static bool functionWillReturn(const Function &F) {
@@ -1461,19 +1451,16 @@ static bool functionWillReturn(const Function &F) {
}
// Set the willreturn function attribute if possible.
-static bool addWillReturn(const SCCNodeSet &SCCNodes) {
- bool Changed = false;
-
+static void addWillReturn(const SCCNodeSet &SCCNodes,
+ SmallSetVector<Function *, 8> &Changed) {
for (Function *F : SCCNodes) {
if (!F || F->willReturn() || !functionWillReturn(*F))
continue;
F->setWillReturn();
NumWillReturn++;
- Changed = true;
+ Changed.insert(F);
}
-
- return Changed;
}
// Return true if this is an atomic which has an ordering stronger than
@@ -1532,7 +1519,8 @@ static bool InstrBreaksNoSync(Instruction &I, const SCCNodeSet &SCCNodes) {
}
// Infer the nosync attribute.
-static bool addNoSyncAttr(const SCCNodeSet &SCCNodes) {
+static void addNoSyncAttr(const SCCNodeSet &SCCNodes,
+ SmallSetVector<Function *, 8> &Changed) {
AttributeInferer AI;
AI.registerAttrInference(AttributeInferer::InferenceDescriptor{
Attribute::NoSync,
@@ -1549,7 +1537,7 @@ static bool addNoSyncAttr(const SCCNodeSet &SCCNodes) {
++NumNoSync;
},
/* RequiresExactDefinition= */ true});
- return AI.run(SCCNodes);
+ AI.run(SCCNodes, Changed);
}
static SCCNodesResult createSCCNodeSet(ArrayRef<Function *> Functions) {
@@ -1582,32 +1570,33 @@ static SCCNodesResult createSCCNodeSet(ArrayRef<Function *> Functions) {
}
template <typename AARGetterT>
-static bool deriveAttrsInPostOrder(ArrayRef<Function *> Functions,
- AARGetterT &&AARGetter) {
+static SmallSetVector<Function *, 8>
+deriveAttrsInPostOrder(ArrayRef<Function *> Functions, AARGetterT &&AARGetter) {
SCCNodesResult Nodes = createSCCNodeSet(Functions);
- bool Changed = false;
// Bail if the SCC only contains optnone functions.
if (Nodes.SCCNodes.empty())
- return Changed;
+ return {};
- Changed |= addArgumentReturnedAttrs(Nodes.SCCNodes);
- Changed |= addReadAttrs(Nodes.SCCNodes, AARGetter);
- Changed |= addArgumentAttrs(Nodes.SCCNodes);
- Changed |= inferConvergent(Nodes.SCCNodes);
- Changed |= addNoReturnAttrs(Nodes.SCCNodes);
- Changed |= addWillReturn(Nodes.SCCNodes);
+ SmallSetVector<Function *, 8> Changed;
+
+ addArgumentReturnedAttrs(Nodes.SCCNodes, Changed);
+ addReadAttrs(Nodes.SCCNodes, AARGetter, Changed);
+ addArgumentAttrs(Nodes.SCCNodes, Changed);
+ inferConvergent(Nodes.SCCNodes, Changed);
+ addNoReturnAttrs(Nodes.SCCNodes, Changed);
+ addWillReturn(Nodes.SCCNodes, Changed);
// If we have no external nodes participating in the SCC, we can deduce some
// more precise attributes as well.
if (!Nodes.HasUnknownCall) {
- Changed |= addNoAliasAttrs(Nodes.SCCNodes);
- Changed |= addNonNullAttrs(Nodes.SCCNodes);
- Changed |= inferAttrsFromFunctionBodies(Nodes.SCCNodes);
- Changed |= addNoRecurseAttrs(Nodes.SCCNodes);
+ addNoAliasAttrs(Nodes.SCCNodes, Changed);
+ addNonNullAttrs(Nodes.SCCNodes, Changed);
+ inferAttrsFromFunctionBodies(Nodes.SCCNodes, Changed);
+ addNoRecurseAttrs(Nodes.SCCNodes, Changed);
}
- Changed |= addNoSyncAttr(Nodes.SCCNodes);
+ addNoSyncAttr(Nodes.SCCNodes, Changed);
// Finally, infer the maximal set of attributes from the ones we've inferred
// above. This is handling the cases where one attribute on a signature
@@ -1615,7 +1604,8 @@ static bool deriveAttrsInPostOrder(ArrayRef<Function *> Functions,
// the later is missing (or simply less sophisticated).
for (Function *F : Nodes.SCCNodes)
if (F)
- Changed |= inferAttributesFromOthers(*F);
+ if (inferAttributesFromOthers(*F))
+ Changed.insert(F);
return Changed;
}
@@ -1638,14 +1628,24 @@ PreservedAnalyses PostOrderFunctionAttrsPass::run(LazyCallGraph::SCC &C,
Functions.push_back(&N.getFunction());
}
- if (deriveAttrsInPostOrder(Functions, AARGetter)) {
- // We have not changed the call graph or removed/added functions.
- PreservedAnalyses PA;
- PA.preserve<FunctionAnalysisManagerCGSCCProxy>();
- return PA;
+ auto ChangedFunctions = deriveAttrsInPostOrder(Functions, AARGetter);
+ if (ChangedFunctions.empty()) {
+ return PreservedAnalyses::all();
}
- return PreservedAnalyses::all();
+ // Invalidate analyses for modified functions so that we don't have to
+ // invalidate all analyses for all functions in this SCC.
+ PreservedAnalyses FuncPA;
+ // We haven't changed the CFG for modified functions.
+ FuncPA.preserveSet<CFGAnalyses>();
+ for (Function *Changed : ChangedFunctions)
+ FAM.invalidate(*Changed, FuncPA);
+
+ // We have not changed the call graph or removed/added functions.
+ PreservedAnalyses PA;
+ PA.preserve<FunctionAnalysisManagerCGSCCProxy>();
+ PA.preserveSet<AllAnalysesOn<Function>>();
+ return PA;
}
namespace {
@@ -1690,7 +1690,7 @@ static bool runImpl(CallGraphSCC &SCC, AARGetterT AARGetter) {
Functions.push_back(I->getFunction());
}
- return deriveAttrsInPostOrder(Functions, AARGetter);
+ return !deriveAttrsInPostOrder(Functions, AARGetter).empty();
}
bool PostOrderFunctionAttrsLegacyPass::runOnSCC(CallGraphSCC &SCC) {
diff --git a/llvm/lib/Transforms/IPO/Inliner.cpp b/llvm/lib/Transforms/IPO/Inliner.cpp
index 1c9800f69e6aa..b0a1f05c3de70 100644
--- a/llvm/lib/Transforms/IPO/Inliner.cpp
+++ b/llvm/lib/Transforms/IPO/Inliner.cpp
@@ -951,6 +951,10 @@ PreservedAnalyses InlinerPass::run(LazyCallGraph::SCC &InitialC,
UR.InlinedInternalEdges.insert({&N, OldC});
}
InlinedCallees.clear();
+
+ // Invalidate analyses for this function now so that we don't have to
+ // invalidate analyses for all functions in this SCC later.
+ FAM.invalidate(F, PreservedAnalyses::none());
}
// Now that we've finished inlining all of the calls across this SCC, delete
@@ -990,10 +994,12 @@ PreservedAnalyses InlinerPass::run(LazyCallGraph::SCC &InitialC,
if (!Changed)
return PreservedAnalyses::all();
+ PreservedAnalyses PA;
// Even if we change the IR, we update the core CGSCC data structures and so
// can preserve the proxy to the function analysis manager.
- PreservedAnalyses PA;
PA.preserve<FunctionAnalysisManagerCGSCCProxy>();
+ // We have already invalidated all analyses on modified functions.
+ PA.preserveSet<AllAnalysesOn<Function>>();
return PA;
}
diff --git a/llvm/test/Other/opt-O3-pipeline.ll b/llvm/test/Other/opt-O3-pipeline.ll
index 6802d58bc0954..a97f28231ab19 100644
--- a/llvm/test/Other/opt-O3-pipeline.ll
+++ b/llvm/test/Other/opt-O3-pipeline.ll
@@ -399,13 +399,9 @@
; NEWPM-NEXT: OpenMPOptCGSCCPass on (f)
; NEWPM-NEXT: CGSCCToFunctionPassAdaptor on (f)
; NEWPM-NEXT: PassManager<{{.*}}> on f
-; NEWPM-NEXT: PreservedCFGCheckerAnalysis analysis on f
; NEWPM-NEXT: SROA on f
-; NEWPM-NEXT: DominatorTreeAnalysis analysis on f
; NEWPM-NEXT: EarlyCSEPass on f
; NEWPM-NEXT: MemorySSAAnalysis analysis on f
-; NEWPM-NEXT: AAManager analysis on f
-; NEWPM-NEXT: BasicAA analysis on f
; NEWPM-NEXT: SpeculativeExecutionPass on f
; NEWPM-NEXT: JumpThreadingPass on f
; NEWPM-NEXT: LazyValueAnalysis analysis on f
@@ -459,27 +455,19 @@
; NEWPM-NEXT: RequireAnalysisPass<{{.*}}> on [module]
; NEWPM-NEXT: ModuleToFunctionPassAdaptor on [module]
; NEWPM-NEXT: PassManager<{{.*}}> on f
-; NEWPM-NEXT: PreservedCFGCheckerAnalysis analysis on f
; NEWPM-NEXT: Float2IntPass on f
-; NEWPM-NEXT: DominatorTreeAnalysis analysis on f
; NEWPM-NEXT: LowerConstantIntrinsicsPass on f
; NEWPM-NEXT: FunctionToLoopPassAdaptor on f
; NEWPM-NEXT: PassManager<{{.*}}> on f
; NEWPM-NEXT: LoopSimplifyPass on f
-; NEWPM-NEXT: LoopAnalysis analysis on f
; NEWPM-NEXT: LCSSAPass on f
; NEWPM-NEXT: LoopDistributePass on f
; NEWPM-NEXT: ScalarEvolutionAnalysis analysis on f
-; NEWPM-NEXT: AAManager analysis on f
-; NEWPM-NEXT: BasicAA analysis on f
; NEWPM-NEXT: InnerAnalysisManagerProxy<{{.*}}> analysis on f
; NEWPM-NEXT: InjectTLIMappings on f
; NEWPM-NEXT: LoopVectorizePass on f
; NEWPM-NEXT: BlockFrequencyAnalysis analysis on f
; NEWPM-NEXT: BranchProbabilityAnalysis analysis on f
-; NEWPM-NEXT: PostDominatorTreeAnalysis analysis on f
-; NEWPM-NEXT: DemandedBitsAnalysis analysis on f
-; NEWPM-NEXT: MemorySSAAnalysis analysis on f
; NEWPM-NEXT: LoopLoadEliminationPass on f
; NEWPM-NEXT: InstCombinePass on f
; NEWPM-NEXT: SimplifyCFGPass on f
diff --git a/llvm/test/Transforms/Inline/analysis-invalidation.ll b/llvm/test/Transforms/Inline/analysis-invalidation.ll
new file mode 100644
index 0000000000000..ff7b12fd6a3ab
--- /dev/null
+++ b/llvm/test/Transforms/Inline/analysis-invalidation.ll
@@ -0,0 +1,17 @@
+; RUN: opt -passes=inliner-wrapper < %s -disable-output -debug-pass-manager 2>&1 | FileCheck %s
+
+; We shouldn't invalidate any function analyses on g since it's never modified.
+
+; CHECK-NOT: Invalidating{{.*}} on g
+; CHECK: Invalidating{{.*}} on f
+; CHECK-NOT: Invalidating{{.*}} on g
+
+define void @f() {
+ call void @g()
+ ret void
+}
+
+define void @g() alwaysinline {
+ call void @f()
+ ret void
+}
diff --git a/llvm/test/Transforms/Inline/cgscc-incremental-invalidate.ll b/llvm/test/Transforms/Inline/cgscc-incremental-invalidate.ll
index 6e2c1c3e7ea4b..66eec6a59dcf4 100644
--- a/llvm/test/Transforms/Inline/cgscc-incremental-invalidate.ll
+++ b/llvm/test/Transforms/Inline/cgscc-incremental-invalidate.ll
@@ -9,11 +9,11 @@
; CHECK-LABEL: Starting llvm::Module pass manager run.
; CHECK: Running pass: InlinerPass on (test1_f, test1_g, test1_h)
; CHECK: Running analysis: DominatorTreeAnalysis on test1_f
-; CHECK: Running analysis: DominatorTreeAnalysis on test1_g
; CHECK: Invalidating analysis: DominatorTreeAnalysis on test1_f
; CHECK: Invalidating analysis: LoopAnalysis on test1_f
; CHECK: Invalidating analysis: BranchProbabilityAnalysis on test1_f
; CHECK: Invalidating analysis: BlockFrequencyAnalysis on test1_f
+; CHECK: Running analysis: DominatorTreeAnalysis on test1_g
; CHECK: Invalidating analysis: DominatorTreeAnalysis on test1_g
; CHECK: Invalidating analysis: LoopAnalysis on test1_g
; CHECK: Invalidating analysis: BranchProbabilityAnalysis on test1_g
@@ -34,7 +34,6 @@
; CHECK-NEXT: Finished llvm::Function pass manager run.
; CHECK-NOT: Invalidating analysis:
; CHECK: Running pass: DominatorTreeVerifierPass on test1_f
-; CHECK-NEXT: Running analysis: DominatorTreeAnalysis on test1_f
; An external function used to control branches.
declare i1 @flag()
More information about the llvm-commits
mailing list