[PATCH] D144849: [SCEV] Only add uses to worklist for instrs in map, loop or phis.

[Florian Hahn via Phabricator via llvm-commits]

fhahn created this revision.
fhahn added reviewers: mkazantsev, nikic, reames.
Herald added subscribers: StephenFan, javed.absar, hiraditya.
Herald added a project: All.
fhahn requested review of this revision.
Herald added a project: LLVM.
Herald added a subscriber: llvm-commits.

This patch limits the def-use traversal in forgetValue to defs that have
a value in ValueExprMap, are in the same loop as the starting
instructions or are phis. This should ensure that we invalidate SCEVs as
needed, but avoid traversing instructions for which no SCEVs exist or
which have already been invalidated.

In particular, it should still invalidate

- SCEVs for instructions if any of its operands have a SCEV which is invalidate,
- any add-recs that depend on a given value in the current loop,
- and LCSSA loop exit phis.

There's assertions to try to catch any cases where invalidation is
missed now, which should help to surface any edge-cases, if any.

This can cut down the time spent invalidating SCEVs. For some internal
workloads with very large functions, this reduces the time spent in
passes using SCEV by 2-5x.

For CTmark, there are also some notable improvements, mostly in ClamAV
(-0.91% with -O3):

- NewPM-O3: -0.12%
- NewPM-ReleaseThinLTO: -0.10%
- NewPM-ReleaseLTO-g: -0.07%

https://llvm-compile-time-tracker.com/compare.php?from=ccea4e9b0494787e56809f3a8d261f49c829b149&to=d41584481b8f4f0501ed8197590048bd70fa1e2c&stat=instructions:u

Depends on D144848 <https://reviews.llvm.org/D144848>.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D144849

Files:
  llvm/include/llvm/Analysis/ScalarEvolution.h
  llvm/lib/Analysis/ScalarEvolution.cpp


Index: llvm/lib/Analysis/ScalarEvolution.cpp
===================================================================
--- llvm/lib/Analysis/ScalarEvolution.cpp
+++ llvm/lib/Analysis/ScalarEvolution.cpp
@@ -8438,7 +8438,7 @@
 void ScalarEvolution::visitAndClearUsers(
     SmallVectorImpl<Instruction *> &Worklist,
     SmallPtrSetImpl<Instruction *> &Visited,
-    SmallVectorImpl<const SCEV *> &ToForget) {
+    SmallVectorImpl<const SCEV *> &ToForget, const Loop *CurrL) {
   while (!Worklist.empty()) {
     Instruction *I = Worklist.pop_back_val();
     if (!isSCEVable(I->getType()))
@@ -8451,9 +8451,10 @@
       ToForget.push_back(It->second);
       if (PHINode *PN = dyn_cast<PHINode>(I))
         ConstantEvolutionLoopExitValue.erase(PN);
-    }
-
-    PushDefUseChildren(I, Worklist, Visited);
+      PushDefUseChildren(I, Worklist, Visited);
+    } else if ((CurrL && LI.getLoopFor(I->getParent()) == CurrL) ||
+               isa<PHINode>(I))
+      PushDefUseChildren(I, Worklist, Visited);
   }
 }
 
@@ -8512,7 +8513,7 @@
 #ifndef NDEBUG
     PushLoopPHIs(CurrL, VerificationWorklist, VerificationVisited);
 #endif
-    visitAndClearUsers(Worklist, Visited, ToForget);
+    visitAndClearUsers(Worklist, Visited, ToForget, CurrL);
 
     LoopPropertiesCache.erase(CurrL);
     // Forget all contained loops too, to avoid dangling entries in the
@@ -8540,7 +8541,8 @@
   SmallVector<const SCEV *, 8> ToForget;
   Worklist.push_back(I);
   Visited.insert(I);
-  visitAndClearUsers(Worklist, Visited, ToForget);
+  visitAndClearUsers(Worklist, Visited, ToForget,
+                     LI.getLoopFor(I->getParent()));
 
   forgetMemoizedResults(ToForget);
 
Index: llvm/include/llvm/Analysis/ScalarEvolution.h
===================================================================
--- llvm/include/llvm/Analysis/ScalarEvolution.h
+++ llvm/include/llvm/Analysis/ScalarEvolution.h
@@ -2043,7 +2043,8 @@
   /// from ValueExprMap and collect SCEV expressions in \p ToForget
   void visitAndClearUsers(SmallVectorImpl<Instruction *> &Worklist,
                           SmallPtrSetImpl<Instruction *> &Visited,
-                          SmallVectorImpl<const SCEV *> &ToForget);
+                          SmallVectorImpl<const SCEV *> &ToForget,
+                          const Loop *CurrL);
 
 #ifndef NDEBUG
   /// Iterate over instructions in \p Worlist and their users. Assert all users


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D144849.500690.patch
Type: text/x-patch
Size: 2392 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20230227/b2c95a43/attachment.bin>