[PATCH] D73054: Prevent explosion of debug intrinsics during jump threading

[Stephen Tozer via Phabricator via llvm-commits]

StephenTozer created this revision.
StephenTozer added reviewers: probinson, aprantl, vsk, markus, djtodoro, jmorse.
Herald added subscribers: llvm-commits, hiraditya.
Herald added a project: LLVM.
StephenTozer added a child revision: D70318: Recover debug intrinsics when killing duplicate or empty basic blocks.

This patch is a fix following the revert of 72ce759 <https://reviews.llvm.org/rG72ce759928e6dfee6a9efa310b966c19722352ba> (https://reviews.llvm.org/rG72ce759928e6dfee6a9efa310b966c19722352ba) and fixes the failure that it caused.

The above patch failed on the Thread Sanitizer buildbot with an out of memory error. After an investigation, the cause was identified as an explosion in debug intrinsics while running the Jump Threading pass on ModuleMap.ll. The above patched prevented debug intrinsics from being dropped when their Basic Block was deleted due to being "empty". In this case, one of the functions in ModuleMap.ll had (after many optimization passes) a very large number of debug intrinsics representing a set of repeatedly inlined variables. Previously the vast majority of these were silently dropped during Jump Threading when their blocks were deleted, but as of the above patch they survived for longer, causing a large increase in the number of debug intrinsics. These intrinsics were then repeatedly cloned by the Jump Threading pass as edges were threaded, multiplying the intrinsic count further. The memory consumed by this process spiralled out of control, crashing the buildbot that uses TSan (which has an estimated 5-10x memory overhead compared to non-sanitized builds).

In order to rein in the number of debug intrinsics, I have added RemoveRedundantDbgInstrs to the Jump Threading pass. This recently added function (https://reviews.llvm.org/D71478) processes a Basic Block and removes any debug intrinsics that are "redundant", i.e. they are either identical to a previous intrinsic for the same source variable in the same block, or immediately overwritten by another intrinsic with no intermediate instructions. As many intrinsics for the same variable end up propagated into the same blocks during this case, the removal of redundant intrinsics from each block as they are processed is effective are reducing the number of intrinsics from exploding in the same manner.

This seems to bring the number of intrinsics for this case down to a manageable level in practice, and should be sufficient to unblock the above patch for remerging. It does seem likely to me that there are other passes for which this function would also be appropriate; this is probably worth looking into after the immediate problem is solved.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D73054

Files:
  llvm/lib/Transforms/Scalar/JumpThreading.cpp


Index: llvm/lib/Transforms/Scalar/JumpThreading.cpp
===================================================================
--- llvm/lib/Transforms/Scalar/JumpThreading.cpp
+++ llvm/lib/Transforms/Scalar/JumpThreading.cpp
@@ -418,17 +418,21 @@
       // ProcessBlock doesn't thread BBs with unconditional TIs. However, if BB
       // is "almost empty", we attempt to merge BB with its sole successor.
       auto *BI = dyn_cast<BranchInst>(BB.getTerminator());
-      if (BI && BI->isUnconditional() &&
-          // The terminator must be the only non-phi instruction in BB.
-          BB.getFirstNonPHIOrDbg()->isTerminator() &&
-          // Don't alter Loop headers and latches to ensure another pass can
-          // detect and transform nested loops later.
-          !LoopHeaders.count(&BB) && !LoopHeaders.count(BI->getSuccessor(0)) &&
-          TryToSimplifyUncondBranchFromEmptyBlock(&BB, DTU)) {
-        // BB is valid for cleanup here because we passed in DTU. F remains
-        // BB's parent until a DTU->getDomTree() event.
-        LVI->eraseBlock(&BB);
-        Changed = true;
+      if (BI && BI->isUnconditional()) {
+        BasicBlock *Succ = BI->getSuccessor(0);
+        if (
+            // The terminator must be the only non-phi instruction in BB.
+            BB.getFirstNonPHIOrDbg()->isTerminator() &&
+            // Don't alter Loop headers and latches to ensure another pass can
+            // detect and transform nested loops later.
+            !LoopHeaders.count(&BB) && !LoopHeaders.count(Succ) &&
+            TryToSimplifyUncondBranchFromEmptyBlock(&BB, DTU)) {
+          // BB is valid for cleanup here because we passed in DTU. F remains
+          // BB's parent until a DTU->getDomTree() event.
+          RemoveRedundantDbgInstrs(Succ);
+          LVI->eraseBlock(&BB);
+          Changed = true;
+        }
       }
     }
     EverChanged |= Changed;


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D73054.239152.patch
Type: text/x-patch
Size: 1933 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20200120/7013b9f9/attachment.bin>