[llvm] r254760 - [LegacyPassManager] Reduce memory usage for AnalysisUsage
Mike Aizatsky via llvm-commits
llvm-commits at lists.llvm.org
Fri Dec 4 13:37:49 PST 2015
The code seems to have a memory leak. Reported by asan build:
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/8348/steps/check-clang%20asan/logs/stdio
=================================================================
==17198==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 520 byte(s) in 1 object(s) allocated from:
#0 0x8514a8 in __interceptor_malloc
/mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:40
#1 0x3c428fd in llvm::SmallVectorBase::grow_pod(void*, unsigned
long, unsigned long)
/mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/lib/Support/SmallVector.cpp:28:15
#2 0x32771b2 in grow_pod
/mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/include/llvm/ADT/SmallVector.h:81:5
#3 0x32771b2 in grow
/mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/include/llvm/ADT/SmallVector.h:334
#4 0x32771b2 in operator=
/mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/include/llvm/ADT/SmallVector.h:771
#5 0x32771b2 in SmallVector
/mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/include/llvm/ADT/SmallVector.h:897
#6 0x32771b2 in AnalysisUsage
/mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/include/llvm/PassAnalysisSupport.h:37
#7 0x32771b2 in AUFoldingSetNode
/mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/include/llvm/IR/LegacyPassManagers.h:259
#8 0x32771b2 in
llvm::PMTopLevelManager::findAnalysisUsage(llvm::Pass*)
/mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/lib/IR/LegacyPassManager.cpp:592
#9 0x3278ed8 in llvm::PMTopLevelManager::schedulePass(llvm::Pass*)
/mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/lib/IR/LegacyPassManager.cpp:623:28
#10 0x2ba9b0f in llvm::TargetPassConfig::addPass(llvm::Pass*,
bool, bool) /mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/lib/CodeGen/Passes.cpp:327:5
#11 0x2bad50a in
llvm::TargetPassConfig::addOptimizedRegAlloc(llvm::FunctionPass*)
/mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/lib/CodeGen/Passes.cpp:759:5
#12 0x2babdd0 in llvm::TargetPassConfig::addMachinePasses()
/mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/lib/CodeGen/Passes.cpp:553:5
#13 0x298e905 in addPassesToGenerateCode(llvm::LLVMTargetMachine*,
llvm::legacy::PassManagerBase&, bool, void const*, void const*, void
const*, llvm::MachineFunctionInitializer*)
/mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/lib/CodeGen/LLVMTargetMachine.cpp:138:3
#14 0x298d12e in
llvm::LLVMTargetMachine::addPassesToEmitFile(llvm::legacy::PassManagerBase&,
llvm::raw_pwrite_stream&, llvm::TargetMachine::CodeGenFileType, bool,
void const*, void const*, void const*,
llvm::MachineFunctionInitializer*)
/mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/lib/CodeGen/LLVMTargetMachine.cpp:151:7
#15 0x4082655 in AddEmitPasses
/mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/lib/CodeGen/BackendUtil.cpp:587:7
#16 0x4082655 in EmitAssembly
/mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/lib/CodeGen/BackendUtil.cpp:627
#17 0x4082655 in
clang::EmitBackendOutput(clang::DiagnosticsEngine&,
clang::CodeGenOptions const&, clang::TargetOptions const&,
clang::LangOptions const&, llvm::StringRef, llvm::Module*,
clang::BackendAction, llvm::raw_pwrite_stream*)
/mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/lib/CodeGen/BackendUtil.cpp:666
#18 0x5329076 in
clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&)
/mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/lib/CodeGen/CodeGenAction.cpp:189:7
#19 0x5de04f4 in clang::ParseAST(clang::Sema&, bool, bool)
/mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/lib/Parse/ParseAST.cpp:168:3
#20 0x532541c in clang::CodeGenAction::ExecuteAction()
/mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/lib/CodeGen/CodeGenAction.cpp:797:3
#21 0x4a2949b in clang::FrontendAction::Execute()
/mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/lib/Frontend/FrontendAction.cpp:439:8
#22 0x498c2d4 in
clang::CompilerInstance::ExecuteAction(clang::FrontendAction&)
/mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/lib/Frontend/CompilerInstance.cpp:841:7
#23 0x4bd55ee in
clang::ExecuteCompilerInvocation(clang::CompilerInstance*)
/mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/lib/FrontendTool/ExecuteCompilerInvocation.cpp:222:18
#24 0x89019a in cc1_main(llvm::ArrayRef<char const*>, char const*,
void*) /mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/tools/driver/cc1_main.cpp:116:13
#25 0x88a5b2 in ExecuteCC1Tool
/mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/tools/driver/driver.cpp:301:12
#26 0x88a5b2 in main
/mnt/b/sanitizer-buildbot3/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/tools/driver/driver.cpp:366
#27 0x7fdd62e94ec4 in __libc_start_main
(/lib/x86_64-linux-gnu/libc.so.6+0x21ec4)
SUMMARY: AddressSanitizer: 520 byte(s) leaked in 1 allocation(s).
On Fri, Dec 4, 2015 at 12:08 PM Philip Reames via llvm-commits <
llvm-commits at lists.llvm.org> wrote:
> Author: reames
> Date: Fri Dec 4 14:05:04 2015
> New Revision: 254760
>
> URL: http://llvm.org/viewvc/llvm-project?rev=254760&view=rev
> Log:
> [LegacyPassManager] Reduce memory usage for AnalysisUsage
>
> The LegacyPassManager was storing an instance of AnalysisUsage for each
> instance of each pass. In practice, most instances of a single pass class
> share the same dependencies. We can't rely on this because passes can (and
> some do) have dynamic dependencies based on instance options.
>
> We can exploit the likely commonality by uniqueing the usage information
> after querying the pass, but before storing it into the pass manager. This
> greatly reduces memory consumption by the AnalysisUsage objects. For a long
> pass pipeline, I measured a decrease in memory consumption for this storage
> of about 50%. I have not measured on the default O3 pipeline, but I suspect
> it will see some benefit as well since many passes are repeated (e.g.
> InstCombine).
>
> Differential Revision: http://reviews.llvm.org/D14677
>
>
> Modified:
> llvm/trunk/include/llvm/IR/LegacyPassManagers.h
> llvm/trunk/lib/IR/LegacyPassManager.cpp
>
> Modified: llvm/trunk/include/llvm/IR/LegacyPassManagers.h
> URL:
> http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/IR/LegacyPassManagers.h?rev=254760&r1=254759&r2=254760&view=diff
>
> ==============================================================================
> --- llvm/trunk/include/llvm/IR/LegacyPassManagers.h (original)
> +++ llvm/trunk/include/llvm/IR/LegacyPassManagers.h Fri Dec 4 14:05:04
> 2015
> @@ -16,6 +16,7 @@
>
> #include "llvm/ADT/ArrayRef.h"
> #include "llvm/ADT/DenseMap.h"
> +#include "llvm/ADT/FoldingSet.h"
> #include "llvm/ADT/SmallPtrSet.h"
> #include "llvm/ADT/SmallVector.h"
> #include "llvm/Pass.h"
> @@ -250,7 +251,40 @@ private:
> /// Map from ID to immutable passes.
> SmallDenseMap<AnalysisID, ImmutablePass *, 8> ImmutablePassMap;
>
> - DenseMap<Pass *, AnalysisUsage *> AnUsageMap;
> +
> + /// A wrapper around AnalysisUsage for the purpose of uniqueing. The
> wrapper
> + /// is used to avoid needing to make AnalysisUsage itself a folding set
> node.
> + struct AUFoldingSetNode : public FoldingSetNode {
> + AnalysisUsage AU;
> + AUFoldingSetNode(const AnalysisUsage &AU) : AU(AU) {}
> + void Profile(FoldingSetNodeID &ID) const {
> + Profile(ID, AU);
> + }
> + static void Profile(FoldingSetNodeID &ID, const AnalysisUsage &AU) {
> + // TODO: We could consider sorting the dependency arrays within the
> + // AnalysisUsage (since they are conceptually unordered).
> + ID.AddBoolean(AU.getPreservesAll());
> + for (auto &Vec : {AU.getRequiredSet(),
> AU.getRequiredTransitiveSet(),
> + AU.getPreservedSet(), AU.getUsedSet()}) {
> + ID.AddInteger(Vec.size());
> + for(AnalysisID AID : Vec)
> + ID.AddPointer(AID);
> + }
> + }
> + };
> +
> + // Contains all of the unique combinations of AnalysisUsage. This is
> helpful
> + // when we have multiple instances of the same pass since they'll
> usually
> + // have the same analysis usage and can share storage.
> + FoldingSet<AUFoldingSetNode> UniqueAnalysisUsages;
> +
> + // Allocator used for allocating UAFoldingSetNodes. This handles
> deletion of
> + // all allocated nodes in one fell swoop.
> + BumpPtrAllocator AUFoldingSetNodeAllocator;
> +
> + // Maps from a pass to it's associated entry in UniqueAnalysisUsages.
> Does
> + // not own the storage associated with either key or value..
> + DenseMap<Pass *, AnalysisUsage*> AnUsageMap;
>
> /// Collection of PassInfo objects found via analysis IDs and in this
> top
> /// level manager. This is used to memoize queries to the pass registry.
>
> Modified: llvm/trunk/lib/IR/LegacyPassManager.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/IR/LegacyPassManager.cpp?rev=254760&r1=254759&r2=254760&view=diff
>
> ==============================================================================
> --- llvm/trunk/lib/IR/LegacyPassManager.cpp (original)
> +++ llvm/trunk/lib/IR/LegacyPassManager.cpp Fri Dec 4 14:05:04 2015
> @@ -569,13 +569,33 @@ void PMTopLevelManager::collectLastUses(
>
> AnalysisUsage *PMTopLevelManager::findAnalysisUsage(Pass *P) {
> AnalysisUsage *AnUsage = nullptr;
> - DenseMap<Pass *, AnalysisUsage *>::iterator DMI = AnUsageMap.find(P);
> + auto DMI = AnUsageMap.find(P);
> if (DMI != AnUsageMap.end())
> AnUsage = DMI->second;
> else {
> - AnUsage = new AnalysisUsage();
> - P->getAnalysisUsage(*AnUsage);
> - AnUsageMap[P] = AnUsage;
> + // Look up the analysis usage from the pass instance (different
> instances
> + // of the same pass can produce different results), but unique the
> + // resulting object to reduce memory usage. This helps to greatly
> reduce
> + // memory usage when we have many instances of only a few pass types
> + // (e.g. instcombine, simplifycfg, etc...) which tend to share a
> fixed set
> + // of dependencies.
> + AnalysisUsage AU;
> + P->getAnalysisUsage(AU);
> +
> + AUFoldingSetNode* Node = nullptr;
> + FoldingSetNodeID ID;
> + AUFoldingSetNode::Profile(ID, AU);
> + void *IP = nullptr;
> + if (auto *N = UniqueAnalysisUsages.FindNodeOrInsertPos(ID, IP))
> + Node = N;
> + else {
> + Node = new (AUFoldingSetNodeAllocator) AUFoldingSetNode(AU);
> + UniqueAnalysisUsages.InsertNode(Node, IP);
> + }
> + assert(Node && "cached analysis usage must be non null");
> +
> + AnUsageMap[P] = &Node->AU;
> + AnUsage = &Node->AU;;
> }
> return AnUsage;
> }
> @@ -798,10 +818,6 @@ PMTopLevelManager::~PMTopLevelManager()
> for (SmallVectorImpl<ImmutablePass *>::iterator
> I = ImmutablePasses.begin(), E = ImmutablePasses.end(); I != E;
> ++I)
> delete *I;
> -
> - for (DenseMap<Pass *, AnalysisUsage *>::iterator DMI =
> AnUsageMap.begin(),
> - DME = AnUsageMap.end(); DMI != DME; ++DMI)
> - delete DMI->second;
> }
>
>
> //===----------------------------------------------------------------------===//
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20151204/6355984a/attachment.html>
More information about the llvm-commits
mailing list