[llvm-dev] OrcV1 removal
Andres Freund via llvm-dev
llvm-dev at lists.llvm.org
Thu Oct 1 16:42:47 PDT 2020
Hi,
On 2020-10-01 15:29:12 -0700, Lang Hames wrote:
> 24bytes / object -- Looks like I managed module ownership correctly but
> leaked the ThreadSafeModule container. This should be fixed in 5044196b412f.
That helped a bit, but not yet fully. Looks like it might be still
reachable memory, so leakcheck isn't that helpful.
Oooh. I think I see. For various reasons the symbol names we generate
are unique over time. But the interned strings aren't cleared ever. Is
that right?
With massif, I see:
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
22 62,022,701,974 2,766,072 2,664,931 101,141 0
...
->16.97% (469,433B) 0xB29E471: llvm::allocate_buffer(unsigned long, unsigned long) (MemAlloc.cpp:15)
| ->03.54% (97,812B) 0x831FB40: llvm::MallocAllocator::Allocate(unsigned long, unsigned long) (AllocatorBase.h:85)
| | ->03.54% (97,812B) 0x8330268: llvm::StringMapEntry<std::atomic<unsigned long> >* llvm::StringMapEntry<std::atomic<unsigned long> >::Create<llvm::MallocAllocator, int>(llvm::StringRef, llvm::MallocAllocator&, int&&) (StringMapEntry.h:101)
| | ->03.54% (97,812B) 0x8322B26: std::pair<llvm::StringMapIterator<std::atomic<unsigned long> >, bool> llvm::StringMap<std::atomic<unsigned long>, llvm::MallocAllocator>::try_emplace<int>(llvm::StringRef, int&&) (StringMap.h:322)
| | ->03.54% (97,812B) 0x831FD62: llvm::orc::SymbolStringPool::intern(llvm::StringRef) (SymbolStringPool.h:159)
| | ->03.54% (97,812B) 0x8320AF5: llvm::orc::ExecutionSession::intern(llvm::StringRef) (Core.h:1225)
| | ->03.53% (97,542B) 0x83EFBAE: llvm::orc::MangleAndInterner::operator()(llvm::StringRef) (Mangling.cpp:30)
| | | ->03.53% (97,542B) 0x83B20B2: llvm::orc::IRMaterializationUnit::IRMaterializationUnit(llvm::orc::ExecutionSession&, llvm::orc::IRSymbolMapper::ManglingOptions const&, llvm::orc::ThreadSafeModule)::{lambda(llvm::Module&)
| | | ->03.53% (97,542B) 0x83B2376: decltype(auto) llvm::orc::ThreadSafeModule::withModuleDo<llvm::orc::IRMaterializationUnit::IRMaterializationUnit(llvm::orc::ExecutionSession&, llvm::orc::IRSymbolMapper::ManglingOptions const&, llvm::orc::ThreadSafeModule)::{lambda(llvm::Module&)
| | | ->03.53% (97,542B) 0x83B24DA: llvm::orc::IRMaterializationUnit::IRMaterializationUnit(llvm::orc::ExecutionSession&, llvm::orc::IRSymbolMapper::ManglingOptions const&, llvm::orc::ThreadSafeModule) (Layer.cpp:41)
| | | ->03.53% (97,542B) 0x83B29B3: llvm::orc::BasicIRLayerMaterializationUnit::BasicIRLayerMaterializationUnit(llvm::orc::IRLayer&, llvm::orc::IRSymbolMapper::ManglingOptions const&, llvm::orc::ThreadSafeModule) (Layer.cpp:131)
| | | ->03.53% (97,542B) 0x83B357D: std::_MakeUniq<llvm::orc::BasicIRLayerMaterializationUnit>::__single_object std::make_unique<llvm::orc::BasicIRLayerMaterializationUnit, llvm::orc::IRLayer&, llvm::orc::IRSymbolMapper::ManglingOptions const&, llvm::orc::ThreadSafeModule>(llvm::orc::IRLayer&, llvm::o>
| | | ->03.53% (97,542B) 0x83B1C7D: llvm::orc::IRLayer::add(llvm::IntrusiveRefCntPtr<llvm::orc::ResourceTracker>, llvm::orc::ThreadSafeModule) (Layer.cpp:28)
| | | ->03.53% (97,542B) 0x83BCBF5: llvm::orc::LLJIT::addIRModule(llvm::orc::JITDylib&, llvm::orc::ThreadSafeModule, llvm::IntrusiveRefCntPtr<llvm::orc::ResourceTracker>) (LLJIT.cpp:990)
| | | ->03.52% (97,354B) 0x840C95B: LLVMOrcLLJITAddLLVMIRModule (OrcV2CBindings.cpp:263)
| | | | ->03.52% (97,354B) 0x80639E9: llvm_compile_module (llvmjit.c:608)
| | | | ->03.52% (97,354B) 0x8063000: llvm_get_function (llvmjit.c:275)
| | | | ->03.52% (97,354B) 0x80758E1: ExecRunCompiledExpr (llvmjit_expr.c:2410)
52 148,049,498,480 3,039,472 2,901,891 137,581 0
...
->20.01% (608,089B) 0xB29E471: llvm::allocate_buffer(unsigned long, unsigned long) (MemAlloc.cpp:15)
| ->07.78% (236,468B) 0x831FB40: llvm::MallocAllocator::Allocate(unsigned long, unsigned long) (AllocatorBase.h:85)
| | ->07.78% (236,468B) 0x8330268: llvm::StringMapEntry<std::atomic<unsigned long> >* llvm::StringMapEntry<std::atomic<unsigned long> >::Create<llvm::MallocAllocator, int>(llvm::StringRef, llvm::MallocAllocator&, int&&) (StringMapEntry.h:101)
| | ->07.78% (236,468B) 0x8322B26: std::pair<llvm::StringMapIterator<std::atomic<unsigned long> >, bool> llvm::StringMap<std::atomic<unsigned long>, llvm::MallocAllocator>::try_emplace<int>(llvm::StringRef, int&&) (StringMap.h:322)
| | ->07.78% (236,468B) 0x831FD62: llvm::orc::SymbolStringPool::intern(llvm::StringRef) (SymbolStringPool.h:159)
| | ->07.78% (236,468B) 0x8320AF5: llvm::orc::ExecutionSession::intern(llvm::StringRef) (Core.h:1225)
| | ->07.77% (236,198B) 0x83EFBAE: llvm::orc::MangleAndInterner::operator()(llvm::StringRef) (Mangling.cpp:30)
| | | ->07.77% (236,198B) 0x83B20B2: llvm::orc::IRMaterializationUnit::IRMaterializationUnit(llvm::orc::ExecutionSession&, llvm::orc::IRSymbolMapper::ManglingOptions const&, llvm::orc::ThreadSafeModule)::{lambda(llvm::Module&)
| | | ->07.77% (236,198B) 0x83B2376: decltype(auto) llvm::orc::ThreadSafeModule::withModuleDo<llvm::orc::IRMaterializationUnit::IRMaterializationUnit(llvm::orc::ExecutionSession&, llvm::orc::IRSymbolMapper::ManglingOptions const&, llvm::orc::ThreadSafeModule)::{lambda(llvm::Module&)
| | | ->07.77% (236,198B) 0x83B24DA: llvm::orc::IRMaterializationUnit::IRMaterializationUnit(llvm::orc::ExecutionSession&, llvm::orc::IRSymbolMapper::ManglingOptions const&, llvm::orc::ThreadSafeModule) (Layer.cpp:41)
| | | ->07.77% (236,198B) 0x83B29B3: llvm::orc::BasicIRLayerMaterializationUnit::BasicIRLayerMaterializationUnit(llvm::orc::IRLayer&, llvm::orc::IRSymbolMapper::ManglingOptions const&, llvm::orc::ThreadSafeModule) (Layer.cpp:131)
| | | ->07.77% (236,198B) 0x83B357D: std::_MakeUniq<llvm::orc::BasicIRLayerMaterializationUnit>::__single_object std::make_unique<llvm::orc::BasicIRLayerMaterializationUnit, llvm::orc::IRLayer&, llvm::orc::IRSymbolMapper::ManglingOptions const&, llvm::orc::ThreadSafeModule>(llvm::orc::IRLayer&, llvm::>
| | | ->07.77% (236,198B) 0x83B1C7D: llvm::orc::IRLayer::add(llvm::IntrusiveRefCntPtr<llvm::orc::ResourceTracker>, llvm::orc::ThreadSafeModule) (Layer.cpp:28)
| | | ->07.77% (236,198B) 0x83BCBF5: llvm::orc::LLJIT::addIRModule(llvm::orc::JITDylib&, llvm::orc::ThreadSafeModule, llvm::IntrusiveRefCntPtr<llvm::orc::ResourceTracker>) (LLJIT.cpp:990)
| | | ->07.76% (236,010B) 0x840C95B: LLVMOrcLLJITAddLLVMIRModule (OrcV2CBindings.cpp:263)
| | | | ->07.76% (236,010B) 0x80639E9: llvm_compile_module (llvmjit.c:608)
| | | | ->07.76% (236,010B) 0x8063000: llvm_get_function (llvmjit.c:275)
| | | | ->07.76% (236,010B) 0x80758E1: ExecRunCompiledExpr (llvmjit_expr.c:2410)
i.e. you can see that over time (snapshot 22 vs 52) the percentage of
memory used by string interning increases. And that the total bytes used
for that increases too.
Regards,
Andres
More information about the llvm-dev
mailing list