[llvm-dev] llvm-symbolizer memory usage
David Blaikie via llvm-dev
llvm-dev at lists.llvm.org
Tue Jan 14 14:32:26 PST 2020
(Adding Hyoun who's been looking at memory use of llvm-symbolizer recently
On Tue, Jan 14, 2020 at 11:07 AM Francis Ricci via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> I work on a linux program with restricted RSS limits (a couple hundred
> MB), and one of the things this program does is symbolication. Ideally,
> we'd like to use llvm-symbolizer for this symbolication (because we get
> things like function inlining that we can't get from cheaper symbolizers),
> but for large binaries, the memory usage gets pretty huge.
> Based on some memory profiling, it looks like the majority of this memory
> cost comes from mmap-ing the binary to be symbolized (via
> `llvm::object::createBinary"). This alone comes with hundreds of MB of cost
> in many cases.
> I have 2 questions here:
> 1) Does it seem feasible to make llvm-symbolizer work *without* loading
> the full binary into memory (perhaps just reading sections from disk as
> needed, at the cost of some extra CPU)?
Does memory mapping the file actually use real memory? Or is it just
reading from the file, effectively? I don't think the mapped file was part
of the memory usage Hyoun and I encountered when doing memory accounting.
What we were talking about was an LRU cache of DwarfCompileUnits, or
something like that - to strip out the DIEArrays and other associated data
structures after they were used.
Are you running llvm-symbolizer on many input addresses in a single run?
Only a single address? Optimized or unoptimized build of llvm-symbolizer?
> 2) If we figured this out, and put it behind something like a
> "--low-memory" flag, would it be something the upstream community would
Maybe, though I'm hoping we can avoid having to have too much of a perf
tradeoff for low memory usage, so we can keep it all together without a
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the llvm-dev