[PATCH] D119784: [Symbolize] LRU cache binaries in llvm-symbolizer.

Tue Feb 15 11:55:26 PST 2022

dblaikie added a comment.

In D119784#3324002 <https://reviews.llvm.org/D119784#3324002>, @mysterymath wrote:

> In D119784#3323786 <https://reviews.llvm.org/D119784#3323786>, @dblaikie wrote:
>
>> Perhaps you could commandeer/pick up https://reviews.llvm.org/D90006 ? or summarize in this (or the other, D78950 <https://reviews.llvm.org/D78950>) that historical context/what's new/different/justified in this new review?
>
> The context for this change is to try to fix OOMs discovered when symbolizing system logs with llvm-symbolizer.
> As is, llvm-symbolizer essentially leaks memory when using it's STDIN format. It's specified in a line-at-a-time fashion, but data is never cleaned up from one line to the next. Eventually, you either have to either stop feeding it lines or restart the job.
>
> It seems like the Symbolizer is the right place to address this; it *could* mmap in and parse the debug binaries anew for each symbolization request, but it doesn't, because it's more efficient to keep this data around. At least, right up until physical memory is exhausted.
> Accordingly, it shouldn't really matter what kind of derived data the Symbolizer keeps; everything should be fair game for eviction; otherwise whatever we don't try to evict will leak from line to line.

Does this solution subsume the previous patches? What does it provide that they didn't/why this rather than those directions? (does it not subsume that functionality? (ie: it addresses some issues, but not some/all of the issues the other patches were interested in) if so, it'd be unfortunate to have to solve related issues in two different places/ways - would be good to understand why we can't find a common solution to these issues)

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D119784/new/

https://reviews.llvm.org/D119784