[Lldb-commits] [PATCH] D78489: [lldb/DWARF] Trust CU DW_AT_low/high_pc information when building address tables

Thu Apr 23 10:48:24 PDT 2020

dblaikie added a comment.

In D78489#1993806 <https://reviews.llvm.org/D78489#1993806>, @clayborg wrote:

> One thing to think about that my flawed example above did show: dead stripped functions cause problems for debug info. Both in DW_AT_ranges on the compile unit and in all of the DWARF below this. We might want to check any and all address ranges to ensure that they are in sections that read + execute permissions and if they don't throw them out. It is easy to come up with a minimal address range from the main object file prior to parsing any address information and use that to weed out the bad entries. If functions have had their address set to zero, then we will usually correctly weed these out as zero is often part of an image base load address, but not in a section (not program header or top level section in LLDB's ObjectFileELF, but in a section header section) with r+x permissions. It would be nice to be able to weed these addresses out so they don't cause problems possibly as a follow up patch.

In D78489#1999110 <https://reviews.llvm.org/D78489#1999110>, @labath wrote:

> Thanks for the review.
>
> In D78489#1998060 <https://reviews.llvm.org/D78489#1998060>, @clayborg wrote:
>
> > In D78489#1996834 <https://reviews.llvm.org/D78489#1996834>, @labath wrote:
> >
> > > I've done a similar benchmark to the last one, but measured memory usage ("RssAnon" as reported by linux). One notable difference is that I
> > >
> > > Currently (information retrieved from DW_AT_ranges) we use about ~330 MB of memory. If I switch to dwarf dies as the source, the memory goes all the way to 2890 MB. This number is suspiciously large -- it either means that our die freeing is not working properly, or that glibc is very bad at releasing memory back to the OS. Given the magnitude of the increase, i think it's a little bit of both. With line tables as the source the memory usage is 706 MB. It's an increase from 330, but definitely smaller than 2.8 GB. (the number 330 is kind of misleading here since we're not considering removing that option -- it will always be used if it is available).
> >
> >
> > Since we mmap in the entire DWARF, I am not surprised by taking up new memory because we touch those pages and won't get those back. If you remove the DIE freeing code, I will bet you see much more memory used. We definitely free the memory for the DIEs and give that back, so I would be willing to bet the increase you are seeing is from mmap loading pages in that we touch.
>
>
> I don't think it's as simple as that. "RssAnon" should not include file-backed memory (which why I chose to measure it). According to `man proc`:
>
>   * RssAnon: Size of resident anonymous memory.  (since Linux 4.5).
>   * RssFile: Size of resident file mappings.  (since Linux 4.5).
>   
>
> I also don't think it's as simple as not freeing the DIE memory at all. There has to be some more complex interaction going on. If I had more time, I would be very interested in learning what it is, but for now I'll contend myself with "it's not a problem of this patch".

FWIW, I can recommend Valgrind's massif for memory usage analysis - I think it works at the level of intercepting memory allocation (malloc/free/etc) level, so in some ways that makes it more actionable (it doesn't get sidetracked by the actual libc memory management implementation), but also less accurate (in that, yeah, your process is still going to hold that memory if your libc isn't releasing it back to teh OS) - but perhaps if you tune your application code for what massif tells you, then if there's still a big delta between that and the reality, see if there are tuning options for your memory allocator.

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D78489/new/

https://reviews.llvm.org/D78489