[llvm-dev] Reducing DWARF emitter memory consumption

[Mehdi Amini via llvm-dev]

> On Feb 5, 2016, at 5:40 PM, Peter Collingbourne <peter at pcc.me.uk> wrote:
> 
> On Fri, Feb 05, 2016 at 04:58:45PM -0800, Mehdi Amini wrote:
>> 
>>> On Feb 5, 2016, at 3:17 PM, Peter Collingbourne via llvm-dev <llvm-dev at lists.llvm.org> wrote:
>>> 
>>> Hi all,
>>> 
>>> We have profiled [1] the memory usage in LLVM when LTO'ing Chromium, and
>>> we've found that one of the top consumers of memory is the DWARF emitter in
>>> lib/CodeGen/AsmPrinter/Dwarf*.
>> 
>> I'm staring at the profile attached to the post #15 on the link you posted, can you confirm that the Dwarf emitter accounts for 6.7%+15.6%=22.3% of the the total allocated memory?
>> If I understand correctly the numbers, this does not tell anything about how much the Dwarf emitter accounts on the *peak memory* usage (could be more, could be nothing...).
> 
> I think these nodes represent allocations from the DWARF emitter:
> 
> DwarfDebug::DwarfDebug 9.5%
> DwarfDebug::endFunction 15.6%
> DIEValueList::addValue 9.1%
> total 34.2%
> 
> I believe they are totals, but my reading of the code is that the DWARF
> emitter does not deallocate its memory until the end of code generation,

That's sad :(

> so total ~= peak in this case.

Assuming the peak occurs during CodeGen (which is what I on my profile), that sounds pretty reasonable!

Thanks for the information (and the work!).

Another question I have, is how worse the split codegen make the situation? Naively there will be a lot of redundancy in the split modules, for ThinLTO Teresa has to proceed with care to limit the amount of duplication.

Mehdi


> 
> I am not surprised by these figures -- see e.g. DIEValueList::Node which in
> the worst case can use up to 24 bytes on a 1-byte DWARF attribute record.
> 
> Ivan was the person who collected the numbers, he may be able to comment more.
> 
> Thanks,
> -- 
> Peter