<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Feb 5, 2016, at 5:53 PM, Eric Christopher <<a href="mailto:echristo@gmail.com" class="">echristo@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""><br class=""><div class="gmail_quote"><div dir="ltr" class="">On Fri, Feb 5, 2016 at 5:51 PM Mehdi Amini via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><br class="">> On Feb 5, 2016, at 5:40 PM, Peter Collingbourne <<a href="mailto:peter@pcc.me.uk" target="_blank" class="">peter@pcc.me.uk</a>> wrote:<br class="">><br class="">> On Fri, Feb 05, 2016 at 04:58:45PM -0800, Mehdi Amini wrote:<br class="">>><br class="">>>> On Feb 5, 2016, at 3:17 PM, Peter Collingbourne via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>> wrote:<br class="">>>><br class="">>>> Hi all,<br class="">>>><br class="">>>> We have profiled [1] the memory usage in LLVM when LTO'ing Chromium, and<br class="">>>> we've found that one of the top consumers of memory is the DWARF emitter in<br class="">>>> lib/CodeGen/AsmPrinter/Dwarf*.<br class="">>><br class="">>> I'm staring at the profile attached to the post #15 on the link you posted, can you confirm that the Dwarf emitter accounts for 6.7%+15.6%=22.3% of the the total allocated memory?<br class="">>> If I understand correctly the numbers, this does not tell anything about how much the Dwarf emitter accounts on the *peak memory* usage (could be more, could be nothing...).<br class="">><br class="">> I think these nodes represent allocations from the DWARF emitter:<br class="">><br class="">> DwarfDebug::DwarfDebug 9.5%<br class="">> DwarfDebug::endFunction 15.6%<br class="">> DIEValueList::addValue 9.1%<br class="">> total 34.2%<br class="">><br class="">> I believe they are totals, but my reading of the code is that the DWARF<br class="">> emitter does not deallocate its memory until the end of code generation,<br class=""><br class="">That's sad :(<br class=""><br class="">> so total ~= peak in this case.<br class=""><br class="">Assuming the peak occurs during CodeGen (which is what I on my profile), that sounds pretty reasonable!<br class=""><br class="">Thanks for the information (and the work!).<br class=""><br class="">Another question I have, is how worse the split codegen make the situation? Naively there will be a lot of redundancy in the split modules, for ThinLTO Teresa has to proceed with care to limit the amount of duplication.<br class=""><br class=""></blockquote><div class=""><br class=""></div><div class="">Hmm? Can you reword this slightly? I'm not sure what you're asking here.</div></div></div></div></blockquote><div><br class=""></div><div>The parallel split codegen will take the big LTO module with all the debug info and produce multiple modules.</div><div>When splitting in multiple modules, you may have functions from the same DICompileUnit ending up in multiple modules. All the retained types would be pulled in.</div><div>(this is assuming you are already taking care of not pulling the DICompileUnit when no functions referencing it is in the split module).</div><div>Then each thread would do redundant work processing this type hierarchy (and other debug info).</div><div><br class=""></div><div>For ThinLTO, Teresa is taking care (review waiting here: <a href="http://reviews.llvm.org/D16440" class="">http://reviews.llvm.org/D16440</a> ) to try to import as little as possible, and turn type definition into declaration when possible.</div><div><br class=""></div><div>-- </div><div>Mehdi</div><div><br class=""></div></div></body></html>