<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Hi James, Matthias<div class=""><br class=""></div><div class="">I recently proposed the idea of deleting the IR for a function after its reached the AsmPrinter. This was before the idea of MachineModulePasses. I was seeing peak memory savings of 20% when LTOing clang itself.</div><div class=""><br class=""></div><div class="">So it might be possible to trade the memory so that IR and MachineIR aren’t live at the same time for all functions. But that depends on whether a MachineModulePass would want access to the IR. AA is the typical example of MachineIR referencing back to IR. I knew it was safe to delete IR when we hit the AsmPrinter because nothing at (or beyond) that point needs AA.</div><div class=""><br class=""></div><div class="">Cheers,</div><div class="">Pete<br class=""><div><blockquote type="cite" class=""><div class="">On Jul 17, 2016, at 10:57 AM, Matthias Braun via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><blockquote type="cite" class=""><div class=""><br class="Apple-interchange-newline">On Jul 17, 2016, at 9:34 AM, James Molloy via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div class="">Hi,</div><div class=""><br class=""></div><div class="">[Apologies to those receiving this mail twice - used the old list address by accident]</div><div class=""><br class=""></div><div class="">In LLVM it is currently not possible to write a Module-level pass (a pass that modifies or analyzes multiple MachineFunctions) after DAG formation. This inhibits some optimizations[1] and is something I'd like to see changed.</div><div class=""><br class=""></div><div class="">The problem is that in the backend, we emit a function at a time, from DAG formation to object emission. So no two MachineFunctions ever exist at any one time. Changing this necessarily means increasing memory usage.</div><div class=""><br class=""></div><div class="">I've prototyped this change and have measured peak memory usage in the worst case scenario - LTO'ing llc and clang. Without further ado:</div><div class=""><br class=""></div><div class=""> llvm-lto llc: before: 1.44GB maximum resident set size</div><div class=""> after: 1.68GB (+17%)</div><div class=""><br class=""></div><div class=""> llvm-lto clang: before: 2.48GB maximum resident set size</div><div class=""> after: 3.42GB (+33%)</div><div class=""><br class=""></div><div class="">The increases are very large. This is worst-case (non-LTO builds would see the peak usage of the backend masked by the peak of the midend) but still - pretty big. Thoughts? Is this completely no-go? is this something that we *just need* to do? Is crippling the backend architecture to keep memory down justified? Is this something we could enable under an option?</div></div></div></blockquote><div class="">I also recently did a prototype that enables MachineModulePasses (<a href="https://github.com/MatzeB/llvm/tree/MachineModulePass" class="">https://github.com/MatzeB/llvm/tree/MachineModulePass</a>). I assume your patch looks similar (I did not do any measurements with mine).</div><div class=""><br class=""></div><div class="">I believe this could be done in a way so the memory stays at current levels if no machine module pass issued (simply remove the memory used by the MachineFunction after we AsmPrinted it). So that would at least make the infrastructure available for people to experiment with. So maybe that is a good first step?</div><div class=""><br class=""></div><div class="">So far all the uses I have seen did not convince me that the increased memory consumption is worth it (by default). But IMO if we can provide by infrastructure for MachineModulePasses with nearly zero cost in case no machinemodulepass is used, then I’d say we should go for it.</div><div class=""><br class=""></div><div class="">- Matthias</div><div class=""><br class=""></div></div><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">_______________________________________________</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">LLVM Developers mailing list</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><a href="mailto:llvm-dev@lists.llvm.org" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">llvm-dev@lists.llvm.org</a><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a></div></blockquote></div><br class=""></div></body></html>