<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">I submitted a cleaned up patch here: <a href="https://reviews.llvm.org/D23736" class="">https://reviews.llvm.org/D23736</a></div><div class=""><br class=""></div><div class="">- Matthias</div><br class=""><div><blockquote type="cite" class=""><div class="">On Jul 27, 2016, at 6:41 AM, Hal Finkel via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-family: arial, helvetica, sans-serif; font-size: 10pt;" class=""><br class="Apple-interchange-newline"><hr id="zwchr" class=""><blockquote style="border-left-width: 2px; border-left-style: solid; border-left-color: rgb(16, 16, 255); margin-left: 5px; padding-left: 5px; font-weight: normal; font-style: normal; text-decoration: none; font-family: Helvetica, Arial, sans-serif; font-size: 12pt;" class=""><b class="">From:<span class="Apple-converted-space"> </span></b>"James Molloy" <<a href="mailto:james@jamesmolloy.co.uk" class="">james@jamesmolloy.co.uk</a>><br class=""><b class="">To:<span class="Apple-converted-space"> </span></b>"Justin Bogner" <<a href="mailto:mail@justinbogner.com" class="">mail@justinbogner.com</a>>, "James Molloy via llvm-dev" <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>><br class=""><b class="">Cc:<span class="Apple-converted-space"> </span></b>"Hal Finkel" <<a href="mailto:hfinkel@anl.gov" class="">hfinkel@anl.gov</a>>, "Chandler Carruth" <<a href="mailto:chandlerc@google.com" class="">chandlerc@google.com</a>>, "Matthias Braun" <<a href="mailto:matze@braunis.de" class="">matze@braunis.de</a>>, "Pete Cooper" <<a href="mailto:peter_cooper@apple.com" class="">peter_cooper@apple.com</a>><br class=""><b class="">Sent:<span class="Apple-converted-space"> </span></b>Tuesday, July 19, 2016 9:16:02 AM<br class=""><b class="">Subject:<span class="Apple-converted-space"> </span></b>Re: [llvm-dev] RFC: Enabling Module passes post-ISel<br class=""><br class=""><div dir="ltr" class="">Hi all,<div class=""><br class=""></div><div class="">I like all the ideas so far. Here are my thoughts:</div><div class=""><br class=""></div><div id="DWT10922" class="">I think that fundamentally users of LLVM should be able to opt-in to more aggressive or intensive computation at compile time if they wish. Users' needs differ, and while a 33% increase in clang LTO is absolutely out of the question for some people, for those developing microcontrollers or HPC applications that may well be irrelevant.</div></div></blockquote>I agree. A 33% increase is absorbable in many environments.<br class=""><blockquote style="border-left-width: 2px; border-left-style: solid; border-left-color: rgb(16, 16, 255); margin-left: 5px; padding-left: 5px; font-weight: normal; font-style: normal; text-decoration: none; font-family: Helvetica, Arial, sans-serif; font-size: 12pt;" class=""><div dir="ltr" class=""><div class="">Either the volume of code expected is significantly smaller or they're happy to trade off compile time for expensive server time. That does not mean that we shouldn't strive for a solution that can be acceptable by all users. On the other hand making something opt-in makes it non-default, and that increases the testing surface.</div><div class=""><br class=""></div><div class="">Tangentially I think that LLVM currently doesn't have the right tuning knobs to allow the user to select their desired tradeoff. We have one optimization flag -O{s,z,0,1,2,3} which encodes both optimization *goal* (a point on the pareto curve between size and speed) and amount of effort to expend at compile time achieving that goal. Anyway, that's besides the point.</div><div class=""><br class=""></div><div class="">I like Justin's idea of removing IR from the backend to free up memory. I think it's a very long term project though, one that requires significant (re)design; alias analysis access in the backend would be completely broken and BasicAA among others depends on seeing the IR at query time. We'd need to work out a way of providing alias analysis with no IR present. I don't think that is feasible for the near future.</div><div class=""><br class=""></div><div id="DWT10923" class="">So my suggestion is that we go with Matthias' idea - do the small amount of refactoring needed to allow MachineModulePasses on an opt-in basis. The knobs to enable that opt-in might need some more bikeshedding.</div></div></blockquote>This makes sense to me. I expect that targets will be able to opt-in in some optimization-level-dependent fashion.<br class=""><br class=""> -Hal<br class=""><blockquote style="border-left-width: 2px; border-left-style: solid; border-left-color: rgb(16, 16, 255); margin-left: 5px; padding-left: 5px; font-weight: normal; font-style: normal; text-decoration: none; font-family: Helvetica, Arial, sans-serif; font-size: 12pt;" class=""><div dir="ltr" class=""><div class=""></div><div class=""><br class=""></div><div class="">Cheers,</div><div class=""><br class=""></div><div class="">James</div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="">On Tue, 19 Jul 2016 at 08:21 Justin Bogner <<a href="mailto:mail@justinbogner.com" target="_blank" class="">mail@justinbogner.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;">James Molloy via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>> writes:<br class="">> In LLVM it is currently not possible to write a Module-level pass (a pass that<br class="">> modifies or analyzes multiple MachineFunctions) after DAG formation. This<br class="">> inhibits some optimizations[1] and is something I'd like to see changed.<br class="">><br class="">> The problem is that in the backend, we emit a function at a time, from DAG<br class="">> formation to object emission. So no two MachineFunctions ever exist at any one<br class="">> time. Changing this necessarily means increasing memory usage.<br class="">><br class="">> I've prototyped this change and have measured peak memory usage in the worst<br class="">> case scenario - LTO'ing llc and clang. Without further ado:<br class="">><br class="">> llvm-lto llc: before: 1.44GB maximum resident set size<br class="">> after: 1.68GB (+17%)<br class="">><br class="">> llvm-lto clang: before: 2.48GB maximum resident set size<br class="">> after: 3.42GB (+33%)<br class="">><br class="">> The increases are very large. This is worst-case (non-LTO builds would see the<br class="">> peak usage of the backend masked by the peak of the midend) but still - pretty<br class="">> big. Thoughts? Is this completely no-go? is this something that we *just need*<br class="">> to do? Is crippling the backend architecture to keep memory down justified? Is<br class="">> this something we could enable under an option?<br class=""><br class="">Personally, I think this price is too high. I think that if we want to<br class="">enable machine module passes (which we probably do) we need to turn<br class="">MachineFunction into more of a first class object that isn't just a<br class="">wrapper around IR.<br class=""><br class="">This can and should be designed to work something like Pete's solution,<br class="">where we get rid of the IR and just have machine level stuff in memory.<br class="">This way, we may still increase the memory usage here, but it should be<br class="">far less dramatic.<br class=""><br class="">You'll note that doing this also has tangential benefits - it should be<br class="">helpful for simplifying MIR and generally improving testability of the<br class="">backends.<br class=""></blockquote></div></blockquote><br class=""><br class=""><br class="">--<span class="Apple-converted-space"> </span><br class=""><div class=""><span name="x" class=""></span>Hal Finkel<br class="">Assistant Computational Scientist<br class="">Leadership Computing Facility<br class="">Argonne National Laboratory<span name="x" class=""></span><br class=""></div></div><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">_______________________________________________</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">LLVM Developers mailing list</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><a href="mailto:llvm-dev@lists.llvm.org" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">llvm-dev@lists.llvm.org</a><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a></div></blockquote></div><br class=""></body></html>