<div dir="ltr">Hi Andrey,<div> Questions and feedback are very much welcome. <br>- The explanation as to why the improvements can vary between the IR and MIR outliner mainly boil down to the level of abstraction that each are working at. The MIR level has very accurate heuristics and is effectively the last post ISel target independent code gen pass. The IR outliner on the other hand has more estimation in the cost model, and can affect the decisions of function simplification passes, instruction selection, RA, etc. Taking this into account can lead to different results. Its important to remember the differences that being at a higher abstraction level can cause.</div><div>- As for the spec(it is 2006, sorry I missed that) command line options, as well as any other benchmark, I only added "-Oz -mno-red-zone(to keep comparisons fair) -(whatever enables each transform)" to the default flags needed for compilation. I'll try to get the exact command line options used and add them.</div><div>- Debug information(line numbers) is currently only kept if it is the same across all occurrences. This was simply a design choice and can be changed if keeping one instance is the desired behavior.</div><div>- The behavior described with the profile data is already implemented, I will update the numbers to include the results after including pgo data. </div><div>- The LTO results are very experimental given that there isn't a size pipeline for LTO yet(there should be). The %improvements can be similar to non LTO but because the LTO binary is generally much smaller the actual decrease in size is also much smaller. I'll add more detailed LTO numbers as well.</div><div>Thanks,</div><div>River Riddle</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Jul 22, 2017 at 3:23 PM, Andrey Bokhanko <span dir="ltr"><<a href="mailto:andreybokhanko@gmail.com" target="_blank">andreybokhanko@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi River,<div><br></div><div>Very impressive! -- thanks for working on this.</div><div><br></div><div>A few questions, if you don't mind.</div><div><br></div><div>First, on results (from <font color="#000000"><span style="white-space:pre-wrap"><a href="http://goo.gl/5k6wsP" target="_blank">goo.gl/5k6wsP</a>). Some of them are quite surprising. In theory, "top improvements" should be quite similar in all three approaches ("Early&Late Outlining", "Late Outlining" and "Machine Outliner"), with E&LO capturing most of the cases. Yet, they are very different:</span></font></div><div><span style="color:rgb(0,0,0);font-family:Arial;font-size:9pt;white-space:pre-wrap;background-color:transparent"><br></span></div><div><span style="color:rgb(0,0,0);font-family:Arial;font-size:9pt;white-space:pre-wrap;background-color:transparent">Test Suite, top improvements:</span></div><div><span style="color:rgb(0,0,0);font-family:Arial;font-size:9pt;white-space:pre-wrap;background-color:transparent">E&LO:</span></div><div><span id="m_-3659905828330367631gmail-docs-internal-guid-ec77862f-6c4e-6dfc-ecbc-d87136c4cbe1"><ul style="margin-top:0pt;margin-bottom:0pt"><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">enc-3des: 66.31%</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">StatementReordering-dbl: 51.45%</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Symbolics-dbl: 51.42%</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Recurrences-dbl: 51.38%</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Packing-dbl: 51.33%</span></p></li></ul></span></div><div><span style="color:rgb(0,0,0);font-family:Arial;font-size:9pt;white-space:pre-wrap;background-color:transparent">LO:</span></div><div><span id="m_-3659905828330367631gmail-docs-internal-guid-ec77862f-6c4f-6a1d-0c19-ac0d5f161f20"><ul style="margin-top:0pt;margin-bottom:0pt"><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">enc-3des: 50.7%</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">ecbdes: 46.27%</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">security-rjindael:45.13%</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">ControlFlow-flt: 25.79%</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">ControlFlow-dbl: 25.74%</span></p></li></ul></span></div><div>MO:<br></div><div><span id="m_-3659905828330367631gmail-docs-internal-guid-ec77862f-6c50-7cfc-bbca-3947c48bcbd2"><ul style="margin-top:0pt;margin-bottom:0pt"><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">ecbdes: 28.22%</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Expansion-flt: 22.56%</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Recurrences-flt: 22.19%</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">StatementReordering-flt: 22.15%</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Searching-flt: 21.96%</span></p></li></ul></span></div><div><br></div><div>SPEC, top improvements:</div><div>E&LO:</div><div><span id="m_-3659905828330367631gmail-docs-internal-guid-ec77862f-6c51-fe5f-01a2-97c6571e7f67"><ul style="margin-top:0pt;margin-bottom:0pt"><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">bzip2: 9.15%</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">gcc: 4.03%</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">sphinx3: 3.8%</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">H264ref: 3.24%</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Perlbench: 3%</span></p></li></ul></span></div><div>LO:</div><div><span id="m_-3659905828330367631gmail-docs-internal-guid-ec77862f-6c52-63f6-f74c-2bd443642b60"><ul style="margin-top:0pt;margin-bottom:0pt"><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">bzip2: 7.27%</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">sphinx3: 3.65%</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Namd: 3.08%</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Gcc: 3.06%</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">H264ref: 3.05%</span></p></li></ul></span></div><div>MO:</div><div><span id="m_-3659905828330367631gmail-docs-internal-guid-ec77862f-6c53-8627-f7c2-a01d9fb5544e"><ul style="margin-top:0pt;margin-bottom:0pt"><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Namd: 7.8%</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">bzip2: 7.27%</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">libquantum: 2.99%</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:9pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:9pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">h264ref: 2%</span></p></li></ul></span></div><div><br></div><div>Do you understand why so?</div><div><br></div><div>I'm especially interested in cases where MO managed to find redundancies while E&O+LO didn't. For example, 2.99% on libquantum (or is it simply below "top 5 results" for E&O+LO?) -- did you investigated this?</div><div><br></div><div>Also, it would be nice to specify full options list for SPEC (I assume SPEC CPU2006?), similar to how results are reported on <a href="http://spec.org" target="_blank">spec.org</a>.</div><div><br></div><div>And a few questions on the RFC:</div><span class=""><div><br></div><div>On Fri, Jul 21, 2017 at 12:47 AM, River Riddle via llvm-dev <span dir="ltr"><<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>></span> wrote:<br></div></span><div class="gmail_extra"><div class="gmail_quote"><span class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><span id="m_-3659905828330367631m_-8118405716264276722gmail-docs-internal-guid-128cceb0-622c-6ca0-9998-bc07500bb3bc"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-family:Arial;font-size:11pt;white-space:pre-wrap">* Debug Info:</span><br></p></span></div></blockquote><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><span id="m_-3659905828330367631m_-8118405716264276722gmail-docs-internal-guid-128cceb0-622c-6ca0-9998-bc07500bb3bc"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">Debug information is preserved for the calls to functions which have been outlined but all debug info from the original outlined portions is removed, making them harder to debug.</span></p></span></div></blockquote><div><br></div></span><div>Just to check I understand it correctly: you remove *all* debug info in outlined functions, essentially making them undebuggable -- correct? Did you considered copying debug info from one of outlined fragments instead? -- at least line numbers?</div><span class=""><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><span id="m_-3659905828330367631m_-8118405716264276722gmail-docs-internal-guid-128cceb0-622c-6ca0-9998-bc07500bb3bc"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-family:Arial;font-size:11pt;white-space:pre-wrap;background-color:transparent">The execution time results are to be expected given that the outliner, without profile data, will extract from whatever region it deems profitable. Extracting from the hot path can lead to a noticeable performance regression on any platform, which can be somewhat avoided by providing profile data during outlining.</span></p></span></div></blockquote><div><br></div></span><div>Some of regressions are quite severe. It would be interesting to implement what you stated above and measure -- both code size reductions and performance degradations -- again.</div><span class=""><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><span id="m_-3659905828330367631m_-8118405716264276722gmail-docs-internal-guid-128cceb0-622c-6ca0-9998-bc07500bb3bc"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-family:Arial;font-size:11pt;white-space:pre-wrap;background-color:transparent">* LTO:</span><br></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;background-color:transparent;vertical-align:baseline;white-space:pre-wrap">    - LTO doesn’t have a code size pipeline, but %reductions over LTO are comparable to non LTO.</span></p></span></div></blockquote><div><br></div></span><div>LTO is known to affect code size significantly (for example, by removing redundant functions), so I'm frankly quite surprised that the results are the same...</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><span id="m_-3659905828330367631m_-8118405716264276722gmail-docs-internal-guid-128cceb0-622c-6ca0-9998-bc07500bb3bc"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"></p></span></div></blockquote></div>Yours,</div><div class="gmail_extra">Andrey</div><div class="gmail_extra">===</div><div class="gmail_extra">Compiler Architect</div><div class="gmail_extra">NXP</div><div class="gmail_extra"><br></div></div>
</blockquote></div><br></div>