<div dir="ltr"><div class="gmail_quote"><div dir="ltr">On Thu, Feb 16, 2017 at 7:06 PM Xinliang David Li <<a href="mailto:xinliangli@gmail.com">xinliangli@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_msg"><div class="gmail_quote gmail_msg"><div class="gmail_msg">On Thu, Feb 16, 2017 at 5:43 PM Mehdi Amini <<a href="mailto:mehdi.amini@apple.com" class="gmail_msg" target="_blank">mehdi.amini@apple.com</a>> wrote:<br class="gmail_msg"></div><blockquote class="gmail_quote gmail_msg" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class="gmail_msg"><div class="gmail_msg"><blockquote type="cite" class="gmail_msg"><div class="gmail_msg">On Feb 16, 2017, at 4:41 PM, Xinliang David Li via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="gmail_msg" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:</div><br class="m_-2955104811854048874m_5107660674424646341Apple-interchange-newline gmail_msg"></blockquote></div></div><div style="word-wrap:break-word" class="gmail_msg"><div class="gmail_msg"><blockquote type="cite" class="gmail_msg"><div class="gmail_msg"><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class="gmail_msg"><div class="gmail_extra gmail_msg"><div class="gmail_quote gmail_msg">On Thu, Feb 16, 2017 at 3:45 PM, Chandler Carruth via llvm-dev<span class="m_-2955104811854048874m_5107660674424646341Apple-converted-space gmail_msg"> </span><span class="gmail_msg"><<a href="mailto:llvm-dev@lists.llvm.org" class="gmail_msg" target="_blank">llvm-dev@lists.llvm.org</a>></span><span class="m_-2955104811854048874m_5107660674424646341Apple-converted-space gmail_msg"> </span>wrote:<br class="gmail_msg"></div></div></div></div></blockquote></div></div><div style="word-wrap:break-word" class="gmail_msg"><div class="gmail_msg"><blockquote type="cite" class="gmail_msg"><div class="gmail_msg"><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class="gmail_msg"><div class="gmail_extra gmail_msg"><div class="gmail_quote gmail_msg"><blockquote class="gmail_quote gmail_msg" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div class="gmail_msg">First off, I just want to say wow and thank you. This kind of data is amazing. =D<br class="gmail_msg"><br class="gmail_msg"></div></blockquote></div></div></div></div></blockquote></div></div><div style="word-wrap:break-word" class="gmail_msg"><div class="gmail_msg"><blockquote type="cite" class="gmail_msg"><div class="gmail_msg"><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class="gmail_msg"><div class="gmail_extra gmail_msg"><div class="gmail_quote gmail_msg"><blockquote class="gmail_quote gmail_msg" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div class="gmail_msg"><div class="gmail_quote gmail_msg"><span class="gmail_msg"><div class="gmail_msg">On Thu, Feb 16, 2017 at 2:46 AM Kristof Beyls <<a href="mailto:Kristof.Beyls@arm.com" class="gmail_msg" target="_blank">Kristof.Beyls@arm.com</a>> wrote:</div><blockquote class="gmail_quote gmail_msg" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg" style="word-wrap:break-word"><div class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg"><div class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg">The biggest relative code size increases indeed didn't happen for the biggest programs, but instead for a few programs weighing in at about 100KB.</div><div class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg">I'm assuming the Google benchmark set covers much bigger programs than the ones displayed here.</div><div class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg">FWIW, the cluster of programs where code size increases between 60% to 80% with a size of about 100KB, all come from MultiSource/Benchmarks/TSVC. Interestingly, these programs seem to have float and double variants,  e.g. (MultiSource/Benchmarks/TSVC/Searching-flt/Searching-flt and MultiSource/Benchmarks/TSVC/Searching-dbl/Searching-dbl), and the code size bloat only happens for the double variants.</div></div></div></blockquote><div class="gmail_msg"><br class="gmail_msg"></div></span><div class="gmail_msg">I think we should definitely look at this (as it seems likely to be a bug somewhere), but I'm also not overly concerned with size regressions in the TSVC benchmarks which are unusually loop heavy and small. We've have several other changes that caused big fluctuations here.</div><span class="gmail_msg"><div class="gmail_msg"> </div><div class="gmail_msg"> </div><blockquote class="gmail_quote gmail_msg" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg" style="word-wrap:break-word"><div class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg"><div class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg">I think it may still be worthwhile to check if this also happens on other architectures, and why it happens only for the double-variants, not the float-variants.</div></div></div></blockquote><div class="gmail_msg"><br class="gmail_msg"></div></span><div class="gmail_msg">+1</div><span class="gmail_msg"><div class="gmail_msg"><br class="gmail_msg"></div><blockquote class="gmail_quote gmail_msg" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg" style="word-wrap:break-word"><div class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg"><div class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg">The second chart shows relative code size increase (vertical axis) vs relative performance improvement (horizontal axis):<br class="gmail_msg"></div><div class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg">I manually checked the cause of the 3 biggest performance regressions (proprietary benchmark1: -13.70%; MultiSource/Applications/hexxagon/hexxagon: -10.10%; MultiSource/Benchmarks/FreeBench/fourinarow/fourinarow<span class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082m_-1689292437810271661Apple-tab-span m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg" style="white-space:pre-wrap">
</span>-5.23%).</div><div class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg">For the proprietary benchmark and hexxagon, the code generation didn't change for the hottest parts, so probably is caused by micro-architectural effects of code layout changes.</div></div></div></blockquote><div class="gmail_msg"><br class="gmail_msg"></div></span><div class="gmail_msg">This is always good to know, even though it is frustrating. =]</div><span class="gmail_msg"><div class="gmail_msg"> </div><blockquote class="gmail_quote gmail_msg" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg" style="word-wrap:break-word"><div class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg"><div class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg">For fourinarow, there seemed to be a lot more spill/fill code, so probably due to non-optimality of register allocation.</div></div></div></blockquote><div class="gmail_msg"><br class="gmail_msg"></div></span><div class="gmail_msg">This is something we should probably look at. If you have the output lying around, maybe file a PR about it?</div><div class="gmail_msg"><br class="gmail_msg"></div></div></div></blockquote></div></div></div></div></blockquote></div></div><div style="word-wrap:break-word" class="gmail_msg"><div class="gmail_msg"><blockquote type="cite" class="gmail_msg"><div class="gmail_msg"><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class="gmail_msg"><div class="gmail_extra gmail_msg"><div class="gmail_quote gmail_msg"><blockquote class="gmail_quote gmail_msg" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div class="gmail_msg"><div class="gmail_quote gmail_msg"><blockquote class="gmail_quote gmail_msg" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg" style="word-wrap:break-word"><div class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg"><span class="gmail_msg"><div class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg">The third chart below just zooms in on the above chart to the -5% to 5% performance improvement range:<br class="gmail_msg"></div></span><div class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg"><span id="m_-2955104811854048874m_5107660674424646341cid:15a4855865b9015c7b73" class="gmail_msg"><unroll_codesize_vs_performance_zoom.png></span></div><span class="gmail_msg"><div class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg"><br class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg"></div><div class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg"><br class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg"></div><div class="m_-2955104811854048874m_5107660674424646341m_-5579311497984940082gmail_msg gmail_msg">Whether to enable the increase in unroll threshold only at O3 or also at O2: I don't have a strong opinion based on the above data.</div></span></div></div></blockquote></div></div></blockquote></div></div></div></div></blockquote></div></div><div style="word-wrap:break-word" class="gmail_msg"><div class="gmail_msg"><blockquote type="cite" class="gmail_msg"><div class="gmail_msg"><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class="gmail_msg"><div class="gmail_extra gmail_msg"><div class="gmail_quote gmail_msg"><blockquote class="gmail_quote gmail_msg" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div class="gmail_msg"><div class="gmail_quote gmail_msg"><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg">FWIW, this data seems to clearly indicate that we don't get performance wins with any consistency when the code size goes up (and thus the change has impact). As a consequence, I pretty strongly suspect that this should be *just* used at O3 at least for now.</div></div></div></blockquote></div></div></div></div></blockquote></div></div><div style="word-wrap:break-word" class="gmail_msg"><div class="gmail_msg"><blockquote type="cite" class="gmail_msg"><div class="gmail_msg"><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class="gmail_msg"><div class="gmail_extra gmail_msg"><div class="gmail_quote gmail_msg"><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg">The correlation is there -- when there is performance improvement, there is size increase.</div></div></div></div></div></blockquote></div></div><div style="word-wrap:break-word" class="gmail_msg"><div class="gmail_msg"><blockquote type="cite" class="gmail_msg"><div class="gmail_msg"><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class="gmail_msg"><div class="gmail_extra gmail_msg"><div class="gmail_quote gmail_msg"></div></div></div></div></blockquote><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg">I didn’t quite get this impression from the graph, the highest improvement didn’t come with code size increase:</div><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg"><img id="m_-2955104811854048874m_510766067442464634117FCBC2A-04A2-4737-B05E-5D7B8522FA0E" class="gmail_msg"></div><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg">And on the other hand there were many code-size increase without any runtime improvement. </div></div></div></blockquote><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg"><br class="gmail_msg"></div></div></div><div class="gmail_msg"><div class="gmail_quote gmail_msg"><div class="gmail_msg">the zoomed in graph does not show the trend indeed, but the original graph does 😀 </div></div></div></blockquote><div><br></div><div>I kind of agree with how Mehdi is interpreting this data. =]</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_msg"><div class="gmail_quote gmail_msg"><blockquote class="gmail_quote gmail_msg" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class="gmail_msg"><div class="gmail_msg"><div class="gmail_msg"></div></div></div><div style="word-wrap:break-word" class="gmail_msg"><div class="gmail_msg"><div class="gmail_msg"><br class="gmail_msg"></div><br class="gmail_msg"><blockquote type="cite" class="gmail_msg"><div class="gmail_msg"><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class="gmail_msg"><div class="gmail_extra gmail_msg"><div class="gmail_quote gmail_msg"><div class="gmail_msg"> The opposite is not true -- but that is expected. If the speedup is in the cold path, there won't be visible performance improvement but size increase.</div><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg">Put it another way. If we reduce the threshold, there will be sizable size improvement for many benchmarks without regressing performance, shall we use the reduced threshold for O2 instead?</div></div></div></div></div></blockquote><div class="gmail_msg"><br class="gmail_msg"></div></div></div><div style="word-wrap:break-word" class="gmail_msg"><div class="gmail_msg"><div class="gmail_msg">Yes, all the ones here IIUC:</div><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg"><img id="m_-2955104811854048874m_51076606744246463410A8FD80C-5340-40F2-9364-F0AA4F1F912F" class="gmail_msg"></div><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg">However it is likely that we could consider these “small” benchmarks should use -Os if they're sensitive to size, and so O2 would be fine with the more aggressive threshold (as larger program aren’t affected).</div><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg">With good heuristic we’d have every dot forming a straight line   code_size_increase = m * runtime_perf (with m as small as possible). The current lack of shape (or the exact opposite distribution to the ideal I imagine above) seems to show that our "profitability” heuristics are pretty bad and the current threshold knob is bad predictor of the runtime performance.</div></div></div></blockquote><div class="gmail_msg"><br class="gmail_msg"></div></div></div><div class="gmail_msg"><div class="gmail_quote gmail_msg"><div class="gmail_msg">To get that, profile data is needed. Downstream component bugs also makes it hard to achieve.</div></div></div></blockquote><div><br></div><div>Sure. I think what I'm saying (don't want to put words in Mehdi's mouth) is that until the bugs are fixed, this probably belongs at O3 rather than O2 based on this data.</div><div><br></div><div>FWIW, I suspect there might be some bug fixes that would change this even without profile data, but its of course impossible to know until the bugs are analyzed.</div><div><br></div><div><br></div><div>I don't think this should be a show stopper, I think the change is still a really good one at O3.</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_msg"><div class="gmail_quote gmail_msg"><blockquote class="gmail_quote gmail_msg" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class="gmail_msg"><div class="gmail_msg"><blockquote type="cite" class="gmail_msg"></blockquote></div></div></blockquote></div></div></blockquote></div></div>