<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<div class=""><br class="">
</div>
<div>
<blockquote type="cite" class="">
<div class="">On 15 Feb 2017, at 19:10, Chandler Carruth <<a href="mailto:chandlerc@gmail.com" class="">chandlerc@gmail.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">Thanks for running these Kristof!
<div class=""><br class="">
</div>
<div class="">I'd still like to hear from Apple, and if we can get a few more x86 micro-architectures covered that'd be great, but it looks like -O3 is uncontroversial, and the question is whether this makes sense at O2...</div>
<div class=""><br class="">
</div>
<div class="">To me, it would help a lot to know the actual breakdown of benchmarks such as yours Kristof (as they seem to have more codesize impact than others have mentioned). Specificially, are the runtime improvements correlated with the codesize increases?
And what are the absolute size deltas? For *very* small benchmarks, a 5% code size fluctuation seems less concerning than for a larger benchmark. If the larger code size changes are mostly smaller benchmarks and reasonably correlated to the ones likely to
see improvement from the change (this seemed to be the case w/ Dehao's data on x86 for example) that would to me indicate this makes sense at O2.</div>
<div class=""><br class="">
</div>
<div class="">Note that I'm fine if you have to list the benchmarks as "1, 2, 3, ..." or whatever, much like we did for Google-internal benchmarks. It's still useful to know the shape of the change.</div>
</div>
</div>
</blockquote>
<div><br class="">
</div>
<div>With this being data from a few hundred programs, I don't think listing the data in a long table really helps in getting a feel for the overall structure of the data.</div>
<div>Instead, I created a few scatter plots that hopefully helps in getting a better feel for the overall effect of the patch. The charts below are for the Cortex-A57 numbers. I decided not to produce a chart for Cortex-A53 as the shape of the data didn't seem
very different. The optimization level used is -O3 -fomit-frame-pointer, targeting AArch64 linux.</div>
<div><br class="">
</div>
<div>The first chart shows relative code size increase (vertical axis) vs absolute code size:</div>
<div>The biggest relative code size increases indeed didn't happen for the biggest programs, but instead for a few programs weighing in at about 100KB.</div>
<div>I'm assuming the Google benchmark set covers much bigger programs than the ones displayed here.</div>
<div>FWIW, the cluster of programs where code size increases between 60% to 80% with a size of about 100KB, all come from MultiSource/Benchmarks/TSVC. Interestingly, these programs seem to have float and double variants, e.g. (MultiSource/Benchmarks/TSVC/Searching-flt/Searching-flt
and MultiSource/Benchmarks/TSVC/Searching-dbl/Searching-dbl), and the code size bloat only happens for the double variants. I think it may still be worthwhile to check if this also happens on other architectures, and why it happens only for the double-variants,
not the float-variants.</div>
<div><br class="">
</div>
<div><br class="">
</div>
<div><img height="679" width="1194" apple-width="yes" apple-height="yes" apple-inline="yes" id="68A38B61-9110-4909-8B53-165F60AA8434" src="cid:C557D770-9D82-45EA-AA84-A5CB28B190EA" class=""></div>
<div><br class="">
</div>
<div><br class="">
</div>
<div>The second chart shows relative code size increase (vertical axis) vs relative performance improvement (horizontal axis):</div>
<div>I manually checked the cause of the 3 biggest performance regressions (proprietary benchmark1: -13.70%; MultiSource/Applications/hexxagon/hexxagon: -10.10%; MultiSource/Benchmarks/FreeBench/fourinarow/fourinarow<span class="Apple-tab-span" style="white-space: pre;">
</span>-5.23%).</div>
<div>For the proprietary benchmark and hexxagon, the code generation didn't change for the hottest parts, so probably is caused by micro-architectural effects of code layout changes.</div>
<div>For fourinarow, there seemed to be a lot more spill/fill code, so probably due to non-optimality of register allocation.</div>
<div><br class="">
</div>
<div><img height="707" width="1194" apple-width="yes" apple-height="yes" apple-inline="yes" id="93FDBE24-4F33-40DD-8573-C8B7EE638C33" src="cid:35438EFB-1337-4478-88C7-B8A718B61681" class=""></div>
<div><br class="">
</div>
<div>The third chart below just zooms in on the above chart to the -5% to 5% performance improvement range:</div>
<div><img height="812" width="1194" apple-width="yes" apple-height="yes" apple-inline="yes" id="CB31B1EF-0716-4CDC-A6BB-1367A50EA286" src="cid:C7AB0398-ED09-448D-BF28-5FD328D90350" class=""></div>
<div><br class="">
</div>
<div><br class="">
</div>
<div>Whether to enable the increase in unroll threshold only at O3 or also at O2: I don't have a strong opinion based on the above data.</div>
<div>Maybe the compile time impact is what should be driving that discussion the most? I'm afraid I don't have compile time numbers.</div>
<div>Ultimately, I guess this boils down to what exactly the difference is in intent between O2 and O3, which seems like a never-ending discussion...</div>
<div><br class="">
</div>
Hoping you find this useful,</div>
<div><br class="">
</div>
<div>Kristof</div>
<div><br class="">
<blockquote type="cite" class="">
<div class=""><br class="">
<div class="gmail_quote">
<div dir="ltr" class="">On Tue, Feb 14, 2017 at 1:06 PM Kristof Beyls via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> wrote:<br class="">
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word" class="gmail_msg">I've run the patch on <a href="https://reviews.llvm.org/D28368" class="gmail_msg" target="_blank">https://reviews.llvm.org/D28368</a> on the test-suite and other benchmarks, for AArch64 -O3 -fomit-frame-pointer,
both for Cortex-A53 and Cortex-A57.
<div class="gmail_msg"><br class="gmail_msg">
</div>
<div class="gmail_msg">The geomean over the few hundred programs in there is roughly the same for Cortex-A53 and Cortex-A57: a bit over 1% improvement in execution speed for a bit over 5% increase in code size.</div>
<div class="gmail_msg">Obviously I wouldn't want this for optimization levels where code size is of any concern, like -Os or -Oz, but don't have a problem with this going in for other optimization levels where this isn't a concern.</div>
<div class="gmail_msg"><br class="gmail_msg">
</div>
<div class="gmail_msg">Thanks,</div>
<div class="gmail_msg"><br class="gmail_msg">
</div>
<div class="gmail_msg">Kristof</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
<br class="">
</body>
</html>