<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.hoenzb
{mso-style-name:hoenzb;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-GB link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Yes, absolutely!<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Right now, we’re slowly getting to the point where we’re seeing the significant performance changes on <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_perf&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=mQ4LZ2PUj9hpadE3cDHZnIdEwhEBrbAstXeMaFoB9tg&m=LnuSvbqpTiVBtU2bsGFubHL0946rQLgIEqfQVoI2wYw&s=iSV9Duky8_kXhX2dwWWRx0Mhf5ZoTnKU8vBupH0c5es&e=">http://llvm.org/perf</a>.<br>I’m currently focussing mainly on improving the daily report page (<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_perf_db-5Fdefault_v4_nts_daily-5Freport&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=mQ4LZ2PUj9hpadE3cDHZnIdEwhEBrbAstXeMaFoB9tg&m=LnuSvbqpTiVBtU2bsGFubHL0946rQLgIEqfQVoI2wYw&s=JsXzuAlS5jQjSMra7XrjF4Ee2t-RbBQB0dYbGiabmVM&e=">http://llvm.org/perf/db_default/v4/nts/daily_report</a>) as that<br>seems the most useful report to get an overview of the significant changes.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>But once that is working well, the next step is to make it easy to understand what caused the significant performance change.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Indeed, something visual that helps to intuitively understand the main changes in the profile + a way to also see the corresponding<br>changes in assembly code.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Measuring the profile + recording the assembly code for all programs at every run will likely be prohibitively expensive – both<br>due to the performance tracking bots slowing down and the storage capacity needed to store the information for all runs on the server.<br>But maybe the rerun-functionality could be extended so that for the significant changes, the bot is asked to do extra profiling runs<br>of the programs with significant performance deltas and is asked to also submit the assembly produced. That will need to be done<br>for both the current version of clang under test + the previous version. How to make sure the bot caches a number of previous versions<br>(or rebuilds it) is probably not entirely trivial. We’ll also need to make sure that the profiles and the assembly code also don’t<br>massively increase the storage needs on the server side. But all-in-all, off the top of my head, this seems to be the most straightforward<br>way to get to a system like you describe below.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>So, yes, absolutely, being able to detect significant performance deltas is not the end goal, but rather a system like you describe below.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Kristof<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div style='border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt'><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal><b><span lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Sean Silva [mailto:chisophugis@gmail.com] <br><b>Sent:</b> 25 June 2015 03:23<br><b>To:</b> Kristof Beyls<br><b>Cc:</b> llvm-commits; Tobias Grosser<br><b>Subject:</b> Re: [PATCH][LNT] Add support for more detailed profiling to lnt.<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>It would be awesome if when we detect a performance change of one of the benchmarks (not the LNT server), we did a similar analysis across the two versions and presented a graph like this, but where the weight is the performance diff of the function. Bonus points for making the graph nodes clickable and showing an annotated assembly diff.<o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>-- Sean Silva<o:p></o:p></p></div></div><div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>On Wed, Jun 24, 2015 at 10:15 AM, Kristof Beyls <<a href="mailto:kristof.beyls@arm.com" target="_blank">kristof.beyls@arm.com</a>> wrote:<o:p></o:p></p><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Hi,<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>The attached patch adds support for getting call-graph profiles from lnt runserver. It does this by adding<br>a new –profiler-dir command line option, which results in a pstat profile to be stored to a separate file<br>in that sub directory, per web request. I find the extra call graph info very<br>useful when trying to understand what the bottlenecks are in the rendering of pages by the<br>lnt webserver.<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>The pstat files can easily be converted into something graphical by a command like:<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>$ gprof2dot -f pstats profile_dir/POST.submitRun.000591ms.1435075964.prof | dot -Tsvg > submitRun.svg<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>I’ve attached such an svg graph for a request of the daily report page to give an idea of<br>the kind of information this more detailed profiling can give.<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Does the patch look OK to commit?<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Thanks,<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='color:#888888'> <o:p></o:p></span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='color:#888888'>Kristof<o:p></o:p></span></p></div></div><p class=MsoNormal style='margin-bottom:12.0pt'><br>_______________________________________________<br>llvm-commits mailing list<br><a href="mailto:llvm-commits@cs.uiuc.edu">llvm-commits@cs.uiuc.edu</a><br><a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits</a><o:p></o:p></p></div><p class=MsoNormal><o:p> </o:p></p></div></div></div></body></html>