<div dir="ltr"><br>Hello George,<div><br></div><div>Sorry, somehow hit a send button too soon. Please ignore the previous e-mail.</div><div><br></div><div>The bot does 10 runs for each of the benchmarks (those dots in the logs are meaningful). We can increase the number of runs if proven that this would significantly increase the accuracy. I didn't see the increase in accuracy <span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">when have been staging the bot,</span>

 which would justify the extra time and larger gaps between the tested revisions.  10 runs seems give a good balance. But I'm open for suggestions.</div><div><br></div><div>

<div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial">It seems the statistics are quite stable if you would look over number of revisions.</div></div><div>And in this particular case the picture seems quite clear.</div><div><br></div><div>At <a href="http://lnt.llvm.org/db_default/v4/link/104" target="_blank">http://lnt.llvm.org/db_default<wbr>/v4/link/104</a>, the list of Performance Regressions suggests that the most hit was with the linux-kernel. The regressed metrics - <span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">branches, </span>branch-misses<span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">, ins<wbr>tructions</span>, cycles, seconds-elapsed, task-<wbr>clock. Some other benchmarks shows regressions in <span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;background-color:rgb(255,255,255);float:none;display:inline">branches and </span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">branch-misses, some shows improvements.</span></div><div><br></div><div>The metrics are consistent before and after the commit, so, I do not think this one is an outliner.</div><div>For example, if one would take a look at the linux-kernel branches - <a href="http://lnt.llvm.org/db_default/v4/link/graph?plot.0=1.12.2&highlight_run=104" target="_blank">http://lnt.llvm.org/db_default<wbr>/v4/link/graph?plot.0=1.12.2&h<wbr>ighlight_run=104</a>, it gets obvious that the number of branches increased significantly as a result of the r325313. The metric is very stable around the impacted commit and does not go down after. The 

<span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">branch-misses</span> is more volatile, but still consistently shows the regression as the result of this commit.<br></div><div><br></div><div>Now someone should see why this particular commit has resulted in significant increase of branching with the Linux Kernel.</div><div><br></div><div>As of how to use LNT web UI, I'm sure you have checked that, but, just in case, here is the link to the LNT doc - <a href="http://llvm.org/docs/lnt/contents.html" target="_blank">http://llvm.org/docs/lnt/con<wbr>tents.html</a>.</div><div><br></div><div>> <span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">task-clock results are available for "linux-kernel" and "llvm-as-fsds" only and all other</span><br style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">> tests has blank field. Should it mean there was no noticable difference in results ?</span>

</div><div><br></div><div>If you would go to <a href="http://lnt.llvm.org/db_default/v4/link/104#task-clock" target="_blank">http://lnt.llvm.org/db_defa<wbr>ult/v4/link/104#task-clock</a> (or go to 

<a href="http://lnt.llvm.org/db_default/v4/link/104" target="_blank">http://lnt.llvm.org/db_default<wbr>/v4/link/104</a> and select the <span style="font-size:12.8px">task-clock on the left, which is the same), you would see the list of actual values in the "Current" column. All of them populated, none is blank. The column "%" contains the difference from the previous run in percents, or dash for no measured difference.</span><br></div><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"></span></div><div><br></div><div>> 

<span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Also, "Graph" and "Matrix" buttons whatever they should do show errors atm.</span>

</div><div><br></div><div>I guess you didn't select what to graph or what to show as a matrix, did you?</div><div><br></div><div>Besides reporting to the <a href="http://lnt.llvm.org" target="_blank">lnt.llvm.org</a>, each build contains in the log all the reported data, so you could process it whatever you want and find helpful.</div><div><br></div><div>Hope this helps.</div><div><br></div><div>Thanks</div><div><br></div><div>Galina</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Feb 16, 2018 at 1:55 AM, George Rimar via llvm-dev <span dir="ltr"><<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">>Hello everyone,<br>
><br>
>I have added a new public LLD performance builder at<br>
><a href="http://lab.llvm.org:8011/builders/lld-perf-testsuite" rel="noreferrer" target="_blank">http://lab.llvm.org:8011/<wbr>builders/lld-perf-testsuite</a>.<br>
>It builds LLVM and LLD by the latest releaed Clang and runs a set of<br>
>perfromance tests.<br>
><br>
>The builder is reliable. Please pay attention on the failures.<br>
><br>
>The performance statistics are here:<br>
><a href="http://lnt.llvm.org/db_default/v4/link/recent_activity" rel="noreferrer" target="_blank">http://lnt.llvm.org/db_<wbr>default/v4/link/recent_<wbr>activity</a><br>
><br>
>Thanks<br>
><br>
>Galina<br>
<br>
</div></div>Great news, thanks !<br>
<br>
Looking on results I am not sure how to explain them though.<br>
<br>
For example r325313 fixes "use after free", it should not give any performance<br>
slowdowns or boosts. Though if I read results right, they show 23.65% slowdown<br>
for time of linking linux kernel (<a href="http://lnt.llvm.org/db_default/v4/link/104" rel="noreferrer" target="_blank">http://lnt.llvm.org/db_<wbr>default/v4/link/104</a>).<br>
<br>
I guess such variation can happen for example if bot do only single link iteration for tests,<br>
so that final time is just a error mostly probably.<br>
<br>
task-clock results are available for "linux-kernel" and "llvm-as-fsds" only and all other<br>
tests has blank field. Should it mean there was no noticable difference in results ?<br>
<br>
Also, "Graph" and "Matrix" buttons whatever they should do show errors atm.<br>
("Nothing to graph." and "Not Found: Request requires some data arguments.").<br>
<br>
Best regards,<br>
George | Developer | Access Softek, Inc<br>
______________________________<wbr>_________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/llvm-dev</a><br>
</blockquote></div><br></div>