<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<br class="">
<div>
<blockquote type="cite" class="">
<div class="">On 22 Apr 2016, at 11:14, Mehdi Amini <<a href="mailto:mehdi.amini@apple.com" class="">mehdi.amini@apple.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<blockquote type="cite" class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<div class=""><br class="Apple-interchange-newline">
On Apr 22, 2016, at 12:45 AM, Kristof Beyls via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<blockquote type="cite" class="">
<div class=""><br class="Apple-interchange-newline">
On 21 Apr 2016, at 17:44, Sergey Yakoushkin <<a href="mailto:sergey.yakoushkin@gmail.com" class="">sergey.yakoushkin@gmail.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">Hi Kristof,
<div class=""><br class="">
</div>
<div class=""><font color="#500050" face="Calibri, sans-serif" class=""><span class="" style="font-size: 14.6667px;"> The way we use LNT, we would run different configuration (e.g. -O3 vs -Os) as different "machines" in LNT's model.</span></font></div>
<div class=""><br class="">
</div>
<div class="">O2/O3 is indeed bad example. We're also using different machines for Os/O3 - such parameters apply to all tests and we don't propose major changes.</div>
<div class="">
<div class="">Elena was only extending LNT interface a bit to ease LLVM-testsuite execution with different compiler or HW flags.<br class="">
</div>
</div>
</div>
</div>
</blockquote>
<div class=""><br class="">
</div>
<div class="">Oh I see, this boils down to extending the lnt runtest interface to be able to specify a set of configurations, rather than a single configuration and making</div>
<div class="">sure configurations get submitted under different machine names? We kick off the different configuration runs through a script invoking lnt runtest multiple</div>
<div class="">times. I don't see a big problem with extending the lnt runtest interface to do this, assuming it doesn't break the underlying concepts assumed throughout</div>
<div class="">LNT. Maybe the only downside is that this will add even more command line options to lnt runtest, which already has a lot (too many?) command line</div>
<div class="">options.</div>
<br class="">
<blockquote type="cite" class="">
<div class="">
<div dir="ltr" class="">
<div class="">
<div class=""></div>
<div class="">Maybe some changes are required to analyze and compare metrics between "machines": e.g. code size/performance between Os/O2/O3.</div>
<div class="">Do you perform such comparisons?</div>
</div>
</div>
</div>
</blockquote>
<div class=""><br class="">
</div>
<div class="">We typically do these kinds of comparisons when we test our patches pre-commit, i.e. comparing for example '-O3' with '-O3 'mllvm -enable-my-new-pass'.</div>
<div class="">To stick with the LNT concepts, tests enabling new passes are stored as a different "machine".</div>
<div class="">The only way I know to be able to do a comparison between runs on 2 different "machine"s is to manually edit the URL for run vs run comparison</div>
<div class="">and fill in the runids of the 2 runs you want to compare.</div>
<div class="">For example, the following URL is a comparison of green-dragon-07-x86_64-O3-flto vs green-dragon-06-x86_64-O0-g on the public<span class="Apple-converted-space"> </span><a href="http://llvm.org/perf" class="">llvm.org/perf</a> server:</div>
<div class=""><a href="http://llvm.org/perf/db_default/v4/nts/70644?compare_to=70634" class="">http://llvm.org/perf/db_default/v4/nts/70644?compare_to=70634</a></div>
<div class="">I had to manually look up and fill in the run ids 70644 and 70634.</div>
<div class="">It would be great if there was a better way to be able to do these kind of comparisons - i.e. not having to manually fill in run ids, but having a webui to easily find and pick the runs you want to compare.</div>
<div class="">(As an aside: I find it intriguing that the URL above suggests that there are quite a few cases where "-O0 -g" produces faster code than "-O3 -flto").</div>
</div>
</div>
</blockquote>
<div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<br class="">
</div>
<div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
Can you be more explicit which ones? I don't see any regression (other than compared to the baseline, or on the compile time).</div>
<div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<br class="">
</div>
<div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
-- </div>
<div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
Mehdi</div>
</div>
</blockquote>
<br class="">
</div>
<div>D'Oh! I was misinterpreting the compile time differences as execution time differences. Indeed, there is no unexpected result in there.</div>
<div>Sorry for the noise!</div>
<div><br class="">
</div>
<div>Kristof</div>
<br class="">
</body>
</html>