<div dir="ltr">On 9 August 2013 22:29, Chris Matthews <span dir="ltr"><<a href="mailto:chris.matthews@apple.com" target="_blank">chris.matthews@apple.com</a>></span> wrote:<br><div class="gmail_extra"><div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><div>First when an LNT run is finished and after the initial submission, rerun any benchmarks which the server reports back as regressed.</div>
</div></blockquote><div><br></div><div>This is great, I think it will increase the quality without re-running everything.</div><div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word"><div>This would not catch changes in the system during reruns, but it would between runs.<br></div></div></blockquote><div><br></div><div>That's the idea, yes. A re-run should be considered "the same run" for any statistical purposes.</div>
<div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><div></div><div><div><div style="text-indent:0px;letter-spacing:normal;text-align:start;text-transform:none;white-space:normal;word-wrap:break-word;word-spacing:0px">
<div style="text-indent:0px;letter-spacing:normal;text-align:start;text-transform:none;white-space:normal;word-wrap:break-word;word-spacing:0px"><div style="text-indent:0px;letter-spacing:normal;text-align:start;text-transform:none;white-space:normal;word-wrap:break-word;word-spacing:0px">
Both Daniel and myself have some scripts for helping with these sort of performance comparisons you are talking about, basically just looking at A/B situations and comparing the performance that way.</div></div></div></div>
</div></div></blockquote><div><br></div><div>Excellent! Thanks for working on that!</div><div><br></div><div>cheers,</div><div>--renato</div></div></div></div>