[www] r176209 - Add LNT statistics project

[Renato Golin]

On 28 February 2013 11:20, David Tweed <david.tweed at arm.com> wrote:

> Hi Renato, sounds like a useful project to have. However, since unlike
> correctness regressions there's less time pressure, there are other
> statistical measures with greater resolving power you could use if you
> don't
> need to only consider the latest sample, eg, you can use change point
> detection tests between 5 samples and the next five samples. (Of course
> this
> will still only produce estimates of when there was a big perf change in
> one
> go, moving averages may be the way to deal with the "0.5% here, 0.5%, ...,
> soon it adds up to real slowdowns".
>

Hi David,

I agree that this is just the starting point. Once we have the
infrastructure in place we can go to great lengths to spot regressions.

My main focus now is to get rid of the 3000% regression/improvement reports
for sub-second benchmarks. Another approach would also be to make all
sub-second benchmarks run for longer on the fastest machine, to say at
least 10s, so that the noise would naturally become less important.

All ideas are welcome, feel free to add things to that project on the doc.
:D

cheers,
--renato
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20130228/88107157/attachment.html>