[www] r176209 - Add LNT statistics project

[Renato Golin]

On 1 March 2013 11:47, David Tweed <david.tweed at arm.com> wrote:

> The trick, of course, is knowing when it's reasonable to discard a big
> value


Hi Dave,

It depends on the data distribution. I don't think there is a definite rule
that will apply to all cases.

What I've done in the past was to get mean+stdev, discard anything after N
stdev (eg. 4, then 3 if none) and re-calculate mean+stdev.

Fix percentage has little meaning if you don't know the distribution. You
could be discarding good values, which means you'll have to re-run the test
more times than needed just to get a good stdev.

And calculating ave+stdev twice is normally quicker than re-running a test,
so you can do it after every test until your stdev is at an acceptable
fraction of your mean (or you've hit the maximum number of runs).

cheers,
--renato
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20130301/64978fbe/attachment.html>