[www] r176209 - Add LNT statistics project

[David Tweed]

On 1 March 2013 16:31, David Tweed <David.Tweed at arm.com> wrote:

benchmarking is a science in the sense that once a user has decided what
goals/properties they consider important, then one set of statistical
procedure can be determined to be better at achieving those goals than
another.

 

| When you know the question, the answer is obvious. The art is knowing
which question is more important... ;)

 

So, my recent personal experience (and admittedly this has been
"microbenchmarking" in that I was looking for detecting small differences
due to different code generation strategies within an automated optimization
loop) is that the kind of data you get looks to have a "dense" distribution
from the starting point up to some level, with a few significantly bigger
outliers. Reanalysing one set of data for various code options, the
arithmetic mean varies from essentially the same as the median to up to 10%
(and in one case 15% larger). Personally I'm reasonably convinced  that the
using a trimmed deviation around a median is giving a better estimate of the
"inherently variable program execution" random variable, but that's just in
the context of the kind of stuff I've been doing.

 

Cheers,

Dave
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20130304/973784f4/attachment.html>