[PATCH] [LNT] Use Mann-Whitney U test to identify changes

[Renato Golin]

On 2 May 2014 10:32, Yi Kong <Yi.Kong at arm.com> wrote:
> By looking at Mann-Whitney test, seems it requires data group to be
> distributed in the same way, so your suggestion probably won't work.
> Maybe we need to ask Anton to explain.

What I understood is that the MWW doesn't expect the data to be
generated with any order or around any mean value (non-parametric), so
this shouldn't be a problem.


> I do have a local installation on my desktop, but it is way too noisy
> compared to ones Tobias is running. Thus the results are not very
> representative. I'm trying to set up an ARM Highbank board at the moment.

This is why he needed the confidence level so high. ;) I think this
should be a parameter per buildbot, but I'm not sure where one would
set that up.


> Also can you share the results you are getting from your ARM runs? I am
> not sure what magnitude of noise you are seeing.

http://llvm.org/perf/db_default/v4/nts/machine/10

cheers,
--renato