[PATCH] [LNT] Use Mann-Whitney U test to identify changes

[Renato Golin]

On 1 May 2014 22:27, Yi Kong <Yi.Kong at arm.com> wrote:
> This patch adds Mann-Whitney U tests to identify changes, as suggested by Tobias and Anton. User is able to configure the desired confidence level.

Hi Yi,

>From what I understood, this test will only check for disjoint groups,
so larger noise levels will still trigger differences. Also, if the
number of runs is less than 4, this test will not be ran, so ARM
benchmarks won't "benefit" from the check.

For runs with less than 4 data points, you could consider the last N
runs as one group, and the last N-1 runs + the current as another and
run the same check.

Have you used it on a perf installation locally? What's the impact?

cheers,
--renato