[LLVMdev] [LNT] Question about results reliability in LNT infrustructure

[Anton Korobeynikov]

> Getting 10 samples at different commits will give you similar accuracy if
> behaviour doesn't change, and you can rely on 10-point blocks before and > after each change to have the same result.
Right. But this way you will have 10-commits delay. So, you will need
3-4 additional test runs to pinpoint the offending commit in the worst
case.

> This is why I proposed something like moving averages.
Moving average will "smooth" the result. So, only really big changes
will be caught by it.

--
With best regards, Anton Korobeynikov
Faculty of Mathematics and Mechanics, Saint Petersburg State University