[LLVMdev] Why is the default LNT aggregation function min instead of mean
tobias at grosser.es
Fri Jan 17 16:57:53 PST 2014
On 01/17/2014 08:58 AM, Chris Matthews wrote:
> Is it the case that you converge on the min faster than the mean?
Sorry, I do not fully understand what you mean here. What exactly would
I need to do to check this? Should I just pick a couple of test/run
pairs and see after how many samples the min/mean does not change any more?
What conclusion can I take from this?
> Right now there is no way to set a per-tester aggregation function.
> I had spent a little time trying to detect regressions using k-means clustering. It looked promising. That was outside LNT though.
Interesting idea and that would most likely be helpful in configurations
where we actually get run-times clustered in different groups. I was
initially assuming this, but after Chandlers comments I have the feeling
we actually only have a single cluster where the statements are just
grouped due to the limited resolution of the analysis.
More information about the llvm-dev