[PATCH] [LNT] Add geometric mean to run report

[Chris Matthews]

Patch LGTM.

On May 16, 2014, at 10:46 AM, Hal Finkel <hfinkel at anl.gov> wrote:

> ----- Original Message -----
>> From: "Chris Matthews" <chris.matthews at apple.com>
>> To: "Hal Finkel" <hfinkel at anl.gov>
>> Cc: "Yi Kong" <kongy.dev at gmail.com>, llvm-commits at cs.uiuc.edu, "Anton Korobeynikov" <anton at korobeynikov.info>
>> Sent: Friday, May 16, 2014 12:42:47 PM
>> Subject: Re: [PATCH] [LNT] Add geometric mean to run report
>> 
>> I wonder if we filter, if benchmarks that move around the filter
>> point, so that sometimes they are included, and other times not,
>> would cause problems in the geomean?
> 
> I'm not sure that it would cause too much of a problem; on the other hand, Yi's proposal to add 1 to everything (and subtract it at the end) might be the most numerically-stable adjustment.
> 
> -Hal
> 
>> 
>> 
>> On May 16, 2014, at 10:39 AM, Chris Matthews
>> <chris.matthews at apple.com> wrote:
>> 
>>> I agree that it would be better to filter them in some way.
>>> 
>>> I really like the idea of trying this though.
>>> 
>>> On May 16, 2014, at 10:37 AM, Hal Finkel <hfinkel at anl.gov> wrote:
>>> 
>>>> ----- Original Message -----
>>>>> From: "Hal Finkel" <hfinkel at anl.gov>
>>>>> To: "Yi Kong" <kongy.dev at gmail.com>
>>>>> Cc: llvm-commits at cs.uiuc.edu, "Chris Matthews"
>>>>> <chris.matthews at apple.com>, "Anton Korobeynikov"
>>>>> <anton at korobeynikov.info>
>>>>> Sent: Friday, May 16, 2014 12:29:21 PM
>>>>> Subject: Re: [PATCH] [LNT] Add geometric mean to run report
>>>>> 
>>>>> ----- Original Message -----
>>>>>> From: "Yi Kong" <kongy.dev at gmail.com>
>>>>>> To: "Chris Matthews" <chris.matthews at apple.com>, "Anton
>>>>>> Korobeynikov" <anton at korobeynikov.info>
>>>>>> Cc: llvm-commits at cs.uiuc.edu
>>>>>> Sent: Friday, May 16, 2014 12:08:54 PM
>>>>>> Subject: [PATCH] [LNT] Add geometric mean to run report
>>>>>> 
>>>>>> Hi all,
>>>>>> 
>>>>>> This patch implements geometric mean in run report.
>>>>> 
>>>>> You mean that this is for all tests in the entire test suite
>>>>> aggregated into a single number?
>>>>> 
>>>>>> 
>>>>>> The idea of geomean is to represent the entire data set and not
>>>>>> to
>>>>>> get
>>>>>> skewed by long running tests. However the major problem with it
>>>>>> is,
>>>>>> zero time is not allowed. I used a workaround by adding 1 to
>>>>>> each
>>>>>> value and subtract 1 from the final geomean result.  The
>>>>>> workaround
>>>>>> certainly break the geomean number, but we don't really care the
>>>>>> number itself, rather the change of it. It's certainly not the
>>>>>> ideal.
>>>>>> Is there any better solution or statistics we can use?
>>>>> 
>>>>> I don't understand this at all. Is the geometric mean not
>>>>> (mul_i^n
>>>>> a_i)^(1/n)?
>>>> 
>>>> On the other hand, I suppose if you include any zeros, then the
>>>> answer is always zero. Why don't you just exclude values less
>>>> than 0.1s? Maybe this is no better than what you've proposed.
>>>> 
>>>> -Hal
>>>> 
>>>>> 
>>>>> Thanks again,
>>>>> Hal
>>>>> 
>>>>>> 
>>>>>> Thanks,
>>>>>> Yi Kong
>>>>>> 
>>>>>> _______________________________________________
>>>>>> llvm-commits mailing list
>>>>>> llvm-commits at cs.uiuc.edu
>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>>>>>> 
>>>>> 
>>>>> --
>>>>> Hal Finkel
>>>>> Assistant Computational Scientist
>>>>> Leadership Computing Facility
>>>>> Argonne National Laboratory
>>>>> 
>>>> 
>>>> --
>>>> Hal Finkel
>>>> Assistant Computational Scientist
>>>> Leadership Computing Facility
>>>> Argonne National Laboratory
>>> 
>> 
>> 
> 
> -- 
> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20140516/415a0fe0/attachment.html>