[PATCH] [LNT] Add geometric mean to run report

[Chris Matthews]

Sometimes.  What time impact are you seeing?


On May 16, 2014, at 12:06 PM, Yi Kong <kongy.dev at gmail.com> wrote:

> Chris, does the LNT server have high work load? I'm testing graphing
> support for geomean, but it seems to be quite expensive for both
> processor and IO.
> 
> On 16 May 2014 18:50, Chris Matthews <chris.matthews at apple.com> wrote:
>> Patch LGTM.
>> 
>> On May 16, 2014, at 10:46 AM, Hal Finkel <hfinkel at anl.gov> wrote:
>> 
>> ----- Original Message -----
>> 
>> From: "Chris Matthews" <chris.matthews at apple.com>
>> To: "Hal Finkel" <hfinkel at anl.gov>
>> Cc: "Yi Kong" <kongy.dev at gmail.com>, llvm-commits at cs.uiuc.edu, "Anton
>> Korobeynikov" <anton at korobeynikov.info>
>> Sent: Friday, May 16, 2014 12:42:47 PM
>> Subject: Re: [PATCH] [LNT] Add geometric mean to run report
>> 
>> I wonder if we filter, if benchmarks that move around the filter
>> point, so that sometimes they are included, and other times not,
>> would cause problems in the geomean?
>> 
>> 
>> I'm not sure that it would cause too much of a problem; on the other hand,
>> Yi's proposal to add 1 to everything (and subtract it at the end) might be
>> the most numerically-stable adjustment.
>> 
>> -Hal
>> 
>> 
>> 
>> On May 16, 2014, at 10:39 AM, Chris Matthews
>> <chris.matthews at apple.com> wrote:
>> 
>> I agree that it would be better to filter them in some way.
>> 
>> I really like the idea of trying this though.
>> 
>> On May 16, 2014, at 10:37 AM, Hal Finkel <hfinkel at anl.gov> wrote:
>> 
>> ----- Original Message -----
>> 
>> From: "Hal Finkel" <hfinkel at anl.gov>
>> To: "Yi Kong" <kongy.dev at gmail.com>
>> Cc: llvm-commits at cs.uiuc.edu, "Chris Matthews"
>> <chris.matthews at apple.com>, "Anton Korobeynikov"
>> <anton at korobeynikov.info>
>> Sent: Friday, May 16, 2014 12:29:21 PM
>> Subject: Re: [PATCH] [LNT] Add geometric mean to run report
>> 
>> ----- Original Message -----
>> 
>> From: "Yi Kong" <kongy.dev at gmail.com>
>> To: "Chris Matthews" <chris.matthews at apple.com>, "Anton
>> Korobeynikov" <anton at korobeynikov.info>
>> Cc: llvm-commits at cs.uiuc.edu
>> Sent: Friday, May 16, 2014 12:08:54 PM
>> Subject: [PATCH] [LNT] Add geometric mean to run report
>> 
>> Hi all,
>> 
>> This patch implements geometric mean in run report.
>> 
>> 
>> You mean that this is for all tests in the entire test suite
>> aggregated into a single number?
>> 
>> 
>> The idea of geomean is to represent the entire data set and not
>> to
>> get
>> skewed by long running tests. However the major problem with it
>> is,
>> zero time is not allowed. I used a workaround by adding 1 to
>> each
>> value and subtract 1 from the final geomean result.  The
>> workaround
>> certainly break the geomean number, but we don't really care the
>> number itself, rather the change of it. It's certainly not the
>> ideal.
>> Is there any better solution or statistics we can use?
>> 
>> 
>> I don't understand this at all. Is the geometric mean not
>> (mul_i^n
>> a_i)^(1/n)?
>> 
>> 
>> On the other hand, I suppose if you include any zeros, then the
>> answer is always zero. Why don't you just exclude values less
>> than 0.1s? Maybe this is no better than what you've proposed.
>> 
>> -Hal
>> 
>> 
>> Thanks again,
>> Hal
>> 
>> 
>> Thanks,
>> Yi Kong
>> 
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>> 
>> 
>> --
>> Hal Finkel
>> Assistant Computational Scientist
>> Leadership Computing Facility
>> Argonne National Laboratory
>> 
>> 
>> --
>> Hal Finkel
>> Assistant Computational Scientist
>> Leadership Computing Facility
>> Argonne National Laboratory
>> 
>> 
>> 
>> 
>> 
>> --
>> Hal Finkel
>> Assistant Computational Scientist
>> Leadership Computing Facility
>> Argonne National Laboratory
>> 
>>