[llvm-commits] [PATCH][LNT] Add support for selection of timing statistic (user or real time)

[Daniel Dunbar]

Hi David,

Sorry for review lag, I've tried to fix my filters so I see this stuff sooner.

The LNT side of the patch looks fine, but I think the test-suite side
would be substantially simplified if you just added an option to
timeit-target to report the real time instead of the user time as the
"program time". This would avoid needing most of the other changes.

Also, I can't imagine how your patch actually works given that it
deletes the --summary option, but the test suite still uses it?

 - Daniel

On Mon, Aug 6, 2012 at 2:54 PM, David Blaikie <dblaikie at gmail.com> wrote:
> Bump and...
>
> [+chandler]
>
> Chandler - you mentioned on IRC that you thought there might be a way
> to get higher-granularity user time in Linux? Any pointers/thoughts as
> to how that might be achieved? (I'm not sure that necessarily
> invalidates this change - it's probably going to be helpful to be able
> to plumb more than one result up through more layers of the test suite
> anyway even if this particular value is very valuable)
>
> On Mon, Jul 23, 2012 at 3:13 PM, David Dean <david_dean at apple.com> wrote:
>> These LGTM.
>>
>> On 23 Jul 2012, at 11:13 AM, David Blaikie <dblaikie at gmail.com> wrote:
>>
>>> +David Dean who's generously offered to take a look.
>>>
>>> Thanks David - 'preciate the help. I realize you might not have the
>>> context of some of the discussions I had with Daniel months ago about
>>> these timing issues - so certainly feel free to ask any questions
>>> about the direction/design/point of this change that I might've failed
>>> to provide in the original email (visible below) or patch.
>>>
>>> On Fri, Jul 20, 2012 at 11:46 AM, David Blaikie <dblaikie at gmail.com> wrote:
>>>> Ping.
>>>>
>>>> +Tobias, in case it interests him too
>>>> +Daniel's other email address, in case that helps
>>>>
>>>> Nearly the 3 month anniversary of this patch.
>>>>
>>>> On Mon, Jul 2, 2012 at 9:34 AM, David Blaikie <dblaikie at gmail.com> wrote:
>>>>> Bump. [5th ping over 2 months]
>>>>>
>>>>> Any feedback would be appreciated - it's by no means a huge change
>>>>> (both in size and scope/value) but will hopefully make it a little
>>>>> easier to investigate other timing properties (& paves the way for
>>>>> collecting more timing data up into LNT at some point, should anyone
>>>>> feel there's value in that).
>>>>>
>>>>> On Tue, Jun 5, 2012 at 10:17 AM, David Blaikie <dblaikie at gmail.com> wrote:
>>>>>> Once more, with feeling: Ping.
>>>>>>
>>>>>> On Wed, May 23, 2012 at 11:49 AM, David Blaikie <dblaikie at gmail.com> wrote:
>>>>>>> Ping.
>>>>>>>
>>>>>>> On Mon, May 14, 2012 at 8:54 AM, David Blaikie <dblaikie at gmail.com> wrote:
>>>>>>>> Ping.
>>>>>>>>
>>>>>>>> On Sun, Apr 29, 2012 at 9:43 PM, David Blaikie <dblaikie at gmail.com> wrote:
>>>>>>>>> Bump.
>>>>>>>>>
>>>>>>>>> On Mon, Apr 23, 2012 at 4:26 PM, David Blaikie <dblaikie at gmail.com> wrote:
>>>>>>>>>> Hi Daniel (& others),
>>>>>>>>>>
>>>>>>>>>> Here's a couple of patches that add an option to the LNT nt runner to
>>>>>>>>>> allow the selection of statistic to report (user time or real time) &
>>>>>>>>>> includes the necessary plumbing through the test-suite to propagate
>>>>>>>>>> this information (rather than passing a flag down through the
>>>>>>>>>> test-suite strata, this elevates both stats (we could add sys time for
>>>>>>>>>> symmetry at some point - and/or, more broadly, we could also stop
>>>>>>>>>> doing the aggregation (of multisource per-file stats into single
>>>>>>>>>> per-test stats) down in the test-suite & let LNT have more fine
>>>>>>>>>> grained information about the sub-parts of a test case's timing) up
>>>>>>>>>> into the simple report.
>>>>>>>>>>
>>>>>>>>>> [I removed one instance of the 'summary' parsing in timeit.c - it
>>>>>>>>>> seems there were two instances of it. If there's a reason for the
>>>>>>>>>> apparent duplication I'd love to hear it - then add it back in with a
>>>>>>>>>> comment explaining that. I also removed the 'program' timing there -
>>>>>>>>>> the reports currently can just deliberately pick which statistics they
>>>>>>>>>> want. It seems like only the nightly and simple reports were using
>>>>>>>>>> these values, so I've updated those. I was wondering if I could also
>>>>>>>>>> remove the 'exit' lines so the summaries just look like time(1)
>>>>>>>>>> output, but I haven't got there yet - the exit lines are used from
>>>>>>>>>> these files in some places. Perhaps there are other sources they could
>>>>>>>>>> use instead]
>>>>>>>>>>
>>>>>>>>>> Anyone have some thoughts on this approach or future work along these lines?
>>>>>>>>>>
>>>>>>>>>> - David
>>> <lnt-time-stat.diff><test-suite-time-stat.diff>
>>
>> -David
>>
>>