[lldb-dev] New test summary results formatter

[Todd Fiala via lldb-dev]

I'll also move those couple counts (total tests run and the rerun count)
under the summary.  I missed that when evolving the code.

On Wed, Dec 2, 2015 at 11:20 AM, Todd Fiala <todd.fiala at gmail.com> wrote:

> Yeah I'd be good with that.  I can change that as well.
>
> -Todd
>
> On Wed, Dec 2, 2015 at 11:10 AM, Zachary Turner <zturner at google.com>
> wrote:
>
>> Also another stylistic suggestion.  I've been thinking about how to more
>> logically organize all the source files now that we have a package.  So it
>> makes sense conceptually to group all of the different result formatters
>> under a subpackage called formatters.  So right now you've got
>> lldbsuite.test.basic_results_formatter.BasicResultsFormatter but it
>> might make sense for this to be
>> lldbsuite.test.formatters.basic.BasicResultsFormatter.  If you do things
>> this way, it can actually result in a substantially shorter command line,
>> because the --results-formatter option can use lldbsuite.test.formatters as
>> a starting point.  So you could instead write:
>>
>> test/dotest.py --results-formatter basic
>>
>> dotest then looks for a `basic.py` module in the
>> `lldbsuite.test.formatters` package, looks for a class inside with a
>> @result_formatter decorator, and instantiates that.
>>
>> This has the advantage of making the command line shorter *and* a more
>> logical source file organization.
>>
>> On Wed, Dec 2, 2015 at 11:04 AM Zachary Turner <zturner at google.com>
>> wrote:
>>
>>> Can --results-file=stdout be the default so that we don't have to
>>> specify that?
>>>
>>> On Wed, Dec 2, 2015 at 11:02 AM Todd Fiala via lldb-dev <
>>> lldb-dev at lists.llvm.org> wrote:
>>>
>>>> Also, all the text in the summary is fixed-width lined up nicely, which
>>>> may not show in the commit message description if you're using a
>>>> variable-width font.  On a terminal it looks nice.
>>>>
>>>> On Wed, Dec 2, 2015 at 11:01 AM, Todd Fiala <todd.fiala at gmail.com>
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Wed, Dec 2, 2015 at 10:57 AM, Todd Fiala <todd.fiala at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> I just put up an optional test results formatter that is a prototype
>>>>>> of what we may move towards for our default test summary results.  It went
>>>>>> in here:
>>>>>>
>>>>>> r254530
>>>>>>
>>>>>> and you can try it out with something like:
>>>>>>
>>>>>> time test/dotest.py --executable `pwd`/build/Debug/lldb
>>>>>> --results-formatter
>>>>>> lldbsuite.test.basic_results_formatter.BasicResultsFormatter --results-file
>>>>>> st
>>>>>> out
>>>>>>
>>>>>>
>>>>> I cut and paste my line, but more than likely for most people you'd
>>>>> just want this:
>>>>>
>>>>> test/dotest.py --results-formatter
>>>>> lldbsuite.test.basic_results_formatter.BasicResultsFormatter --results-file
>>>>> stdout
>>>>>
>>>>> The other stuff was specific to my setup.  That line assumes you run
>>>>> from the lldb source dir root.
>>>>>
>>>>>
>>>>> Let me know if this satisfies the basic needs of counts and whatnot.
>>>>>> It counts test method runs rather than all the oddball "file, class, etc."
>>>>>> counts we had before.
>>>>>>
>>>>>> It prints out the Details section when there are details, and keeps
>>>>>> it nice and clean when there are none.
>>>>>>
>>>>>> It also mentions a bit about test reruns up top, but that won't come
>>>>>> into play until I get the multi-test-pass, single-worker/low-load mechanism
>>>>>> in place, which will depend on newer rerun count awareness support.
>>>>>>
>>>>>> The change also cleans up places where the test event framework was
>>>>>> using string codes and replaces them with symbolic constants.
>>>>>>
>>>>>> Let me know what you think.  I can tweak it as needed to address
>>>>>> testbot and other needs.  Once it looks reasonable, I'd like to move over
>>>>>> to using it by default in the parallel test runner rather than the legacy
>>>>>> support.
>>>>>>
>>>>>> Thanks!
>>>>>> --
>>>>>> -Todd
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> -Todd
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> -Todd
>>>> _______________________________________________
>>>> lldb-dev mailing list
>>>> lldb-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>>
>>>
>
>
> --
> -Todd
>



-- 
-Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20151202/3c573619/attachment.html>