[lldb-dev] New test summary results formatter

[Todd Fiala via lldb-dev]

Yeah I'd be good with that.  I can change that as well.

-Todd

On Wed, Dec 2, 2015 at 11:10 AM, Zachary Turner <zturner at google.com> wrote:

> Also another stylistic suggestion.  I've been thinking about how to more
> logically organize all the source files now that we have a package.  So it
> makes sense conceptually to group all of the different result formatters
> under a subpackage called formatters.  So right now you've got
> lldbsuite.test.basic_results_formatter.BasicResultsFormatter but it might
> make sense for this to be
> lldbsuite.test.formatters.basic.BasicResultsFormatter.  If you do things
> this way, it can actually result in a substantially shorter command line,
> because the --results-formatter option can use lldbsuite.test.formatters as
> a starting point.  So you could instead write:
>
> test/dotest.py --results-formatter basic
>
> dotest then looks for a `basic.py` module in the
> `lldbsuite.test.formatters` package, looks for a class inside with a
> @result_formatter decorator, and instantiates that.
>
> This has the advantage of making the command line shorter *and* a more
> logical source file organization.
>
> On Wed, Dec 2, 2015 at 11:04 AM Zachary Turner <zturner at google.com> wrote:
>
>> Can --results-file=stdout be the default so that we don't have to specify
>> that?
>>
>> On Wed, Dec 2, 2015 at 11:02 AM Todd Fiala via lldb-dev <
>> lldb-dev at lists.llvm.org> wrote:
>>
>>> Also, all the text in the summary is fixed-width lined up nicely, which
>>> may not show in the commit message description if you're using a
>>> variable-width font.  On a terminal it looks nice.
>>>
>>> On Wed, Dec 2, 2015 at 11:01 AM, Todd Fiala <todd.fiala at gmail.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On Wed, Dec 2, 2015 at 10:57 AM, Todd Fiala <todd.fiala at gmail.com>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I just put up an optional test results formatter that is a prototype
>>>>> of what we may move towards for our default test summary results.  It went
>>>>> in here:
>>>>>
>>>>> r254530
>>>>>
>>>>> and you can try it out with something like:
>>>>>
>>>>> time test/dotest.py --executable `pwd`/build/Debug/lldb
>>>>> --results-formatter
>>>>> lldbsuite.test.basic_results_formatter.BasicResultsFormatter --results-file
>>>>> st
>>>>> out
>>>>>
>>>>>
>>>> I cut and paste my line, but more than likely for most people you'd
>>>> just want this:
>>>>
>>>> test/dotest.py --results-formatter
>>>> lldbsuite.test.basic_results_formatter.BasicResultsFormatter --results-file
>>>> stdout
>>>>
>>>> The other stuff was specific to my setup.  That line assumes you run
>>>> from the lldb source dir root.
>>>>
>>>>
>>>> Let me know if this satisfies the basic needs of counts and whatnot.
>>>>> It counts test method runs rather than all the oddball "file, class, etc."
>>>>> counts we had before.
>>>>>
>>>>> It prints out the Details section when there are details, and keeps it
>>>>> nice and clean when there are none.
>>>>>
>>>>> It also mentions a bit about test reruns up top, but that won't come
>>>>> into play until I get the multi-test-pass, single-worker/low-load mechanism
>>>>> in place, which will depend on newer rerun count awareness support.
>>>>>
>>>>> The change also cleans up places where the test event framework was
>>>>> using string codes and replaces them with symbolic constants.
>>>>>
>>>>> Let me know what you think.  I can tweak it as needed to address
>>>>> testbot and other needs.  Once it looks reasonable, I'd like to move over
>>>>> to using it by default in the parallel test runner rather than the legacy
>>>>> support.
>>>>>
>>>>> Thanks!
>>>>> --
>>>>> -Todd
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> -Todd
>>>>
>>>
>>>
>>>
>>> --
>>> -Todd
>>> _______________________________________________
>>> lldb-dev mailing list
>>> lldb-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>
>>


-- 
-Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20151202/2d403d40/attachment-0001.html>