[lldb-dev] BasicResultsFormatter - new test results summary

[Pavel Labath via lldb-dev]

Todd, I've had to disable the new result formatter as it was not
working with the expected timeout logic we have for the old one. The
old XTIMEOUT code is a massive hack and I will be extremely glad when
we get rid of it, but we can't keep our buildbot red until then, so
I've switched it off.

I am ready to start working on this, but I wanted to run this idea
here first. I thought we could have a test annotation like:
@expectedTimeout(oslist=["linux"], ...)

Then, when the child runner would encounter this annotation, it would
set a flag in the "test is starting" message indicating that this test
may time out. Then if the test really times out, the parent would know
about this, and it could avoid flagging the test as error.

Alternatively, if we want to avoid the proliferation test result
states, we could key this off the standard @expectedFailure
annotation, then a "time out" would become just another way it which a
test can fail, and XTIMEOUT would become XFAIL.

What do you think ?

pl

PS: I am pretty new to this part of code, so any pointers you have
towards implementing this would be extremely helpful.



On 10 December 2015 at 23:20, Todd Fiala <todd.fiala at gmail.com> wrote:
> Checked this in as r255310.  Let me know if you find any issues with that,
> Tamas.
>
> You will need '-v' to enable it.  Otherwise, it will just print the method
> name.
>
> -Todd
>
> On Thu, Dec 10, 2015 at 2:39 PM, Todd Fiala <todd.fiala at gmail.com> wrote:
>>
>> Sure, I can do that.
>>
>> Tamas, okay to give more detail on -v?  I'll give it a shot to see what
>> else comes out if we do that.
>>
>> -Todd
>>
>> On Thu, Dec 10, 2015 at 12:58 PM, Zachary Turner <zturner at google.com>
>> wrote:
>>>
>>>
>>>
>>> On Thu, Dec 10, 2015 at 12:54 PM Todd Fiala <todd.fiala at gmail.com> wrote:
>>>>
>>>> Hi Tamas,
>>>>
>>>>
>>>>
>>>> On Thu, Dec 10, 2015 at 2:52 AM, Tamas Berghammer
>>>> <tberghammer at google.com> wrote:
>>>>>
>>>>> HI Todd,
>>>>>
>>>>> You changed the way the test failure list is printed in a way that now
>>>>> we only print the name of the test function failing with the name of the
>>>>> test file in parenthesis. Can we add back the name of the test class to this
>>>>> list?
>>>>
>>>>
>>>> Sure.  I originally planned to have that in there but there was some
>>>> discussion about it being too much info.  I'm happy to add that back.
>>>
>>> Can we have it tied to verbosity level?  We have -t and -v, maybe one of
>>> those could trigger more detail in the summary view.
>>
>>
>>
>>
>> --
>> -Todd
>
>
>
>
> --
> -Todd