[lldb-dev] RFC: Making unit tests run by default on ninja check-lldb

[Todd Fiala via lldb-dev]

Hi Ying,

Our dotest.py lldb test results go through that lit test parser system?  I
see XPASS happen frequently (and in fact is my whole reason for starting a
thread on getting rid of flakey tests, or making them run enough times so
that their output can be a useful signal rather than useless).  According
to this script, an XPASS would be listed as failure.  I'm not seeing us
treat XPASS as failures AFAICT.

Are we just saying that our gtests get processed by that?

-Todd

On Tue, Oct 20, 2015 at 4:51 PM, Ying Chen via lldb-dev <
lldb-dev at lists.llvm.org> wrote:

> Hi Zachary,
>
> The big unknown here is how to make the buildbots understand unit test
>> failures and trigger a failure when ninja check-lldb-unit fails.
>>
>
> There're two conditions buildbot will identity a test step as failure.
> One is that the command has non-zero return code.
> The other is that there're failing codes in stdout message. (Refer to
> LitTestCommand::evaluateCommand in this file
> <https://github.com/llvm-mirror/zorg/blob/master/zorg/buildbot/commands/LitTestCommand.py>
> .)
> Failing codes are defined as:
>     failingCodes = set(['FAIL', 'XPASS', 'KPASS', 'UNRESOLVED', 'TIMEOUT'])
>
> So if the failures are print out as '^FAIL: (.*) \(.*\)', buildbot will
> understand it's failing even if ninja check-lldb-unit returns 0.
> Or we could add some logic to the above file to handle the output of unit
> test.
>
> Thanks,
> Ying
>
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
>


-- 
-Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20151020/697f6250/attachment.html>