[lldb-dev] RFC: Making unit tests run by default on ninja check-lldb
Ying Chen via lldb-dev
lldb-dev at lists.llvm.org
Wed Oct 21 10:01:13 PDT 2015
Yes, the output of dotest.py goes through LitTestCommand parse.
The parser is matching for "XPASS", but dotest output is using "UNEXPECTED
On Tue, Oct 20, 2015 at 6:34 PM, Todd Fiala <todd.fiala at gmail.com> wrote:
> Hi Ying,
> Our dotest.py lldb test results go through that lit test parser system? I
> see XPASS happen frequently (and in fact is my whole reason for starting a
> thread on getting rid of flakey tests, or making them run enough times so
> that their output can be a useful signal rather than useless). According
> to this script, an XPASS would be listed as failure. I'm not seeing us
> treat XPASS as failures AFAICT.
> Are we just saying that our gtests get processed by that?
> On Tue, Oct 20, 2015 at 4:51 PM, Ying Chen via lldb-dev <
> lldb-dev at lists.llvm.org> wrote:
>> Hi Zachary,
>> The big unknown here is how to make the buildbots understand unit test
>>> failures and trigger a failure when ninja check-lldb-unit fails.
>> There're two conditions buildbot will identity a test step as failure.
>> One is that the command has non-zero return code.
>> The other is that there're failing codes in stdout message. (Refer to
>> LitTestCommand::evaluateCommand in this file
>> Failing codes are defined as:
>> failingCodes = set(['FAIL', 'XPASS', 'KPASS', 'UNRESOLVED',
>> So if the failures are print out as '^FAIL: (.*) \(.*\)', buildbot will
>> understand it's failing even if ninja check-lldb-unit returns 0.
>> Or we could add some logic to the above file to handle the output of unit
>> lldb-dev mailing list
>> lldb-dev at lists.llvm.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the lldb-dev