[lldb-dev] RFC: Making unit tests run by default on ninja check-lldb
Todd Fiala via lldb-dev
lldb-dev at lists.llvm.org
Wed Oct 21 22:39:26 PDT 2015
Oh haha okay. :-)
Thanks for explaining, Ying!
On Wed, Oct 21, 2015 at 10:01 AM, Ying Chen <chying at google.com> wrote:
> Yes, the output of dotest.py goes through LitTestCommand parse.
> The parser is matching for "XPASS", but dotest output is using "UNEXPECTED
> SUCCESS". :)
> On Tue, Oct 20, 2015 at 6:34 PM, Todd Fiala <todd.fiala at gmail.com> wrote:
>> Hi Ying,
>> Our dotest.py lldb test results go through that lit test parser system?
>> I see XPASS happen frequently (and in fact is my whole reason for starting
>> a thread on getting rid of flakey tests, or making them run enough times so
>> that their output can be a useful signal rather than useless). According
>> to this script, an XPASS would be listed as failure. I'm not seeing us
>> treat XPASS as failures AFAICT.
>> Are we just saying that our gtests get processed by that?
>> On Tue, Oct 20, 2015 at 4:51 PM, Ying Chen via lldb-dev <
>> lldb-dev at lists.llvm.org> wrote:
>>> Hi Zachary,
>>> The big unknown here is how to make the buildbots understand unit test
>>>> failures and trigger a failure when ninja check-lldb-unit fails.
>>> There're two conditions buildbot will identity a test step as failure.
>>> One is that the command has non-zero return code.
>>> The other is that there're failing codes in stdout message. (Refer to
>>> LitTestCommand::evaluateCommand in this file
>>> Failing codes are defined as:
>>> failingCodes = set(['FAIL', 'XPASS', 'KPASS', 'UNRESOLVED',
>>> So if the failures are print out as '^FAIL: (.*) \(.*\)', buildbot will
>>> understand it's failing even if ninja check-lldb-unit returns 0.
>>> Or we could add some logic to the above file to handle the output of
>>> unit test.
>>> lldb-dev mailing list
>>> lldb-dev at lists.llvm.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the lldb-dev