[lldb-dev] Heads up - test runner fix

[Todd Fiala]

The failures I currently get on a stock Ubuntu 14.04 x86_64 Desktop VM
running under Fusion 7.0 using TOT as of this morning are these:

Failing Tests (14)

FAIL: LLDB (suite) :: TestGdbRemoteProcessInfo.py (Linux vagrant
3.13.0-32-generic #57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64)

FAIL: LLDB (suite) :: TestGdbRemoteAttach.py (Linux vagrant
3.13.0-32-generic #57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64)

FAIL: LLDB (suite) :: TestLldbGdbServer.py (Linux vagrant 3.13.0-32-generic
#57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64)

FAIL: LLDB (suite) :: TestGdbRemoteKill.py (Linux vagrant 3.13.0-32-generic
#57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64)

FAIL: LLDB (suite) :: TestDataFormatterStdVector.py (Linux vagrant
3.13.0-32-generic #57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64)

FAIL: LLDB (suite) :: TestDataFormatterStdVBool.py (Linux vagrant
3.13.0-32-generic #57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64)

FAIL: LLDB (suite) :: TestDataFormatterStdMap.py (Linux vagrant
3.13.0-32-generic #57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64)

FAIL: LLDB (suite) :: TestDataFormatterSynthVal.py (Linux vagrant
3.13.0-32-generic #57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64)

FAIL: LLDB (suite) :: TestReportData.py (Linux vagrant 3.13.0-32-generic
#57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64)

FAIL: LLDB (suite) :: TestMemoryHistory.py (Linux vagrant 3.13.0-32-generic
#57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64)

FAIL: LLDB (suite) :: TestUniqueTypes.py (Linux vagrant 3.13.0-32-generic
#57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64)

FAIL: LLDB (suite) :: TestAnonymous.py (Linux vagrant 3.13.0-32-generic
#57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64)

FAIL: LLDB (suite) :: TestSharedLib.py (Linux vagrant 3.13.0-32-generic
#57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64)

FAIL: LLDB (suite) :: TestSharedLibStrippedSymbols.py (Linux vagrant
3.13.0-32-generic #57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64)


The first 4 or so are mine.

On Tue, Oct 14, 2014 at 9:35 AM, Todd Fiala <todd.fiala at gmail.com> wrote:

> For example, here is the set of test failures I'm getting on MacOSX 10.10
> GM Candidate 3, with the latest Xcode 6.1 beta:
>
> Ran 313 tests.
>
> Failing Tests (6)
>
> FAIL: LLDB (suite) :: TestFormatters.py (Darwin snappy-2.local 14.0.0
> Darwin Kernel Version 14.0.0: Tue Sep 23 23:01:41 PDT 2014;
> root:xnu-2782.1.97~6/RELEASE_X86_64 x86_64 i386)
>
> FAIL: LLDB (suite) :: TestCallWithTimeout.py (Darwin snappy-2.local 14.0.0
> Darwin Kernel Version 14.0.0: Tue Sep 23 23:01:41 PDT 2014;
> root:xnu-2782.1.97~6/RELEASE_X86_64 x86_64 i386)
>
> FAIL: LLDB (suite) :: TestMemoryHistory.py (Darwin snappy-2.local 14.0.0
> Darwin Kernel Version 14.0.0: Tue Sep 23 23:01:41 PDT 2014;
> root:xnu-2782.1.97~6/RELEASE_X86_64 x86_64 i386)
>
> FAIL: LLDB (suite) :: TestReportData.py (Darwin snappy-2.local 14.0.0
> Darwin Kernel Version 14.0.0: Tue Sep 23 23:01:41 PDT 2014;
> root:xnu-2782.1.97~6/RELEASE_X86_64 x86_64 i386)
>
> FAIL: LLDB (suite) :: TestDataFormatterStdMap.py (Darwin snappy-2.local
> 14.0.0 Darwin Kernel Version 14.0.0: Tue Sep 23 23:01:41 PDT 2014;
> root:xnu-2782.1.97~6/RELEASE_X86_64 x86_64 i386)
>
> FAIL: LLDB (suite) :: TestDataFormatterStdVector.py (Darwin snappy-2.local
> 14.0.0 Darwin Kernel Version 14.0.0: Tue Sep 23 23:01:41 PDT 2014;
> root:xnu-2782.1.97~6/RELEASE_X86_64 x86_64 i386)
>
> On Tue, Oct 14, 2014 at 9:33 AM, Todd Fiala <todd.fiala at gmail.com> wrote:
>
>> > This is the accumulation of all build breaks
>>
>> test breaks, rather...
>>
>> On Tue, Oct 14, 2014 at 9:32 AM, Todd Fiala <todd.fiala at gmail.com> wrote:
>>
>>> Hey all,
>>>
>>> I have good news and bad news.
>>>
>>> Good: our test runners now report failures correctly.
>>> Bad news: our test runners have not been reporting failures since Aug 8
>>> 2014.
>>>
>>> As of svn r219689, dotest.py will once again report errors/failures via
>>> non-zero return value, which will then propagate correctly through test run
>>> results as reported by 'make test', 'ninja check-lldb', etc.
>>>
>>> You will almost certainly see test failures show up after r219689 that
>>> you were not seeing before.  This is the accumulation of all build breaks
>>> we essentially made starting with r215256.  We will need to fix the errors,
>>> not revert r219689.  (That would be akin to shooting the messenger).
>>>
>>> The lesson here is make sure we don't break our test runner, and ensure
>>> return values from the test runner are preserved on exit.  That is a hugely
>>> critical facility.  I will look into adding a test runner test so we don't
>>> break that again.
>>> --
>>> -Todd
>>>
>>
>>
>>
>> --
>> -Todd
>>
>
>
>
> --
> -Todd
>



-- 
-Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20141016/aeae5ef7/attachment.html>