[lldb-dev] test results look typical?

Mon Aug 24 16:01:23 PDT 2015

The TestDataFormatterLibcc* tests require libc++-dev:

$ sudo apt-get install libc++-dev

On Mon, Aug 24, 2015 at 3:42 PM, Todd Fiala via lldb-dev <
lldb-dev at lists.llvm.org> wrote:

>
> On Mon, Aug 24, 2015 at 3:39 PM, Zachary Turner <zturner at google.com>
> wrote:
>
>> Can't comment on the failures for Linux, but I don't think we have a good
>> handle on the unexpected successes.  I only added that information to the
>> output about a week ago, before that unexpected successes were actually
>> going unnoticed.
>>
>
> Okay, thanks Zachary.   A while back we had some flapping tests that would
> oscillate between unexpected success and failure on Linux.  Some of those
> might still be in that state but maybe (!) are fixed.
>
> Anyone on the Linux end who happens to know if the fails in particular
> look normal, that'd be good to know.
>
> Thanks!
>
>
>>
>> It's likely that someone could just go in there and remove the XFAIL from
>> those tests.
>>
>> On Mon, Aug 24, 2015 at 3:37 PM Todd Fiala via lldb-dev <
>> lldb-dev at lists.llvm.org> wrote:
>>
>>> Hi all,
>>>
>>> I'm just trying to get a handle on current lldb test failures across
>>> different platforms.
>>>
>>> On Linux on non-virtualized hardware, I currently see the failures below
>>> on Ubuntu 14.04.2 using a setup like this:
>>> * stock linker (ld.bfd),
>>> * g++ 4.9.2
>>> * cmake
>>> * ninja
>>> * libstdc++
>>>
>>> ninja check-lldb output:
>>>
>>> Ran 394 test suites (15 failed) (3.807107%)
>>> Ran 474 test cases (17 failed) (3.586498%)
>>> Failing Tests (15)
>>> FAIL: LLDB (suite) :: TestCPPThis.py (Linux rad 3.13.0-57-generic
>>> #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestDataFormatterLibccIterator.py (Linux rad
>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestDataFormatterLibccMap.py (Linux rad
>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestDataFormatterLibccMultiMap.py (Linux rad
>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxMultiSet.py (Linux rad
>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxSet.py (Linux rad
>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxString.py (Linux rad
>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestDataFormatterSkipSummary.py (Linux rad
>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestDataFormatterUnordered.py (Linux rad
>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestMiGdbSetShowPrint.py (Linux rad
>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestRegisterVariables.py (Linux rad
>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestStaticVariables.py (Linux rad
>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestStepNoDebug.py (Linux rad 3.13.0-57-generic
>>> #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestTypedefArray.py (Linux rad 3.13.0-57-generic
>>> #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestVectorTypesFormatting.py (Linux rad
>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>
>>> Unexpected Successes (10)
>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestBatchMode.py (Linux rad
>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestEvents.py (Linux rad
>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestExitDuringStep.py (Linux rad
>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestFdLeak.py (Linux rad
>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestInferiorAssert.py (Linux rad
>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiGdbSetShow.py (Linux rad
>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiInterpreterExec.py (Linux rad
>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiSyntax.py (Linux rad
>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestRaise.py (Linux rad
>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestStubSetSID.py (Linux rad
>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>> ninja: build stopped: subcommand failed.
>>>
>>>
>>>
>>> On a similar setup, although bumped up to Ubuntu 14.04.3 and now on a
>>> VMWare VM, everything else the same, I see a similar report from 'ninja
>>> check-lldb':
>>>
>>> Ran 394 test suites (17 failed) (4.314721%)
>>> Ran 474 test cases (19 failed) (4.008439%)
>>> Failing Tests (17)
>>> FAIL: LLDB (suite) :: TestAttachResume.py (Linux lldb 3.19.0-26-generic
>>> #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestCPPThis.py (Linux lldb 3.19.0-26-generic
>>> #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestDataFormatterLibccIterator.py (Linux lldb
>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
>>> x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestDataFormatterLibccMap.py (Linux lldb
>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
>>> x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestDataFormatterLibccMultiMap.py (Linux lldb
>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
>>> x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxMultiSet.py (Linux lldb
>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
>>> x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxSet.py (Linux lldb
>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
>>> x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxString.py (Linux lldb
>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
>>> x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestDataFormatterSkipSummary.py (Linux lldb
>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
>>> x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestDataFormatterUnordered.py (Linux lldb
>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
>>> x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestMiGdbSetShowPrint.py (Linux lldb
>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
>>> x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestRegisterVariables.py (Linux lldb
>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
>>> x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestStaticVariables.py (Linux lldb
>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
>>> x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestStepNoDebug.py (Linux lldb 3.19.0-26-generic
>>> #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestStepOverWatchpoint.py (Linux lldb
>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
>>> x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestTypedefArray.py (Linux lldb 3.19.0-26-generic
>>> #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64 x86_64)
>>> FAIL: LLDB (suite) :: TestVectorTypesFormatting.py (Linux lldb
>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
>>> x86_64 x86_64)
>>>
>>> Unexpected Successes (11)
>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestBatchMode.py (Linux lldb
>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
>>> x86_64 x86_64)
>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestEvents.py (Linux lldb
>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
>>> x86_64 x86_64)
>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestExitDuringStep.py (Linux lldb
>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
>>> x86_64 x86_64)
>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestFdLeak.py (Linux lldb
>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
>>> x86_64 x86_64)
>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestInferiorAssert.py (Linux lldb
>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
>>> x86_64 x86_64)
>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiGdbSetShow.py (Linux lldb
>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
>>> x86_64 x86_64)
>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiInterpreterExec.py (Linux lldb
>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
>>> x86_64 x86_64)
>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiSyntax.py (Linux lldb
>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
>>> x86_64 x86_64)
>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestRaise.py (Linux lldb
>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
>>> x86_64 x86_64)
>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestStubSetSID.py (Linux lldb
>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
>>> x86_64 x86_64)
>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestWatchedVarHitWhenInScope.py
>>> (Linux lldb 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17
>>> UTC 2015 x86_64 x86_64)
>>> ninja: build stopped: subcommand failed.
>>>
>>>
>>> Do these more or less match the current state of affairs for Linux
>>> tests?  If not, are there any suggestions on best practices for upgrades?
>>> A while back I had tried things like clang and libc++ but I had arrived at
>>> the above setup (albeit typically with the gold linker) simply due to what
>>> worked, what was easy to debug and what built fast.
>>>
>>> Thanks for any confirmation on that!
>>>
>>> --
>>> -Todd
>>> _______________________________________________
>>> lldb-dev mailing list
>>> lldb-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>
>>
>
>
> --
> -Todd
>
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20150824/17d57850/attachment-0001.html>