[lldb-dev] test results look typical?

Tue Aug 25 09:14:20 PDT 2015

There's no need to do anything fancy (yet :) ). For initial diagnosis
the output of `./dotest.py $your_usual_options -p SomeLibcxxTest.py
-t` should suffice.

pl

On 25 August 2015 at 16:45, Todd Fiala <todd.fiala at gmail.com> wrote:
> Thanks, Pavel!  I'll dig that up and get back.
>
> On Tue, Aug 25, 2015 at 8:30 AM, Pavel Labath <labath at google.com> wrote:
>>
>> There is no separate option, it should just work. :)
>>
>> I'm betting you are still missing some package there (we should
>> document the prerequisites better). Could you send the error message
>> you are getting so we can have a look.
>>
>> cheers,
>> pl
>>
>>
>> On 25 August 2015 at 16:20, Todd Fiala via lldb-dev
>> <lldb-dev at lists.llvm.org> wrote:
>> >
>> >
>> > On Mon, Aug 24, 2015 at 4:11 PM, Todd Fiala <todd.fiala at gmail.com>
>> > wrote:
>> >>
>> >>
>> >>
>> >> On Mon, Aug 24, 2015 at 4:01 PM, Chaoren Lin <chaorenl at google.com>
>> >> wrote:
>> >>>
>> >>> The TestDataFormatterLibcc* tests require libc++-dev:
>> >>>
>> >>> $ sudo apt-get install libc++-dev
>> >>>
>> >>
>> >> Ah okay, so we are working with libc++ on Ubuntu, that's good to hear.
>> >> Pre-14.04 I gave up on it.
>> >>
>> >> Will cmake automatically choose libc++ if it is present?  Or do I need
>> >> to
>> >> pass something to cmake to use libc++?
>> >
>> >
>> > Hmm it appears I need to do more than just install libc++-dev.  I did a
>> > clean build with that installed, then ran the tests, and I still have
>> > the
>> > Libcxc/Libcxx tests failing.  Is there some flag expected, either to
>> > pass
>> > along for the compile options to dotest.py to override/specify which c++
>> > lib
>> > it is using?
>> >
>> >>
>> >>
>> >> Thanks, Chaoren!
>> >>
>> >> -Todd
>> >>
>> >>>
>> >>> On Mon, Aug 24, 2015 at 3:42 PM, Todd Fiala via lldb-dev
>> >>> <lldb-dev at lists.llvm.org> wrote:
>> >>>>
>> >>>>
>> >>>> On Mon, Aug 24, 2015 at 3:39 PM, Zachary Turner <zturner at google.com>
>> >>>> wrote:
>> >>>>>
>> >>>>> Can't comment on the failures for Linux, but I don't think we have a
>> >>>>> good handle on the unexpected successes.  I only added that
>> >>>>> information to
>> >>>>> the output about a week ago, before that unexpected successes were
>> >>>>> actually
>> >>>>> going unnoticed.
>> >>>>
>> >>>>
>> >>>> Okay, thanks Zachary.   A while back we had some flapping tests that
>> >>>> would oscillate between unexpected success and failure on Linux.
>> >>>> Some of
>> >>>> those might still be in that state but maybe (!) are fixed.
>> >>>>
>> >>>> Anyone on the Linux end who happens to know if the fails in
>> >>>> particular
>> >>>> look normal, that'd be good to know.
>> >>>>
>> >>>> Thanks!
>> >>>>
>> >>>>>
>> >>>>>
>> >>>>> It's likely that someone could just go in there and remove the XFAIL
>> >>>>> from those tests.
>> >>>>>
>> >>>>> On Mon, Aug 24, 2015 at 3:37 PM Todd Fiala via lldb-dev
>> >>>>> <lldb-dev at lists.llvm.org> wrote:
>> >>>>>>
>> >>>>>> Hi all,
>> >>>>>>
>> >>>>>> I'm just trying to get a handle on current lldb test failures
>> >>>>>> across
>> >>>>>> different platforms.
>> >>>>>>
>> >>>>>> On Linux on non-virtualized hardware, I currently see the failures
>> >>>>>> below on Ubuntu 14.04.2 using a setup like this:
>> >>>>>> * stock linker (ld.bfd),
>> >>>>>> * g++ 4.9.2
>> >>>>>> * cmake
>> >>>>>> * ninja
>> >>>>>> * libstdc++
>> >>>>>>
>> >>>>>> ninja check-lldb output:
>> >>>>>>
>> >>>>>> Ran 394 test suites (15 failed) (3.807107%)
>> >>>>>> Ran 474 test cases (17 failed) (3.586498%)
>> >>>>>> Failing Tests (15)
>> >>>>>> FAIL: LLDB (suite) :: TestCPPThis.py (Linux rad 3.13.0-57-generic
>> >>>>>> #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccIterator.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> >>>>>> x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccMap.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> >>>>>> x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccMultiMap.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> >>>>>> x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxMultiSet.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> >>>>>> x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxSet.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> >>>>>> x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxString.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> >>>>>> x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterSkipSummary.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> >>>>>> x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterUnordered.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> >>>>>> x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestMiGdbSetShowPrint.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> >>>>>> x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestRegisterVariables.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> >>>>>> x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestStaticVariables.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> >>>>>> x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestStepNoDebug.py (Linux rad
>> >>>>>> 3.13.0-57-generic
>> >>>>>> #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestTypedefArray.py (Linux rad
>> >>>>>> 3.13.0-57-generic
>> >>>>>> #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestVectorTypesFormatting.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> >>>>>> x86_64 x86_64)
>> >>>>>>
>> >>>>>> Unexpected Successes (10)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestBatchMode.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> >>>>>> x86_64 x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestEvents.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> >>>>>> x86_64 x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestExitDuringStep.py (Linux
>> >>>>>> rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> >>>>>> x86_64 x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestFdLeak.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> >>>>>> x86_64 x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestInferiorAssert.py (Linux
>> >>>>>> rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> >>>>>> x86_64 x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiGdbSetShow.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> >>>>>> x86_64 x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiInterpreterExec.py (Linux
>> >>>>>> rad 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> >>>>>> x86_64
>> >>>>>> x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiSyntax.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> >>>>>> x86_64 x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestRaise.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> >>>>>> x86_64 x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestStubSetSID.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> >>>>>> x86_64 x86_64)
>> >>>>>> ninja: build stopped: subcommand failed.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> On a similar setup, although bumped up to Ubuntu 14.04.3 and now on
>> >>>>>> a
>> >>>>>> VMWare VM, everything else the same, I see a similar report from
>> >>>>>> 'ninja
>> >>>>>> check-lldb':
>> >>>>>>
>> >>>>>> Ran 394 test suites (17 failed) (4.314721%)
>> >>>>>> Ran 474 test cases (19 failed) (4.008439%)
>> >>>>>> Failing Tests (17)
>> >>>>>> FAIL: LLDB (suite) :: TestAttachResume.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> >>>>>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestCPPThis.py (Linux lldb 3.19.0-26-generic
>> >>>>>> #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccIterator.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> >>>>>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccMap.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> >>>>>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccMultiMap.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> >>>>>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxMultiSet.py (Linux
>> >>>>>> lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> >>>>>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxSet.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> >>>>>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxString.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> >>>>>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterSkipSummary.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> >>>>>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterUnordered.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> >>>>>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestMiGdbSetShowPrint.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> >>>>>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestRegisterVariables.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> >>>>>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestStaticVariables.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> >>>>>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestStepNoDebug.py (Linux lldb
>> >>>>>> 3.19.0-26-generic
>> >>>>>> #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestStepOverWatchpoint.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> >>>>>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestTypedefArray.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> >>>>>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestVectorTypesFormatting.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> >>>>>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>>
>> >>>>>> Unexpected Successes (11)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestBatchMode.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> >>>>>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestEvents.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> >>>>>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestExitDuringStep.py (Linux
>> >>>>>> lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> >>>>>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestFdLeak.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> >>>>>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestInferiorAssert.py (Linux
>> >>>>>> lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> >>>>>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiGdbSetShow.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> >>>>>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiInterpreterExec.py (Linux
>> >>>>>> lldb 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17
>> >>>>>> UTC 2015
>> >>>>>> x86_64 x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiSyntax.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> >>>>>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestRaise.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> >>>>>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestStubSetSID.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> >>>>>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestWatchedVarHitWhenInScope.py
>> >>>>>> (Linux lldb 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>> >>>>>> 14:09:17 UTC
>> >>>>>> 2015 x86_64 x86_64)
>> >>>>>> ninja: build stopped: subcommand failed.
>> >>>>>>
>> >>>>>>
>> >>>>>> Do these more or less match the current state of affairs for Linux
>> >>>>>> tests?  If not, are there any suggestions on best practices for
>> >>>>>> upgrades?  A
>> >>>>>> while back I had tried things like clang and libc++ but I had
>> >>>>>> arrived at the
>> >>>>>> above setup (albeit typically with the gold linker) simply due to
>> >>>>>> what
>> >>>>>> worked, what was easy to debug and what built fast.
>> >>>>>>
>> >>>>>> Thanks for any confirmation on that!
>> >>>>>>
>> >>>>>> --
>> >>>>>> -Todd
>> >>>>>> _______________________________________________
>> >>>>>> lldb-dev mailing list
>> >>>>>> lldb-dev at lists.llvm.org
>> >>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> -Todd
>> >>>>
>> >>>> _______________________________________________
>> >>>> lldb-dev mailing list
>> >>>> lldb-dev at lists.llvm.org
>> >>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>> >>>>
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> -Todd
>> >
>> >
>> >
>> >
>> > --
>> > -Todd
>> >
>> > _______________________________________________
>> > lldb-dev mailing list
>> > lldb-dev at lists.llvm.org
>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>> >
>
>
>
>
> --
> -Todd