[lldb-dev] test results look typical?
Todd Fiala via lldb-dev
lldb-dev at lists.llvm.org
Tue Aug 25 08:45:15 PDT 2015
Thanks, Pavel! I'll dig that up and get back.
On Tue, Aug 25, 2015 at 8:30 AM, Pavel Labath <labath at google.com> wrote:
> There is no separate option, it should just work. :)
>
> I'm betting you are still missing some package there (we should
> document the prerequisites better). Could you send the error message
> you are getting so we can have a look.
>
> cheers,
> pl
>
>
> On 25 August 2015 at 16:20, Todd Fiala via lldb-dev
> <lldb-dev at lists.llvm.org> wrote:
> >
> >
> > On Mon, Aug 24, 2015 at 4:11 PM, Todd Fiala <todd.fiala at gmail.com>
> wrote:
> >>
> >>
> >>
> >> On Mon, Aug 24, 2015 at 4:01 PM, Chaoren Lin <chaorenl at google.com>
> wrote:
> >>>
> >>> The TestDataFormatterLibcc* tests require libc++-dev:
> >>>
> >>> $ sudo apt-get install libc++-dev
> >>>
> >>
> >> Ah okay, so we are working with libc++ on Ubuntu, that's good to hear.
> >> Pre-14.04 I gave up on it.
> >>
> >> Will cmake automatically choose libc++ if it is present? Or do I need
> to
> >> pass something to cmake to use libc++?
> >
> >
> > Hmm it appears I need to do more than just install libc++-dev. I did a
> > clean build with that installed, then ran the tests, and I still have the
> > Libcxc/Libcxx tests failing. Is there some flag expected, either to pass
> > along for the compile options to dotest.py to override/specify which c++
> lib
> > it is using?
> >
> >>
> >>
> >> Thanks, Chaoren!
> >>
> >> -Todd
> >>
> >>>
> >>> On Mon, Aug 24, 2015 at 3:42 PM, Todd Fiala via lldb-dev
> >>> <lldb-dev at lists.llvm.org> wrote:
> >>>>
> >>>>
> >>>> On Mon, Aug 24, 2015 at 3:39 PM, Zachary Turner <zturner at google.com>
> >>>> wrote:
> >>>>>
> >>>>> Can't comment on the failures for Linux, but I don't think we have a
> >>>>> good handle on the unexpected successes. I only added that
> information to
> >>>>> the output about a week ago, before that unexpected successes were
> actually
> >>>>> going unnoticed.
> >>>>
> >>>>
> >>>> Okay, thanks Zachary. A while back we had some flapping tests that
> >>>> would oscillate between unexpected success and failure on Linux.
> Some of
> >>>> those might still be in that state but maybe (!) are fixed.
> >>>>
> >>>> Anyone on the Linux end who happens to know if the fails in particular
> >>>> look normal, that'd be good to know.
> >>>>
> >>>> Thanks!
> >>>>
> >>>>>
> >>>>>
> >>>>> It's likely that someone could just go in there and remove the XFAIL
> >>>>> from those tests.
> >>>>>
> >>>>> On Mon, Aug 24, 2015 at 3:37 PM Todd Fiala via lldb-dev
> >>>>> <lldb-dev at lists.llvm.org> wrote:
> >>>>>>
> >>>>>> Hi all,
> >>>>>>
> >>>>>> I'm just trying to get a handle on current lldb test failures across
> >>>>>> different platforms.
> >>>>>>
> >>>>>> On Linux on non-virtualized hardware, I currently see the failures
> >>>>>> below on Ubuntu 14.04.2 using a setup like this:
> >>>>>> * stock linker (ld.bfd),
> >>>>>> * g++ 4.9.2
> >>>>>> * cmake
> >>>>>> * ninja
> >>>>>> * libstdc++
> >>>>>>
> >>>>>> ninja check-lldb output:
> >>>>>>
> >>>>>> Ran 394 test suites (15 failed) (3.807107%)
> >>>>>> Ran 474 test cases (17 failed) (3.586498%)
> >>>>>> Failing Tests (15)
> >>>>>> FAIL: LLDB (suite) :: TestCPPThis.py (Linux rad 3.13.0-57-generic
> >>>>>> #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccIterator.py (Linux rad
> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
> x86_64 x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccMap.py (Linux rad
> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
> x86_64 x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccMultiMap.py (Linux rad
> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
> x86_64 x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxMultiSet.py (Linux rad
> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
> x86_64 x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxSet.py (Linux rad
> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
> x86_64 x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxString.py (Linux rad
> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
> x86_64 x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterSkipSummary.py (Linux rad
> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
> x86_64 x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterUnordered.py (Linux rad
> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
> x86_64 x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestMiGdbSetShowPrint.py (Linux rad
> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
> x86_64 x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestRegisterVariables.py (Linux rad
> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
> x86_64 x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestStaticVariables.py (Linux rad
> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
> x86_64 x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestStepNoDebug.py (Linux rad
> 3.13.0-57-generic
> >>>>>> #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestTypedefArray.py (Linux rad
> 3.13.0-57-generic
> >>>>>> #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestVectorTypesFormatting.py (Linux rad
> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
> x86_64 x86_64)
> >>>>>>
> >>>>>> Unexpected Successes (10)
> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestBatchMode.py (Linux rad
> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
> x86_64 x86_64)
> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestEvents.py (Linux rad
> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
> x86_64 x86_64)
> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestExitDuringStep.py (Linux rad
> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
> x86_64 x86_64)
> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestFdLeak.py (Linux rad
> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
> x86_64 x86_64)
> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestInferiorAssert.py (Linux rad
> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
> x86_64 x86_64)
> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiGdbSetShow.py (Linux rad
> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
> x86_64 x86_64)
> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiInterpreterExec.py (Linux
> >>>>>> rad 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
> x86_64
> >>>>>> x86_64)
> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiSyntax.py (Linux rad
> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
> x86_64 x86_64)
> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestRaise.py (Linux rad
> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
> x86_64 x86_64)
> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestStubSetSID.py (Linux rad
> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
> x86_64 x86_64)
> >>>>>> ninja: build stopped: subcommand failed.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On a similar setup, although bumped up to Ubuntu 14.04.3 and now on
> a
> >>>>>> VMWare VM, everything else the same, I see a similar report from
> 'ninja
> >>>>>> check-lldb':
> >>>>>>
> >>>>>> Ran 394 test suites (17 failed) (4.314721%)
> >>>>>> Ran 474 test cases (19 failed) (4.008439%)
> >>>>>> Failing Tests (17)
> >>>>>> FAIL: LLDB (suite) :: TestAttachResume.py (Linux lldb
> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
> 2015 x86_64
> >>>>>> x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestCPPThis.py (Linux lldb 3.19.0-26-generic
> >>>>>> #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64 x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccIterator.py (Linux lldb
> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
> 2015 x86_64
> >>>>>> x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccMap.py (Linux lldb
> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
> 2015 x86_64
> >>>>>> x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccMultiMap.py (Linux lldb
> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
> 2015 x86_64
> >>>>>> x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxMultiSet.py (Linux lldb
> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
> 2015 x86_64
> >>>>>> x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxSet.py (Linux lldb
> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
> 2015 x86_64
> >>>>>> x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxString.py (Linux lldb
> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
> 2015 x86_64
> >>>>>> x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterSkipSummary.py (Linux lldb
> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
> 2015 x86_64
> >>>>>> x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterUnordered.py (Linux lldb
> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
> 2015 x86_64
> >>>>>> x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestMiGdbSetShowPrint.py (Linux lldb
> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
> 2015 x86_64
> >>>>>> x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestRegisterVariables.py (Linux lldb
> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
> 2015 x86_64
> >>>>>> x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestStaticVariables.py (Linux lldb
> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
> 2015 x86_64
> >>>>>> x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestStepNoDebug.py (Linux lldb
> 3.19.0-26-generic
> >>>>>> #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64 x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestStepOverWatchpoint.py (Linux lldb
> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
> 2015 x86_64
> >>>>>> x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestTypedefArray.py (Linux lldb
> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
> 2015 x86_64
> >>>>>> x86_64)
> >>>>>> FAIL: LLDB (suite) :: TestVectorTypesFormatting.py (Linux lldb
> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
> 2015 x86_64
> >>>>>> x86_64)
> >>>>>>
> >>>>>> Unexpected Successes (11)
> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestBatchMode.py (Linux lldb
> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
> 2015 x86_64
> >>>>>> x86_64)
> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestEvents.py (Linux lldb
> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
> 2015 x86_64
> >>>>>> x86_64)
> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestExitDuringStep.py (Linux
> lldb
> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
> 2015 x86_64
> >>>>>> x86_64)
> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestFdLeak.py (Linux lldb
> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
> 2015 x86_64
> >>>>>> x86_64)
> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestInferiorAssert.py (Linux
> lldb
> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
> 2015 x86_64
> >>>>>> x86_64)
> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiGdbSetShow.py (Linux lldb
> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
> 2015 x86_64
> >>>>>> x86_64)
> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiInterpreterExec.py (Linux
> >>>>>> lldb 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17
> UTC 2015
> >>>>>> x86_64 x86_64)
> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiSyntax.py (Linux lldb
> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
> 2015 x86_64
> >>>>>> x86_64)
> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestRaise.py (Linux lldb
> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
> 2015 x86_64
> >>>>>> x86_64)
> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestStubSetSID.py (Linux lldb
> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
> 2015 x86_64
> >>>>>> x86_64)
> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestWatchedVarHitWhenInScope.py
> >>>>>> (Linux lldb 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
> 14:09:17 UTC
> >>>>>> 2015 x86_64 x86_64)
> >>>>>> ninja: build stopped: subcommand failed.
> >>>>>>
> >>>>>>
> >>>>>> Do these more or less match the current state of affairs for Linux
> >>>>>> tests? If not, are there any suggestions on best practices for
> upgrades? A
> >>>>>> while back I had tried things like clang and libc++ but I had
> arrived at the
> >>>>>> above setup (albeit typically with the gold linker) simply due to
> what
> >>>>>> worked, what was easy to debug and what built fast.
> >>>>>>
> >>>>>> Thanks for any confirmation on that!
> >>>>>>
> >>>>>> --
> >>>>>> -Todd
> >>>>>> _______________________________________________
> >>>>>> lldb-dev mailing list
> >>>>>> lldb-dev at lists.llvm.org
> >>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> -Todd
> >>>>
> >>>> _______________________________________________
> >>>> lldb-dev mailing list
> >>>> lldb-dev at lists.llvm.org
> >>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> >>>>
> >>>
> >>
> >>
> >>
> >> --
> >> -Todd
> >
> >
> >
> >
> > --
> > -Todd
> >
> > _______________________________________________
> > lldb-dev mailing list
> > lldb-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> >
>
--
-Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20150825/c954a1a5/attachment-0001.html>
More information about the lldb-dev
mailing list