[lldb-dev] test results look typical?

Tue Aug 25 14:29:58 PDT 2015

On Tue, Aug 25, 2015 at 8:45 AM, Tamas Berghammer <tberghammer at google.com>
wrote:

> Hi Todd,
>
> I am using a clang-3.5 build release LLDB to debug an other clang-3.5
> build debug LLDB on Linux x86_64 and it works pretty well for me (works
> better then using GDB). The most issue I am hitting is around expression
> evaluation when I can't execute very small functions in std:: objects, but
> I can get around it with accessing the internal data representation
> (primarily for shared_ptr, unique_ptr and vector). We are still using gcc
> for compiling lldb-server for android because the android clang have some
> issues (atomic not supported) but I don't know anybody who testing a gcc
> built LLDB on Linux.
>
> Tamas
>
>
Okay, thanks for the details on your setup, Tamas!

-Todd

>
> On Tue, Aug 25, 2015 at 4:31 PM Pavel Labath via lldb-dev <
> lldb-dev at lists.llvm.org> wrote:
>
>> There is no separate option, it should just work. :)
>>
>> I'm betting you are still missing some package there (we should
>> document the prerequisites better). Could you send the error message
>> you are getting so we can have a look.
>>
>> cheers,
>> pl
>>
>>
>> On 25 August 2015 at 16:20, Todd Fiala via lldb-dev
>> <lldb-dev at lists.llvm.org> wrote:
>> >
>> >
>> > On Mon, Aug 24, 2015 at 4:11 PM, Todd Fiala <todd.fiala at gmail.com>
>> wrote:
>> >>
>> >>
>> >>
>> >> On Mon, Aug 24, 2015 at 4:01 PM, Chaoren Lin <chaorenl at google.com>
>> wrote:
>> >>>
>> >>> The TestDataFormatterLibcc* tests require libc++-dev:
>> >>>
>> >>> $ sudo apt-get install libc++-dev
>> >>>
>> >>
>> >> Ah okay, so we are working with libc++ on Ubuntu, that's good to hear.
>> >> Pre-14.04 I gave up on it.
>> >>
>> >> Will cmake automatically choose libc++ if it is present?  Or do I need
>> to
>> >> pass something to cmake to use libc++?
>> >
>> >
>> > Hmm it appears I need to do more than just install libc++-dev.  I did a
>> > clean build with that installed, then ran the tests, and I still have
>> the
>> > Libcxc/Libcxx tests failing.  Is there some flag expected, either to
>> pass
>> > along for the compile options to dotest.py to override/specify which
>> c++ lib
>> > it is using?
>> >
>> >>
>> >>
>> >> Thanks, Chaoren!
>> >>
>> >> -Todd
>> >>
>> >>>
>> >>> On Mon, Aug 24, 2015 at 3:42 PM, Todd Fiala via lldb-dev
>> >>> <lldb-dev at lists.llvm.org> wrote:
>> >>>>
>> >>>>
>> >>>> On Mon, Aug 24, 2015 at 3:39 PM, Zachary Turner <zturner at google.com>
>> >>>> wrote:
>> >>>>>
>> >>>>> Can't comment on the failures for Linux, but I don't think we have a
>> >>>>> good handle on the unexpected successes.  I only added that
>> information to
>> >>>>> the output about a week ago, before that unexpected successes were
>> actually
>> >>>>> going unnoticed.
>> >>>>
>> >>>>
>> >>>> Okay, thanks Zachary.   A while back we had some flapping tests that
>> >>>> would oscillate between unexpected success and failure on Linux.
>> Some of
>> >>>> those might still be in that state but maybe (!) are fixed.
>> >>>>
>> >>>> Anyone on the Linux end who happens to know if the fails in
>> particular
>> >>>> look normal, that'd be good to know.
>> >>>>
>> >>>> Thanks!
>> >>>>
>> >>>>>
>> >>>>>
>> >>>>> It's likely that someone could just go in there and remove the XFAIL
>> >>>>> from those tests.
>> >>>>>
>> >>>>> On Mon, Aug 24, 2015 at 3:37 PM Todd Fiala via lldb-dev
>> >>>>> <lldb-dev at lists.llvm.org> wrote:
>> >>>>>>
>> >>>>>> Hi all,
>> >>>>>>
>> >>>>>> I'm just trying to get a handle on current lldb test failures
>> across
>> >>>>>> different platforms.
>> >>>>>>
>> >>>>>> On Linux on non-virtualized hardware, I currently see the failures
>> >>>>>> below on Ubuntu 14.04.2 using a setup like this:
>> >>>>>> * stock linker (ld.bfd),
>> >>>>>> * g++ 4.9.2
>> >>>>>> * cmake
>> >>>>>> * ninja
>> >>>>>> * libstdc++
>> >>>>>>
>> >>>>>> ninja check-lldb output:
>> >>>>>>
>> >>>>>> Ran 394 test suites (15 failed) (3.807107%)
>> >>>>>> Ran 474 test cases (17 failed) (3.586498%)
>> >>>>>> Failing Tests (15)
>> >>>>>> FAIL: LLDB (suite) :: TestCPPThis.py (Linux rad 3.13.0-57-generic
>> >>>>>> #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccIterator.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccMap.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccMultiMap.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxMultiSet.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxSet.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxString.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterSkipSummary.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterUnordered.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestMiGdbSetShowPrint.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestRegisterVariables.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestStaticVariables.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestStepNoDebug.py (Linux rad
>> 3.13.0-57-generic
>> >>>>>> #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestTypedefArray.py (Linux rad
>> 3.13.0-57-generic
>> >>>>>> #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestVectorTypesFormatting.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> x86_64 x86_64)
>> >>>>>>
>> >>>>>> Unexpected Successes (10)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestBatchMode.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> x86_64 x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestEvents.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> x86_64 x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestExitDuringStep.py (Linux
>> rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> x86_64 x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestFdLeak.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> x86_64 x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestInferiorAssert.py (Linux
>> rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> x86_64 x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiGdbSetShow.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> x86_64 x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiInterpreterExec.py (Linux
>> >>>>>> rad 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> x86_64
>> >>>>>> x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiSyntax.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> x86_64 x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestRaise.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> x86_64 x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestStubSetSID.py (Linux rad
>> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015
>> x86_64 x86_64)
>> >>>>>> ninja: build stopped: subcommand failed.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> On a similar setup, although bumped up to Ubuntu 14.04.3 and now
>> on a
>> >>>>>> VMWare VM, everything else the same, I see a similar report from
>> 'ninja
>> >>>>>> check-lldb':
>> >>>>>>
>> >>>>>> Ran 394 test suites (17 failed) (4.314721%)
>> >>>>>> Ran 474 test cases (19 failed) (4.008439%)
>> >>>>>> Failing Tests (17)
>> >>>>>> FAIL: LLDB (suite) :: TestAttachResume.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestCPPThis.py (Linux lldb 3.19.0-26-generic
>> >>>>>> #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccIterator.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccMap.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccMultiMap.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxMultiSet.py (Linux
>> lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxSet.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxString.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterSkipSummary.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterUnordered.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestMiGdbSetShowPrint.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestRegisterVariables.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestStaticVariables.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestStepNoDebug.py (Linux lldb
>> 3.19.0-26-generic
>> >>>>>> #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64 x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestStepOverWatchpoint.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestTypedefArray.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> FAIL: LLDB (suite) :: TestVectorTypesFormatting.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>>
>> >>>>>> Unexpected Successes (11)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestBatchMode.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestEvents.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestExitDuringStep.py (Linux
>> lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestFdLeak.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestInferiorAssert.py (Linux
>> lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiGdbSetShow.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiInterpreterExec.py (Linux
>> >>>>>> lldb 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17
>> UTC 2015
>> >>>>>> x86_64 x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiSyntax.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestRaise.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestStubSetSID.py (Linux lldb
>> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>> 2015 x86_64
>> >>>>>> x86_64)
>> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestWatchedVarHitWhenInScope.py
>> >>>>>> (Linux lldb 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>> 14:09:17 UTC
>> >>>>>> 2015 x86_64 x86_64)
>> >>>>>> ninja: build stopped: subcommand failed.
>> >>>>>>
>> >>>>>>
>> >>>>>> Do these more or less match the current state of affairs for Linux
>> >>>>>> tests?  If not, are there any suggestions on best practices for
>> upgrades?  A
>> >>>>>> while back I had tried things like clang and libc++ but I had
>> arrived at the
>> >>>>>> above setup (albeit typically with the gold linker) simply due to
>> what
>> >>>>>> worked, what was easy to debug and what built fast.
>> >>>>>>
>> >>>>>> Thanks for any confirmation on that!
>> >>>>>>
>> >>>>>> --
>> >>>>>> -Todd
>> >>>>>> _______________________________________________
>> >>>>>> lldb-dev mailing list
>> >>>>>> lldb-dev at lists.llvm.org
>> >>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> -Todd
>> >>>>
>> >>>> _______________________________________________
>> >>>> lldb-dev mailing list
>> >>>> lldb-dev at lists.llvm.org
>> >>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>> >>>>
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> -Todd
>> >
>> >
>> >
>> >
>> > --
>> > -Todd
>> >
>> > _______________________________________________
>> > lldb-dev mailing list
>> > lldb-dev at lists.llvm.org
>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>> >
>> _______________________________________________
>> lldb-dev mailing list
>> lldb-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>
>

-- 
-Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20150825/0d2e1973/attachment-0001.html>