[lldb-dev] test results look typical?

Tue Aug 25 08:30:40 PDT 2015

There is no separate option, it should just work. :)

I'm betting you are still missing some package there (we should
document the prerequisites better). Could you send the error message
you are getting so we can have a look.

cheers,
pl

On 25 August 2015 at 16:20, Todd Fiala via lldb-dev
<lldb-dev at lists.llvm.org> wrote:
>
>
> On Mon, Aug 24, 2015 at 4:11 PM, Todd Fiala <todd.fiala at gmail.com> wrote:
>>
>>
>>
>> On Mon, Aug 24, 2015 at 4:01 PM, Chaoren Lin <chaorenl at google.com> wrote:
>>>
>>> The TestDataFormatterLibcc* tests require libc++-dev:
>>>
>>> $ sudo apt-get install libc++-dev
>>>
>>
>> Ah okay, so we are working with libc++ on Ubuntu, that's good to hear.
>> Pre-14.04 I gave up on it.
>>
>> Will cmake automatically choose libc++ if it is present?  Or do I need to
>> pass something to cmake to use libc++?
>
>
> Hmm it appears I need to do more than just install libc++-dev.  I did a
> clean build with that installed, then ran the tests, and I still have the
> Libcxc/Libcxx tests failing.  Is there some flag expected, either to pass
> along for the compile options to dotest.py to override/specify which c++ lib
> it is using?
>
>>
>>
>> Thanks, Chaoren!
>>
>> -Todd
>>
>>>
>>> On Mon, Aug 24, 2015 at 3:42 PM, Todd Fiala via lldb-dev
>>> <lldb-dev at lists.llvm.org> wrote:
>>>>
>>>>
>>>> On Mon, Aug 24, 2015 at 3:39 PM, Zachary Turner <zturner at google.com>
>>>> wrote:
>>>>>
>>>>> Can't comment on the failures for Linux, but I don't think we have a
>>>>> good handle on the unexpected successes.  I only added that information to
>>>>> the output about a week ago, before that unexpected successes were actually
>>>>> going unnoticed.
>>>>
>>>>
>>>> Okay, thanks Zachary.   A while back we had some flapping tests that
>>>> would oscillate between unexpected success and failure on Linux.  Some of
>>>> those might still be in that state but maybe (!) are fixed.
>>>>
>>>> Anyone on the Linux end who happens to know if the fails in particular
>>>> look normal, that'd be good to know.
>>>>
>>>> Thanks!
>>>>
>>>>>
>>>>>
>>>>> It's likely that someone could just go in there and remove the XFAIL
>>>>> from those tests.
>>>>>
>>>>> On Mon, Aug 24, 2015 at 3:37 PM Todd Fiala via lldb-dev
>>>>> <lldb-dev at lists.llvm.org> wrote:
>>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> I'm just trying to get a handle on current lldb test failures across
>>>>>> different platforms.
>>>>>>
>>>>>> On Linux on non-virtualized hardware, I currently see the failures
>>>>>> below on Ubuntu 14.04.2 using a setup like this:
>>>>>> * stock linker (ld.bfd),
>>>>>> * g++ 4.9.2
>>>>>> * cmake
>>>>>> * ninja
>>>>>> * libstdc++
>>>>>>
>>>>>> ninja check-lldb output:
>>>>>>
>>>>>> Ran 394 test suites (15 failed) (3.807107%)
>>>>>> Ran 474 test cases (17 failed) (3.586498%)
>>>>>> Failing Tests (15)
>>>>>> FAIL: LLDB (suite) :: TestCPPThis.py (Linux rad 3.13.0-57-generic
>>>>>> #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccIterator.py (Linux rad
>>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccMap.py (Linux rad
>>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccMultiMap.py (Linux rad
>>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxMultiSet.py (Linux rad
>>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxSet.py (Linux rad
>>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxString.py (Linux rad
>>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> FAIL: LLDB (suite) :: TestDataFormatterSkipSummary.py (Linux rad
>>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> FAIL: LLDB (suite) :: TestDataFormatterUnordered.py (Linux rad
>>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> FAIL: LLDB (suite) :: TestMiGdbSetShowPrint.py (Linux rad
>>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> FAIL: LLDB (suite) :: TestRegisterVariables.py (Linux rad
>>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> FAIL: LLDB (suite) :: TestStaticVariables.py (Linux rad
>>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> FAIL: LLDB (suite) :: TestStepNoDebug.py (Linux rad 3.13.0-57-generic
>>>>>> #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> FAIL: LLDB (suite) :: TestTypedefArray.py (Linux rad 3.13.0-57-generic
>>>>>> #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> FAIL: LLDB (suite) :: TestVectorTypesFormatting.py (Linux rad
>>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>>
>>>>>> Unexpected Successes (10)
>>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestBatchMode.py (Linux rad
>>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestEvents.py (Linux rad
>>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestExitDuringStep.py (Linux rad
>>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestFdLeak.py (Linux rad
>>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestInferiorAssert.py (Linux rad
>>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiGdbSetShow.py (Linux rad
>>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiInterpreterExec.py (Linux
>>>>>> rad 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiSyntax.py (Linux rad
>>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestRaise.py (Linux rad
>>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestStubSetSID.py (Linux rad
>>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> ninja: build stopped: subcommand failed.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On a similar setup, although bumped up to Ubuntu 14.04.3 and now on a
>>>>>> VMWare VM, everything else the same, I see a similar report from 'ninja
>>>>>> check-lldb':
>>>>>>
>>>>>> Ran 394 test suites (17 failed) (4.314721%)
>>>>>> Ran 474 test cases (19 failed) (4.008439%)
>>>>>> Failing Tests (17)
>>>>>> FAIL: LLDB (suite) :: TestAttachResume.py (Linux lldb
>>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> FAIL: LLDB (suite) :: TestCPPThis.py (Linux lldb 3.19.0-26-generic
>>>>>> #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64 x86_64)
>>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccIterator.py (Linux lldb
>>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccMap.py (Linux lldb
>>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccMultiMap.py (Linux lldb
>>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxMultiSet.py (Linux lldb
>>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxSet.py (Linux lldb
>>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxString.py (Linux lldb
>>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> FAIL: LLDB (suite) :: TestDataFormatterSkipSummary.py (Linux lldb
>>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> FAIL: LLDB (suite) :: TestDataFormatterUnordered.py (Linux lldb
>>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> FAIL: LLDB (suite) :: TestMiGdbSetShowPrint.py (Linux lldb
>>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> FAIL: LLDB (suite) :: TestRegisterVariables.py (Linux lldb
>>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> FAIL: LLDB (suite) :: TestStaticVariables.py (Linux lldb
>>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> FAIL: LLDB (suite) :: TestStepNoDebug.py (Linux lldb 3.19.0-26-generic
>>>>>> #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64 x86_64)
>>>>>> FAIL: LLDB (suite) :: TestStepOverWatchpoint.py (Linux lldb
>>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> FAIL: LLDB (suite) :: TestTypedefArray.py (Linux lldb
>>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> FAIL: LLDB (suite) :: TestVectorTypesFormatting.py (Linux lldb
>>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>>
>>>>>> Unexpected Successes (11)
>>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestBatchMode.py (Linux lldb
>>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestEvents.py (Linux lldb
>>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestExitDuringStep.py (Linux lldb
>>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestFdLeak.py (Linux lldb
>>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestInferiorAssert.py (Linux lldb
>>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiGdbSetShow.py (Linux lldb
>>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiInterpreterExec.py (Linux
>>>>>> lldb 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
>>>>>> x86_64 x86_64)
>>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiSyntax.py (Linux lldb
>>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestRaise.py (Linux lldb
>>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestStubSetSID.py (Linux lldb
>>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestWatchedVarHitWhenInScope.py
>>>>>> (Linux lldb 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC
>>>>>> 2015 x86_64 x86_64)
>>>>>> ninja: build stopped: subcommand failed.
>>>>>>
>>>>>>
>>>>>> Do these more or less match the current state of affairs for Linux
>>>>>> tests?  If not, are there any suggestions on best practices for upgrades?  A
>>>>>> while back I had tried things like clang and libc++ but I had arrived at the
>>>>>> above setup (albeit typically with the gold linker) simply due to what
>>>>>> worked, what was easy to debug and what built fast.
>>>>>>
>>>>>> Thanks for any confirmation on that!
>>>>>>
>>>>>> --
>>>>>> -Todd
>>>>>> _______________________________________________
>>>>>> lldb-dev mailing list
>>>>>> lldb-dev at lists.llvm.org
>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> -Todd
>>>>
>>>> _______________________________________________
>>>> lldb-dev mailing list
>>>> lldb-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>>
>>>
>>
>>
>>
>> --
>> -Todd
>
>
>
>
> --
> -Todd
>
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>