[lldb-dev] test results look typical?

Todd Fiala via lldb-dev lldb-dev at lists.llvm.org
Tue Aug 25 16:17:05 PDT 2015 

One more data point:

Building/testing on Ubuntu 14.04.3 built with clang-3.6 and the ld.gold
linker yielded the following test results, bringing me down to a single
failure (and was 1.6x faster than a Debug build with gcc-4.9 and ld.bfd, 12
GB RAM and 6 cores allocated):

Failing Tests (1)
FAIL: LLDB (suite) :: TestStepOverWatchpoint.py (Linux lldb
3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
x86_64 x86_64)

Unexpected Successes (12)
UNEXPECTED SUCCESS: LLDB (suite) :: TestBatchMode.py (Linux lldb
3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
x86_64 x86_64)
UNEXPECTED SUCCESS: LLDB (suite) :: TestConstVariables.py (Linux lldb
3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
x86_64 x86_64)
UNEXPECTED SUCCESS: LLDB (suite) :: TestEvents.py (Linux lldb
3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
x86_64 x86_64)
UNEXPECTED SUCCESS: LLDB (suite) :: TestExitDuringStep.py (Linux lldb
3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
x86_64 x86_64)
UNEXPECTED SUCCESS: LLDB (suite) :: TestFdLeak.py (Linux lldb
3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
x86_64 x86_64)
UNEXPECTED SUCCESS: LLDB (suite) :: TestInferiorAssert.py (Linux lldb
3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
x86_64 x86_64)
UNEXPECTED SUCCESS: LLDB (suite) :: TestMiGdbSetShow.py (Linux lldb
3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
x86_64 x86_64)
UNEXPECTED SUCCESS: LLDB (suite) :: TestMiInterpreterExec.py (Linux lldb
3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
x86_64 x86_64)
UNEXPECTED SUCCESS: LLDB (suite) :: TestMiSyntax.py (Linux lldb
3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
x86_64 x86_64)
UNEXPECTED SUCCESS: LLDB (suite) :: TestRaise.py (Linux lldb
3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
x86_64 x86_64)
UNEXPECTED SUCCESS: LLDB (suite) :: TestStubSetSID.py (Linux lldb
3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
x86_64 x86_64)
UNEXPECTED SUCCESS: LLDB (suite) :: TestWatchedVarHitWhenInScope.py (Linux
lldb 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
x86_64 x86_64)


I'm not yet sure if that's stable, but it's what I'm seeing on my VM.

-Todd

On Tue, Aug 25, 2015 at 1:56 PM, Todd Fiala <todd.fiala at gmail.com> wrote:

> So specifying CC=/usr/bin/gcc CXX=/usr/bin/g++ cmake -GNinja ...
>
> did the trick for getting rid of the libc++ issues.  I think I may try to
> see if we can get those tests to make a run-time check to see if the
> inferior is linked against libc++, and if not, to skip it.  We can have
> lldb do it by looking at the image list.  Sound reasonable?  That seems
> more fool-proof than guessing based on the compiler.
>
> An alternative I considered and probably also might be valid to do anyway
> for cases where we look at the compiler binary is to fully resolve symbolic
> links before making decisions based on the binary.
>
> Thoughts?
>
> Separately, with the tests correctly seeing gcc now, I am down to the
> following errors:
>
> Ran 394 test suites (5 failed) (1.269036%)
> Ran 451 test cases (5 failed) (1.108647%)
> Failing Tests (5)
> FAIL: LLDB (suite) :: TestExitDuringStep.py (Linux lldb 3.19.0-26-generic
> #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64 x86_64)
> FAIL: LLDB (suite) :: TestNumThreads.py (Linux lldb 3.19.0-26-generic
> #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64 x86_64)
> FAIL: LLDB (suite) :: TestRegisterVariables.py (Linux lldb
> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
> x86_64 x86_64)
> FAIL: LLDB (suite) :: TestStepOverWatchpoint.py (Linux lldb
> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
> x86_64 x86_64)
> FAIL: LLDB (suite) :: TestThreadExit.py (Linux lldb 3.19.0-26-generic
> #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64 x86_64)
>
> Unexpected Successes (10)
> UNEXPECTED SUCCESS: LLDB (suite) :: TestBatchMode.py (Linux lldb
> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
> x86_64 x86_64)
> UNEXPECTED SUCCESS: LLDB (suite) :: TestEvents.py (Linux lldb
> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
> x86_64 x86_64)
> UNEXPECTED SUCCESS: LLDB (suite) :: TestFdLeak.py (Linux lldb
> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
> x86_64 x86_64)
> UNEXPECTED SUCCESS: LLDB (suite) :: TestInferiorAssert.py (Linux lldb
> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
> x86_64 x86_64)
> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiGdbSetShow.py (Linux lldb
> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
> x86_64 x86_64)
> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiInterpreterExec.py (Linux lldb
> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
> x86_64 x86_64)
> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiSyntax.py (Linux lldb
> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
> x86_64 x86_64)
> UNEXPECTED SUCCESS: LLDB (suite) :: TestRaise.py (Linux lldb
> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
> x86_64 x86_64)
> UNEXPECTED SUCCESS: LLDB (suite) :: TestStubSetSID.py (Linux lldb
> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
> x86_64 x86_64)
> UNEXPECTED SUCCESS: LLDB (suite) :: TestWatchedVarHitWhenInScope.py (Linux
> lldb 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
> x86_64 x86_64)
>
>
> Some of those failures look like old friends that were failing a year
> ago.  Can anybody tell me anything about those failures on Linux?  Are they
> being looked at?  Any hunches at to what is wrong?
>
> Thanks!
>
> -Todd
>
> On Tue, Aug 25, 2015 at 10:04 AM, Todd Fiala <todd.fiala at gmail.com> wrote:
>
>> Okay.
>>
>> So the culprit then is that I'm using:
>> cmake -GNinja ../llvm
>>
>> with one extra flag for build type.  And cmake is then just choosing
>> /usr/bin/cc.
>>
>> We could improve this by having the compiler symbolic links fully
>> resolved:
>> /usr/bin/cc -> /etc/alternatives/cc -> /usr/bin/gcc, which would have
>> then caught that it doesn't support libc++.
>>
>> Couldn't we use gcc with libc++?  (i.e. is it sufficient to assume we
>> don't have libc++ if we're using gcc?)  I have never tried that combo but I
>> don't know that it is impossible.  (After all, I just added libc++-dev to
>> the system, which presumably I can link against).
>>
>> On Tue, Aug 25, 2015 at 9:48 AM, Tamas Berghammer <tberghammer at google.com
>> > wrote:
>>
>>> In theory the test should be skipped when you are using gcc (cc is an
>>> alias for it) but we detect the type of the compiler based on the
>>> executable name and in case of cc we don't recognize that it is a gcc, so
>>> we don't skip the test.
>>>
>>> On Tue, Aug 25, 2015 at 5:45 PM Chaoren Lin via lldb-dev <
>>> lldb-dev at lists.llvm.org> wrote:
>>>
>>>> You're using CC="/usr/bin/cc". It needs to be clang for USE_LIBCPP to
>>>> do anything. :/
>>>>
>>>> On Tue, Aug 25, 2015 at 9:20 AM, Todd Fiala <todd.fiala at gmail.com>
>>>> wrote:
>>>>
>>>>> Here are a couple of the failures that came up (the log output from
>>>>> the full dosep.py run).
>>>>>
>>>>> Let me know if that is not sufficient!
>>>>>
>>>>> On Tue, Aug 25, 2015 at 9:14 AM, Pavel Labath <labath at google.com>
>>>>> wrote:
>>>>>
>>>>>> There's no need to do anything fancy (yet :) ). For initial diagnosis
>>>>>> the output of `./dotest.py $your_usual_options -p SomeLibcxxTest.py
>>>>>> -t` should suffice.
>>>>>>
>>>>>> pl
>>>>>>
>>>>>> On 25 August 2015 at 16:45, Todd Fiala <todd.fiala at gmail.com> wrote:
>>>>>> > Thanks, Pavel!  I'll dig that up and get back.
>>>>>> >
>>>>>> > On Tue, Aug 25, 2015 at 8:30 AM, Pavel Labath <labath at google.com>
>>>>>> wrote:
>>>>>> >>
>>>>>> >> There is no separate option, it should just work. :)
>>>>>> >>
>>>>>> >> I'm betting you are still missing some package there (we should
>>>>>> >> document the prerequisites better). Could you send the error
>>>>>> message
>>>>>> >> you are getting so we can have a look.
>>>>>> >>
>>>>>> >> cheers,
>>>>>> >> pl
>>>>>> >>
>>>>>> >>
>>>>>> >> On 25 August 2015 at 16:20, Todd Fiala via lldb-dev
>>>>>> >> <lldb-dev at lists.llvm.org> wrote:
>>>>>> >> >
>>>>>> >> >
>>>>>> >> > On Mon, Aug 24, 2015 at 4:11 PM, Todd Fiala <
>>>>>> todd.fiala at gmail.com>
>>>>>> >> > wrote:
>>>>>> >> >>
>>>>>> >> >>
>>>>>> >> >>
>>>>>> >> >> On Mon, Aug 24, 2015 at 4:01 PM, Chaoren Lin <
>>>>>> chaorenl at google.com>
>>>>>> >> >> wrote:
>>>>>> >> >>>
>>>>>> >> >>> The TestDataFormatterLibcc* tests require libc++-dev:
>>>>>> >> >>>
>>>>>> >> >>> $ sudo apt-get install libc++-dev
>>>>>> >> >>>
>>>>>> >> >>
>>>>>> >> >> Ah okay, so we are working with libc++ on Ubuntu, that's good
>>>>>> to hear.
>>>>>> >> >> Pre-14.04 I gave up on it.
>>>>>> >> >>
>>>>>> >> >> Will cmake automatically choose libc++ if it is present?  Or do
>>>>>> I need
>>>>>> >> >> to
>>>>>> >> >> pass something to cmake to use libc++?
>>>>>> >> >
>>>>>> >> >
>>>>>> >> > Hmm it appears I need to do more than just install libc++-dev.
>>>>>> I did a
>>>>>> >> > clean build with that installed, then ran the tests, and I still
>>>>>> have
>>>>>> >> > the
>>>>>> >> > Libcxc/Libcxx tests failing.  Is there some flag expected,
>>>>>> either to
>>>>>> >> > pass
>>>>>> >> > along for the compile options to dotest.py to override/specify
>>>>>> which c++
>>>>>> >> > lib
>>>>>> >> > it is using?
>>>>>> >> >
>>>>>> >> >>
>>>>>> >> >>
>>>>>> >> >> Thanks, Chaoren!
>>>>>> >> >>
>>>>>> >> >> -Todd
>>>>>> >> >>
>>>>>> >> >>>
>>>>>> >> >>> On Mon, Aug 24, 2015 at 3:42 PM, Todd Fiala via lldb-dev
>>>>>> >> >>> <lldb-dev at lists.llvm.org> wrote:
>>>>>> >> >>>>
>>>>>> >> >>>>
>>>>>> >> >>>> On Mon, Aug 24, 2015 at 3:39 PM, Zachary Turner <
>>>>>> zturner at google.com>
>>>>>> >> >>>> wrote:
>>>>>> >> >>>>>
>>>>>> >> >>>>> Can't comment on the failures for Linux, but I don't think
>>>>>> we have a
>>>>>> >> >>>>> good handle on the unexpected successes.  I only added that
>>>>>> >> >>>>> information to
>>>>>> >> >>>>> the output about a week ago, before that unexpected
>>>>>> successes were
>>>>>> >> >>>>> actually
>>>>>> >> >>>>> going unnoticed.
>>>>>> >> >>>>
>>>>>> >> >>>>
>>>>>> >> >>>> Okay, thanks Zachary.   A while back we had some flapping
>>>>>> tests that
>>>>>> >> >>>> would oscillate between unexpected success and failure on
>>>>>> Linux.
>>>>>> >> >>>> Some of
>>>>>> >> >>>> those might still be in that state but maybe (!) are fixed.
>>>>>> >> >>>>
>>>>>> >> >>>> Anyone on the Linux end who happens to know if the fails in
>>>>>> >> >>>> particular
>>>>>> >> >>>> look normal, that'd be good to know.
>>>>>> >> >>>>
>>>>>> >> >>>> Thanks!
>>>>>> >> >>>>
>>>>>> >> >>>>>
>>>>>> >> >>>>>
>>>>>> >> >>>>> It's likely that someone could just go in there and remove
>>>>>> the XFAIL
>>>>>> >> >>>>> from those tests.
>>>>>> >> >>>>>
>>>>>> >> >>>>> On Mon, Aug 24, 2015 at 3:37 PM Todd Fiala via lldb-dev
>>>>>> >> >>>>> <lldb-dev at lists.llvm.org> wrote:
>>>>>> >> >>>>>>
>>>>>> >> >>>>>> Hi all,
>>>>>> >> >>>>>>
>>>>>> >> >>>>>> I'm just trying to get a handle on current lldb test
>>>>>> failures
>>>>>> >> >>>>>> across
>>>>>> >> >>>>>> different platforms.
>>>>>> >> >>>>>>
>>>>>> >> >>>>>> On Linux on non-virtualized hardware, I currently see the
>>>>>> failures
>>>>>> >> >>>>>> below on Ubuntu 14.04.2 using a setup like this:
>>>>>> >> >>>>>> * stock linker (ld.bfd),
>>>>>> >> >>>>>> * g++ 4.9.2
>>>>>> >> >>>>>> * cmake
>>>>>> >> >>>>>> * ninja
>>>>>> >> >>>>>> * libstdc++
>>>>>> >> >>>>>>
>>>>>> >> >>>>>> ninja check-lldb output:
>>>>>> >> >>>>>>
>>>>>> >> >>>>>> Ran 394 test suites (15 failed) (3.807107%)
>>>>>> >> >>>>>> Ran 474 test cases (17 failed) (3.586498%)
>>>>>> >> >>>>>> Failing Tests (15)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestCPPThis.py (Linux rad
>>>>>> 3.13.0-57-generic
>>>>>> >> >>>>>> #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccIterator.py
>>>>>> (Linux rad
>>>>>> >> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC
>>>>>> 2015
>>>>>> >> >>>>>> x86_64 x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccMap.py (Linux
>>>>>> rad
>>>>>> >> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC
>>>>>> 2015
>>>>>> >> >>>>>> x86_64 x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccMultiMap.py
>>>>>> (Linux rad
>>>>>> >> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC
>>>>>> 2015
>>>>>> >> >>>>>> x86_64 x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxMultiSet.py
>>>>>> (Linux rad
>>>>>> >> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC
>>>>>> 2015
>>>>>> >> >>>>>> x86_64 x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxSet.py (Linux
>>>>>> rad
>>>>>> >> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC
>>>>>> 2015
>>>>>> >> >>>>>> x86_64 x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxString.py
>>>>>> (Linux rad
>>>>>> >> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC
>>>>>> 2015
>>>>>> >> >>>>>> x86_64 x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterSkipSummary.py
>>>>>> (Linux rad
>>>>>> >> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC
>>>>>> 2015
>>>>>> >> >>>>>> x86_64 x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterUnordered.py (Linux
>>>>>> rad
>>>>>> >> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC
>>>>>> 2015
>>>>>> >> >>>>>> x86_64 x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestMiGdbSetShowPrint.py (Linux rad
>>>>>> >> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC
>>>>>> 2015
>>>>>> >> >>>>>> x86_64 x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestRegisterVariables.py (Linux rad
>>>>>> >> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC
>>>>>> 2015
>>>>>> >> >>>>>> x86_64 x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestStaticVariables.py (Linux rad
>>>>>> >> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC
>>>>>> 2015
>>>>>> >> >>>>>> x86_64 x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestStepNoDebug.py (Linux rad
>>>>>> >> >>>>>> 3.13.0-57-generic
>>>>>> >> >>>>>> #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestTypedefArray.py (Linux rad
>>>>>> >> >>>>>> 3.13.0-57-generic
>>>>>> >> >>>>>> #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestVectorTypesFormatting.py (Linux
>>>>>> rad
>>>>>> >> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC
>>>>>> 2015
>>>>>> >> >>>>>> x86_64 x86_64)
>>>>>> >> >>>>>>
>>>>>> >> >>>>>> Unexpected Successes (10)
>>>>>> >> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestBatchMode.py (Linux
>>>>>> rad
>>>>>> >> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC
>>>>>> 2015
>>>>>> >> >>>>>> x86_64 x86_64)
>>>>>> >> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestEvents.py (Linux rad
>>>>>> >> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC
>>>>>> 2015
>>>>>> >> >>>>>> x86_64 x86_64)
>>>>>> >> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestExitDuringStep.py
>>>>>> (Linux
>>>>>> >> >>>>>> rad
>>>>>> >> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC
>>>>>> 2015
>>>>>> >> >>>>>> x86_64 x86_64)
>>>>>> >> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestFdLeak.py (Linux rad
>>>>>> >> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC
>>>>>> 2015
>>>>>> >> >>>>>> x86_64 x86_64)
>>>>>> >> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestInferiorAssert.py
>>>>>> (Linux
>>>>>> >> >>>>>> rad
>>>>>> >> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC
>>>>>> 2015
>>>>>> >> >>>>>> x86_64 x86_64)
>>>>>> >> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiGdbSetShow.py
>>>>>> (Linux rad
>>>>>> >> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC
>>>>>> 2015
>>>>>> >> >>>>>> x86_64 x86_64)
>>>>>> >> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) ::
>>>>>> TestMiInterpreterExec.py (Linux
>>>>>> >> >>>>>> rad 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15
>>>>>> UTC 2015
>>>>>> >> >>>>>> x86_64
>>>>>> >> >>>>>> x86_64)
>>>>>> >> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiSyntax.py (Linux
>>>>>> rad
>>>>>> >> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC
>>>>>> 2015
>>>>>> >> >>>>>> x86_64 x86_64)
>>>>>> >> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestRaise.py (Linux rad
>>>>>> >> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC
>>>>>> 2015
>>>>>> >> >>>>>> x86_64 x86_64)
>>>>>> >> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestStubSetSID.py
>>>>>> (Linux rad
>>>>>> >> >>>>>> 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC
>>>>>> 2015
>>>>>> >> >>>>>> x86_64 x86_64)
>>>>>> >> >>>>>> ninja: build stopped: subcommand failed.
>>>>>> >> >>>>>>
>>>>>> >> >>>>>>
>>>>>> >> >>>>>>
>>>>>> >> >>>>>> On a similar setup, although bumped up to Ubuntu 14.04.3
>>>>>> and now on
>>>>>> >> >>>>>> a
>>>>>> >> >>>>>> VMWare VM, everything else the same, I see a similar report
>>>>>> from
>>>>>> >> >>>>>> 'ninja
>>>>>> >> >>>>>> check-lldb':
>>>>>> >> >>>>>>
>>>>>> >> >>>>>> Ran 394 test suites (17 failed) (4.314721%)
>>>>>> >> >>>>>> Ran 474 test cases (19 failed) (4.008439%)
>>>>>> >> >>>>>> Failing Tests (17)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestAttachResume.py (Linux lldb
>>>>>> >> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>>>>>> 14:09:17 UTC
>>>>>> >> >>>>>> 2015 x86_64
>>>>>> >> >>>>>> x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestCPPThis.py (Linux lldb
>>>>>> 3.19.0-26-generic
>>>>>> >> >>>>>> #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccIterator.py
>>>>>> (Linux lldb
>>>>>> >> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>>>>>> 14:09:17 UTC
>>>>>> >> >>>>>> 2015 x86_64
>>>>>> >> >>>>>> x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccMap.py (Linux
>>>>>> lldb
>>>>>> >> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>>>>>> 14:09:17 UTC
>>>>>> >> >>>>>> 2015 x86_64
>>>>>> >> >>>>>> x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibccMultiMap.py
>>>>>> (Linux lldb
>>>>>> >> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>>>>>> 14:09:17 UTC
>>>>>> >> >>>>>> 2015 x86_64
>>>>>> >> >>>>>> x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxMultiSet.py
>>>>>> (Linux
>>>>>> >> >>>>>> lldb
>>>>>> >> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>>>>>> 14:09:17 UTC
>>>>>> >> >>>>>> 2015 x86_64
>>>>>> >> >>>>>> x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxSet.py (Linux
>>>>>> lldb
>>>>>> >> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>>>>>> 14:09:17 UTC
>>>>>> >> >>>>>> 2015 x86_64
>>>>>> >> >>>>>> x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterLibcxxString.py
>>>>>> (Linux lldb
>>>>>> >> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>>>>>> 14:09:17 UTC
>>>>>> >> >>>>>> 2015 x86_64
>>>>>> >> >>>>>> x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterSkipSummary.py
>>>>>> (Linux lldb
>>>>>> >> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>>>>>> 14:09:17 UTC
>>>>>> >> >>>>>> 2015 x86_64
>>>>>> >> >>>>>> x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestDataFormatterUnordered.py (Linux
>>>>>> lldb
>>>>>> >> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>>>>>> 14:09:17 UTC
>>>>>> >> >>>>>> 2015 x86_64
>>>>>> >> >>>>>> x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestMiGdbSetShowPrint.py (Linux lldb
>>>>>> >> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>>>>>> 14:09:17 UTC
>>>>>> >> >>>>>> 2015 x86_64
>>>>>> >> >>>>>> x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestRegisterVariables.py (Linux lldb
>>>>>> >> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>>>>>> 14:09:17 UTC
>>>>>> >> >>>>>> 2015 x86_64
>>>>>> >> >>>>>> x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestStaticVariables.py (Linux lldb
>>>>>> >> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>>>>>> 14:09:17 UTC
>>>>>> >> >>>>>> 2015 x86_64
>>>>>> >> >>>>>> x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestStepNoDebug.py (Linux lldb
>>>>>> >> >>>>>> 3.19.0-26-generic
>>>>>> >> >>>>>> #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64
>>>>>> x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestStepOverWatchpoint.py (Linux lldb
>>>>>> >> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>>>>>> 14:09:17 UTC
>>>>>> >> >>>>>> 2015 x86_64
>>>>>> >> >>>>>> x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestTypedefArray.py (Linux lldb
>>>>>> >> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>>>>>> 14:09:17 UTC
>>>>>> >> >>>>>> 2015 x86_64
>>>>>> >> >>>>>> x86_64)
>>>>>> >> >>>>>> FAIL: LLDB (suite) :: TestVectorTypesFormatting.py (Linux
>>>>>> lldb
>>>>>> >> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>>>>>> 14:09:17 UTC
>>>>>> >> >>>>>> 2015 x86_64
>>>>>> >> >>>>>> x86_64)
>>>>>> >> >>>>>>
>>>>>> >> >>>>>> Unexpected Successes (11)
>>>>>> >> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestBatchMode.py (Linux
>>>>>> lldb
>>>>>> >> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>>>>>> 14:09:17 UTC
>>>>>> >> >>>>>> 2015 x86_64
>>>>>> >> >>>>>> x86_64)
>>>>>> >> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestEvents.py (Linux
>>>>>> lldb
>>>>>> >> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>>>>>> 14:09:17 UTC
>>>>>> >> >>>>>> 2015 x86_64
>>>>>> >> >>>>>> x86_64)
>>>>>> >> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestExitDuringStep.py
>>>>>> (Linux
>>>>>> >> >>>>>> lldb
>>>>>> >> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>>>>>> 14:09:17 UTC
>>>>>> >> >>>>>> 2015 x86_64
>>>>>> >> >>>>>> x86_64)
>>>>>> >> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestFdLeak.py (Linux
>>>>>> lldb
>>>>>> >> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>>>>>> 14:09:17 UTC
>>>>>> >> >>>>>> 2015 x86_64
>>>>>> >> >>>>>> x86_64)
>>>>>> >> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestInferiorAssert.py
>>>>>> (Linux
>>>>>> >> >>>>>> lldb
>>>>>> >> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>>>>>> 14:09:17 UTC
>>>>>> >> >>>>>> 2015 x86_64
>>>>>> >> >>>>>> x86_64)
>>>>>> >> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiGdbSetShow.py
>>>>>> (Linux lldb
>>>>>> >> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>>>>>> 14:09:17 UTC
>>>>>> >> >>>>>> 2015 x86_64
>>>>>> >> >>>>>> x86_64)
>>>>>> >> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) ::
>>>>>> TestMiInterpreterExec.py (Linux
>>>>>> >> >>>>>> lldb 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>>>>>> 14:09:17
>>>>>> >> >>>>>> UTC 2015
>>>>>> >> >>>>>> x86_64 x86_64)
>>>>>> >> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiSyntax.py (Linux
>>>>>> lldb
>>>>>> >> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>>>>>> 14:09:17 UTC
>>>>>> >> >>>>>> 2015 x86_64
>>>>>> >> >>>>>> x86_64)
>>>>>> >> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestRaise.py (Linux lldb
>>>>>> >> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>>>>>> 14:09:17 UTC
>>>>>> >> >>>>>> 2015 x86_64
>>>>>> >> >>>>>> x86_64)
>>>>>> >> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) :: TestStubSetSID.py
>>>>>> (Linux lldb
>>>>>> >> >>>>>> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
>>>>>> 14:09:17 UTC
>>>>>> >> >>>>>> 2015 x86_64
>>>>>> >> >>>>>> x86_64)
>>>>>> >> >>>>>> UNEXPECTED SUCCESS: LLDB (suite) ::
>>>>>> TestWatchedVarHitWhenInScope.py
>>>>>> >> >>>>>> (Linux lldb 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed
>>>>>> Aug 12
>>>>>> >> >>>>>> 14:09:17 UTC
>>>>>> >> >>>>>> 2015 x86_64 x86_64)
>>>>>> >> >>>>>> ninja: build stopped: subcommand failed.
>>>>>> >> >>>>>>
>>>>>> >> >>>>>>
>>>>>> >> >>>>>> Do these more or less match the current state of affairs
>>>>>> for Linux
>>>>>> >> >>>>>> tests?  If not, are there any suggestions on best practices
>>>>>> for
>>>>>> >> >>>>>> upgrades?  A
>>>>>> >> >>>>>> while back I had tried things like clang and libc++ but I
>>>>>> had
>>>>>> >> >>>>>> arrived at the
>>>>>> >> >>>>>> above setup (albeit typically with the gold linker) simply
>>>>>> due to
>>>>>> >> >>>>>> what
>>>>>> >> >>>>>> worked, what was easy to debug and what built fast.
>>>>>> >> >>>>>>
>>>>>> >> >>>>>> Thanks for any confirmation on that!
>>>>>> >> >>>>>>
>>>>>> >> >>>>>> --
>>>>>> >> >>>>>> -Todd
>>>>>> >> >>>>>> _______________________________________________
>>>>>> >> >>>>>> lldb-dev mailing list
>>>>>> >> >>>>>> lldb-dev at lists.llvm.org
>>>>>> >> >>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>>>> >> >>>>
>>>>>> >> >>>>
>>>>>> >> >>>>
>>>>>> >> >>>>
>>>>>> >> >>>> --
>>>>>> >> >>>> -Todd
>>>>>> >> >>>>
>>>>>> >> >>>> _______________________________________________
>>>>>> >> >>>> lldb-dev mailing list
>>>>>> >> >>>> lldb-dev at lists.llvm.org
>>>>>> >> >>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>>>> >> >>>>
>>>>>> >> >>>
>>>>>> >> >>
>>>>>> >> >>
>>>>>> >> >>
>>>>>> >> >> --
>>>>>> >> >> -Todd
>>>>>> >> >
>>>>>> >> >
>>>>>> >> >
>>>>>> >> >
>>>>>> >> > --
>>>>>> >> > -Todd
>>>>>> >> >
>>>>>> >> > _______________________________________________
>>>>>> >> > lldb-dev mailing list
>>>>>> >> > lldb-dev at lists.llvm.org
>>>>>> >> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>>>> >> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > --
>>>>>> > -Todd
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> -Todd
>>>>>
>>>>
>>>> _______________________________________________
>>>> lldb-dev mailing list
>>>> lldb-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>>
>>>
>>
>>
>> --
>> -Todd
>>
>
>
>
> --
> -Todd
>



-- 
-Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20150825/e668905d/attachment-0001.html>

More information about the lldb-dev mailing list