[lldb-dev] Digging into Linux unexpected successes

[Tamas Berghammer via lldb-dev]

Hi Todd,

I attached the statistic of the last 100 test run on the Linux x86_64
builder (http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake).
The data might be a little bit noisy because of the actual test failures
happening because of a temporary regression, but they should give you a
general idea about what is happening.

I will try to create a statistic where the results are displayed separately
for each compiler and architecture to get a bit more detailed view, but it
will take some time. If you want I can include the list of build numbers
for all outcome, but it will be a very log list (currently only included
for Timeout and Failure)

Tamas

On Mon, Sep 14, 2015 at 11:24 PM Todd Fiala via lldb-dev <
lldb-dev at lists.llvm.org> wrote:

> On an Ubuntu 14.04 x86_64 system, I'm seeing the following results:
>
> *cmake/ninja/clang-3.6:*
>
> Testing: 395 test suites, 24 threads
> 395 out of 395 test suites processed - TestGdbRemoteKill.py
> Ran 395 test suites (0 failed) (0.000000%)
> Ran 478 test cases (0 failed) (0.000000%)
>
> Unexpected Successes (6)
> UNEXPECTED SUCCESS: LLDB (suite) :: TestConstVariables.py
> UNEXPECTED SUCCESS: LLDB (suite) :: TestEvents.py
> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiBreak.py
> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiGdbSetShow.py
> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiInterpreterExec.py
> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiSyntax.py
>
>
> *cmake/ninja/gcc-4.9.2:*
>
> 395 out of 395 test suites processed - TestMultithreaded.py
> Ran 395 test suites (1 failed) (0.253165%)
> Ran 457 test cases (1 failed) (0.218818%)
> Failing Tests (1)
> FAIL: LLDB (suite) :: TestRegisterVariables.py
>
> Unexpected Successes (6)
> UNEXPECTED SUCCESS: LLDB (suite) :: TestDataFormatterSynth.py
> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiBreak.py
> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiGdbSetShow.py
> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiInterpreterExec.py
> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiSyntax.py
> UNEXPECTED SUCCESS: LLDB (suite) :: TestRaise.py
>
>
> I will look into those.  I suspect some of them are compiler-version
> specific, much like some of the OS X ones I dug into earlier.
> --
> -Todd
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20150915/b2d03a59/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test-statistics
Type: application/octet-stream
Size: 27107 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20150915/b2d03a59/attachment-0001.obj>