[lldb-dev] Digging into Linux unexpected successes

[Todd Fiala via lldb-dev]

On Tue, Sep 15, 2015 at 2:57 AM, Tamas Berghammer <tberghammer at google.com>
wrote:

> Hi Todd,
>
> I attached the statistic of the last 100 test run on the Linux x86_64
> builder (http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake).
> The data might be a little bit noisy because of the actual test failures
> happening because of a temporary regression, but they should give you a
> general idea about what is happening.
>
>
Thanks, Tamas!  I'll have a look.


> I will try to create a statistic where the results are displayed
> separately for each compiler and architecture to get a bit more detailed
> view, but it will take some time. If you want I can include the list of
> build numbers for all outcome, but it will be a very log list (currently
> only included for Timeout and Failure)
>
>
I'll know better when I have a look at what you provided.  The hole I see
right now is we're not adequately dealing with unexpected successes for
different configurations.  Any reporting around that is helpful.

Thanks!


> Tamas
>
> On Mon, Sep 14, 2015 at 11:24 PM Todd Fiala via lldb-dev <
> lldb-dev at lists.llvm.org> wrote:
>
>> On an Ubuntu 14.04 x86_64 system, I'm seeing the following results:
>>
>> *cmake/ninja/clang-3.6:*
>>
>> Testing: 395 test suites, 24 threads
>> 395 out of 395 test suites processed - TestGdbRemoteKill.py
>> Ran 395 test suites (0 failed) (0.000000%)
>> Ran 478 test cases (0 failed) (0.000000%)
>>
>> Unexpected Successes (6)
>> UNEXPECTED SUCCESS: LLDB (suite) :: TestConstVariables.py
>> UNEXPECTED SUCCESS: LLDB (suite) :: TestEvents.py
>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiBreak.py
>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiGdbSetShow.py
>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiInterpreterExec.py
>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiSyntax.py
>>
>>
>> *cmake/ninja/gcc-4.9.2:*
>>
>> 395 out of 395 test suites processed - TestMultithreaded.py
>> Ran 395 test suites (1 failed) (0.253165%)
>> Ran 457 test cases (1 failed) (0.218818%)
>> Failing Tests (1)
>> FAIL: LLDB (suite) :: TestRegisterVariables.py
>>
>> Unexpected Successes (6)
>> UNEXPECTED SUCCESS: LLDB (suite) :: TestDataFormatterSynth.py
>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiBreak.py
>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiGdbSetShow.py
>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiInterpreterExec.py
>> UNEXPECTED SUCCESS: LLDB (suite) :: TestMiSyntax.py
>> UNEXPECTED SUCCESS: LLDB (suite) :: TestRaise.py
>>
>>
>> I will look into those.  I suspect some of them are compiler-version
>> specific, much like some of the OS X ones I dug into earlier.
>> --
>> -Todd
>> _______________________________________________
>> lldb-dev mailing list
>> lldb-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>
>


-- 
-Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20150915/6cb2c95d/attachment.html>