[lldb-dev] Running check-lldb

[Pavel Labath via lldb-dev]

On 19 April 2017 at 19:15, Scott Smith <scott.smith at purestorage.com> wrote:

> A combination of:
> 1. Updating to a known good release according to buildbot
> 2. using Ubuntu 14.04
> 3. compiling release using clang-4.0
>
I'd hope that the compiler used to build lldb does not matter. If you see
any differences due to this factor, please let me know.

4. using the dotest command line that buildbot uses
>
The exact command line the buildbot uses is not important.. The only
important distinction from the check-lldb target is the compiler used. By
default it uses the host compiler used to build lldb. As no-one builds
tests using clang-4.0 it's quite possible that some things may be broken
(or just not properly annotated).


> 5. specifying gcc-4.8 instead of the locally compiled clang
>
> has most of the tests passing, with a handful of unexpected successes:
>
> UNEXPECTED SUCCESS: TestRegisterVariables.RegisterVariableTestCase.test_and_run_command_dwarf
> (lang/c/register_variables/TestRegisterVariables.py)
> UNEXPECTED SUCCESS: TestRegisterVariables.RegisterVariableTestCase.test_and_run_command_dwo
> (lang/c/register_variables/TestRegisterVariables.py)
> UNEXPECTED SUCCESS: TestExitDuringBreak.ExitDuringBreakpointTestCase.test_dwarf
> (functionalities/thread/exit_during_break/TestExitDuringBreak.py)
> UNEXPECTED SUCCESS: TestExitDuringBreak.ExitDuringBreakpointTestCase.test_dwo
> (functionalities/thread/exit_during_break/TestExitDuringBreak.py)
> UNEXPECTED SUCCESS: TestThreadStates.ThreadStateTestCase.test_process_interrupt_dwarf
> (functionalities/thread/state/TestThreadStates.py)
> UNEXPECTED SUCCESS: TestThreadStates.ThreadStateTestCase.test_process_interrupt_dwo
> (functionalities/thread/state/TestThreadStates.py)
> UNEXPECTED SUCCESS: TestRaise.RaiseTestCase.test_restart_bug_dwarf
> (functionalities/signal/raise/TestRaise.py)
> UNEXPECTED SUCCESS: TestRaise.RaiseTestCase.test_restart_bug_dwo
> (functionalities/signal/raise/TestRaise.py)
> UNEXPECTED SUCCESS: TestMultithreaded.SBBreakpointCallbackCase.test_sb_api_listener_resume_dwarf
> (api/multithreaded/TestMultithreaded.py)
> UNEXPECTED SUCCESS: TestMultithreaded.SBBreakpointCallbackCase.test_sb_api_listener_resume_dwo
> (api/multithreaded/TestMultithreaded.py)
> UNEXPECTED SUCCESS: lldbsuite.test.lldbtest.TestPrintf.test_with_dwarf
> (lang/cpp/printf/TestPrintf.py)
> UNEXPECTED SUCCESS: lldbsuite.test.lldbtest.TestPrintf.test_with_dwo
> (lang/cpp/printf/TestPrintf.py)
>
The unexpected successes are expected, unfortunately. :) What happens here
is that the tests are flaky and they fail like 1% of the time, so they are
marked as xfail.


>
> This looks different than another user's issue: http://lists.llvm.org/
> pipermail/lldb-dev/2016-February/009504.html
>
> I also tried gcc-4.9.4 (via the ubuntu-toolchain-r ppa) and got a
> different set of problems:
>
> FAIL: TestNamespaceDefinitions.NamespaceDefinitionsTestCase.test_expr_dwarf
> (lang/cpp/namespace_definitions/TestNamespaceDefinitions.py)
> FAIL: TestNamespaceDefinitions.NamespaceDefinitionsTestCase.test_expr_dwo
> (lang/cpp/namespace_definitions/TestNamespaceDefinitions.py)
> FAIL: TestTopLevelExprs.TopLevelExpressionsTestCase.
> test_top_level_expressions_dwarf (expression_command/top-level/
> TestTopLevelExprs.py)
> FAIL: TestTopLevelExprs.TopLevelExpressionsTestCase.test_top_level_expressions_dwo
> (expression_command/top-level/TestTopLevelExprs.py)
> UNEXPECTED SUCCESS: TestExitDuringBreak.ExitDuringBreakpointTestCase.test_dwarf
> (functionalities/thread/exit_during_break/TestExitDuringBreak.py)
> UNEXPECTED SUCCESS: TestExitDuringBreak.ExitDuringBreakpointTestCase.test_dwo
> (functionalities/thread/exit_during_break/TestExitDuringBreak.py)
> UNEXPECTED SUCCESS: TestThreadStates.ThreadStateTestCase.test_process_interrupt_dwarf
> (functionalities/thread/state/TestThreadStates.py)
> UNEXPECTED SUCCESS: TestRaise.RaiseTestCase.test_restart_bug_dwarf
> (functionalities/signal/raise/TestRaise.py)
> UNEXPECTED SUCCESS: TestRaise.RaiseTestCase.test_restart_bug_dwo
> (functionalities/signal/raise/TestRaise.py)
> UNEXPECTED SUCCESS: TestMultithreaded.SBBreakpointCallbackCase.test_sb_api_listener_resume_dwarf
> (api/multithreaded/TestMultithreaded.py)
> UNEXPECTED SUCCESS: TestMultithreaded.SBBreakpointCallbackCase.test_sb_api_listener_resume_dwo
> (api/multithreaded/TestMultithreaded.py)
> UNEXPECTED SUCCESS: lldbsuite.test.lldbtest.TestPrintf.test_with_dwarf
> (lang/cpp/printf/TestPrintf.py)
> UNEXPECTED SUCCESS: lldbsuite.test.lldbtest.TestPrintf.test_with_dwo
> (lang/cpp/printf/TestPrintf.py)
>
> Well, at least the list is consistent, which gives me a base to start
> testing race conditions :-)
>

Interesting, the bot uses a locally-built gcc-4.9.2 and those tests seem to
work there... I'll see if I can find the time to check that out.

Could you please also share the list of failures when using the top-of-tree
clang. I've tried that configuration yesterday on my home machine (x86_64
gentoo) and it had only a couple of failures -- most of them should be
fixed now, only a single failure in TestExprs2 should remain that I am
planning to have a look at.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20170420/8251b358/attachment-0001.html>