[lldb-dev] Problem running the test suite on Linux.

[Todd Fiala via lldb-dev]

We've occasionally discussed the concept of unexpected successes being an
issue.  Last time we landed on this being the best we can do for now as we
want the code to be executed, even if it fails occasionally and/or under
load.  The alternative (aside from fixing it) is to skip it, which then
prevents us from seeing if something horrible like a crash occurs when
going down that code path.

On Wed, Feb 3, 2016 at 2:57 PM, Siva Chandra via lldb-dev <
lldb-dev at lists.llvm.org> wrote:

> Our bot is running on Ubuntu 14.04 and is green:
> http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake
>
> One thing though, the bot does not run the testsuite with clang-3.6.
> About the unexpected successes, they are very likely tests which were
> found to be flaky and marked as expectedFailure (or something similar)
> to keep the bot green. Even the bot logs show these unexpected
> successes.
>
> On Wed, Feb 3, 2016 at 2:50 PM, Zachary Turner via lldb-dev
> <lldb-dev at lists.llvm.org> wrote:
> >
> > On Linux I get the following test results:
> >
> > UNEXPECTED SUCCESS: test_and_run_command_dwarf
> > (lang/c/const_variables/TestConstVariables.py)
> > UNEXPECTED SUCCESS: test_and_run_command_dwo
> > (lang/c/const_variables/TestConstVariables.py)
> > UNEXPECTED SUCCESS: test_command_script_immediate_output_dwarf
> >
> (functionalities/command_script_immediate_output/TestCommandScriptImmediateOutput.py)
> > UNEXPECTED SUCCESS: test_command_script_immediate_output_dwo
> >
> (functionalities/command_script_immediate_output/TestCommandScriptImmediateOutput.py)
> > UNEXPECTED SUCCESS: test_fd_leak_basic_dwarf
> > (functionalities/avoids-fd-leak/TestFdLeak.py)
> > UNEXPECTED SUCCESS: test_fd_leak_basic_dwo
> > (functionalities/avoids-fd-leak/TestFdLeak.py)
> > UNEXPECTED SUCCESS: test_fd_leak_log_dwarf
> > (functionalities/avoids-fd-leak/TestFdLeak.py)
> > UNEXPECTED SUCCESS: test_fd_leak_log_dwo
> > (functionalities/avoids-fd-leak/TestFdLeak.py)
> > UNEXPECTED SUCCESS: test_fd_leak_multitarget_dwarf
> > (functionalities/avoids-fd-leak/TestFdLeak.py)
> > UNEXPECTED SUCCESS: test_fd_leak_multitarget_dwo
> > (functionalities/avoids-fd-leak/TestFdLeak.py)
> > UNEXPECTED SUCCESS: test_file_scope_lookup_with_run_command_dwarf
> > (lang/cpp/namespace/TestNamespaceLookup.py)
> > UNEXPECTED SUCCESS: test_file_scope_lookup_with_run_command_dwo
> > (lang/cpp/namespace/TestNamespaceLookup.py)
> > UNEXPECTED SUCCESS: test_lldbmi_gdb_set_target_async_off
> > (tools/lldb-mi/TestMiGdbSetShow.py)
> > UNEXPECTED SUCCESS: test_lldbmi_process_output
> > (tools/lldb-mi/syntax/TestMiSyntax.py)
> > UNEXPECTED SUCCESS: test_lldbmi_settings_set_target_run_args_after
> > (tools/lldb-mi/interpreter/TestMiInterpreterExec.py)
> > UNEXPECTED SUCCESS: test_lldbmi_settings_set_target_run_args_before
> > (tools/lldb-mi/interpreter/TestMiInterpreterExec.py)
> > UNEXPECTED SUCCESS: test_restart_bug_dwarf
> > (functionalities/signal/raise/TestRaise.py)
> > UNEXPECTED SUCCESS: test_restart_bug_dwo
> > (functionalities/signal/raise/TestRaise.py)
> > UNEXPECTED SUCCESS: test_scope_lookup_before_using_with_run_command_dwo
> > (lang/cpp/namespace/TestNamespaceLookup.py)
> > TIMEOUT: test_qThreadInfo_matches_qC_attach_llgs_dwo
> > (tools/lldb-server/TestLldbGdbServer.py)
> > TIMEOUT: test_watchpoint_delay_watchpoint_one_breakpoint_dwarf
> > (functionalities/thread/concurrent_events/TestConcurrentEvents.py)
> >
> >
> > This is a ton of unexpected successes.  Does anyone regularly run the
> test
> > suite on Linux?  Is this normal?  I also notice that it takes almost 30
> > minutes to complete, and I get these timeouts:
> >
> > TIMEOUT: test_qThreadInfo_matches_qC_attach_llgs_dwo
> > (tools/lldb-server/TestLldbGdbServer.py)
> > TIMEOUT: test_watchpoint_delay_watchpoint_one_breakpoint_dwarf
> > (functionalities/thread/concurrent_events/TestConcurrentEvents.py)
> >
> > Are these known issues?  I'm using Ubuntu 14.04 and building tests with
> > Clang 3.6
> >
> > _______________________________________________
> > lldb-dev mailing list
> > lldb-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> >
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>



-- 
-Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20160203/8d4cda5c/attachment-0001.html>