<div dir="ltr">We've occasionally discussed the concept of unexpected successes being an issue.  Last time we landed on this being the best we can do for now as we want the code to be executed, even if it fails occasionally and/or under load.  The alternative (aside from fixing it) is to skip it, which then prevents us from seeing if something horrible like a crash occurs when going down that code path.</div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Feb 3, 2016 at 2:57 PM, Siva Chandra via lldb-dev <span dir="ltr"><<a href="mailto:lldb-dev@lists.llvm.org" target="_blank">lldb-dev@lists.llvm.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Our bot is running on Ubuntu 14.04 and is green:<br>
<a href="http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake" rel="noreferrer" target="_blank">http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake</a><br>
<br>
One thing though, the bot does not run the testsuite with clang-3.6.<br>
About the unexpected successes, they are very likely tests which were<br>
found to be flaky and marked as expectedFailure (or something similar)<br>
to keep the bot green. Even the bot logs show these unexpected<br>
successes.<br>
<div><div class="h5"><br>
On Wed, Feb 3, 2016 at 2:50 PM, Zachary Turner via lldb-dev<br>
<<a href="mailto:lldb-dev@lists.llvm.org">lldb-dev@lists.llvm.org</a>> wrote:<br>
><br>
> On Linux I get the following test results:<br>
><br>
> UNEXPECTED SUCCESS: test_and_run_command_dwarf<br>
> (lang/c/const_variables/TestConstVariables.py)<br>
> UNEXPECTED SUCCESS: test_and_run_command_dwo<br>
> (lang/c/const_variables/TestConstVariables.py)<br>
> UNEXPECTED SUCCESS: test_command_script_immediate_output_dwarf<br>
> (functionalities/command_script_immediate_output/TestCommandScriptImmediateOutput.py)<br>
> UNEXPECTED SUCCESS: test_command_script_immediate_output_dwo<br>
> (functionalities/command_script_immediate_output/TestCommandScriptImmediateOutput.py)<br>
> UNEXPECTED SUCCESS: test_fd_leak_basic_dwarf<br>
> (functionalities/avoids-fd-leak/TestFdLeak.py)<br>
> UNEXPECTED SUCCESS: test_fd_leak_basic_dwo<br>
> (functionalities/avoids-fd-leak/TestFdLeak.py)<br>
> UNEXPECTED SUCCESS: test_fd_leak_log_dwarf<br>
> (functionalities/avoids-fd-leak/TestFdLeak.py)<br>
> UNEXPECTED SUCCESS: test_fd_leak_log_dwo<br>
> (functionalities/avoids-fd-leak/TestFdLeak.py)<br>
> UNEXPECTED SUCCESS: test_fd_leak_multitarget_dwarf<br>
> (functionalities/avoids-fd-leak/TestFdLeak.py)<br>
> UNEXPECTED SUCCESS: test_fd_leak_multitarget_dwo<br>
> (functionalities/avoids-fd-leak/TestFdLeak.py)<br>
> UNEXPECTED SUCCESS: test_file_scope_lookup_with_run_command_dwarf<br>
> (lang/cpp/namespace/TestNamespaceLookup.py)<br>
> UNEXPECTED SUCCESS: test_file_scope_lookup_with_run_command_dwo<br>
> (lang/cpp/namespace/TestNamespaceLookup.py)<br>
> UNEXPECTED SUCCESS: test_lldbmi_gdb_set_target_async_off<br>
> (tools/lldb-mi/TestMiGdbSetShow.py)<br>
> UNEXPECTED SUCCESS: test_lldbmi_process_output<br>
> (tools/lldb-mi/syntax/TestMiSyntax.py)<br>
> UNEXPECTED SUCCESS: test_lldbmi_settings_set_target_run_args_after<br>
> (tools/lldb-mi/interpreter/TestMiInterpreterExec.py)<br>
> UNEXPECTED SUCCESS: test_lldbmi_settings_set_target_run_args_before<br>
> (tools/lldb-mi/interpreter/TestMiInterpreterExec.py)<br>
> UNEXPECTED SUCCESS: test_restart_bug_dwarf<br>
> (functionalities/signal/raise/TestRaise.py)<br>
> UNEXPECTED SUCCESS: test_restart_bug_dwo<br>
> (functionalities/signal/raise/TestRaise.py)<br>
> UNEXPECTED SUCCESS: test_scope_lookup_before_using_with_run_command_dwo<br>
> (lang/cpp/namespace/TestNamespaceLookup.py)<br>
> TIMEOUT: test_qThreadInfo_matches_qC_attach_llgs_dwo<br>
> (tools/lldb-server/TestLldbGdbServer.py)<br>
> TIMEOUT: test_watchpoint_delay_watchpoint_one_breakpoint_dwarf<br>
> (functionalities/thread/concurrent_events/TestConcurrentEvents.py)<br>
><br>
><br>
> This is a ton of unexpected successes.  Does anyone regularly run the test<br>
> suite on Linux?  Is this normal?  I also notice that it takes almost 30<br>
> minutes to complete, and I get these timeouts:<br>
><br>
> TIMEOUT: test_qThreadInfo_matches_qC_attach_llgs_dwo<br>
> (tools/lldb-server/TestLldbGdbServer.py)<br>
> TIMEOUT: test_watchpoint_delay_watchpoint_one_breakpoint_dwarf<br>
> (functionalities/thread/concurrent_events/TestConcurrentEvents.py)<br>
><br>
> Are these known issues?  I'm using Ubuntu 14.04 and building tests with<br>
> Clang 3.6<br>
><br>
</div></div>> _______________________________________________<br>
> lldb-dev mailing list<br>
> <a href="mailto:lldb-dev@lists.llvm.org">lldb-dev@lists.llvm.org</a><br>
> <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev</a><br>
><br>
_______________________________________________<br>
lldb-dev mailing list<br>
<a href="mailto:lldb-dev@lists.llvm.org">lldb-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev</a><br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr">-Todd</div></div>
</div>