<div dir="ltr">For the record, our bots don't run any of the lldb-mi tests. I don't know if it is an issue with pexpect, the way we are using, or genuine issues, but I found pretty much all of lldb-mi tests flaky, and it's code too hairy to understand what is going on.</div><br><br><div class="gmail_quote"><div dir="ltr">On Mon, 5 Mar 2018 at 20:19, Vedant Kumar via lldb-commits <<a href="mailto:lldb-commits@lists.llvm.org">lldb-commits@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Author: vedantk<br>
Date: Mon Mar 5 12:16:52 2018<br>
New Revision: 326739<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=326739&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project?rev=326739&view=rev</a><br>
Log:<br>
[test] Skip pexpect-based lldb-mi tests on Darwin<br>
<br>
These tests fail with a relatively frequently on Darwin machines with<br>
errors such as:<br>
<br>
File ".../lldb/third_party/Python/module/pexpect-2.4/pexpect.py", line 1444, in expect_loop<br>
raise EOF(str(e) + '\n' + str(self))<br>
EOF: End Of File (EOF) in read_nonblocking(). Empty string style platform.<br>
<br>
The unpredictable failures make these tests noisy.<br>
<br>
rdar://37046976<br>
<br>
Modified:<br>
lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-mi/TestMiExit.py<br>
lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-mi/TestMiGdbSetShow.py<br>
lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-mi/data/TestMiData.py<br>
lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-mi/signal/TestMiSignal.py<br>
<br>
Modified: lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-mi/TestMiExit.py<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-mi/TestMiExit.py?rev=326739&r1=326738&r2=326739&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-mi/TestMiExit.py?rev=326739&r1=326738&r2=326739&view=diff</a><br>
==============================================================================<br>
--- lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-mi/TestMiExit.py (original)<br>
+++ lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-mi/TestMiExit.py Mon Mar 5 12:16:52 2018<br>
@@ -17,6 +17,7 @@ class MiExitTestCase(lldbmi_testcase.MiT<br>
@expectedFailureAll(<br>
oslist=["windows"],<br>
bugnumber="<a href="http://llvm.org/pr22274" rel="noreferrer" target="_blank">llvm.org/pr22274</a>: need a pexpect replacement for windows")<br>
+ @skipIfDarwin # pexpect is known to be unreliable on Darwin<br>
@skipIfFreeBSD # <a href="http://llvm.org/pr22411" rel="noreferrer" target="_blank">llvm.org/pr22411</a>: Failure presumably due to known thread races<br>
@skipIfRemote # We do not currently support remote debugging via the MI.<br>
def test_lldbmi_gdb_exit(self):<br>
@@ -44,6 +45,7 @@ class MiExitTestCase(lldbmi_testcase.MiT<br>
@expectedFailureAll(<br>
oslist=["windows"],<br>
bugnumber="<a href="http://llvm.org/pr22274" rel="noreferrer" target="_blank">llvm.org/pr22274</a>: need a pexpect replacement for windows")<br>
+ @skipIfDarwin # pexpect is known to be unreliable on Darwin<br>
@skipIfFreeBSD # <a href="http://llvm.org/pr22411" rel="noreferrer" target="_blank">llvm.org/pr22411</a>: Failure presumably due to known thread races<br>
@skipIfRemote # We do not currently support remote debugging via the MI.<br>
def test_lldbmi_quit(self):<br>
@@ -70,6 +72,7 @@ class MiExitTestCase(lldbmi_testcase.MiT<br>
@expectedFailureAll(<br>
oslist=["windows"],<br>
bugnumber="<a href="http://llvm.org/pr22274" rel="noreferrer" target="_blank">llvm.org/pr22274</a>: need a pexpect replacement for windows")<br>
+ @skipIfDarwin # pexpect is known to be unreliable on Darwin<br>
@skipIfFreeBSD # <a href="http://llvm.org/pr22411" rel="noreferrer" target="_blank">llvm.org/pr22411</a>: Failure presumably due to known thread races<br>
@skipIfRemote # We do not currently support remote debugging via the MI.<br>
def test_lldbmi_q(self):<br>
<br>
Modified: lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-mi/TestMiGdbSetShow.py<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-mi/TestMiGdbSetShow.py?rev=326739&r1=326738&r2=326739&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-mi/TestMiGdbSetShow.py?rev=326739&r1=326738&r2=326739&view=diff</a><br>
==============================================================================<br>
--- lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-mi/TestMiGdbSetShow.py (original)<br>
+++ lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-mi/TestMiGdbSetShow.py Mon Mar 5 12:16:52 2018<br>
@@ -19,6 +19,7 @@ class MiGdbSetShowTestCase(lldbmi_testca<br>
@expectedFailureAll(<br>
oslist=["windows"],<br>
bugnumber="<a href="http://llvm.org/pr22274" rel="noreferrer" target="_blank">llvm.org/pr22274</a>: need a pexpect replacement for windows")<br>
+ @skipIfDarwin # pexpect is known to be unreliable on Darwin<br>
@skipIfFreeBSD # <a href="http://llvm.org/pr22411" rel="noreferrer" target="_blank">llvm.org/pr22411</a>: Failure presumably due to known thread races<br>
@skipIfRemote # We do not currently support remote debugging via the MI.<br>
def test_lldbmi_gdb_set_target_async_default(self):<br>
@@ -41,6 +42,7 @@ class MiGdbSetShowTestCase(lldbmi_testca<br>
@expectedFailureAll(<br>
oslist=["windows"],<br>
bugnumber="<a href="http://llvm.org/pr22274" rel="noreferrer" target="_blank">llvm.org/pr22274</a>: need a pexpect replacement for windows")<br>
+ @skipIfDarwin # pexpect is known to be unreliable on Darwin<br>
@skipIfFreeBSD # <a href="http://llvm.org/pr22411" rel="noreferrer" target="_blank">llvm.org/pr22411</a>: Failure presumably due to known thread races<br>
@expectedFlakeyLinux("<a href="http://llvm.org/pr26028" rel="noreferrer" target="_blank">llvm.org/pr26028</a>") # Fails in ~1% of cases<br>
@skipIfRemote # We do not currently support remote debugging via the MI.<br>
@@ -74,6 +76,7 @@ class MiGdbSetShowTestCase(lldbmi_testca<br>
oslist=["windows"],<br>
bugnumber="<a href="http://llvm.org/pr22274" rel="noreferrer" target="_blank">llvm.org/pr22274</a>: need a pexpect replacement for windows")<br>
@skipIfFreeBSD # <a href="http://llvm.org/pr22411" rel="noreferrer" target="_blank">llvm.org/pr22411</a>: Failure presumably due to known thread races<br>
+ @skipIfDarwin # pexpect is known to be unreliable on Darwin<br>
@expectedFailureAll(<br>
oslist=["linux"],<br>
bugnumber="Failing in ~11/600 dosep runs (build 3120-3122)")<br>
@@ -103,6 +106,7 @@ class MiGdbSetShowTestCase(lldbmi_testca<br>
@expectedFailureAll(<br>
oslist=["windows"],<br>
bugnumber="<a href="http://llvm.org/pr22274" rel="noreferrer" target="_blank">llvm.org/pr22274</a>: need a pexpect replacement for windows")<br>
+ @skipIfDarwin # pexpect is known to be unreliable on Darwin<br>
@skipIfFreeBSD # <a href="http://llvm.org/pr22411" rel="noreferrer" target="_blank">llvm.org/pr22411</a>: Failure presumably due to known thread races<br>
@skipIfRemote # We do not currently support remote debugging via the MI.<br>
def test_lldbmi_gdb_show_target_async(self):<br>
@@ -117,6 +121,7 @@ class MiGdbSetShowTestCase(lldbmi_testca<br>
@expectedFailureAll(<br>
oslist=["windows"],<br>
bugnumber="<a href="http://llvm.org/pr22274" rel="noreferrer" target="_blank">llvm.org/pr22274</a>: need a pexpect replacement for windows")<br>
+ @skipIfDarwin # pexpect is known to be unreliable on Darwin<br>
@skipIfFreeBSD # <a href="http://llvm.org/pr22411" rel="noreferrer" target="_blank">llvm.org/pr22411</a>: Failure presumably due to known thread races<br>
@skipIfRemote # We do not currently support remote debugging via the MI.<br>
def test_lldbmi_gdb_show_language(self):<br>
@@ -142,6 +147,7 @@ class MiGdbSetShowTestCase(lldbmi_testca<br>
@expectedFailureAll(<br>
oslist=["windows"],<br>
bugnumber="<a href="http://llvm.org/pr22274" rel="noreferrer" target="_blank">llvm.org/pr22274</a>: need a pexpect replacement for windows")<br>
+ @skipIfDarwin # pexpect is known to be unreliable on Darwin<br>
@unittest2.expectedFailure("-gdb-set ignores unknown properties")<br>
@skipIfRemote # We do not currently support remote debugging via the MI.<br>
def test_lldbmi_gdb_set_unknown(self):<br>
@@ -156,6 +162,7 @@ class MiGdbSetShowTestCase(lldbmi_testca<br>
@expectedFailureAll(<br>
oslist=["windows"],<br>
bugnumber="<a href="http://llvm.org/pr22274" rel="noreferrer" target="_blank">llvm.org/pr22274</a>: need a pexpect replacement for windows")<br>
+ @skipIfDarwin # pexpect is known to be unreliable on Darwin<br>
@unittest2.expectedFailure("-gdb-show ignores unknown properties")<br>
@skipIfRemote # We do not currently support remote debugging via the MI.<br>
def test_lldbmi_gdb_show_unknown(self):<br>
@@ -170,6 +177,7 @@ class MiGdbSetShowTestCase(lldbmi_testca<br>
@expectedFailureAll(<br>
oslist=["windows"],<br>
bugnumber="<a href="http://llvm.org/pr22274" rel="noreferrer" target="_blank">llvm.org/pr22274</a>: need a pexpect replacement for windows")<br>
+ @skipIfDarwin # pexpect is known to be unreliable on Darwin<br>
@skipIfFreeBSD # <a href="http://llvm.org/pr22411" rel="noreferrer" target="_blank">llvm.org/pr22411</a>: Failure presumably due to known thread races<br>
@skipIfLinux # <a href="http://llvm.org/pr22841" rel="noreferrer" target="_blank">llvm.org/pr22841</a>: lldb-mi tests fail on all Linux buildbots<br>
@skipIfRemote # We do not currently support remote debugging via the MI.<br>
@@ -219,6 +227,7 @@ class MiGdbSetShowTestCase(lldbmi_testca<br>
<br>
@skipIfWindows # <a href="http://llvm.org/pr24452" rel="noreferrer" target="_blank">llvm.org/pr24452</a>: Get lldb-mi tests working on Windows<br>
@skipIfFreeBSD # <a href="http://llvm.org/pr22411" rel="noreferrer" target="_blank">llvm.org/pr22411</a>: Failure presumably due to known thread races<br>
+ @skipIfDarwin # pexpect is known to be unreliable on Darwin<br>
@skipIfRemote # We do not currently support remote debugging via the MI.<br>
@expectedFailureAll(<br>
bugnumber="<a href="http://llvm.org/pr31485" rel="noreferrer" target="_blank">llvm.org/pr31485</a>: data-disassemble doesn't follow flavor settings")<br>
<br>
Modified: lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-mi/data/TestMiData.py<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-mi/data/TestMiData.py?rev=326739&r1=326738&r2=326739&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-mi/data/TestMiData.py?rev=326739&r1=326738&r2=326739&view=diff</a><br>
==============================================================================<br>
--- lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-mi/data/TestMiData.py (original)<br>
+++ lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-mi/data/TestMiData.py Mon Mar 5 12:16:52 2018<br>
@@ -18,6 +18,7 @@ class MiDataTestCase(lldbmi_testcase.MiT<br>
<br>
@skipIfRemote # We do not currently support remote debugging via the MI.<br>
@skipIfWindows # <a href="http://llvm.org/pr24452" rel="noreferrer" target="_blank">llvm.org/pr24452</a>: Get lldb-mi tests working on Windows<br>
+ @skipIfDarwin # pexpect is known to be unreliable on Darwin<br>
@skipIfFreeBSD # <a href="http://llvm.org/pr22411" rel="noreferrer" target="_blank">llvm.org/pr22411</a>: Failure presumably due to known thread races<br>
def test_lldbmi_data_disassemble(self):<br>
"""Test that 'lldb-mi --interpreter' works for -data-disassemble."""<br>
@@ -89,6 +90,7 @@ class MiDataTestCase(lldbmi_testcase.MiT<br>
<br>
@skipIfRemote # We do not currently support remote debugging via the MI.<br>
@skipIfWindows # <a href="http://llvm.org/pr24452" rel="noreferrer" target="_blank">llvm.org/pr24452</a>: Get lldb-mi tests working on Windows<br>
+ @skipIfDarwin # pexpect is known to be unreliable on Darwin<br>
@skipIfFreeBSD # <a href="http://llvm.org/pr22411" rel="noreferrer" target="_blank">llvm.org/pr22411</a>: Failure presumably due to known thread races<br>
def test_lldbmi_data_read_memory_bytes_global(self):<br>
"""Test that -data-read-memory-bytes can access global buffers."""<br>
@@ -132,6 +134,7 @@ class MiDataTestCase(lldbmi_testcase.MiT<br>
<br>
@skipIfRemote # We do not currently support remote debugging via the MI.<br>
@skipIfWindows # <a href="http://llvm.org/pr24452" rel="noreferrer" target="_blank">llvm.org/pr24452</a>: Get lldb-mi tests working on Windows<br>
+ @skipIfDarwin # pexpect is known to be unreliable on Darwin<br>
@skipIfFreeBSD # <a href="http://llvm.org/pr22411" rel="noreferrer" target="_blank">llvm.org/pr22411</a>: Failure presumably due to known thread races<br>
def test_lldbmi_data_read_memory_bytes_local(self):<br>
"""Test that -data-read-memory-bytes can access local buffers."""<br>
@@ -272,6 +275,7 @@ class MiDataTestCase(lldbmi_testcase.MiT<br>
<br>
@skipIfRemote # We do not currently support remote debugging via the MI.<br>
@skipIfWindows # <a href="http://llvm.org/pr24452" rel="noreferrer" target="_blank">llvm.org/pr24452</a>: Get lldb-mi tests working on Windows<br>
+ @skipIfDarwin # pexpect is known to be unreliable on Darwin<br>
@skipIfFreeBSD # <a href="http://llvm.org/pr22411" rel="noreferrer" target="_blank">llvm.org/pr22411</a>: Failure presumably due to known thread races<br>
def test_lldbmi_data_list_register_names(self):<br>
"""Test that 'lldb-mi --interpreter' works for -data-list-register-names."""<br>
@@ -299,6 +303,7 @@ class MiDataTestCase(lldbmi_testcase.MiT<br>
<br>
@skipIfRemote # We do not currently support remote debugging via the MI.<br>
@skipIfWindows # <a href="http://llvm.org/pr24452" rel="noreferrer" target="_blank">llvm.org/pr24452</a>: Get lldb-mi tests working on Windows<br>
+ @skipIfDarwin # pexpect is known to be unreliable on Darwin<br>
@skipIfFreeBSD # <a href="http://llvm.org/pr22411" rel="noreferrer" target="_blank">llvm.org/pr22411</a>: Failure presumably due to known thread races<br>
def test_lldbmi_data_list_register_values(self):<br>
"""Test that 'lldb-mi --interpreter' works for -data-list-register-values."""<br>
@@ -328,6 +333,7 @@ class MiDataTestCase(lldbmi_testcase.MiT<br>
<br>
@skipIfRemote # We do not currently support remote debugging via the MI.<br>
@skipIfWindows # <a href="http://llvm.org/pr24452" rel="noreferrer" target="_blank">llvm.org/pr24452</a>: Get lldb-mi tests working on Windows<br>
+ @skipIfDarwin # pexpect is known to be unreliable on Darwin<br>
@skipIfFreeBSD # <a href="http://llvm.org/pr22411" rel="noreferrer" target="_blank">llvm.org/pr22411</a>: Failure presumably due to known thread races<br>
def test_lldbmi_data_info_line(self):<br>
"""Test that 'lldb-mi --interpreter' works for -data-info-line."""<br>
@@ -383,6 +389,7 @@ class MiDataTestCase(lldbmi_testcase.MiT<br>
<br>
@skipIfRemote # We do not currently support remote debugging via the MI.<br>
@skipIfWindows # <a href="http://llvm.org/pr24452" rel="noreferrer" target="_blank">llvm.org/pr24452</a>: Get lldb-mi tests working on Windows<br>
+ @skipIfDarwin # pexpect is known to be unreliable on Darwin<br>
@skipIfFreeBSD # <a href="http://llvm.org/pr22411" rel="noreferrer" target="_blank">llvm.org/pr22411</a>: Failure presumably due to known thread races<br>
def test_lldbmi_data_evaluate_expression(self):<br>
"""Test that 'lldb-mi --interpreter' works for -data-evaluate-expression."""<br>
<br>
Modified: lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-mi/signal/TestMiSignal.py<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-mi/signal/TestMiSignal.py?rev=326739&r1=326738&r2=326739&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-mi/signal/TestMiSignal.py?rev=326739&r1=326738&r2=326739&view=diff</a><br>
==============================================================================<br>
--- lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-mi/signal/TestMiSignal.py (original)<br>
+++ lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-mi/signal/TestMiSignal.py Mon Mar 5 12:16:52 2018<br>
@@ -89,7 +89,7 @@ class MiSignalTestCase(lldbmi_testcase.M<br>
self.expect("\*stopped,reason=\"breakpoint-hit\"")<br>
<br>
@skipIfWindows # <a href="http://llvm.org/pr24452" rel="noreferrer" target="_blank">llvm.org/pr24452</a>: Get lldb-mi tests working on Windows<br>
- @skipUnlessDarwin<br>
+ @skipIfDarwin # pexpect is known to be unreliable on Darwin<br>
@skipIfRemote # We do not currently support remote debugging via the MI.<br>
def test_lldbmi_stopped_when_stopatentry_remote(self):<br>
"""Test that 'lldb-mi --interpreter' notifies after it was stopped on entry (remote)."""<br>
<br>
<br>
_______________________________________________<br>
lldb-commits mailing list<br>
<a href="mailto:lldb-commits@lists.llvm.org" target="_blank">lldb-commits@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits</a><br>
</blockquote></div>