[Lldb-commits] [lldb] r338769 - [lldbsuite, windows] Mark tests as XFAIL on Windows or skip them
Stella Stamenova via lldb-commits
lldb-commits at lists.llvm.org
Thu Aug 2 14:26:20 PDT 2018
Author: stella.stamenova
Date: Thu Aug 2 14:26:19 2018
New Revision: 338769
URL: http://llvm.org/viewvc/llvm-project?rev=338769&view=rev
Log:
[lldbsuite, windows] Mark tests as XFAIL on Windows or skip them
Summary:
1) Several tests that are flakey on windows fail the run even if they are marked as expected to be flakey. This is because they fail frequently enough that even a retry won't help
2) Skip several tests on Windows that will occasionally hang rather than failing or exiting. This is causing the entire test suite to hang
Reviewers: asmith, labath, zturner
Reviewed By: zturner
Differential Revision: https://reviews.llvm.org/D50198
Modified:
lldb/trunk/packages/Python/lldbsuite/test/functionalities/attach_resume/TestAttachResume.py
lldb/trunk/packages/Python/lldbsuite/test/functionalities/breakpoint/breakpoint_ignore_count/TestBreakpointIgnoreCount.py
lldb/trunk/packages/Python/lldbsuite/test/functionalities/memory/cache/TestMemoryCache.py
lldb/trunk/packages/Python/lldbsuite/test/functionalities/thread/exit_during_step/TestExitDuringStep.py
lldb/trunk/packages/Python/lldbsuite/test/functionalities/thread/step_out/TestThreadStepOut.py
lldb/trunk/packages/Python/lldbsuite/test/functionalities/thread/thread_exit/TestThreadExit.py
lldb/trunk/packages/Python/lldbsuite/test/lang/cpp/virtual/TestVirtual.py
lldb/trunk/packages/Python/lldbsuite/test/python_api/event/TestEvents.py
Modified: lldb/trunk/packages/Python/lldbsuite/test/functionalities/attach_resume/TestAttachResume.py
URL: http://llvm.org/viewvc/llvm-project/lldb/trunk/packages/Python/lldbsuite/test/functionalities/attach_resume/TestAttachResume.py?rev=338769&r1=338768&r2=338769&view=diff
==============================================================================
--- lldb/trunk/packages/Python/lldbsuite/test/functionalities/attach_resume/TestAttachResume.py (original)
+++ lldb/trunk/packages/Python/lldbsuite/test/functionalities/attach_resume/TestAttachResume.py Thu Aug 2 14:26:19 2018
@@ -21,7 +21,7 @@ class AttachResumeTestCase(TestBase):
@skipIfRemote
@expectedFailureAll(oslist=['freebsd'], bugnumber='llvm.org/pr19310')
- @expectedFailureAll(oslist=["windows"], bugnumber="llvm.org/pr24778")
+ @skipIfWindows # llvm.org/pr24778, llvm.org/pr21753
def test_attach_continue_interrupt_detach(self):
"""Test attach/continue/interrupt/detach"""
self.build()
Modified: lldb/trunk/packages/Python/lldbsuite/test/functionalities/breakpoint/breakpoint_ignore_count/TestBreakpointIgnoreCount.py
URL: http://llvm.org/viewvc/llvm-project/lldb/trunk/packages/Python/lldbsuite/test/functionalities/breakpoint/breakpoint_ignore_count/TestBreakpointIgnoreCount.py?rev=338769&r1=338768&r2=338769&view=diff
==============================================================================
--- lldb/trunk/packages/Python/lldbsuite/test/functionalities/breakpoint/breakpoint_ignore_count/TestBreakpointIgnoreCount.py (original)
+++ lldb/trunk/packages/Python/lldbsuite/test/functionalities/breakpoint/breakpoint_ignore_count/TestBreakpointIgnoreCount.py Thu Aug 2 14:26:19 2018
@@ -18,12 +18,14 @@ class BreakpointIgnoreCountTestCase(Test
mydir = TestBase.compute_mydir(__file__)
+ @skipIfWindows # This test will hang on windows llvm.org/pr21753
def test_with_run_command(self):
"""Exercise breakpoint ignore count with 'breakpoint set -i <count>'."""
self.build()
self.breakpoint_ignore_count()
@add_test_categories(['pyapi'])
+ @skipIfWindows # This test will hang on windows llvm.org/pr21753
def test_with_python_api(self):
"""Use Python APIs to set breakpoint ignore count."""
self.build()
Modified: lldb/trunk/packages/Python/lldbsuite/test/functionalities/memory/cache/TestMemoryCache.py
URL: http://llvm.org/viewvc/llvm-project/lldb/trunk/packages/Python/lldbsuite/test/functionalities/memory/cache/TestMemoryCache.py?rev=338769&r1=338768&r2=338769&view=diff
==============================================================================
--- lldb/trunk/packages/Python/lldbsuite/test/functionalities/memory/cache/TestMemoryCache.py (original)
+++ lldb/trunk/packages/Python/lldbsuite/test/functionalities/memory/cache/TestMemoryCache.py Thu Aug 2 14:26:19 2018
@@ -24,7 +24,9 @@ class MemoryCacheTestCase(TestBase):
# Find the line number to break inside main().
self.line = line_number('main.cpp', '// Set break point at this line.')
- @expectedFlakeyOS(oslist=["windows"])
+ # The test is actually flakey on Windows, failing every dozen or so runs, but even with the flakey
+ # decorator it still fails
+ @expectedFailureAll(oslist=["windows"], bugnumber="llvm.org/pr38373")
def test_memory_cache(self):
"""Test the MemoryCache class with a sequence of 'memory read' and 'memory write' operations."""
self.build()
Modified: lldb/trunk/packages/Python/lldbsuite/test/functionalities/thread/exit_during_step/TestExitDuringStep.py
URL: http://llvm.org/viewvc/llvm-project/lldb/trunk/packages/Python/lldbsuite/test/functionalities/thread/exit_during_step/TestExitDuringStep.py?rev=338769&r1=338768&r2=338769&view=diff
==============================================================================
--- lldb/trunk/packages/Python/lldbsuite/test/functionalities/thread/exit_during_step/TestExitDuringStep.py (original)
+++ lldb/trunk/packages/Python/lldbsuite/test/functionalities/thread/exit_during_step/TestExitDuringStep.py Thu Aug 2 14:26:19 2018
@@ -18,6 +18,9 @@ class ExitDuringStepTestCase(TestBase):
mydir = TestBase.compute_mydir(__file__)
@skipIfFreeBSD # llvm.org/pr21411: test is hanging
+ # The test is actually flakey on Windows, failing every dozen or so runs, but even with the flakey
+ # decorator it still fails
+ @expectedFailureAll(oslist=["windows"], bugnumber="llvm.org/pr38373")
def test(self):
"""Test thread exit during step handling."""
self.build(dictionary=self.getBuildFlags())
@@ -27,6 +30,9 @@ class ExitDuringStepTestCase(TestBase):
True)
@skipIfFreeBSD # llvm.org/pr21411: test is hanging
+ # The test is actually flakey on Windows, failing every dozen or so runs, but even with the flakey
+ # decorator it still fails
+ @expectedFailureAll(oslist=["windows"], bugnumber="llvm.org/pr38373")
def test_step_over(self):
"""Test thread exit during step-over handling."""
self.build(dictionary=self.getBuildFlags())
@@ -36,6 +42,9 @@ class ExitDuringStepTestCase(TestBase):
False)
@skipIfFreeBSD # llvm.org/pr21411: test is hanging
+ # The test is actually flakey on Windows, failing every dozen or so runs, but even with the flakey
+ # decorator it still fails
+ @expectedFailureAll(oslist=["windows"], bugnumber="llvm.org/pr38373")
def test_step_in(self):
"""Test thread exit during step-in handling."""
self.build(dictionary=self.getBuildFlags())
Modified: lldb/trunk/packages/Python/lldbsuite/test/functionalities/thread/step_out/TestThreadStepOut.py
URL: http://llvm.org/viewvc/llvm-project/lldb/trunk/packages/Python/lldbsuite/test/functionalities/thread/step_out/TestThreadStepOut.py?rev=338769&r1=338768&r2=338769&view=diff
==============================================================================
--- lldb/trunk/packages/Python/lldbsuite/test/functionalities/thread/step_out/TestThreadStepOut.py (original)
+++ lldb/trunk/packages/Python/lldbsuite/test/functionalities/thread/step_out/TestThreadStepOut.py Thu Aug 2 14:26:19 2018
@@ -25,6 +25,7 @@ class ThreadStepOutTestCase(TestBase):
@expectedFailureAll(
oslist=["freebsd"],
bugnumber="llvm.org/pr18066 inferior does not exit")
+ @skipIfWindows # This test will hang on windows llvm.org/pr21753
@expectedFailureAll(oslist=["windows"])
def test_step_single_thread(self):
"""Test thread step out on one thread via command interpreter. """
@@ -39,6 +40,7 @@ class ThreadStepOutTestCase(TestBase):
@expectedFailureAll(
oslist=["freebsd"],
bugnumber="llvm.org/pr19347 2nd thread stops at breakpoint")
+ @skipIfWindows # This test will hang on windows llvm.org/pr21753
@expectedFailureAll(oslist=["windows"])
@expectedFailureAll(oslist=["watchos"], archs=['armv7k'], bugnumber="rdar://problem/34674488") # stop reason is trace when it should be step-out
def test_step_all_threads(self):
@@ -54,6 +56,7 @@ class ThreadStepOutTestCase(TestBase):
@expectedFailureAll(
oslist=["freebsd"],
bugnumber="llvm.org/pr19347 2nd thread stops at breakpoint")
+ @skipIfWindows # This test will hang on windows llvm.org/pr21753
@expectedFailureAll(oslist=["windows"], bugnumber="llvm.org/pr24681")
def test_python(self):
"""Test thread step out on one thread via Python API (dwarf)."""
Modified: lldb/trunk/packages/Python/lldbsuite/test/functionalities/thread/thread_exit/TestThreadExit.py
URL: http://llvm.org/viewvc/llvm-project/lldb/trunk/packages/Python/lldbsuite/test/functionalities/thread/thread_exit/TestThreadExit.py?rev=338769&r1=338768&r2=338769&view=diff
==============================================================================
--- lldb/trunk/packages/Python/lldbsuite/test/functionalities/thread/thread_exit/TestThreadExit.py (original)
+++ lldb/trunk/packages/Python/lldbsuite/test/functionalities/thread/thread_exit/TestThreadExit.py Thu Aug 2 14:26:19 2018
@@ -8,6 +8,7 @@ from __future__ import print_function
import os
import time
import lldb
+from lldbsuite.test.decorators import *
from lldbsuite.test.lldbtest import *
import lldbsuite.test.lldbutil as lldbutil
@@ -25,6 +26,9 @@ class ThreadExitTestCase(TestBase):
self.break_3 = line_number('main.cpp', '// Set third breakpoint here')
self.break_4 = line_number('main.cpp', '// Set fourth breakpoint here')
+ # The test is actually flakey on Windows, failing every dozen or so runs, but even with the flakey
+ # decorator it still fails
+ @expectedFailureAll(oslist=["windows"], bugnumber="llvm.org/pr38373")
def test(self):
"""Test thread exit handling."""
self.build(dictionary=self.getBuildFlags())
Modified: lldb/trunk/packages/Python/lldbsuite/test/lang/cpp/virtual/TestVirtual.py
URL: http://llvm.org/viewvc/llvm-project/lldb/trunk/packages/Python/lldbsuite/test/lang/cpp/virtual/TestVirtual.py?rev=338769&r1=338768&r2=338769&view=diff
==============================================================================
--- lldb/trunk/packages/Python/lldbsuite/test/lang/cpp/virtual/TestVirtual.py (original)
+++ lldb/trunk/packages/Python/lldbsuite/test/lang/cpp/virtual/TestVirtual.py Thu Aug 2 14:26:19 2018
@@ -36,6 +36,7 @@ class CppVirtualMadness(TestBase):
@expectedFailureAll(
compiler="icc",
bugnumber="llvm.org/pr16808 lldb does not call the correct virtual function with icc.")
+ @skipIfWindows # This test will hang on windows llvm.org/pr21753
def test_virtual_madness(self):
"""Test that expression works correctly with virtual inheritance as well as virtual function."""
self.build()
Modified: lldb/trunk/packages/Python/lldbsuite/test/python_api/event/TestEvents.py
URL: http://llvm.org/viewvc/llvm-project/lldb/trunk/packages/Python/lldbsuite/test/python_api/event/TestEvents.py?rev=338769&r1=338768&r2=338769&view=diff
==============================================================================
--- lldb/trunk/packages/Python/lldbsuite/test/python_api/event/TestEvents.py (original)
+++ lldb/trunk/packages/Python/lldbsuite/test/python_api/event/TestEvents.py Thu Aug 2 14:26:19 2018
@@ -31,6 +31,7 @@ class EventAPITestCase(TestBase):
@expectedFailureAll(
oslist=["linux"],
bugnumber="llvm.org/pr23730 Flaky, fails ~1/10 cases")
+ @skipIfWindows # This test will hang on windows llvm.org/pr21753
def test_listen_for_and_print_event(self):
"""Exercise SBEvent API."""
self.build()
More information about the lldb-commits
mailing list