[Lldb-commits] [lldb] [lldb] Support expectedFlakey decorator in dotest (PR #129817)

[David Peixotto via lldb-commits]

https://github.com/dmpots created https://github.com/llvm/llvm-project/pull/129817

The dotest framework had an existing decorator to mark flakey tests. It was not implemented and would simply run the underlying test. This commit modifies the decorator to run the test multiple times in the case of failure and only raises the test failure when all attempts fail.

>From 1d1c3a6bf52ba47a4ff4c13e99209a6ae3d983b3 Mon Sep 17 00:00:00 2001
From: David Peixotto <peix at meta.com>
Date: Tue, 4 Mar 2025 11:39:03 -0800
Subject: [PATCH] [lldb] Support expectedFlakey decorator in dotest

The dotest framework had an existing decorator to mark flakey tests. It
was not implemented and would simply run the underlying test. This
commit modifies the decorator to run the test multiple times in the case
of failure and only raises the test failure when all attempts fail.
---
 .../Python/lldbsuite/test/decorators.py       | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/lldb/packages/Python/lldbsuite/test/decorators.py b/lldb/packages/Python/lldbsuite/test/decorators.py
index c48c0b2f77944..c4de3f8f10751 100644
--- a/lldb/packages/Python/lldbsuite/test/decorators.py
+++ b/lldb/packages/Python/lldbsuite/test/decorators.py
@@ -3,6 +3,7 @@
 from packaging import version
 import ctypes
 import locale
+import logging
 import os
 import platform
 import re
@@ -525,12 +526,24 @@ def expectedFailureWindows(bugnumber=None):
     return expectedFailureOS(["windows"], bugnumber)
 
 
-# TODO: This decorator does not do anything. Remove it.
-def expectedFlakey(expected_fn, bugnumber=None):
+# This decorator can be used to mark a test that can fail non-deterministically.
+# The decorator causes the underlying test to be re-run if it encounters a
+# failure. After `num_retries` attempts the test will be marked as a failure
+# if it has not yet passed.
+def expectedFlakey(expected_fn, bugnumber=None, num_retries=3):
     def expectedFailure_impl(func):
         @wraps(func)
         def wrapper(*args, **kwargs):
-            func(*args, **kwargs)
+            for i in range(1, num_retries+1):
+                try:
+                    return func(*args, **kwargs)
+                except Exception:
+                    logging.warning(
+                        f"expectedFlakey: test {func} failed attempt ({i}/{num_retries})"
+                    )
+                    # If the last attempt fails then re-raise the original failure.
+                    if i == num_retries:
+                        raise
 
         return wrapper