[PATCH] D76288: [lit] Add builtin support for flaky tests in lit

[Louis Dionne via Phabricator via llvm-commits]

ldionne added a reviewer: yln.
ldionne marked 2 inline comments as done.
ldionne added inline comments.


================
Comment at: llvm/utils/lit/lit/TestRunner.py:1508
+    # Re-run failed tests up to test.allowed_retries times.
+    attempts = test.allowed_retries + 1
     for i in range(attempts):
----------------
rnk wrote:
> I'm curious, do you find this logic of having one try and then adding N retries reasonable or confusing? I wasn't sure about it when I wrote it. Should the term "retry" be exclusive from the first try?
I think it is reasonable. For me, a *retry* is something you do after the first try. So the total number of attempts is `retries + 1` -- it makes sense that way in my head.


================
Comment at: llvm/utils/lit/tests/allow-retries.py:3
+
+# RUN: rm -f %t.counter
+# RUN: %{lit} -j 1 %{inputs}/allow-retries/succeeds-within-limit.py -Dcounter=%t.counter -Dpython=%{python} | FileCheck --check-prefix=CHECK-TEST1 %s
----------------
rnk wrote:
> I see, cleaning up the counter file is taken care of at a higher level.
I'm cleaning up *before* I run the test to make sure the counter file does not already exist, which is the case when running the tests over and over again.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D76288/new/

https://reviews.llvm.org/D76288