[all-commits] [llvm/llvm-project] 49923e: [lit] Add --rerun-failed-serially <REGEX> mechanis...

[Adrian Prantl via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 49923e950b3e8f0adb693aa68b99df6a2479418b
      https://github.com/llvm/llvm-project/commit/49923e950b3e8f0adb693aa68b99df6a2479418b
  Author: Adrian Prantl <aprantl at apple.com>
  Date:   2026-09-16 (Wed, 16 Sep 2026)

  Changed paths:
    M llvm/docs/CommandGuide/lit.md
    M llvm/utils/lit/lit/Test.py
    M llvm/utils/lit/lit/cl_arguments.py
    M llvm/utils/lit/lit/main.py
    A llvm/utils/lit/tests/Inputs/rerun-failed-serially-retries/lit.cfg
    A llvm/utils/lit/tests/Inputs/rerun-failed-serially-retries/needs-four-attempts.py
    A llvm/utils/lit/tests/Inputs/rerun-failed-serially/always-fails.py
    A llvm/utils/lit/tests/Inputs/rerun-failed-serially/fails-once.py
    A llvm/utils/lit/tests/Inputs/rerun-failed-serially/lit.cfg
    A llvm/utils/lit/tests/rerun-failed-serially.py

  Log Message:
  -----------
  [lit] Add --rerun-failed-serially <REGEX> mechanism (#223504)

CI systems sometimes run on hardware or operating system versions that
have known problems that need to be worked around. This change in
particular is motivated by an issue with certain versions of macOS that
very rarely (but often enough to be a problem at CI scale) drop a burst
of parallel debugserver's attach requests. The goal is to recognize this
very situation and the re-run the affected tests without opnening the
door for the entire testsuite to become less reliable due to the retry
mechanism.

Tests that pass the second time are reported as FLAKYPASS, which keeps
them out of the failure count and the exit status.

--max-retries-per-test composes with this: the rerun honors
allowed_retries as well, so each of its attempts is retried in place as
before.

The regex argument restricts the rerun to failures whose output matches
a regular expression. Without it, any flaky test that happens to pass on
the rerun is laundered into a pass; with it, only the failures a known
platform bug produces get a second chance.

Assisted-by: Claude



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications