[all-commits] [llvm/llvm-project] b383ef: [lit] Optionally exclude xfail tests (#151191)

[Mircea Trofin via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: b383efc3e3e18b57e1a13bd59aef4b5964410321
      https://github.com/llvm/llvm-project/commit/b383efc3e3e18b57e1a13bd59aef4b5964410321
  Author: Mircea Trofin <mtrofin at google.com>
  Date:   2025-07-31 (Thu, 31 Jul 2025)

  Changed paths:
    M llvm/docs/CommandGuide/lit.rst
    M llvm/utils/lit/lit/Test.py
    M llvm/utils/lit/lit/TestRunner.py
    M llvm/utils/lit/lit/cl_arguments.py
    M llvm/utils/lit/lit/main.py
    A llvm/utils/lit/tests/Inputs/xfail-cl/true-xfail-conditionally.txt
    M llvm/utils/lit/tests/xfail-cl.py

  Log Message:
  -----------
  [lit] Optionally exclude xfail tests (#151191)

See the related issue. We want to set up a build bot where `opt` runs with `-enable-profcheck`, which inserts `MD_prof` before running the rest of the pipeline requested from `opt`, and then validates resulting profile information (more info in the RFC linked by the aforementioned issue)

In that setup, we will also ignore `FileCheck`: while the profile info inserted is, currently, equivalent to the profile info a pass would observe via `BranchProbabilityInfo`/`BlockFrequencyInfo`, (1) we may want to change that, and (2) some tests are quite sensitive to the output IR, and break if, for instance, extra metadata is present (which it would be due to `-enable-profcheck`). Since we're just interested in profile consistency on the upcoming bot, ignoring `FileCheck` is simpler and sufficient. However, this has the effect of passing XFAIL tests. Rather than listing them all, the alternative is to just exclude XFAIL tests.

This PR adds support for that by introducing a `--exclude-xfail` option to `llvm-lit`.

Issue #147390



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications