[PATCH] D96662: [lit] Add --skip (inverse of --filter) and `--xfail`

[Thomas Preud'homme via Phabricator via llvm-commits]

thopre added a comment.

In D96662#2567744 <https://reviews.llvm.org/D96662#2567744>, @jhenderson wrote:

> In D96662#2566903 <https://reviews.llvm.org/D96662#2566903>, @yln wrote:
>
>> I can get behind `--skip` (patch LGTM), but would a little help to understand when `--xfail=path/to/test1;path/to/test2` would be useful (which workflow?).  Thanks!
>
> One example where it would be useful - in our downstream merge process, we may identify a test that has started failing due to an upstream bug. We could modify the test explicitly by adding XFAIL to the start of it, or we could add it to the build script that is used to build and test the changes before they are merged in. The --xfail option allows the latter to happen in such a way that when the issue is fixed by a later upstream change, we'll notice (due to the test starting to pass) and remove the option from the script, whereas with --skip, we won't notice (and thus lose the testing downstream the test gave us). I could imagine other cases along these lines too, where --xfail is superior to --skip.

I was thinking this could be useful to us for the same reason. Some tests fail because they make assumptions about default target or host. We send patch upstream for those but while they are in review and until we merge again, we tend to add xfail in the test. Sometimes the failure is due to the target being different in which case we just mark it unsupported. --skip and --xfail would be quite useful.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D96662/new/

https://reviews.llvm.org/D96662