[llvm-dev] [RFC] FileCheck: (dis)allowing unused prefixes

Mircea Trofin via llvm-dev llvm-dev at lists.llvm.org
Thu Oct 29 10:28:22 PDT 2020

Hello all,

TL;DR; if you used FileCheck --check-prefixes and you missed (misspelled,
for instance) one of the prefixes in your test, FileCheck silently ignores
that and the test passes.

1579 tests have this property.

*The details*
Please refer to https://reviews.llvm.org/D90281 and the discussion there
for more details (make sure you open "older changes" for full context)

The problem is covered by the TL;DR;.

The proposal is to add an explicit flag to FileCheck,
--allow-unused-prefixes, to indicate whether the current behavior is
intended (for instance, jdoerfert contributed a scenario where that is the

We want the default behavior to be 'strict', i.e.
--allow-unused-prefixes=false. Doing that right now would lead to 1500 test

To get there (thanks, maskray, for suggestion), we propose we:
* land D90281 where the flag is introduced, but is flipped to match today's
* employ a 'busy beavers' approach, where test maintainers patch their
  - either leveraging the flag, to explicitly indicate that unused prefixes
is intended (i.e. add --allow-unused-patches=true); or
  - fix the test (e.g. maybe there was a misspelling issue/omission/etc).

A spreadsheet with the failing tests is available here

The request to the community members is to please sign up for their
respective area in the spreadsheet, and then mark it completed when that's
the case (yes/no in the respective column).

When all the tests are fixed, we will then flip --allow-unused-prefixes to
false by default.

Meanwhile, please consider leveraging the flag explicitly when you author
new tests that use --check-prefixes. That can be then cleaned up easily
after we switch to the 'strict' behavior.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20201029/fdb099cc/attachment.html>

More information about the llvm-dev mailing list