[PATCH] D90281: [FileCheck] Report missing prefixes when more than one is provided.

Tue Oct 27 21:32:49 PDT 2020

mtrofin added a comment.

In D90281#2358154 <https://reviews.llvm.org/D90281#2358154>, @jdoerfert wrote:

> So, this will break with the way the Attributor tests are designed. I will describe briefly how that is and why below. That said, I can see this to be useful, however, 1500 failures leads me to believe the Attributor tests are not the only ones that use such a pattern and it might be worth to have a way to keep the old behavior. Maybe we only error on tests that do not have auto generated check lines or we add a flag to FileCheck or something else ;)
>
> ---
>
> Most Attributor tests have 4 run lines: {oldPM, newPM} x {module pass, cgscc pass}
> Since we have 99 tests, 4 run lines each and a lot of things we want to check, we decided early on to use the update scripts.
> Given all the enhancements we had to put in, some still downstream with me (incl. checking for global metadata), I think it is fair to say the Attributor was stress testing the update scripts quite a bit.
> Anyway, to get the number of run lines down, and to make it explicit when the different combinations are equivalent or different, we use the 7 check prefixes per line, for example
> `CHECK,NOT_TUNIT_NPM,NOT_TUNIT_OPM,NOT_CGSCC_NPM,IS__CGSCC____,IS________OPM,IS__CGSCC_OPM`
> As we modify things we can simply run the update script and always find a valid set of check lines that is concise and expressive.
> If we were to strip the unused prefixes (without having an automatic way to put them back) we would have substantially more work during the updates.
> For us, the extra prefixes are not a problem or a bug. Whatever we do here, I would like a way to opt-in/out to keeping them around.

To make sure I understand, is this capturing it: these test files are generated by a script. A test file will always list the exact same --check-prefixes list, but then internally will only use a subset of the prefixes list. Also for the sake of me understanding, an alternative would be to only generate the strict --check-prefixes subset that's used in the test file; but the added complexity here is that some functionality is still downstream.

If there were a (more terse) "--check-at-least-one-of-the-prefixes" with today's semantics, I assume the test generator produce tests using that option instead?

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D90281/new/

https://reviews.llvm.org/D90281