[llvm-dev] [RFC][FileCheck] New option to negate check patterns

[James Henderson via llvm-dev]

Hi all,

There have been a few cases recently where I've noticed two test cases in
the same lit test that do the same thing except invert the CHECK, to show
that something is NOT present. I'm talking about something like the
following:

# RUN: llvm-sometool --print-string | FileCheck %s --check-prefix=STRING
# RUN: llvm-sometool --no-print-string | FileCheck %s
--check-prefix=NO-STRING
# STRING: This is the string
# NO-STRING-NOT: This is the string

In such cases, as can be seen, the CHECK line effectively has to be
duplicated (either in an explicit check like in the above example, or via
--implicit-check-not). Duplication is generally bad, especially in this
sort of case, as it only takes a typo in the NOT pattern, or a careless
developer/reviewer pair changing the output to cause the NOT pattern to no
longer be useful.

I'd like to propose a new FileCheck option (e.g.
--check-not-prefix/--check-not-prefixes) which allows implicitly converting
a check prefix to a -NOT version of the same prefix. That would allow
writing the above example as:

# RUN: llvm-sometool --print-string | FileCheck %s --check-prefix=STRING
# RUN: llvm-sometool --no-print-string | FileCheck %s
--check-not-prefix=STRING
# STRING: This is the string

If there was a typo or somebody changed the string output, this mechanism
would ensure there is no chance of the pattern rotting.

Caveat: I don't know what would be the appropriate way of handling
non-trivial checks, i.e. existing CHECK-NOT/CHECK-NEXT/CHECK-SAME etc. I'd
appreciate any ideas on this.

Thoughts?

James
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200131/83c34049/attachment.html>