[llvm-dev] FileCheck feature request- by default ignore IR-"headers"

[Keane, Erich via llvm-dev]

Alright, I guess it isn’t just my pain then, it makes it feel better ☺  I think that proposed feature would be really nice, since it would encourage people to write tests that have a //CHECK: some-thing-after-header first!

From: Reid Kleckner [mailto:rnk at google.com]
Sent: Thursday, March 30, 2017 11:15 AM
To: Keane, Erich <erich.keane at intel.com>
Cc: llvm-dev at lists.llvm.org
Subject: Re: [llvm-dev] FileCheck feature request- by default ignore IR-"headers"

Yeah, this is a documented shortcoming of FileCheck:
http://llvm.org/docs/TestingGuide.html#fragile-tests

I think the most practical thing to do would be to make FileCheck reject test cases that consist only CHECK-NOT directives, unless a flag is passed to disable this behavior.

On Thu, Mar 30, 2017 at 8:57 AM, Keane, Erich via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
Hi all-
I hope this is the right place for this.

Anyway, the primary usage for the utility "FileCheck" is to pattern match for specific values in a stream.  This is perfectly consistent and deterministic for the most part!  However, when validating clang's LLVM-IR generation, it is possible to make an invalid match against the top few generated lines (the LLVM-IR Headers), which are inconsistent.

For example (from my personal pain), we have a test: clang/test/CodeGen/debug-info-macro.c  that has a 'check-not' on the word "macros".  Unfortunately, my SVN workspace name was "ms_macros", which gave me a false match.  I'm a bit ashamed to say how long this took me to figure out :)

Anywya, to my feature request: to prevent someone else from going through this pain again, would it be possible to have FileCheck ignore these first 4 lines as default behavior?  I suspect there might be a few tests that would want to validate those headers, though I expect those are the vast minority, so I would think this should be opt in.

-Erich
_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170330/12dcd35f/attachment.html>