[llvm-dev] FileCheck feature request- by default ignore IR-"headers"

[Reid Kleckner via llvm-dev]

Yeah, this is a documented shortcoming of FileCheck:
http://llvm.org/docs/TestingGuide.html#fragile-tests

I think the most practical thing to do would be to make FileCheck reject
test cases that consist only CHECK-NOT directives, unless a flag is passed
to disable this behavior.

On Thu, Mar 30, 2017 at 8:57 AM, Keane, Erich via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> Hi all-
> I hope this is the right place for this.
>
> Anyway, the primary usage for the utility "FileCheck" is to pattern match
> for specific values in a stream.  This is perfectly consistent and
> deterministic for the most part!  However, when validating clang's LLVM-IR
> generation, it is possible to make an invalid match against the top few
> generated lines (the LLVM-IR Headers), which are inconsistent.
>
> For example (from my personal pain), we have a test:
> clang/test/CodeGen/debug-info-macro.c  that has a 'check-not' on the word
> "macros".  Unfortunately, my SVN workspace name was "ms_macros", which gave
> me a false match.  I'm a bit ashamed to say how long this took me to figure
> out :)
>
> Anywya, to my feature request: to prevent someone else from going through
> this pain again, would it be possible to have FileCheck ignore these first
> 4 lines as default behavior?  I suspect there might be a few tests that
> would want to validate those headers, though I expect those are the vast
> minority, so I would think this should be opt in.
>
> -Erich
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170330/db44289f/attachment.html>