[llvm-dev] [FileCheck] Fix --strict-whitespace --match-full-lines

[Robinson, Paul via llvm-dev]

> -----Original Message-----
> From: Tom de Vries [mailto:Tom_deVries at mentor.com]
> Sent: Thursday, December 15, 2016 9:12 AM
> To: Robinson, Paul
> Cc: Jonathan Roelofs; llvm-dev at lists.llvm.org
> Subject: Re: [llvm-dev] [FileCheck] Fix --strict-whitespace --match-full-
> lines
> 
> On 15/12/16 17:51, Robinson, Paul wrote:
> >
> >
> >> -----Original Message-----
> >> From: Tom de Vries [mailto:Tom_deVries at mentor.com]
> >> Sent: Thursday, December 15, 2016 2:31 AM
> >> To: Robinson, Paul
> >> Cc: Jonathan Roelofs; llvm-dev at lists.llvm.org
> >> Subject: Re: [llvm-dev] [FileCheck] Fix --strict-whitespace --match-
> full-
> >> lines
> >>
> >> On 14/12/16 18:48, Robinson, Paul wrote:
> >>> Please send patches to llvm-commits not llvm-dev.
> >>>
> >>> Writing FileCheck tests has pitfalls. A test along these lines:
> >>>
> >>> bla0
> >>> CHECK:bla1
> >>>
> >>> will actually pass, because the CHECK pattern is also part of the
> input
> >>> so it will readily match itself.  You want the CHECK lines not to
> match
> >>> themselves, which you can easily do by introducing {{}} into the
> (middle
> >>> of the) pattern.  That is:
> >>>
> >>> bla0
> >>> CHECK:{{bla1}}
> >>>
> >>> will still pass (incorrectly), while
> >>>
> >>> bla0
> >>> CHECK:bla{{1}}
> >>>
> >>> will fail (correctly).  A correct FileCheck test would thus be
> >>>
> >>> bla1
> >>> CHECK:bla{{1}}
> >>>
> >>>
> >>
> >> I see. ISTM though that the FileCheck tests are written in a style that
> >> is not the typical usage of FileCheck.
> >>
> >> As far as I've seen sofar, a typical FileCheck usage looks like:
> >> ...
> >> <comment-token> RUN: ignore-comments-and-translate %s \
> >> <comment-token> RUN: | FileCheck %s
> >>
> >> ... input for translation ...
> >>
> >> <comment-token> CHECK: ... check translation ...
> >> <comment-token> CHECK: ... check translation ...
> >> ...
> >>
> >> So if you run the FileCheck tests in a similar way, like this:
> >> ...
> >> ; RUN: sed 's/^;.*$//' %s \
> >> ; RUN: | FileCheck %s
> >> ...
> >> you don't have to obfuscate the check patterns to work around this
> >> problem.
> >
> > An intriguing trick, unfortunately it didn't work when I tried it on
> > Windows.  The Windows shell doesn't strip the single-quotes, so sed
> > doesn't understand the command; but a Posix shell requires the quotes,
> > to avoid globbing the .* part of the expression.  We could work around
> > that by using 'REQUIRES: shell' but at that point you're basically
> > never testing FileCheck on Windows, which seems like a pretty serious
> > hole in the test coverage.
> >
> 
> What about implicit-check-not.txt?
> 
> It contains the sed command, but with '#' instead of '/':
> ...
> ; RUN: sed 's#^;.*##' %s | FileCheck -check-prefix=CHECK-PASS
> -implicit-check-not=warning: %s
> ...
> 
> Does that also not work on windows?

Huh. Thanks for pointing that out!  sed with single-quote doesn't
work directly on the command line, but it looks like 'lit' will
convert them to double-quotes on Windows, and that does work.

Windows doesn't natively provide 'sed' but we do express a
dependency on some GnuWin32 tools, and 'sed' is one of those. So
while Jon is correct that we're not keen on lots of dependencies,
and in particular you'll see occasional rounds of people removing
the use of 'grep' from tests, a search of the test tree shows that
the 'sed' is used in a variety of places (generally with slashes)
so what you've done here should be fine.

So, LGTM!
Thanks,
--paulr

> 
> Thanks,
> - Tom
>