[llvm-dev] [FileCheck] Fix --strict-whitespace --match-full-lines

[Robinson, Paul via llvm-dev]

Please send patches to llvm-commits not llvm-dev.

Writing FileCheck tests has pitfalls. A test along these lines:

bla0
CHECK:bla1

will actually pass, because the CHECK pattern is also part of the input
so it will readily match itself.  You want the CHECK lines not to match
themselves, which you can easily do by introducing {{}} into the (middle
of the) pattern.  That is:

bla0
CHECK:{{bla1}}

will still pass (incorrectly), while

bla0
CHECK:bla{{1}}

will fail (correctly).  A correct FileCheck test would thus be

bla1
CHECK:bla{{1}}


(I didn't include this in my recent FileCheck Follies lightning talk,
because testing FileCheck itself is kind of an obscure corner of the
LLVM world and I was already bumping up against the 5-minute time limit.)
--paulr


> -----Original Message-----
> From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of Tom
> de Vries via llvm-dev
> Sent: Wednesday, December 14, 2016 4:38 AM
> To: llvm-dev at lists.llvm.org
> Cc: Jonathan Roelofs
> Subject: [llvm-dev] [FileCheck] Fix --strict-whitespace --match-full-lines
> 
> Hi,
> 
> this patch fixes a problem with leading/trailing whitespace matching for
> FileCheck --strict-whitespace --match-full-lines.
> 
> Consider a text file:
> ...
> $ cat DUMP
> bla1
> bla2
>   bla3
> bla4
>   bla5
> ...
> 
> with some leading and trailing spaces, made more visible like this:
> ...
> $ sed 's/ /_/g' DUMP
> bla1
> bla2
> _bla3
> bla4_
> _bla5_
> ...
> 
> and a FileCheck file CHECK to match DUMP:
> ...
> $ cat CHECK
> // CHECK-LABEL:bla1
> // CHECK-NEXT:bla2
> // CHECK-NEXT: bla3
> // CHECK-NEXT:bla4
> // CHECK-NEXT: bla5
> ...
> 
> with whitespace made more visible like this:
> ...
> $ sed 's/ /_/g' CHECK
> //_CHECK-LABEL:bla1
> //_CHECK-NEXT:bla2
> //_CHECK-NEXT:_bla3
> //_CHECK-NEXT:bla4_
> //_CHECK-NEXT:_bla5_
> ...
> 
> When trying the match, it fails:
> ...
> $ cat DUMP | FileCheck CHECK --strict-whitespace --match-full-lines
> CHECK:3:16: error: expected string not found in input
> // CHECK-NEXT: bla3
>                 ^
> <stdin>:3:2: note: scanning from here
>   bla3
>   ^
> ...
> 
> I expect the match to succeed, because I expect leading and trailing
> whitespace _not_ to be ignored, because the documentation states:
> ...
>   --match-full-lines
> 
>      By default, FileCheck allows matches of anywhere on a line. This
> option will require all positive matches to cover an entire line.
> Leading and trailing whitespace is ignored, unless --strict-whitespace
> is also specified.
> ...
> 
> After adding some debug code to FileCheck (which I proposed here on
> llvm-dev ml as '[FileCheck] Add --verbose'), we can see where things go
> wrong:
> ...
> $ cat DUMP | /home/vries/gt/build/./bin/FileCheck CHECK
> --strict-whitespace --match-full-lines --verbose
> CHECK:3:16: note: RegEx string match: '^bla3$'
> // CHECK-NEXT: bla3
> ...
> 
> The resulting regexp string is '^bla3$' instead of '^ bla3$'.
> 
> The patch fixes this, and makes the behavior match the documentation.
> 
> I ran the llvm/test/FileCheck tests, no regressions.
> 
> Any comments? OK for trunk?
> 
> Thanks,
> - Tom