[PATCH] Add support to FileCheck for out-of-order matching

[Michael Liao]

Hi,

Here's a quick-n-dirty support of CHECK-DAG. I tried my best to capture
its behavior suggested/discussed here and revise it to simplify the
implementation.

Basically, CHECK-DAG is quite similar to CHECK-NOT, i.e. it will verify
patterns against the string between two matches (or before the first
match, or after the last match.) There's no specific order among
CHECK-DAG patterns being verified against that string or the order is
implementation specific. For each match, it will have 3 possible
results: 'mismatching', 'matching', or 'deferrred'. 'deferred' means the
matching cannot proceed because the variable referenced or used is not
defined yet. For 'mismatching' pattens, CHECK-DAG match will fail
immediately. For 'matching' patterns, CHECK-DAG match will addd special
mark on them and won't check them again. CHECK-DAG match will skip
'deferred' patterns and re-check them once there're new variables
defined. If there's no chance to re-check 'deferred' patterns again,
e.g. no more patterns defines new variables, CHECK-DAG fails as well.

Jakob, for the example you rasied, it might be necessary for testers to
rewrite it from

	1: CHECK: testfunctionname
        2: CHECK-DAG: add [[REG1:r[0-9]+]], r1, r2
        3: CHECK-DAG: add [[REG2:r[0-9]+]], r3, r4
        4: CHECK: mul [[REG1]], [[REG2]]

to

	1: CHECK: testfunctionname
        2: CHECK-DAG: add [[REG1]], r1, r2
        3: CHECK-DAG: add [[REG2]], r3, r4
        4: CHECK: mul [[REG1:r[0-9]+]], [[REG2:r[0-9]+]]

It's possible to support the former one similar to the two passes you
suggested. E.g. once a CHECK with CHECK-DAG patterns is 'deferred', we
will match its CHECK-DAG patterns from the last match point. After we
match all its CHECK-DAG patterns, we will verify it again. If there's a
match, we will check the match position to see whether it falls behind
all match positions of its CHECK-DAG patterns (or we will verify it from
the farthest match position of its CHECK-DAG patterns). If not, that
pattern fails. This will need more code to handle that, make the
matching failure more obscure IMHO, and add more runtime overhead since
the region to be checked is larger. However, I'm OK if we want to
support the former one.

Thanks
- Michael

On Thu, 2013-04-11 at 09:41 -0700, Jakob Stoklund Olesen wrote:
> On Apr 11, 2013, at 9:10 AM, Eli Bendersky <eliben at google.com> wrote:
> 
> > On Thu, Apr 11, 2013 at 9:00 AM, Jakob Stoklund Olesen <stoklund at 2pi.dk> wrote:
> 
> > Thanks for the explanation. I'm not sure about the -DAG name though; apart from being overloaded in LLVM, it isn't intuitive for this use.
> 
> I don't care what name we use.
> 
> >  On second thought, that would make this invalid:
> > 
> > 1: CHECK: testfunctionname
> > 2: CHECK-DAG: add [[REG1:r[0-9]+]], r1, r2
> > 3: CHECK-DAG: add [[REG2:r[0-9]+]], r3, r4
> > 4: CHECK: mul [[REG1]], [[REG2]]
> > 
> > The last anchor CHECK wouldn't be able to refer to variables defined by CHECK-DAGs preceding it. That seems confusing in a file with only positive tests.
> > 
> > [Currently, CHECK-NOTs can actually use variables defined by the anchor below them. I don't think we use that a lot.]
> > 
> > 
> > Yes, I agree this is confusing. In general, the semantics of mixing such grouping with variable definitions are somewhat moot. If I read this correctly, we can also have multiple valid matches depending on the order the CHECK-DAG lines are searched.
> 
> I don't think it is necessary to do anything other than a simple first-match. There is no need to get into graph theory.
> 
> I can fix the problem with having two passes:
> 
> CHECK-DAGs are matched on the first pass. They search the range from the previous anchor CHECK, or the last CHECK-DAG to define a variable they're using, whichever comes last. They search to the end of the file, like normal CHECKs.
> 
> A CHECK following a sequence of CHECK-DAGs starts searching from the last match in the sequence, which may not be the immediately preceding CHECK-DAG line since they match out of order. This means it will match text following all preceding CHECK-DAGs.
> 
> It's a bit tricky to define what CHECK-NOT does when mixed with CHECK-DAGs. It could be useful to give it the same range as a CHECK-DAG, except terminating at the next anchor CHECK as usual.
> 
> /jakob
> 
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits

-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Add-CHECK-DAG-support.patch
Type: text/x-patch
Size: 16148 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20130422/5340b7ff/attachment.bin>