[PATCH] Add CHECK-UNIQ directive to FileCheck to allow more accurate error messages and error recovery

[Stephen Lin]

(The reason I think it'd be harder is because, to get the same effect,
the error message code would have to backtrack and attempt to deduce
the actual source of the problem, since it might not be the line
actually causing the error, all based on heuristics...)

On Wed, May 29, 2013 at 2:34 PM, Stephen Lin <swlin at post.harvard.edu> wrote:
> Hmm, that could probably work, but it seems like it's putting a bit
> too much domain-specific knowledge into FileCheck? Also, I think that
> implementation is actually harder, but I haven't thought it through
> entirely.
>
> It wouldn't be safe to reuse my code and simply naively convert any
> CHECK of a certain pattern to the internal equivalent of what
> CHECK-UNIQ parses as, because it would break in case there actually is
> more than one line with that pattern.
>
> I was going to do a mass conversion of labels/defines/etc. into
> CHECK-UNIQ as a separate patch, but with sanity checking first to
> verify that uniqueness actually holds.
>
> -Stephen
>
>
>
> On Wed, May 29, 2013 at 2:07 PM, Reid Kleckner <rnk at google.com> wrote:
>> If this only affects error messages, could we simply teach FileCheck some
>> heuristics for bounding the region of the intended match, based on common
>> patterns like 'CHECK: define ...', 'CHECK: asdf:', or 'CHECK: }' ?
>>
>>
>> On Wed, May 29, 2013 at 1:51 PM, Stephen Lin <swlin at post.harvard.edu> wrote:
>>>
>>> Anyone have any thoughts on this? :)
>>> -Stephen
>>>
>>> ---------- Forwarded message ----------
>>> From: Stephen Lin <swlin at post.harvard.edu>
>>> Date: Sat, May 18, 2013 at 5:36 PM
>>> Subject: [PATCH] Add CHECK-UNIQ directive to FileCheck to allow more
>>> accurate error messages and error recovery
>>> To: llvm-commits at cs.uiuc.edu
>>> Cc: Eli Bendersky <eliben at google.com>
>>>
>>>
>>> Hi,
>>>
>>> This patch adds a new directive, CHECK-UNIQ, which is meant to be used
>>> to match unique identifier lines (like labels).
>>>
>>> If present in a match file, FileCheck will use these directives to
>>> split the input into blocks that are independently processed, ensuring
>>> that a CHECK does not inadvertently match a line in a different block
>>> (which can lead to a misleading/useless error message when the error
>>> is eventually caught). Also, FileCheck can now recover from errors
>>> within blocks by continuing to the next block.
>>>
>>> Please let me know if you have any comments or suggestions.
>>>
>>> Also, if people like this idea, I will also script a mass update of
>>> tests to use CHECK-UNIQ in common cases, like assembly labels and IR
>>> function definition lines. (I will include a sanity check to make sure
>>> each pattern does indeed uniquely match a single line of the input
>>> before updating.)
>>>
>>> Stephen
>>>
>>> ---------- Forwarded message ----------
>>> From: Eli Bendersky <eliben at google.com>
>>> Date: Thu, Apr 25, 2013 at 10:37 AM
>>> Subject: Re: [LLVMdev] Minor FileCheck proposal: CHECK-UNIQUE for
>>> labels to improve error messages
>>> To: Stephen Lin <swlin at post.harvard.edu>
>>> Cc: llvmdev <LLVMdev at cs.uiuc.edu>
>>>
>>>
>>> On Wed, Apr 24, 2013 at 8:05 PM, Stephen Lin <swlin at post.harvard.edu>
>>> wrote:
>>> >
>>> > Hi,
>>> >
>>> > Apologies if this has been proposed before; I couldn't find anything
>>> > in basic searching.
>>> >
>>> > I've been writing tests lately and noticed that the error messages are
>>> > not very helpful in cases where a check is incorrect but matches
>>> > something that occurs in a later block: the checker continues assuming
>>> > that the matched line is correct (no matter how much farther ahead it
>>> > occurs) and then provides a misleading error message some time later.
>>> >
>>> > The problem is exacerbated if many similar tests are put in one file,
>>> > since many checks can end up succeeding with unintended matches before
>>> > something ends up failing.
>>> >
>>> > I propose a simple addition to FileCheck syntax to improve error
>>> > messages in this case: a CHECK-UNIQUE directive which indicates that a
>>> > given check is a unique match that should only occur once (and only
>>> > once) in a given file and can only be matched with the given
>>> > CHECK-UNIQUE; this would be used for labels or any other unique
>>> > identifier lines.
>>> >
>>> > The CHECK-UNIQUE lines can then allow a better error message to be
>>> > produced, by basically treating each section between CHECK-UNIQUE
>>> > lines separately, so the actual line causing the error can be
>>> > pinpointed more accurately.
>>> >
>>> > Anyone else thing this is a good idea? I'm happy to work on this if so.
>>>
>>>
>>> I like this idea. There are many places in the test cases where this
>>> idiom is needed and currently CHECK followed by a CHECK-NOT is used,
>>> which is cumbersome.
>>>
>>> Eli
>>>
>>> _______________________________________________
>>> llvm-commits mailing list
>>> llvm-commits at cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>>>
>>