[llvm-dev] RFC: FileCheck Enhancements

Thu May 26 11:34:34 PDT 2016

Hi James, all,

> For all of these changes: why/where are they actually useful?
> Previous enhancements of FileCheck have mostly been added when needed by
a test. It would greatly help your case for adding these new enhancements
to show some tests which would be improved, or have greater clarity, by the
use of these features.

Let me clarify. Elena's proposals are driven by development and testing of
"downstream" commercial toolchain for embedded processors.
The proposed enhancements have practical use cases in internal code base.
We did decide to upstream them.
However, we didn't look for representative examples in upstream repository
(yet).

"3. Expressions repeat  for CHECK"
Tests containing duplicating CHECK lines: e.g. you want to match expression
K times or maybe skip it to find specific expression.
Currently we have to duplicate CHECK or use grep and sed/awk scripts count
patterns and then compare numbers later.
E.g. count number of LD instructions inside function or loop body. This
extension can help to describe that as CHECK pattern.

"5. CHECK-LABEL-DAG"
CHECK-LABEL is used to annotate top-level entities, e.g. functions.
Suppose your tool can reorder output fragments (e.g. functions in ASM)
comparing to original source file.
LABEL-DAG can help.

"8.       Prefix with regular expressions - If statement should be checked
if prefix matches some regular expression, it should be used {{regex}}:,
{{regex}}-NEXT  and etc. "
Our processors are highly configurable and we compile/apply same test for
many ISA extensions options.
Image 100s variants of the same CHECK statements for various ISA
configurations.
CHECK regexp prefixes can explain logic in compact and readable way, making
it easier to understand

Use cases are similar to Daniel's example for MIPS:

               {{MIPS32R[2-5]}} would match MIPS32R2 through to MIPS32R6

                {{MIPS(32|64)R6}} would match MIPS32R6 and MIPS64R6

                {{MIPS64.*}} would match any MIPS64

Thanks,

Sergey

On Thu, May 26, 2016 at 7:13 PM, James Y Knight via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> For all of these changes: why/where are they actually useful?
>
> Previous enhancements of FileCheck have mostly been added when needed by a
> test. It would greatly help your case for adding these new enhancements to
> show some tests which would be improved, or have greater clarity, by the
> use of these features.
>
> I'm particularly skeptical about "3. Expressions repeat  for CHECK", "5. CHECK-LABEL-DAG",
> and (as noted before "7. Wildcard for prefixes" and "8. Prefix with
> regular expressions"
>
> For "6. Check statement for words only" -- I think it might be better to
> just make that be the ONLY behavior, rather than an additional option -- if
> you intended to end a match in the middle of  a word, stick {{[^ ]*}} on it.
>
> On Tue, May 24, 2016 at 9:51 AM, Elena Lepilkina via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Hi everyone,
>>
>>
>>
>> There was idea to add new directives to FileCheck:
>>
>> 1.       Directive to use some patterns as named template with or
>> without parameters.
>>
>> 2.       CHECK-INCLUDE - Directive to include other file with checks to
>> another.
>>
>> 3.       Expressions repeat  for CHECK - If statement should be checked
>> several times repeat modifiers {n}, {n,m} , {,n}, {n,}, *, + can be used.
>>
>> 4.       Repeat in regexs - Repeat with current number should become
>> available by using {n}, {n,m} , {,n}, {n,}
>>
>> 5.       CHECK-LABEL-DAG - Not sequential order of labels.
>>
>> 6.       Check statement for words only - // CHECK-WORD, //
>> CHECK-WORD-NEXT, // CHECK-WORD-SAME, // CHECK-WORD-DAG, // CHECK-WORD-NOT.
>>
>> 7.       Wildcard for prefixes - If some statements should be checked
>> regardless prefix, it should be used //{{*}}, //{{*}}-NEXT, //{{*}}-SAME
>> and etc.
>>
>> 8.       Prefix with regular expressions - If statement should be
>> checked if prefix matches some regular expression, it should be used
>> {{regex}}:, {{regex}}-NEXT  and etc.
>>
>>
>>
>> More information in file
>> https://docs.google.com/document/d/1wAKNzU7-S2EeK1-aADwgP8dEiKfByKNazonybCQW3zs/edit?usp=sharing
>> .
>>
>>
>>
>> Now we have prototype with these features. It’s tested on LLVM 3.8.
>>
>> There was found unsupported before directive in old test. Bug about this
>> - https://llvm.org/bugs/show_bug.cgi?id=27852.
>>
>>
>>
>> There is about 6% slowdown with new features when we tested them on 3.8.
>>
>>
>>
>> I see that there are some changes in FileCheck LLVM 3.9 with new features
>> too. We can publish patch for 3.8 and it can be adapted for LLVM 3.9. Is it
>> interesting for anyone? And how will be better to publish patch as for 3.8
>> or for 3.9?
>>
>>
>>
>> Thanks,
>>
>> Elena.
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160526/374797b1/attachment.html>