[llvm-dev] FileCheck idiom difficulties

James Henderson via llvm-dev llvm-dev at lists.llvm.org
Wed Nov 6 08:44:45 PST 2019


Thanks, I had considered using the CHECK-NOT approach. That would work in
most cases, I believe, but I'm always nervous to use it because CHECK-NOTs
can start failing to check the right thing if output changes. For example,
George Rimar recently made changes that turned some list-like structs into
dictionaries (using curly brackets instead of square brackets) in the
readobj output, which could theoretically have affected the '}' style.

However, the implicit-check-not approach probably works for many cases.
Thanks.

On Wed, 6 Nov 2019 at 15:49, David Blaikie <dblaikie at gmail.com> wrote:

> I'd usually write this sort of test (& there are many in the DWARF
> emission tests):
>
>
> # CHECK:      Name: foo
> # CHECK-NOT: }
> # CHECK: Section: .foo (1)
>
> Or, if you're checking every section, you could use a --implicit-check-not=Symbol
> or similar.
>
> On Wed, Nov 6, 2019 at 4:18 AM James Henderson via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>>
>> Hi all,
>>
>> Many of our lit tests use FileCheck and a tool like llvm-readobj to check
>> properties of a section header/symbol/etc. A typical (pseudoised for
>> brevity) output to match against might be something like the following:
>>
>> Symbols [
>>   Symbol {
>>     Name: foo
>>     Value: 0
>>     Type: Function
>>     Section: .foo (1)
>>   }
>>   Symbol {
>>     Name: bar
>>     Value: 1
>>     Type: Object
>>     Section: .foo (1)
>>   }
>> ]
>>
>> and your lit test might want to check the properties of the foo symbol
>> like so:
>>
>> # CHECK:      Name: foo
>> # CHECK-NEXT: Value: 0
>> # CHECK-NEXT: Type: Function
>> # CHECK-NEXT: Section: .foo (1)
>>
>> This is fine. But what if you only care about the section of a symbol,
>> and not the value or type etc? You could do the following:
>>
>> # CHECK: Name: foo
>> # CHECK: Section: .foo (1)
>>
>> Hopefully some of you will already notice the problem with this approach:
>> if foo was in, say, the .baz section, the test will spuriously pass,
>> because the Section line will match the Section line for .bar. One
>> alternative to this is to explicitly match each field in between, using
>> CHECK-NEXT:
>>
>> # CHECK:      Name: foo
>> # CHECK-NEXT: Value:
>> # CHECK-NEXT: Type:
>> # CHECK-NEXT: Section: .foo (1)
>>
>> This works, but somewhat hides what is really being tested by adding
>> extra noise to the checks. In reality, there are actually other fields too
>> that need to be listed, meaning the "interesting" parts of the test are
>> even more hidden.
>>
>> I recently started using yet another approach:
>>
>> # CHECK: Name: foo
>> # CHECK: Section:
>> # CHECK-SAME:     .foo (1)
>>
>> This works because the Section: matched will be the first one found, i.e.
>> the one belonging to foo, and then .foo will be looked for on the same
>> line. However, I noticed today that this pattern has its own problem,
>> namely that there could be something between the Section tag and .foo. In
>> other words, the above pattern would match "Section: .bar.foo". A couple of
>> solutions to this are:
>>
>> # CHECK: Section:
>> # CHECK-NOT: {{[:graph:]}}
>> # CHECK-SAME: .foo (1)
>>
>> # CHECK: Section:
>> # CHECK-SAME: {{^}} .foo (1)
>>
>> The first one ensures that there's no non-whitespace between the end of
>> "Section:" and the start of ".foo (1)". The second ensures that the start
>> of the CHECK-SAME match is the "start of line", and since the first half of
>> the line has already been consumed, it means " .foo (1)" must immediately
>> follow "Section:". However, the first is even less readable than the
>> current CHECK-SAME approach, whilst the second is somewhat confusing if you
>> don't realise that FileCheck effectively consumes the things it has matched
>> already, so that they effectively don't exist any more.
>>
>> Does anybody have any other suggestions/thoughts/comments? One idea I had
>> was for a new directive something like "CHECK-IMMEDIATE" which is
>> implicitly the same as the final approach I suggested above, but maybe
>> adding a new directive to achieve this isn't the right approach?
>>
>> James
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20191106/3aa8b98f/attachment-0001.html>


More information about the llvm-dev mailing list