On Tue, Oct 2, 2012 at 5:49 PM, Dmitri Gribenko <span dir="ltr"><<a href="mailto:gribozavr@gmail.com" target="_blank">gribozavr@gmail.com</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="HOEnZb"><div class="h5">On Tue, Oct 2, 2012 at 6:46 PM, Alexander Kornienko <<a href="mailto:alexfh@google.com">alexfh@google.com</a>> wrote:<br>
> On Tue, Oct 2, 2012 at 5:36 PM, Dmitri Gribenko <<a href="mailto:gribozavr@gmail.com">gribozavr@gmail.com</a>> wrote:<br>
>><br>
>> On Tue, Oct 2, 2012 at 6:28 PM, Alexander Kornienko <<a href="mailto:alexfh@google.com">alexfh@google.com</a>><br>
>> wrote:<br>
>> > On Tue, Oct 2, 2012 at 5:15 PM, Dmitri Gribenko <<a href="mailto:gribozavr@gmail.com">gribozavr@gmail.com</a>><br>
>> > wrote:<br>
>> >><br>
>> >> On Tue, Oct 2, 2012 at 5:59 PM, Alexander Kornienko <<a href="mailto:alexfh@google.com">alexfh@google.com</a>><br>
>> >> wrote:<br>
>> >> > So I'm definitely for using FileCheck in this case, but it's only me,<br>
>> >> > others<br>
>> >> > may disagree.<br>
>> >><br>
>> >> I'm in favor of ASTMatchers-based tests for anything dumping related,<br>
>> >> based on my experience of maintaining<br>
>> >> test/Index/annotate-comments.cpp. Any small change to the testcases<br>
>> >> forces a change of line numbers for everything that follows it (half<br>
>> >> of tests, on average).<br>
>> ><br>
>> > FileCheck has variables now, maybe we can add a special __LINE__<br>
>> > variable<br>
>> > (possibly with a way to specify offset)? In this case we could put CHECK<br>
>> > lines near the actual test code and this would not be bound to absolute<br>
>> > line<br>
>> > numbers. What do you think?<br>
>><br>
>> That would help somewhat. But nevertheless, annotate-comments.cpp is<br>
>> the slowest testcase in the testsuite (for some reason, FileCheck<br>
>> takes up to 10s to match it in debug version -- spending most of the<br>
>> time in regex matching).<br>
><br>
><br>
> I'll look into adding special variables to FileCheck.<br>
><br>
> As for debug build, can we somehow use release FileCheck even when testing a<br>
> debug configuration?<br>
><br>
>><br>
>> >> With ASTMatchers-based tests we can keep all<br>
>> >> logically related tests in a single file and keep the test case close<br>
>> >> to the expected output while ensuring that different tests are<br>
>> >> isolated from each other.<br>
>> ><br>
>> > Isolation is possible now using separate RUN lines with specific -D<br>
>> > defines<br>
>> > and #ifdef blocks.<br>
>><br>
>> That's more like putting the isolation into a tool that didn't have<br>
>> it, versus having an approach where testcases are intrinsically<br>
>> isolated.<br>
><br>
> My problem with ASTMatchers-based tests for dumping is that there are lots<br>
> of dependencies, that can be avoided. And the test infrastructure (a custom<br>
> test infrastructure, used only for one test) is an additional possible point<br>
> of failure, and definitely requires some support. In my opinion it makes<br>
> sense only in case this infrastructure is generalized, moved elsewhere and<br>
> gets its own tests. But still this seems to be overkill.<br>
<br>
</div></div>The infrastructure is rather simple in my opinion, and can (should!)<br>
be shared between (Decl|Stmt)(Printer|Dumper) tests.</blockquote><div><br></div><div>Let's put it this way: having a way to use "previous line" or "current line + offset" variable in FileCheck, and having a better FileCheck performance in debug mode, what would you prefer?</div>
<div><br></div><div><div><font face="courier new, monospace">TEST(StmtDumper, TestDeclStmt) {<span class="Apple-tab-span" style="white-space:pre"> </span></font></div><div><font face="courier new, monospace"> ASSERT_TRUE(DumpedStmtCXX98Matches(<span class="Apple-tab-span" style="white-space:pre"> </span></font></div>
<div><font face="courier new, monospace"> "void A() {\n"<span class="Apple-tab-span" style="white-space:pre"> </span></font></div><div><font face="courier new, monospace"> " int a = 1, b;\n"<span class="Apple-tab-span" style="white-space:pre"> </span></font></div>
<div><font face="courier new, monospace"> "}",<span class="Apple-tab-span" style="white-space:pre"> </span></font></div><div><font face="courier new, monospace"> "A",<span class="Apple-tab-span" style="white-space:pre"> </span></font></div>
<div><font face="courier new, monospace"> "(DeclStmt 0xXXXXXXXX <input.cc:2:3, col:15>\n"<span class="Apple-tab-span" style="white-space:pre"> </span></font></div><div><font face="courier new, monospace"> " (VarDecl 0xXXXXXXXX <col:3, col:11> a 'int'\n"<span class="Apple-tab-span" style="white-space:pre"> </span></font></div>
<div><font face="courier new, monospace"> " (IntegerLiteral 0xXXXXXXXX <col:11> 'int' 1))\n"<span class="Apple-tab-span" style="white-space:pre"> </span></font></div><div><font face="courier new, monospace"> " (VarDecl 0xXXXXXXXX <col:3, col:14> b 'int'))\n"));<span class="Apple-tab-span" style="white-space:pre"> </span></font></div>
<div><font face="courier new, monospace">}</font></div></div><div><br></div><div>or</div><div><br></div><div><font face="courier new, monospace">// RUN: clang_cc1 -ast-dump --std=c++98 -DTEST33 %s | FileCheck --strict-whitespace --check-prefix CHECK33 %s</font></div>
<div><font face="courier new, monospace">#ifdef TEST33</font></div><div><font face="courier new, monospace">void A() {</font></div><div><font face="courier new, monospace">// CHECK33: {{^}}(DeclStmt 0x{{........}} <input.cc:[[__LINE_MINUS_1__]]:3, col:15></font></div>
<div><font face="courier new, monospace"> int a = 1, b;</font></div><div><font face="courier new, monospace">// CHECK33-NEXT: {{^ }}(VarDecl 0x{{........}} <col:3, col:11> a 'int'</font><span style="font-family:'courier new',monospace">{{$}}</span></div>
<div></div><div><font face="courier new, monospace">// CHECK33-NEXT: {{^ }}(IntegerLiteral 0x{{........}} <col:11> 'int' 1))</font><span style="font-family:'courier new',monospace">{{$}}</span></div>
<div><div><span style="font-family:'courier new',monospace">// CHECK33-NEXT:</span><span style="font-family:'courier new',monospace"> {{^</span><font face="courier new, monospace"> }}(VarDecl 0x{{........}} <col:3, col:14> b 'int')){{$}}</font></div>
</div><div><span style="font-family:'courier new',monospace">}</span></div><div><font face="courier new, monospace">#endif</font></div><div><br></div><div>These both seem ugly to some extent, both use some form of escaping either for code or for check patterns. But the second option has less dependencies, everything is in one place (including compiler arguments), there's no need to include any headers here, no need to use any additional libraries, no compilation required prior to running the test, you can play with it using clang alone, and check it with FileCheck if needed. And the most important thing for me is that it doesn't have its own pattern matching rules, but reuses the tool already widely known and used in llvm. The first option is a bit less verbose, but it uses custom infrastructure, that has no other use-cases except tests for dump stuff.</div>
<div><br></div><div>-- </div><div>Regards,</div><div>Alex</div></div>