<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Tue, May 20, 2014 at 7:34 PM, Alexander Kornienko <span dir="ltr"><<a href="mailto:alexfh@google.com" target="_blank">alexfh@google.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div>We use lit tests for diagnostic messages. Currently they use FileCheck, but it probably makes sense to implement an analog of Clang's -verify in clang-tidy.</div>
<div><br></div>As for the tests for fix-its, I consider unit tests to be more readable for this purpose.</div></blockquote><div><br></div><div>I don't understand this. What could be more readable than something like:</div>
<div><br></div><div><div>namespace {</div><div>} // namespace asdf</div><div>// CHECK: namespace {</div><div>// CHECK-NEXT: } // namespace</div></div><div><br></div><div>?? I certainly prefer it hands-down to:</div><div>
<br></div><div><div>+ EXPECT_EQ("namespace {\n"</div><div>+ "} // namespace",</div><div>+ runCheckOnCode<NamespaceCommentCheck>("namespace {\n"</div><div>+ "} // namespace asdf"));</div>
</div><div><br></div><div>(actually I think FileCheck's default matching of allowing a match of a partial line could be an issue here, but that can be worked around with a regex. I think that even adding a CHECK-EXACT directive or similar to support this in a very readable way might be worth it for this sort of test)</div>
<div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">They are also more isolated and closer to the check itself.</div>
</blockquote><div><br></div><div>I don't understand this. The check is either 1) performing a particular source code transformation or 2) providing a diagnostic message about a questionable construct. In case 1) you want to verify that the output takes a particular plaintext form, i.e. exactly what a FileCheck does. In case 2) you can avoid rolling your own -verify and use the simpler but almost surely sufficient facility built into FileCheck <a href="http://llvm.org/docs/CommandGuide/FileCheck.html#filecheck-expressions">http://llvm.org/docs/CommandGuide/FileCheck.html#filecheck-expressions</a></div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"> In a lit test, if we verify the full text of the output, the FileCheck tests become quite bulky.</div>
</blockquote><div> </div></div><div>Could you give an example? It is generally possible to create minimal test cases so that the actual output (and the input, in fact) are quite compact. yaml2obj is probably a worst case in this respect but we've still found it easy to keep the tests small and readable. E.g.</div>
<div><br></div><div><a href="http://reviews.llvm.org/diffusion/L/browse/llvm/trunk/test/Object/yaml2obj-elf-symbol-LocalGlobalWeak.yaml">http://reviews.llvm.org/diffusion/L/browse/llvm/trunk/test/Object/yaml2obj-elf-symbol-LocalGlobalWeak.yaml</a><br>
</div><div><a href="http://reviews.llvm.org/diffusion/L/browse/llvm/trunk/test/Object/yaml2obj-elf-bits-endian.test">http://reviews.llvm.org/diffusion/L/browse/llvm/trunk/test/Object/yaml2obj-elf-bits-endian.test</a><br></div>
<div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">Another reason to put these checks in the unit tests, is that each new lit test file has non-trivial run-time overhead.</div>
</blockquote><div><br></div><div>Personally I've never found myself waiting on an incremental lit test run (i.e. rerunning a particular failing test or relevant test directory for the code I'm working on). I *have* found myself waiting significant amounts of time for a unittest (gtest) compile and link in that same scenario.</div>
<div><br></div><div>-- Sean Silva</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">
<div class=""><div>
<div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, May 20, 2014 at 7:32 PM, Sean Silva <span dir="ltr"><<a href="mailto:chisophugis@gmail.com" target="_blank">chisophugis@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">Is there a particular reason that this is being tested with unittests instead of the usual lit tests? This kind of thing seems like it would be a lot more readable as a lit test.<span><font color="#888888"><div>
<br></div><div>-- Sean Silva</div></font></span></div></blockquote></div>
</div></div></div></div>
</blockquote></div><br></div></div>