[LLVMdev] [llvm-commits] [PATCH] A "very verbose" mode for FileCheck

[Eli Bendersky]

On Thu, Jan 17, 2013 at 12:37 PM, Sean Silva <silvas at purdue.edu> wrote:
> On Thu, Jan 17, 2013 at 1:30 PM, Eli Bendersky <eliben at google.com> wrote:
>> This is unfortunate. Last month I tweaked TestingGuide.rst to
>> discourage grep in favor of FileCheck. It now says:
>>
>> "The recommended way to examine output to figure out if the test
>> passes it using the FileCheck tool. The usage of grep in RUN lines is
>> discouraged."
>>
>> However, perhaps it's time to remove any mention of grep from that document?
>
> One contributing factor to the continued use of grep may be that the
> section "Writing New Regression Tests" [1] mentions grep multiple
> times, and FileCheck 0 times.
>
> Besides that, I suspect that most new uses of grep come from cargo
> culting, so just giving them nothing to cargo cult is an easy and
> mechanical way to avoid that part of the issue altogether (i.e. just
> removing all uses of grep in the current tests). Also, removing `not`
> and `count` from utils/ would incapacitate grep significantly w.r.t.
> writing tests with it, discouraging its use and making people look for
> a different way.
>
> The cargo culting also points to a lack of adequate documentation
> about how to add a new test (`HowToAddATest.rst`? You might be a good
> person to write that ;)
> <http://www.llvm.org/docs/SphinxQuickstartTemplate.html>). HowTo
> documents are a great way to disseminate best practices.
>
> [1] <http://llvm.org/docs/TestingGuide.html#writing-new-regression-tests>
>

Good points, Sean. I'll take a look what I can do there later this week.

Eli