[LLVMdev] [llvm-commits] [PATCH] A "very verbose" mode for FileCheck

[Sean Silva]

On Thu, Jan 17, 2013 at 1:30 PM, Eli Bendersky <eliben at google.com> wrote:
> This is unfortunate. Last month I tweaked TestingGuide.rst to
> discourage grep in favor of FileCheck. It now says:
>
> "The recommended way to examine output to figure out if the test
> passes it using the FileCheck tool. The usage of grep in RUN lines is
> discouraged."
>
> However, perhaps it's time to remove any mention of grep from that document?

One contributing factor to the continued use of grep may be that the
section "Writing New Regression Tests" [1] mentions grep multiple
times, and FileCheck 0 times.

Besides that, I suspect that most new uses of grep come from cargo
culting, so just giving them nothing to cargo cult is an easy and
mechanical way to avoid that part of the issue altogether (i.e. just
removing all uses of grep in the current tests). Also, removing `not`
and `count` from utils/ would incapacitate grep significantly w.r.t.
writing tests with it, discouraging its use and making people look for
a different way.

The cargo culting also points to a lack of adequate documentation
about how to add a new test (`HowToAddATest.rst`? You might be a good
person to write that ;)
<http://www.llvm.org/docs/SphinxQuickstartTemplate.html>). HowTo
documents are a great way to disseminate best practices.

[1] <http://llvm.org/docs/TestingGuide.html#writing-new-regression-tests>

-- Sean Silva