[LLVMdev] [llvm-commits] [PATCH] A "very verbose" mode for FileCheck

[Eli Bendersky]

On Thu, Jan 17, 2013 at 10:20 AM, Jim Grosbach <grosbach at apple.com> wrote:
>
> On Jan 17, 2013, at 9:57 AM, Dmitri Gribenko <gribozavr at gmail.com> wrote:
>
>> On Thu, Jan 17, 2013 at 7:51 PM, Sean Silva <silvas at purdue.edu> wrote:
>>> On Thu, Jan 17, 2013 at 8:36 AM, Dmitri Gribenko <gribozavr at gmail.com> wrote:
>>>> We have to options:
>>>> (a) replace 'FileCheck' with '%FileCheck' in all tests, and teach
>>>> 'lit' to replace '%FileCheck' with 'FileCheck --dump-input-on-error';
>>>>
>>>> (b) teach 'lit' to replace a plain 'FileCheck'.
>>>>
>>>> The first approach seems cleaner to developers who read and write
>>>> tests (it suggests that they are invoking some "macro" -- but does
>>>> that matter?)  The second approach is much easier to implement since
>>>> tests will be unchanged.
>>>
>>> IMO the biggest issue with (a) is that developers will continue to use
>>> `FileCheck` instead of `%FileCheck`. So IMO (a) should only be
>>> implemented if simultaneously there is a change that makes just plain
>>> `FileCheck` an error.
>>
>> I think that within a month this knowledge will be propagated to all developers.
>
> I'd like to think so, too, but we still get patches that write tests using 'grep' instead of FileCheck.
>

This is unfortunate. Last month I tweaked TestingGuide.rst to
discourage grep in favor of FileCheck. It now says:

"The recommended way to examine output to figure out if the test
passes it using the FileCheck tool. The usage of grep in RUN lines is
discouraged."

However, perhaps it's time to remove any mention of grep from that document?

Eli