[LLVMdev] [PATCH] A "very verbose" mode for FileCheck

[Dmitri Gribenko]

On Thu, Jan 17, 2013 at 8:43 AM, Chris Lattner <clattner at apple.com> wrote:
> On Jan 16, 2013, at 1:19 PM, Dmitri Gribenko <gribozavr at gmail.com> wrote:
>>> I agree that a command line option would be better.  But in that case
>>> all tests should be updated.  It is not an issue for me -- it is
>>> mostly mechanical.  So should I change tests to use %FileCheck?
>>
>> Here's a third attempt.
>
> Thanks in advance for driving this forward.  I'm sorry that such a simple thing is so complicated.
>
>> The new behavior is as follows:
>>
>> 1. In case of errors we always dump output to a temporary file and print
>>
>> Saving input file "<stdin>" to "/tmp/filecheck.txt-Jno73y"
>
> This doesn't make sense to me.  It's really common in a normal development scenario to do something, test and have stuff fail.  It doesn't make sense to dump things into /tmp in this case.
>
>> 2. If --dump-input-on-error option is passed, FileCheck will also dump
>> input to stderr.
>
> This is fine.
>
>> 3. If FILECHECK_DUMP_INPUT_ON_ERROR env var is set, lit will replace
>> "%FileCheck" with "FileCheck --dump-input-on-error".
>
> Sounds good for lit.
>
>> I will fix all tests in LLVM and Clang if we decide this is the way to go.
>
> What tests need to be fixed?  FileCheck -> %FileCheck?  You should check with Daniel, but would it make sense to have lit just "know" FileCheck?

Hello Daniel & llvm-commits,

We are implementing a "very verbose" for FileCheck, so that FileCheck
will dump the input to stderr in case of mismatch.  This will make it
easier to debug failures on remote buildbots.

This will be a separate mode in FileCheck, activated by "FileCheck
--dump-input-on-error".  lit should add that option to FileCheck
invocations on buildbots (lit can see if it is running on a buildbot
by checking an environment variable).

We have to options:
(a) replace 'FileCheck' with '%FileCheck' in all tests, and teach
'lit' to replace '%FileCheck' with 'FileCheck --dump-input-on-error';

(b) teach 'lit' to replace a plain 'FileCheck'.

The first approach seems cleaner to developers who read and write
tests (it suggests that they are invoking some "macro" -- but does
that matter?)  The second approach is much easier to implement since
tests will be unchanged.

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr at gmail.com>*/