[LLVMdev] [PATCH] A "very verbose" mode for FileCheck

[David Blaikie]

On Wed, Jan 16, 2013 at 10:43 PM, Chris Lattner <clattner at apple.com> wrote:
> On Jan 16, 2013, at 1:19 PM, Dmitri Gribenko <gribozavr at gmail.com> wrote:
>>> I agree that a command line option would be better.  But in that case
>>> all tests should be updated.  It is not an issue for me -- it is
>>> mostly mechanical.  So should I change tests to use %FileCheck?
>>
>> Here's a third attempt.
>
> Thanks in advance for driving this forward.  I'm sorry that such a simple thing is so complicated.
>
>> The new behavior is as follows:
>>
>> 1. In case of errors we always dump output to a temporary file and print
>>
>> Saving input file "<stdin>" to "/tmp/filecheck.txt-Jno73y"
>
> This doesn't make sense to me.  It's really common in a normal development scenario to do something, test and have stuff fail.  It doesn't make sense to dump things into /tmp in this case.

That was a suggestion/request from me - my thinking was that, pretty
much whenever that happens, you end up wanting to look at the output
anyway. My usual process involves copy/pasting the command (if I can
tell which one it is, since lit dumps all the RUN lines out, I think)
& running it to look at the output to see what went wrong. It seems
having that output already available would skip that step - is this
less common for other developers? Is there some other solution that
might address this?

(I suggested this in part in the hopes that we could find a unified
solution for local and buildbot execution, but on consideration I
couldn't find a particularly easy way to gather the extra log file
back to the build master so Dmitri's solution still seemed
necessary/convenient - making my suggestion perhaps orthogonal to his
current proposal)

>
>> 2. If --dump-input-on-error option is passed, FileCheck will also dump
>> input to stderr.
>
> This is fine.
>
>> 3. If FILECHECK_DUMP_INPUT_ON_ERROR env var is set, lit will replace
>> "%FileCheck" with "FileCheck --dump-input-on-error".
>
> Sounds good for lit.
>
>> I will fix all tests in LLVM and Clang if we decide this is the way to go.
>
> What tests need to be fixed?  FileCheck -> %FileCheck?  You should check with Daniel, but would it make sense to have lit just "know" FileCheck?
>
> -Chris