[llvm-commits] [PATCH] A "very verbose" mode for FileCheck

[David Blaikie]

On Wed, Jan 16, 2013 at 11:30 AM, Dmitri Gribenko <gribozavr at gmail.com> wrote:
> On Wed, Jan 16, 2013 at 8:47 PM, Dmitri Gribenko <gribozavr at gmail.com> wrote:
>> On Wed, Jan 16, 2013 at 8:42 PM, David Blaikie <dblaikie at gmail.com> wrote:
>>> On Wed, Jan 16, 2013 at 10:32 AM, Dmitri Gribenko <gribozavr at gmail.com> wrote:
>>>> Hello,
>>>>
>>>> When someone breaks a FileCheck-based test on some buildbot, sometimes
>>>> it may not be obvious *why* did it fail.  If the failure can not be
>>>> reproduced locally, it can be very hard to fix.
>>>
>>> Agreed.
>>>
>>>> I propose adding a "very verbose" mode to FileCheck.  In this mode
>>>> FileCheck will dump the input file in case of failure.
>>>
>>> Why have a separate mode? The same problem of FileCheck failure
>>> investigation exists even when developing locally - quite often after
>>> a FileCheck failure the immediate next step is to run the command
>>> without FileCheck just to get the output. Could we streamline this
>>> scenario in general for both local and remote execution? (eg: have
>>> FileCheck always dump a temp file & log the location)
>>
>> I agree that it is useful.  Will implement and post an updated version.
>
> Updated patch attached.  Now FileCheck prints:
>
> <all the usual messages>
> Saving input file "<stdin>" to "/tmp/filecheck.txt-5PEf0q"
> Input file "<stdin>" contains:
> <...>

Does this address my thought/desire that this should be the default
behavior? (I assume people probably don't want the entire contents
printed on each failure by default, so I guess the answer to my
question is "no")

If we had lit just log the file name on failure, we could improve the
buildbot infrastructure to find/match this file name and generate a
separate log file entry for it. This would allow us not to have a
separate/different lit behavior on the bots while providing the full
details to both local and buildbot use cases.

(I'm not sure if anyone has objections to the idea of lit always
dumping its input data, though - this could slow down test execution,
for example)