[llvm-commits] [PATCH] A "very verbose" mode for FileCheck

[Dmitri Gribenko]

On Wed, Jan 16, 2013 at 9:39 PM, Eli Bendersky <eliben at google.com> wrote:
> On Wed, Jan 16, 2013 at 11:34 AM, David Blaikie <dblaikie at gmail.com> wrote:
>> On Wed, Jan 16, 2013 at 11:30 AM, Dmitri Gribenko <gribozavr at gmail.com> wrote:
>>> On Wed, Jan 16, 2013 at 8:47 PM, Dmitri Gribenko <gribozavr at gmail.com> wrote:
>>>> On Wed, Jan 16, 2013 at 8:42 PM, David Blaikie <dblaikie at gmail.com> wrote:
>>>>> On Wed, Jan 16, 2013 at 10:32 AM, Dmitri Gribenko <gribozavr at gmail.com> wrote:
>>>>>> Hello,
>>>>>>
>>>>>> When someone breaks a FileCheck-based test on some buildbot, sometimes
>>>>>> it may not be obvious *why* did it fail.  If the failure can not be
>>>>>> reproduced locally, it can be very hard to fix.
>>>>>
>>>>> Agreed.
>>>>>
>>>>>> I propose adding a "very verbose" mode to FileCheck.  In this mode
>>>>>> FileCheck will dump the input file in case of failure.
>>>>>
>>>>> Why have a separate mode? The same problem of FileCheck failure
>>>>> investigation exists even when developing locally - quite often after
>>>>> a FileCheck failure the immediate next step is to run the command
>>>>> without FileCheck just to get the output. Could we streamline this
>>>>> scenario in general for both local and remote execution? (eg: have
>>>>> FileCheck always dump a temp file & log the location)
>>>>
>>>> I agree that it is useful.  Will implement and post an updated version.
>>>
>>> Updated patch attached.  Now FileCheck prints:
>>>
>>> <all the usual messages>
>>> Saving input file "<stdin>" to "/tmp/filecheck.txt-5PEf0q"
>>> Input file "<stdin>" contains:
>>> <...>
>>
>> Does this address my thought/desire that this should be the default
>> behavior? (I assume people probably don't want the entire contents
>> printed on each failure by default, so I guess the answer to my
>> question is "no")
>>
>> If we had lit just log the file name on failure, we could improve the
>> buildbot infrastructure to find/match this file name and generate a
>> separate log file entry for it. This would allow us not to have a
>> separate/different lit behavior on the bots while providing the full
>> details to both local and buildbot use cases.
>>
>> (I'm not sure if anyone has objections to the idea of lit always
>> dumping its input data, though - this could slow down test execution,
>> for example)
>> _________________________________
>
> Yes, the effect of dumping a (sometimes largish) file on each
> invocation of FileCheck should be measured, since we run thousands
> upon thousands of instances of FileCheck for each "make check-all".

It is done only on errors, so I don't think we care about performance
impact (if any).

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr at gmail.com>*/