[llvm-dev] Super-verbose failure mode for FileCheck

[David Blaikie via llvm-dev]

Could be useful - as Paul mentioned, having it on standard output could be
good - especially for buildbots (where rerunning the command is
impractical). Though a more generalized solution for 'lit' where it dumps
all the inputs and outputs (well, I guess, the textually representable ones
- maybe that's basically just FileCheck) in a way that buildbot could split
them out into separate files (easier to read, download/use, etc than one
big stream), would be extra super awesome.

On Thu, Jul 12, 2018 at 6:31 PM George Karpenkov via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> Hi All,
>
> FileCheck is a great tool for writing tests.
> However, it can be also a great tool for hiding the error cause from you,
> especially if a very common idiom
>
> RUN: ./myprog | FileCheck %s
>
> is used.
> If you need to know the output of “myprog” to figure out why your test is
> failing —  then well, good luck,
> you would need to manually copy’n paste that line, run it, and then note
> the output.
> (produced line “expected/got” are rarely enough to debug the problem)
>
> Would it be useful to have an optional, off-by-default mode for FileCheck,
> turned on by an environment variable,
> which would save all the input it gets, and then dump it on failure?
> I also wish we could get a
>
> The test case I imagine would be to run:
>
> ninja check-X
>
> note the failure,
> and then rerun:
>
> env FILECHECK_SUPER_VERBOSE=1 ninja check-X
>
> Thoughts?
> George
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180713/e08a89bb/attachment.html>