[llvm-dev] Super-verbose failure mode for FileCheck

[George Karpenkov via llvm-dev]

Hi All,

FileCheck is a great tool for writing tests.
However, it can be also a great tool for hiding the error cause from you, especially if a very common idiom

RUN: ./myprog | FileCheck %s

is used.
If you need to know the output of “myprog” to figure out why your test is failing —  then well, good luck,
you would need to manually copy’n paste that line, run it, and then note the output.
(produced line “expected/got” are rarely enough to debug the problem)

Would it be useful to have an optional, off-by-default mode for FileCheck, turned on by an environment variable,
which would save all the input it gets, and then dump it on failure?
I also wish we could get a 

The test case I imagine would be to run:

ninja check-X

note the failure,
and then rerun:

env FILECHECK_SUPER_VERBOSE=1 ninja check-X

Thoughts?
George