[llvm] r337609 - [FileCheck] Provide an option for FileCheck to dump original input to stderr on failure

[George Karpenkov via llvm-commits]

Author: george.karpenkov
Date: Fri Jul 20 13:21:57 2018
New Revision: 337609

URL: http://llvm.org/viewvc/llvm-project?rev=337609&view=rev
Log:
[FileCheck] Provide an option for FileCheck to dump original input to stderr on failure

The option can be either set using environment variable (e.g. env
FILECHECK_DUMP_INPUT_ON_FAILURE=1 ninja check-fuzzer) or with a
FileCheck flag.

This can be extremely useful for debugging, cf.
https://groups.google.com/forum/#!topic/llvm-dev/kLrzg8OM_h8 for
discussion.

Differential Revision: https://reviews.llvm.org/D49328

Added:
    llvm/trunk/test/FileCheck/verbose_mode.txt
Modified:
    llvm/trunk/docs/CommandGuide/FileCheck.rst
    llvm/trunk/utils/FileCheck/FileCheck.cpp
    llvm/trunk/utils/lit/lit/TestingConfig.py

Modified: llvm/trunk/docs/CommandGuide/FileCheck.rst
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/CommandGuide/FileCheck.rst?rev=337609&r1=337608&r2=337609&view=diff
==============================================================================
--- llvm/trunk/docs/CommandGuide/FileCheck.rst (original)
+++ llvm/trunk/docs/CommandGuide/FileCheck.rst Fri Jul 20 13:21:57 2018
@@ -77,6 +77,10 @@ OPTIONS
   -verify``. With this option FileCheck will verify that input does not contain
   warnings not covered by any ``CHECK:`` patterns.
 
+.. option:: --dump-input-on-failure
+
+  When the check fails, dump all of the original input.
+
 .. option:: --enable-var-scope
 
   Enables scope for regex variables.

Added: llvm/trunk/test/FileCheck/verbose_mode.txt
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/FileCheck/verbose_mode.txt?rev=337609&view=auto
==============================================================================
--- llvm/trunk/test/FileCheck/verbose_mode.txt (added)
+++ llvm/trunk/test/FileCheck/verbose_mode.txt Fri Jul 20 13:21:57 2018
@@ -0,0 +1,17 @@
+; RUN: not FileCheck -input-file %s %s --check-prefix=CHECK1 --match-full-lines --dump-input-on-failure 2>&1 | FileCheck %s --check-prefix=CHECKERROR --match-full-lines
+; RUN: env FILECHECK_DUMP_INPUT_ON_FAILURE=1 not FileCheck -input-file %s %s --check-prefix=CHECK1 --match-full-lines 2>&1 |  FileCheck %s --check-prefix=CHECKERROR --match-full-lines
+; RUN: env FILECHECK_DUMP_INPUT_ON_FAILURE=1 not FileCheck -input-file %s %s --check-prefix=CHECK1 --match-full-lines --dump-input-on-failure=0 2>&1 |  FileCheck %s --check-prefix=CHECKERRORNOVERBOSE --match-full-lines
+
+hello
+world
+
+; CHECK1: ciao
+; CHECK1-NEXT: world
+
+; CHECKERROR: Full input was:
+; CHECKERROR-NEXT: <<<<<<
+; CHECKERROR: hello
+; CHECKERROR: world
+; CHECKERROR: >>>>>>
+
+; CHECKERRORNOVERBOSE-NOT: <<<<<<

Modified: llvm/trunk/utils/FileCheck/FileCheck.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/FileCheck/FileCheck.cpp?rev=337609&r1=337608&r2=337609&view=diff
==============================================================================
--- llvm/trunk/utils/FileCheck/FileCheck.cpp (original)
+++ llvm/trunk/utils/FileCheck/FileCheck.cpp Fri Jul 20 13:21:57 2018
@@ -97,6 +97,13 @@ static cl::opt<bool> VerboseVerbose(
     "vv", cl::init(false),
     cl::desc("Print information helpful in diagnosing internal FileCheck\n"
              "issues.  Implies -v.\n"));
+static const char * DumpInputEnv = "FILECHECK_DUMP_INPUT_ON_FAILURE";
+
+static cl::opt<bool> DumpInputOnFailure(
+    "dump-input-on-failure", cl::init(std::getenv(DumpInputEnv)),
+    cl::desc("Dump original input to stderr before failing.\n"
+             "The value can be also controlled using\n"
+             "FILECHECK_DUMP_INPUT_ON_FAILURE environment variable.\n"));
 
 typedef cl::list<std::string>::const_iterator prefix_iterator;
 
@@ -1605,5 +1612,9 @@ int main(int argc, char **argv) {
                             InputFileText, InputFile.getBufferIdentifier()),
                         SMLoc());
 
-  return CheckInput(SM, InputFileText, CheckStrings) ? EXIT_SUCCESS : 1;
+  int ExitCode = CheckInput(SM, InputFileText, CheckStrings) ? EXIT_SUCCESS : 1;
+  if (ExitCode == 1 && DumpInputOnFailure)
+    errs() << "Full input was:\n<<<<<<\n" << InputFileText << "\n>>>>>>\n";
+
+  return ExitCode;
 }

Modified: llvm/trunk/utils/lit/lit/TestingConfig.py
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/lit/TestingConfig.py?rev=337609&r1=337608&r2=337609&view=diff
==============================================================================
--- llvm/trunk/utils/lit/lit/TestingConfig.py (original)
+++ llvm/trunk/utils/lit/lit/TestingConfig.py Fri Jul 20 13:21:57 2018
@@ -25,7 +25,8 @@ class TestingConfig:
                      'CLANG', 'LD_PRELOAD', 'ASAN_OPTIONS', 'UBSAN_OPTIONS',
                      'LSAN_OPTIONS', 'ADB', 'ANDROID_SERIAL',
                      'SANITIZER_IGNORE_CVE_2016_2143', 'TMPDIR', 'TMP', 'TEMP',
-                     'TEMPDIR', 'AVRLIT_BOARD', 'AVRLIT_PORT']
+                     'TEMPDIR', 'AVRLIT_BOARD', 'AVRLIT_PORT',
+                     'FILECHECK_DUMP_INPUT_ON_FAILURE']
         for var in pass_vars:
             val = os.environ.get(var, '')
             # Check for empty string as some variables such as LD_PRELOAD cannot be empty