[llvm] r307938 - [lit] add a -vv option to echo all executed commands.

[George Karpenkov via llvm-commits]

Author: george.karpenkov
Date: Thu Jul 13 12:26:27 2017
New Revision: 307938

URL: http://llvm.org/viewvc/llvm-project?rev=307938&view=rev
Log:
[lit] add a -vv option to echo all executed commands.

Debugging LIT scripts can be rather painful, as LIT directly does not
specify which line has failed.
Rather, FileCheck is expected to report the failing location, but it can
be often ambiguous if multiple commands are tested against the same
prefix.  This change adds a -vv option, which echoes all output.
Then detecting the error becomes straightforward: last printed line is
the failing one.

Of course, it could be desired to try to get failing line number
directly from bash, but it involves excessive hacks on older bash
versions (cf.
https://stackoverflow.com/questions/24398691/how-to-get-the-real-line-number-of-a-failing-bash-command)

Differential Revision: https://reviews.llvm.org/D35330

Modified:
    llvm/trunk/docs/CommandGuide/lit.rst
    llvm/trunk/utils/lit/lit/LitConfig.py
    llvm/trunk/utils/lit/lit/TestRunner.py
    llvm/trunk/utils/lit/lit/main.py

Modified: llvm/trunk/docs/CommandGuide/lit.rst
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/CommandGuide/lit.rst?rev=307938&r1=307937&r2=307938&view=diff
==============================================================================
--- llvm/trunk/docs/CommandGuide/lit.rst (original)
+++ llvm/trunk/docs/CommandGuide/lit.rst Thu Jul 13 12:26:27 2017
@@ -80,6 +80,13 @@ OUTPUT OPTIONS
  Show more information on test failures, for example the entire test output
  instead of just the test result.
 
+.. option:: -vv, --echo-all-commands
+
+ Echo all commands to stdout, as they are being executed.
+ This can be valuable for debugging test failures, as the last echoed command
+ will be the one which has failed.
+ This option implies ``--verbose``.
+
 .. option:: -a, --show-all
 
  Show more information about all tests, for example the entire test

Modified: llvm/trunk/utils/lit/lit/LitConfig.py
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/lit/LitConfig.py?rev=307938&r1=307937&r2=307938&view=diff
==============================================================================
--- llvm/trunk/utils/lit/lit/LitConfig.py (original)
+++ llvm/trunk/utils/lit/lit/LitConfig.py Thu Jul 13 12:26:27 2017
@@ -25,7 +25,8 @@ class LitConfig(object):
                  params, config_prefix = None,
                  maxIndividualTestTime = 0,
                  maxFailures = None,
-                 parallelism_groups = []):
+                 parallelism_groups = [],
+                 echo_all_commands = False):
         # The name of the test runner.
         self.progname = progname
         # The items to add to the PATH environment variable.
@@ -64,6 +65,7 @@ class LitConfig(object):
         self.maxIndividualTestTime = maxIndividualTestTime
         self.maxFailures = maxFailures
         self.parallelism_groups = parallelism_groups
+        self.echo_all_commands = echo_all_commands
 
     @property
     def maxIndividualTestTime(self):

Modified: llvm/trunk/utils/lit/lit/TestRunner.py
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/lit/TestRunner.py?rev=307938&r1=307937&r2=307938&view=diff
==============================================================================
--- llvm/trunk/utils/lit/lit/TestRunner.py (original)
+++ llvm/trunk/utils/lit/lit/TestRunner.py Thu Jul 13 12:26:27 2017
@@ -715,6 +715,8 @@ def executeScript(test, litConfig, tmpBa
     else:
         if test.config.pipefail:
             f.write('set -o pipefail;')
+        if litConfig.echo_all_commands:
+            f.write('set -x;')
         f.write('{ ' + '; } &&\n{ '.join(commands) + '; }')
     f.write('\n')
     f.close()

Modified: llvm/trunk/utils/lit/lit/main.py
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/lit/main.py?rev=307938&r1=307937&r2=307938&view=diff
==============================================================================
--- llvm/trunk/utils/lit/lit/main.py (original)
+++ llvm/trunk/utils/lit/lit/main.py Thu Jul 13 12:26:27 2017
@@ -199,6 +199,12 @@ def main_with_tmp(builtinParameters):
     format_group.add_argument("-v", "--verbose", dest="showOutput",
                      help="Show test output for failures",
                      action="store_true", default=False)
+    format_group.add_argument("-vv", "--echo-all-commands",
+                     dest="echoAllCommands",
+                     action="store_true", default=False,
+                     help="Echo all commands as they are executed to stdout.\
+                     In case of failure, last command shown will be the\
+                     failing one.")
     format_group.add_argument("-a", "--show-all", dest="showAllOutput",
                      help="Display all commandlines and output",
                      action="store_true", default=False)
@@ -303,6 +309,9 @@ def main_with_tmp(builtinParameters):
     if opts.maxFailures == 0:
         parser.error("Setting --max-failures to 0 does not have any effect.")
 
+    if opts.echoAllCommands:
+        opts.showOutput = True
+
     inputs = args
 
     # Create the user defined parameters.
@@ -338,7 +347,8 @@ def main_with_tmp(builtinParameters):
         config_prefix = opts.configPrefix,
         maxIndividualTestTime = maxIndividualTestTime,
         maxFailures = opts.maxFailures,
-        parallelism_groups = {})
+        parallelism_groups = {},
+        echo_all_commands = opts.echoAllCommands)
 
     # Perform test discovery.
     run = lit.run.Run(litConfig,