[llvm] [lit] Echo full RUN lines in case of external shells (PR #65267)

[Joel E. Denny via llvm-commits]

https://github.com/jdenny-ornl created https://github.com/llvm/llvm-project/pull/65267:

Before <https://reviews.llvm.org/D154984> and <https://reviews.llvm.org/D156954>, lit reported full RUN lines in a `Script:` section.  Now, in the case of lit's internal shell, it's the execution trace that includes them.  However, if lit is configured to use an external shell (e.g., bash, windows `cmd`), they aren't reported at all.

A fix was requested at the following:

* <https://reviews.llvm.org/D154984#4627605>
* <https://discourse.llvm.org/t/rfc-improving-lits-debug-output/72839/35?u=jdenny-ornl>

This patch does not correctly address the case when the external shell is windows `cmd`.  As discussed at <https://github.com/llvm/llvm-project/pull/65242>, it's not clear whether that's a use case that people still care about, and it seems to be generally broken anyway.

>From d9ca09051a49262fefb3a978b9749397b597d744 Mon Sep 17 00:00:00 2001
From: "Joel E. Denny" <jdenny.ornl at gmail.com>
Date: Mon, 4 Sep 2023 10:58:00 -0400
Subject: [PATCH] [lit] Echo full RUN lines in case of external shells

Before <https://reviews.llvm.org/D154984> and
<https://reviews.llvm.org/D156954>, lit reported full RUN lines in a
`Script:` section.  Now, in the case of lit's internal shell, it's the
execution trace that includes them.  However, if lit is configured to
use an external shell (e.g., bash, windows `cmd`), they aren't
reported at all.

A fix was requested at the following:

* <https://reviews.llvm.org/D154984#4627605>
* <https://discourse.llvm.org/t/rfc-improving-lits-debug-output/72839/35?u=jdenny-ornl>

This patch does not correctly address the case when the external shell
is windows `cmd`.  As discussed at
<https://github.com/llvm/llvm-project/pull/65242>, it's not clear
whether that's a use case that people still care about, and it seems
to be generally broken anyway.
---
 llvm/utils/lit/lit/TestRunner.py           | 8 +++++++-
 llvm/utils/lit/tests/shtest-run-at-line.py | 8 ++++----
 2 files changed, 11 insertions(+), 5 deletions(-)

diff --git a/llvm/utils/lit/lit/TestRunner.py b/llvm/utils/lit/lit/TestRunner.py
index 461cf63d6b9685f..4d1caad5369c83e 100644
--- a/llvm/utils/lit/lit/TestRunner.py
+++ b/llvm/utils/lit/lit/TestRunner.py
@@ -1153,8 +1153,14 @@ def executeScript(test, litConfig, tmpBase, commands, cwd):
     for j, ln in enumerate(commands):
         match = re.match(kPdbgRegex, ln)
         if match:
+            dbg = match.group(1)
             command = match.group(2)
-            commands[j] = match.expand(": '\\1'; \\2" if command else ": '\\1'")
+            commands[j] = f"echo '{dbg}'"
+            if command:
+                commands[j] += f": {shlex.quote(command.lstrip())} >&2 " \
+                               f"&& {command}"
+            else:
+                commands[j] += " has no command after substitutions"
             if litConfig.per_test_coverage:
                 # Extract the test case name from the test object
                 test_case_name = test.path_in_suite[-1]
diff --git a/llvm/utils/lit/tests/shtest-run-at-line.py b/llvm/utils/lit/tests/shtest-run-at-line.py
index a0626f872c4c9e0..8322c27dc4cc525 100644
--- a/llvm/utils/lit/tests/shtest-run-at-line.py
+++ b/llvm/utils/lit/tests/shtest-run-at-line.py
@@ -14,14 +14,14 @@
 
 # CHECK-LABEL: FAIL: shtest-run-at-line :: external-shell/basic.txt
 
-# CHECK:     RUN: at line 4
-# CHECK:     RUN: at line 5
+# CHECK:     {{^}}RUN: at line 4: true
+# CHECK:     {{^}}RUN: at line 5: false
 # CHECK-NOT: RUN
 
 # CHECK-LABEL: FAIL: shtest-run-at-line :: external-shell/line-continuation.txt
 
-# CHECK:     RUN: at line 4
-# CHECK:     RUN: at line 6
+# CHECK:     {{^}}RUN: at line 4: echo 'foo bar' | FileCheck
+# CHECK:     {{^}}RUN: at line 6: echo 'foo baz' | FileCheck
 # CHECK-NOT: RUN