[PATCH] D80694: Improve lit.py's usability by highlighting failing lines.

[Varun Gandhi via Phabricator via llvm-commits]

varungandhi-apple created this revision.
varungandhi-apple added a reviewer: yln.
Herald added subscribers: llvm-commits, delcypher.
Herald added a project: LLVM.
varungandhi-apple added a comment.
varungandhi-apple edited the summary of this revision.

Consider the test file:

  # RUN: true
  # RUN: false
  # RUN: true

After this patch, the default output becomes:

  ******************** TEST 'lit :: dummy.py' FAILED ********************
  
  Command Output (stdout, truncated):
  --
  $ ":" "RUN: at line 2"
  $ "false"
  note: command had no output on stdout or stderr
  error: command failed with exit status: 1
  
  --
  (NOTE: The failure may depend on preceding RUN lines.
         Use --verbose to see preceding RUN lines and outputs.)

If you use `-v`/`--verbose`, it becomes:

F12017804: Screen Shot 2020-05-28 at 12.10.46 AM.png <https://reviews.llvm.org/F12017804>


- Before:
  - default (no flags): no script, no command output
  - -v: full script, no command output
  - -vv: full script, shows command output up to and including failing line
- After:
  - default (no flags): no script, show only failing line in command output
  - -v: full script, highlight failing line in script, shows command output up to and including failing line, highlight failing output
  - -vv: alias to -v (preserve backwards compatibility)

Background discussion: http://lists.llvm.org/pipermail/llvm-dev/2019-September/135098.html
This patch doesn't follow the exact suggestion in that thread, but I think it's pretty close and arguably better.

I do have a couple of questions:

1. Is it ok to rename echo_all_commands to runWithCommandOutput? I think the latter represents the meaning better (since the behavior is no longer controlled by the --echo-all-commands flag), but that might break compatibility for out-of-tree users (should be a small fix though).
2. Should TerminalController be split out into a different file?

I don't feel strongly about the colors/styling. Happy to change it if there are common color schemes where it would become unreadable.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D80694

Files:
  llvm/utils/lit/lit/LitConfig.py
  llvm/utils/lit/lit/OutputSettings.py
  llvm/utils/lit/lit/TestRunner.py
  llvm/utils/lit/lit/cl_arguments.py
  llvm/utils/lit/lit/display.py
  llvm/utils/lit/lit/main.py
  llvm/utils/lit/tests/shtest-format.py
  llvm/utils/lit/tests/shtest-run-at-line.py

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D80694.266757.patch
Type: text/x-patch
Size: 22502 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20200528/e1913a52/attachment.bin>