[llvm] [lit] Add support for readfile to external shell (PR #159431)

[Mikael Holmen via llvm-commits]

mikaelholmen wrote:

Hi,

Is it only me that see ```utils/lit/tests/shtest-readfile-external.py``` failing randomly every now and then?
I don't know what's happening but there seem to be something flaky about this test because in downstream build bots we sometimes see it failing and then passing again on the same commit.

I just failed for me like
```
FAIL: lit :: shtest-readfile-external.py (89176 of 103680)
******************** TEST 'lit :: shtest-readfile-external.py' FAILED ********************
Exit Code: 1

Command Output (stdout):
--
# RUN: at line 4
env LIT_USE_INTERNAL_SHELL=0 not env -u FILECHECK_OPTS "/app/vbuild/RHEL8-x86_64/python/3.8.0/bin/python3.8" /repo/llvm/utils/lit/lit.py -j1 --order=lexical -a -v Inputs/shtest-readfile | FileCheck -match-full-lines -DTEMP_PATH=/repo/llvm/build-all-builtins/utils/lit/tests/Inputs/shtest-readfile/Output /repo/llvm/build-all-builtins/utils/lit/tests/shtest-readfile-external.py
# executed command: env LIT_USE_INTERNAL_SHELL=0 not env -u FILECHECK_OPTS /app/vbuild/RHEL8-x86_64/python/3.8.0/bin/python3.8 /repo/llvm/utils/lit/lit.py -j1 --order=lexical -a -v Inputs/shtest-readfile
# note: command had no output on stdout or stderr
# executed command: FileCheck -match-full-lines -DTEMP_PATH=/repo/llvm/build-all-builtins/utils/lit/tests/Inputs/shtest-readfile/Output /repo/llvm/build-all-builtins/utils/lit/tests/shtest-readfile-external.py
# .---command stderr------------
# | /repo/llvm/build-all-builtins/utils/lit/tests/shtest-readfile-external.py:22:10: error: CHECK: expected string not found in input
# | # CHECK: + echo hello bye
# |          ^
# | <stdin>:88:526: note: scanning from here
# | echo $(cat /repo/llvm/build-all-builtins/utils/lit/tests/Inputs/shtest-readfile/Output/two-same-line.txt.tmp.1) $(cat /repo/llvm/build-all-builtins/utils/lit/tests/Inputs/shtest-readfile/Output/two-same-line.txt.tmp.2) && test -e /repo/llvm/build-all-builtins/utils/lit/tests/Inputs/shtest-readfile/Output/two-same-line.txt.tmp.1 && test -e /repo/llvm/build-all-builtins/utils/lit/tests/Inputs/shtest-readfile/Output/two-same-line.txt.tmp.2 # RUN: at line 5
# |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              ^
# | <stdin>:91:1: note: possible intended match here
# | + echo hello
# | ^
# |
# | Input file: <stdin>
# | Check file: /repo/llvm/build-all-builtins/utils/lit/tests/shtest-readfile-external.py
# |
# | -dump-input=help explains the following input dump.
# |
# | Input was:
# | <<<<<<
# |             .
# |             .
# |             .
# |            83: --
# |            84: echo -n "hello" > /repo/llvm/build-all-builtins/utils/lit/tests/Inputs/shtest-readfile/Output/two-same-line.txt.tmp.1 # RUN: at line 3
# |            85: + echo -n hello
# |            86: echo -n "bye" > /repo/llvm/build-all-builtins/utils/lit/tests/Inputs/shtest-readfile/Output/two-same-line.txt.tmp.2 # RUN: at line 4
# |            87: + echo -n bye
# |            88: echo $(cat /repo/llvm/build-all-builtins/utils/lit/tests/Inputs/shtest-readfile/Output/two-same-line.txt.tmp.1) $(cat /repo/llvm/build-all-builtins/utils/lit/tests/Inputs/shtest-readfile/Output/two-same-line.txt.tmp.2) && test -e /repo/llvm/build-all-builtins/utils/lit/tests/Inputs/shtest-readfile/Output/two-same-line.txt.tmp.1 && test -e /repo/llvm/build-all-builtins/utils/lit/tests/Inputs/shtest-readfile/Output/two-same-line.txt.tmp.2 # RUN: at line 5
# | check:22'0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  X error: no match found
# |            89: ++ cat /repo/llvm/build-all-builtins/utils/lit/tests/Inputs/shtest-readfile/Output/two-same-line.txt.tmp.1
# | check:22'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# |            90: ++ cat /repo/llvm/build-all-builtins/utils/lit/tests/Inputs/shtest-readfile/Output/two-same-line.txt.tmp.2
# | check:22'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# |            91: + echo hello
# | check:22'0     ~~~~~~~~~~~~~
# | check:22'1     ?             possible intended match
# |            92: + test -e /repo/llvm/build-all-builtins/utils/lit/tests/Inputs/shtest-readfile/Output/two-same-line.txt.tmp.1
# | check:22'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# |            93: + test -e /repo/llvm/build-all-builtins/utils/lit/tests/Inputs/shtest-readfile/Output/two-same-line.txt.tmp.2
# | check:22'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# |            94: not echo return # RUN: at line 8
# | check:22'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# |            95: + not echo return
# | check:22'0     ~~~~~~~~~~~~~~~~~~
# |            96:
# | check:22'0     ~
# |             .
# |             .
# |             .
# | >>>>>>
# `-----------------------------
# error: command failed with exit status: 1

--

********************
********************
Failed Tests (1):
  lit :: shtest-readfile-external.py
```


https://github.com/llvm/llvm-project/pull/159431