[compiler-rt] [compiler-rt][tests] Fix env command not found errors with lit internal shell (PR #105879)

[via llvm-commits]

================
@@ -14,8 +14,9 @@ RUN: echo %t
 # Out-of-process fuzzing with this rig is slow,
 # we can not wait for the fuzzer to find the faulty input.
 # Just run for a bit and observe the corpus expansion.
-RUN: LIBFUZZER_OOP_TARGET="./oop-target > /dev/null 2>&1 " ./oop-fuzzer -max_len=3 OOP_CORPUS -runs=1000 -jobs=4
+RUN: env LIBFUZZER_OOP_TARGET="./oop-target > /dev/null 2>&1 " ./oop-fuzzer -max_len=3 OOP_CORPUS/seed -runs=1000 -jobs=4 > %t/temp_output.txt 2>&1
----------------
Harini0924 wrote:

The command not found error with `LIBFUZZER_OOP_TARGET` is a bit tricky because it involves processing all files in the `OOP_CORPUS/` directory using a wildcard (`*`).  To ensure that the fuzzing process starts correctly, I used `OOP_CORPUS/seed` as a starting point. This seed file helps to initialize the corpus with a known input, allowing the `fuzzer` to expand upon it during execution.

**Regarding the changes in the patch:**
**Using `env` Command:** The environment variable was prefixed with `env` to ensure it is correctly interpreted by the lit internal shell. The lit internal shell can be sensitive to environment variables and command handling, especially with wildcards like `OOP_CORPUS/*`. This change was needed because the original command caused a command not found error.

**Redirecting Output to a Temp File:** I redirected the output to a temporary file (`%t/temp_output.txt`) to capture everything the fuzzer produced. This was to make sure FileCheck could validate all output, as the lit internal shell might not handle direct output properly in complex cases.

I also tested with just the `env` command (kept everything else in its original state), I still ran into an error:
```
# RUN: at line 21
./oop-target OOP_CORPUS/* 2>&1 | FileCheck /usr/local/google/home/harinidonthula/llvm-# RUN: at line 21
./oop-target OOP_CORPUS/* 2>&1 | FileCheck /usr/local/google/home/harinidonthula/llvm-project/compiler-rt/test/fuzzer/out-of-process-fuzz.test
# executed command: ./oop-target 'OOP_CORPUS/*'
# executed command: FileCheck /usr/local/google/home/harinidonthula/llvm-project/compiler-rt/test/fuzzer/out-of-process-fuzz.test
# .---command stderr------------
# | /usr/local/google/home/harinidonthula/llvm-project/compiler-rt/test/fuzzer/out-of-process-fuzz.test:18:8: error: CHECK: expected string not found in input
# | CHECK: Running: OOP_CORPUS/
# |        ^
# | <stdin>:1:1: note: scanning from here
# | StandaloneFuzzTargetMain: running 11 inputs
# | ^
# | <stdin>:2:170: note: possible intended match here
# | Running: /usr/local/google/home/harinidonthula/llvm-project/llvm/build/runtimes/runtimes-bins/compiler-rt/test/fuzzer/X86_64LibcxxLinuxConfig/Output/out-of-process-fuzz.test.tmp/OOP_CORPUS/27d5482eebd075de44389774fce28c69f45c8a75
# |                                                                                                                                                                          ^
# | 
# | Input file: <stdin>
# | Check file: /usr/local/google/home/harinidonthula/llvm-project/compiler-rt/test/fuzzer/out-of-process-fuzz.test
# | 
# | -dump-input=help explains the following input dump.
# | 
# | Input was:
# | <<<<<<
# |             1: StandaloneFuzzTargetMain: running 11 inputs 
# | check:18'0     X~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found
# |             2: Running: /usr/local/google/home/harinidonthula/llvm-project/llvm/build/runtimes/runtimes-bins/compiler-rt/test/fuzzer/X86_64LibcxxLinuxConfig/Output/out-of-process-fuzz.test.tmp/OOP_CORPUS/27d5482eebd075de44389774fce28c69f45c8a75 
# | check:18'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | check:18'1                                                                                                                                                                              ?                                                             possible intended match
# |             3: Done: /usr/local/google/home/harinidonthula/llvm-project/llvm/build/runtimes/runtimes-bins/compiler-rt/test/fuzzer/X86_64LibcxxLinuxConfig/Output/out-of-process-fuzz.test.tmp/OOP_CORPUS/27d5482eebd075de44389774fce28c69f45c8a75: (1 bytes) 
# | check:18'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# |             4: Running: /usr/local/google/home/harinidonthula/llvm-project/llvm/build/runtimes/runtimes-bins/compiler-rt/test/fuzzer/X86_64LibcxxLinuxConfig/Output/out-of-process-fuzz.test.tmp/OOP_CORPUS/28ed3a797da3c48c309a4ef792147f3c56cfec40 
# | check:18'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# |             5: Done: /usr/local/google/home/harinidonthula/llvm-project/llvm/build/runtimes/runtimes-bins/compiler-rt/test/fuzzer/X86_64LibcxxLinuxConfig/Output/out-of-process-fuzz.test.tmp/OOP_CORPUS/28ed3a797da3c48c309a4ef792147f3c56cfec40: (1 bytes) 
# | check:18'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# |             6: Running: /usr/local/google/home/harinidonthula/llvm-project/llvm/build/runtimes/runtimes-bins/compiler-rt/test/fuzzer/X86_64LibcxxLinuxConfig/Output/out-of-process-fuzz.test.tmp/OOP_CORPUS/42099b4af021e53fd8fd4e056c2568d7c2e3ffa8 
# | check:18'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# |             7: Done: /usr/local/google/home/harinidonthula/llvm-project/llvm/build/runtimes/runtimes-bins/compiler-rt/test/fuzzer/X86_64LibcxxLinuxConfig/Output/out-of-process-fuzz.test.tmp/OOP_CORPUS/42099b4af021e53fd8fd4e056c2568d7c2e3ffa8: (1 bytes) 
# | check:18'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# |             .
# |             .
# |             .
# | >>>>>>
# `-----------------------------
# error: command failed with exit status: 1

--

```

This error suggests that the expected output ("Running: OOP_CORPUS/") was not found, which might be due to the files being processed differently than expected by the fuzzer or FileCheck. The wildcard expansion (`OOP_CORPUS/*`) may have led to unexpected results in how the files were processed. It appears that despite the `env` command successfully setting the environment variable, the processing of files within `OOP_CORPUS/*` didn't produce the output expected by FileCheck.

Given this issue, I made additional changes to ensure the test ran smoothly:

1. Using `OOP_CORPUS/seed` as a starting point to initialize the corpus.
2. Redirecting output to a temporary file to ensure all output is captured and validated by FileCheck.

These changes were made to address the problems encountered with the initial approach (just adding `env` command). I hope this explanation clarifies why these changes were necessary. If there are other ways to rewrite this test, I’m open to trying them. Please let me know if you have any suggestions or if this approach makes sense.


https://github.com/llvm/llvm-project/pull/105879