[llvm] 9764cf8 - [llvm-lit] Add precommit test to verify current behavior of glob expansion in lit's internal shell (#106325)

[via llvm-commits]

Author: Harini0924
Date: 2024-08-30T09:39:35-07:00
New Revision: 9764cf888502fe6dd15ab21de5c2f73cae47a2c0

URL: https://github.com/llvm/llvm-project/commit/9764cf888502fe6dd15ab21de5c2f73cae47a2c0
DIFF: https://github.com/llvm/llvm-project/commit/9764cf888502fe6dd15ab21de5c2f73cae47a2c0.diff

LOG: [llvm-lit] Add precommit test to verify current behavior of glob expansion in lit's internal shell (#106325)

This patch introduces a precommit test to verify the current behavior of
glob expansion in lit's internal shell. The motivation for this test
stems from an issue encountered during the BOLT test suite when running
with the lit internal shell using the command:

`LIT_USE_INTERNAL_SHELL=1 ninja check-bolt`

During execution, the following error was observed:
```
File "/usr/local/google/home/harinidonthula/llvm-project/llvm/utils/lit/lit/TestRunner.py", line 416, in executeBuiltinEcho
    stdout.write(encode(maybeUnescape(args[-1])))
TypeError: string argument expected, got 'GlobItem'
```
The `executeBuiltinEcho` function in the lit testing framework expects a
string to be passed to `stdout.write`, but it received a `GlobItem`
object instead. This precommit test is designed to check the current
behavior where the glob pattern isn't correctly expanded, leading to
this `TypeError`.

While this patch doesn't fix the issue, it helps in understanding and
verifying the current behavior. The feedback I received from this
[PR](https://github.com/llvm/llvm-project/pull/105925) suggests using
`cmd.args = expand_glob_expressions(cmd.args, shenv.cwd)` to match the
behavior of `executeBuiltinMkdir` and `executeBuiltinRm`, but it is
recognized that the internal shell should ideally expand globs before
calling any built-in command.

**Request for Feedback:**
I'm looking for feedback on how to improve this precommit test,
specifically regarding the handling and expansion of glob patterns for
commands like mkdir and rm within the internal shell. Currently, the
args are expanded at the beginning of these functions, which should
ensure proper glob expansion. However, I'd appreciate guidance on
whether I should write additional tests to verify that mkdir and rm are
handling glob expansions correctly.

If such tests are recommended, I would also appreciate advice on the
best approach to implement them, considering the existing framework and
the way glob expansion is expected to function in the internal shell.
Should these tests confirm that the current implementation passes, or
are there specific edge cases I should be aware of?

**Next Steps:**

In my follow-up PR, I plan to address the UNRESOLVED error by expanding
the entire command, ensuring correct and consistent behavior across all
commands. The current test checks for an unresolved issue with the glob
expansion, specifically looking for a `TypeError` due to an unexpanded
`GlobItem`. This will be updated to reflect the correct behavior once
the issue is resolved.

This change is relevant for [[RFC] Enabling the Lit Internal Shell by
Default](https://discourse.llvm.org/t/rfc-enabling-the-lit-internal-shell-by-default/80179/3)

Added: 
    llvm/utils/lit/tests/Inputs/shtest-glob/example_file1.input
    llvm/utils/lit/tests/Inputs/shtest-glob/example_file2.input
    llvm/utils/lit/tests/Inputs/shtest-glob/glob-echo.txt
    llvm/utils/lit/tests/Inputs/shtest-glob/glob-mkdir.txt
    llvm/utils/lit/tests/Inputs/shtest-glob/lit.cfg
    llvm/utils/lit/tests/shtest-glob.py

Modified: 
    

Removed: 
    


################################################################################
diff  --git a/llvm/utils/lit/tests/Inputs/shtest-glob/example_file1.input b/llvm/utils/lit/tests/Inputs/shtest-glob/example_file1.input
new file mode 100644
index 00000000000000..0987c9081ca1f3
--- /dev/null
+++ b/llvm/utils/lit/tests/Inputs/shtest-glob/example_file1.input
@@ -0,0 +1,2 @@
+## This is the first example file used for testing glob pattern matching.
+This is the first example file.

diff  --git a/llvm/utils/lit/tests/Inputs/shtest-glob/example_file2.input b/llvm/utils/lit/tests/Inputs/shtest-glob/example_file2.input
new file mode 100644
index 00000000000000..f1a843f308262e
--- /dev/null
+++ b/llvm/utils/lit/tests/Inputs/shtest-glob/example_file2.input
@@ -0,0 +1,2 @@
+## This is the second example file used for testing glob pattern matching.
+This is the second example file.

diff  --git a/llvm/utils/lit/tests/Inputs/shtest-glob/glob-echo.txt b/llvm/utils/lit/tests/Inputs/shtest-glob/glob-echo.txt
new file mode 100644
index 00000000000000..b69f5e74fd7281
--- /dev/null
+++ b/llvm/utils/lit/tests/Inputs/shtest-glob/glob-echo.txt
@@ -0,0 +1,2 @@
+## Tests glob pattern expansion by listing matching files.
+# RUN: echo %S/example_file*.input

diff  --git a/llvm/utils/lit/tests/Inputs/shtest-glob/glob-mkdir.txt b/llvm/utils/lit/tests/Inputs/shtest-glob/glob-mkdir.txt
new file mode 100644
index 00000000000000..d1329f5dbfaaed
--- /dev/null
+++ b/llvm/utils/lit/tests/Inputs/shtest-glob/glob-mkdir.txt
@@ -0,0 +1,2 @@
+## Tests glob pattern handling in the mkdir command.
+# RUN: not mkdir %S/example_file*.input

diff  --git a/llvm/utils/lit/tests/Inputs/shtest-glob/lit.cfg b/llvm/utils/lit/tests/Inputs/shtest-glob/lit.cfg
new file mode 100644
index 00000000000000..4e5f4cac4c4653
--- /dev/null
+++ b/llvm/utils/lit/tests/Inputs/shtest-glob/lit.cfg
@@ -0,0 +1,8 @@
+import lit.formats
+
+config.name = "shtest-glob"
+config.suffixes = [".txt"]
+config.test_format = lit.formats.ShTest()
+config.test_source_root = None
+config.test_exec_root = None
+config.substitutions.append(("%{python}", '"%s"' % (sys.executable)))

diff  --git a/llvm/utils/lit/tests/shtest-glob.py b/llvm/utils/lit/tests/shtest-glob.py
new file mode 100644
index 00000000000000..551331cb38e259
--- /dev/null
+++ b/llvm/utils/lit/tests/shtest-glob.py
@@ -0,0 +1,12 @@
+## Tests glob pattern handling in echo command.
+
+# RUN: not %{lit} -a -v %{inputs}/shtest-glob \
+# RUN: | FileCheck -dump-input=fail -match-full-lines %s
+#
+# END.
+
+# CHECK: UNRESOLVED: shtest-glob :: glob-echo.txt ({{[^)]*}})
+# CHECK: TypeError: string argument expected, got 'GlobItem'
+
+# CHECK: FAIL: shtest-glob :: glob-mkdir.txt ({{[^)]*}}
+# CHECK: # error: command failed with exit status: 1