[llvm] [lit] Are all RUN lines skipped in windows cmd? (PR #65242)

[Joel E. Denny via llvm-commits]

jdenny-ornl wrote:

> I haven't been so much involved in the recent Lit changes, and it's been a while since I poked at these things, so it took me a little bit of fiddling around to get back on track about what the actual status quo is here. I'll try to summarize my findings, please doublecheck if I've missed something:

Thanks for the nice summary.  It looks right to me.

> Running things with win32 cmd.exe as external shell did use to work, but currently doesn't. This actually seems to have broken in [1041a96](https://github.com/llvm/llvm-project/commit/1041a9642ba035fd2685f925911d705e8edf5bb0) already.

You're right, and that landed in April, 2022, so windows `cmd` support has been broken a while.  That part is the reason why the current PR wasn't sufficient to restore correct behavior, as you point out later in your comment.  It looks straight-forward to fix, but maybe doing so isn't worthwhile.
 
> Separate side note; running the LLVM testsuite with bash as external executor (if it happens to be available in path) also seems to fail:
> 
> ```
> Exit Code: 127
> 
> Command Output (stderr):
> --
> + : 'RUN: at line 3'
> + c:codellvm-projectllvmbuildbinllvm-mc.exe -triple aarch64-pc-win32 -filetype=obj C:/code/llvm-project/llvm/test/MC/AArch64/seh.s
> C:\code\\llvm-project\llvm\build\test\MC\AArch64\Output\seh.s.script: line 1: c:codellvm-projectllvmbuildbinllvm-mc.exe: command not found
> + c:codellvm-projectllvmbuildbinllvm-readobj.exe -S -r -u -
> C:\code\\llvm-project\llvm\build\test\MC\AArch64\Output\seh.s.script: line 1: c:codellvm-projectllvmbuildbinllvm-readobj.exe: command not found
> + c:codellvm-projectllvmbuildbinfilecheck.exe C:/code/llvm-project/llvm/test/MC/AArch64/seh.s
> C:\code\\llvm-project\llvm\build\test\MC\AArch64\Output\seh.s.script: line 1: c:codellvm-projectllvmbuildbinfilecheck.exe: command not found
> 
> --
> ```
> 
> (This seems to have been broken throughout this time period at least. However before [39bbfb7](https://github.com/llvm/llvm-project/commit/39bbfb77264a4a7a216921c2b70a30ba0f27eb56), when libcxx tests were using the external executor, they did work when executed with bash as external executor.)

It looks like the `llvm-mc` substitution isn't quoting its value correctly.

> ## Conclusion
> 
> It certainly looks like this configuration is severely broken. I'm not aware of any parts of the configurations that use it by default, and even then, it's generally problematic, so I'm not sure if it's worth spending effort on fixing. (The suggestion that was made somewhere around these discussions, to move most things towards using the internal shell even on unix, sounds to me like a good direction in general .)

Agreed.  What if we raise a python exception on the windows `cmd` code path in lit for now?  If we still see no complaints for a few weeks, we can think about removing the code path entirely.

> That said; for echoing executed commands, when using the internal executor, it's probably very valuable if they're printed in a form that is easy to copypaste and execute in cmd.exe (where possible - at least for the majority of simple cases).

If RUN lines are written for lit's internal shell, I'm not sure if they are generally guaranteed to work in windows `cmd`.  Do you have a specific change in mind here?

https://github.com/llvm/llvm-project/pull/65242