[Lldb-commits] [lldb] [lldb][test] Support remote run of Shell tests (PR #95986)

[Vladislav Dzhidzhoev via lldb-commits]

dzhidzhoev wrote:

> > > That bug shows we're not able to create a test for a very simple thing (an unaligned stack pointer) that works reliably for everyone.
> > 
> > 
> > Honestly, based on [#101710 (comment)](https://github.com/llvm/llvm-project/issues/101710#issuecomment-2291180977), I'd say that the mentioned test has a problem of too many redundant dependencies, rather than that it shows the backside of tests universality.
> 
> I'm afraid I don't know what you mean by that. How would you go about removing those dependencies?
> 
> > >

I think the dependency on clang/clang's default pipeline was excessive, considering that the biggest part of that test was written directly in x86 assembly. Probably, rewriting that test with yaml2obj would have been better https://github.com/llvm/llvm-project/issues/101710#issuecomment-2305446808.


> I think the main difference here is that you think of the Shell tests as kind of an end to end test which verifies that a certain (user-facing) debugger feature is works the way its supposed to. I don't. I think that's what the API tests are for. I think the Shell tests should verify that a specific piece of debugger code (which is most likely not directly accessible by the user) does what it is supposed to do. I.e., they're more akin to a unit test (which happens to be driven by shell commands).

I think of them as a simplified way of writing the same tests that could have been written with Python API. They are described as integration tests in the documentation https://lldb.llvm.org/resources/test.html#test-suite-structure. From my perspective, the difference between API and Shell tests lies in the tests' scenario complexity, not in test type (if we take aside command line-specific tests).

As far as I can see, both `lldb/API` and `lldb/Shell` directories have pretty generic/platform-dependent and specific/platform-independent tests.

https://github.com/llvm/llvm-project/pull/95986