[Lldb-commits] [lldb] [lldb][test] Support remote run of Shell tests (PR #95986)

[Pavel Labath via lldb-commits]

labath wrote:

> > Looks good to me. @labath should give this another look as the biggest skeptic here :)
> 
> My position remains unchanged. I don't think we should have this functionality. I'm not going to block you adding it either..

I was going to leave it at that, but seeing https://github.com/llvm/llvm-project/issues/101710, I have to add this as it illustrates my point brilliantly.

Overall, I am more worried about the reproducibility of lldb tests, then their universality. That bug shows we're not able to create a test for a very simple thing (an unaligned stack pointer) that works reliably for everyone. What makes us think we'll be able to do that for tests that execute remotely?

This is important as it slows down the velocity of all developers (old and new). In the end, that also reduces test coverage, because people will start to (even proactively) add `REQUIRES: my-system` to their tests to avoid breaking other people's CI. I believe that the loss of test coverage due to these proactive annotations is much bigger than anything that can be gained by running these tests remotely, and we'd be better off focusing on making Shell tests as hermetic/reproducible/etc. as possible rather making them generic so they can run remotely. For me, this just sends the wrong message/sets the wrong incentive.

https://github.com/llvm/llvm-project/pull/95986