[Lldb-commits] [PATCH] D46588: [LLDB][lldb-mi] Add possibility to set breakpoints without selecting a target.

Thu May 17 03:22:51 PDT 2018

labath added a comment.

In https://reviews.llvm.org/D46588#1102363, @aprantl wrote:

> >   The advantage of the second one is that we will have the ability to inject commands which depend on the results of previous commands (something that I think we will need, sooner or later).
>
> That is worth considering. To write good tests that depend on previous results, we probably want to have SBAPI-like Python scripting for lldb-mi available. To make that work we would need to introduce an lldbmi.HandleCommand("-mycmd...") interface that runs one command, blocks on it, and returns the output in a string. And then we would need to export that to Python. Since the interface is string-in-string-out, we could probably do it via C without even involving SWIG. As an escape hatch to access features outside of the mi command set, we could implement the -interpreter-exec mi command that passes its input to SBAPI's HandleCommand() if need be.
>  Alternatively we could use gtest to write unittest-style lldb-mi tests directly in C++, if going via Python is too messy. Again, we would do that in a blocking or synchronized I/O mode.

I agree with your analysis. I don't want to give an opinion on the direction, because:
a) it's not clear to me which way to go, each possibility has different tradeoffs involved
b) I don't see myself doing anything related to lldb-mi any time soon

What I can offer as an additional data point is the lldb-server test suite. lldb-server very similar to lldb-mi in a way, as it has an almost-text-based protocol as its main form of interaction with the world. We also have two ways of testing it: a python one (`test/testcases/tools/lldb-server` and a gtest one (`unittests/tools/lldb-server`). the python one is older, but a bit messy (IMO). The gtest one was started because we had a person interested in improving lldb-server testing situation. I supported starting a fresh sub-test suite, because I saw it as a way to solve a couple some fundamental issues in the old one:

- code duplication: the python test suite needs a complete reimplementation of the gdb-remote protocol. the c++ test can just hook it to the existing functionality at an appropriate layer.
- remote testing mess: a very convoluted setup is required to run tests on a remote (android) device. It's weird that you have to build a host lldb and run the tests from there, even though the lldb-server and the executables it debugs run on a different host/arch. The python part of the test needs to run on the host (there's no easily available python for android), so there's a lot of flakyness and complexity coming from the port forwarding/network communication).

The c++ based approach could solve these, as the gtest executable could run on the same device as the lldb-server under test, and it would be possible to run the tests via check-lldb(-server?) from the cross-build tree instead of copying things around between two build trees.

Sadly, the person who was supposed to do this left the team. I have tried to carry this idea on slowly, but haven't yet gotten around to implementing the most important part (running tests remotely), so I couldn't start migrating the tests. In retrospect, I am less excited about this idea than I was a year ago, though I still think it would offer a better testing story for lldb-server.

Overall, I am not sure how much of this relates to lldb-mi (for example, you probably aren't interested in running it in such a constrained environment), but I thought it might be interesting for you to look at these before deciding on a direction. I also don't want to hold up Alexander's internship over this, as I'm not sure this is what he signed up for. I wanted to bring up the testing story, because I think there are some issues there, but I'll leave it up to you (for some definition of "you") to decide on what to do about that.

Repository:
  rL LLVM

https://reviews.llvm.org/D46588