[lldb-dev] [PATCH] D24591: [LIT] First pass of LLDB LIT support

[Zachary Turner via lldb-dev]

lldb-commits to bcc, lldb-dev to cc.

> The biggest feature I see missing here is the ability to run tests
remotely. Remote debugging is the most important use case for our team, and
now we have a number of (experimental) bots running the remote test suite.
We want to make sure we can debug correctly when the host and target have
different architectures

That is a good point.  I thought about this a little bit and I believe I
have a solution to most of the issues you would face when running a remote
test suite, but perhaps it would be better to take that to a different
thread on lldb-dev rather than on this commit.  So I'm taking lldb-commits
off of this thread and adding lldb-dev.   Responses inline.

On Thu, Sep 15, 2016 at 4:02 AM Pavel Labath <labath at google.com> wrote:



>  - the test suite needs to know how to connect to the remote target
> (mostly it's just executing three commands: `platform select XXX`,
> `platform connect` and `platform settings --working-dir YYY`), but it can
> get a bit more complicated in some cases (if we need to install shared
> libraries along with the main executable, if it involves the test
> executable writing something to a file, etc.)

This seems like the easiest of the problems to solve, so I'm mentioning it
first.  I can imagine a couple of command line options to the lldb
executable itself that allow one to configure remote debugging from the
command line.  Then, we could provide one CMake target for each remote
platform.  check-lldb-remote-android, check-lldb-remote-fizzbuzz,
whatever.  And you could configure all of these at CMake time with one
command.  Like cmake -DLLDB_TEST_REMOTE_TARGETS=android,fizzbuzz,etc.
Internally, each target's CMake logic could configure some appropriate
variables that would be picked up by lit so that the lit substitution for
running lldb would supply the appropriate command line to configure the
remote target.

 - this includes the need to pass funny compiler switches based on the
target (some android targets can only run `-pie` executables, some cannot
run `-pie` executables at all).
 This should be handled the same way as above.  Configuring the lit
substitutions for the compiler invocation can depend on which cmake target
you're running lit through.  ninja check-lldb-remote-android could define
one set of lit substitutions, and another target could define another.  Is
this sufficient?  Is it easy to define a mapping between the set of all
remote targets and the set of all command line flag combinations?

- we need to be able to fully specify the toolchain used to compile the
> target executables (specifically, we should not assume the host compiler)
>
Will you want a single run of the test suite to use multiple compilers, or
can we assume there is only 1 compiler for a given cmake output directory?
I think we can assume the latter since even now we only have 1 cmake
variable `LLDB_TEST_COMPILER`, but let me know if this is a wrong assumption


>
> - we need to be careful about strong cmake integration. The same lldb
> binary is capable of debugging all kinds of targets. We should not require
> a fresh cmake build to run the test suite against multiple targets. If we
> need to switch cmake options (`LLDB_TEST_COMPILER`, `LLDB_TEST_ARCH`,
> `LLDB_TEST_URL` or similar) and it does not trigger a rebuild then it's
> usable, but slightly inpractical. For us, it would be best to be able a
> fire off a remote test with a single command (like we can do now).
>
See first response.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20160915/d2a3d769/attachment.html>