[lldb-dev] [llvm-dev] Running lit (googletest) tests remotely

[Matthias Braun via lldb-dev]

> On May 26, 2017, at 8:17 AM, Pavel Labath via llvm-dev <llvm-dev at lists.llvm.org> wrote:
> 
> Hello all,
> 
> we are trying to convert some of the lldb tests to lit (for these
> specific tests we are using the googletest format). One of our
> requirements is that we are able to run these tests remotely, so that
> we are able to verify that we can debug e.g. android arm binaries even
> though our development environment runs x86 linux).
> 
> As far as I am aware, right now there is no support for that in lit:
> running check-lldb-unit target in a cross-compile situation will
> attempt to run the run the test binaries as if they were native and
> fail horribly.
> 
> Based on a not-too-detailed examination of the lit codebase, it does
> not seem that it would be too difficult to add this capability: During
> test discovery phase, we could copy the required files to the remote
> host. Then, when we run the test, we could just prefix the run command
> similarly to how it is done for running the tests under valgrind. It
> would be up to the user to provide a suitable command for copying and
> running files on the remote host (using rsync, ssh, telnet or any
> other transport he chooses).

This seems to be the crux to me: What does "required files" mean?
- All the executables mentioned in the RUN line? What llvm was compiled as a library, will we copy those too?
- Can tests include other files? Do they need special annotations for that?

As another example: The llvm-testsuite can perform remote runs (test-suite/litsupport/remote.py if you want to see the implementation) that code makes the assumption that the remote devices has an NFS mount so the relevant parts of the filesystem look alike on the host and remote device. I'm not sure that is the best solution as NFS introduces its own sort of flakiness and potential skew in I/O heavy benchmarks but it avoids the question of what to copy to the device.

- Matthias