[lldb-dev] Remote debugging with lldb

[Greg Clayton]

On Aug 24, 2011, at 6:11 PM, Filipe Cabecinhas wrote:

> Hi,
> 
> I'm trying to do remote debugging with lldb, but I keep running into problems.
> 
> How should I do it?
> 
> I tried running "debugserver localhost:42000" and then, on the lldb client:
> 
> platform select (remote-macosx | remote-gdb-server?) + platform connect?
> platform select … + process connect?

The platform stuff is still a work in progress which we hope to really perfect over the next year. If you are doing Mac to Mac debugging the best thing to do right now is:

AFP mount the root drive on the other machine. For these examples lets suppose we have machine1 and machine2. So the flow would be:

- machine1: start debugserver:
machine1 % debugserver localhost:12345 /bin/ls

- machine2: mount the root drive from machine2 and launch lldb:
machine2 % ./lldb
(lldb) platform select remote-macosx --sysroot /Volumes/machine1
(lldb) process connect connect://machine1:12345

This should do it. Currently the platforms don't do any file copying from one to the other, so you will need a copy of all executables and shared libraries to exist within the "Volumes/machine1" directory. So if you are running "/bin/ls" it should be available through your mount at: "/Volumes/machine1/bin/ls". Likewise for all shared libraries.


> 
> Should I have the executable on both machines? Do I just need it on the lldb machine and lldb sends it to the server? Do I just need it on the server?

Again, the platforms should eventually support "install" and "copy" for files, but they currently don't.

> 
> That's good for the most basic case: client and server on Mac OS X.
> 
> Now, what about some different combinations?
> debugserver on macosx + client on linux?
> debugserver on Linux + client on macosx?
> gdbserver on "any" machine + client on macosx? Which programs would be supported?

Any of the above as long as you can mount the root drive and any other things you need within one directory as mentioned above.

> 
> I would suppose you need the plugins for the combinations you want to use, but with the gdb-remote stuff, I'm getting confused.
> 
> In lldb_private::Initialize(), we initialize several plugins, depending on the operating system. I'm supposing the DynamicLoader and Process plugins (along with some others) are only needed when running as host, otherwise, we couldn't, on Linux, debug a FreeBSD or Mac OS X target (since their initialization gets ifdeffed out).
> 

None of these should really be ifdefed out, they should be being compiled on all platforms, excluding any "native only" plug-ins.

> I saw that, on Mac OS X, lldb spawns a debugserver that runs the program and they communicate using the plugin ProcessGDBRemote (another question: Why is that plug-in only started on Mac OS X? It has to run on the host, but otherwise I didn't see much platform-specific stuff (I just skimmed the file)). Shouldn't that Process plugin be also used with a PlatformRemoteGDBServer?
> 
The platform (currently on MacOSX built as the "lldb-platform" executable) can currently do the job of being launched on a remote host and it can spawn a new debugserver in order to debug a process on the host, list processes, attach to existing process and do much more. All the code that the "lldb-platform" uses if from the "lldb_private::Host" API layer, so it completely reuses the host code that  can launch processes for debugging, and much more.

The ideal platform looking forward would:
- be a binary that runs natively on the remote machine that hopefully can be automatically launched when the remote host is contacted on a port. On MacOSX, we can add a launchd plist entry that would automatically launch "lldb-platform" (or any equivalent) when the remote host is contacted on a fixed port number.
- upload and download files from the platform and cache them locally (currently we require all files to be there already in the "--sysroot" for the platform)
- list and query info for existing processes (lldb-platform can already do this)
- launch a process whose executable exists on the remote host (possibly just updloaded via the platform) with any args required. This would spawn the process under a debugserver and return the port needed to connect.
- attach to an existing process (spawn a debugserver that will attach, an return the port number across the platform protocol)

So much of this works right now. You can try out the "lldb-platform" binary with the current build and pretend to debug remotely:


% ./lldb-platform --listen 1444
Listening for a connection on 1444...



Then launch lldb and connect:

% ./lldb
(lldb) platform select remote-macosx
  Platform: remote-macosx
 Connected: no
(lldb) platform connect connect://localhost:1444
  Platform: remote-macosx
    Triple: x86_64-apple-darwin
OS Version: 10.7.1
    Kernel: Darwin Kernel Version 11.2.0: Tue Aug  9 20:54:00 PDT 2011; root:xnu-1699.24.8~1/RELEASE_X86_64
  Hostname: myhost.apple.com
 Connected: yes
(lldb) platform process list
40 matching processes were found on "myhost.apple.com"
PID    PARENT USER       ARCH    NAME
====== ====== ========== ======= ============================
46384  46139  gclayton   x86_64  lldb
.....
(lldb) 


There is much work to do, but there is a lot of support already in place to get remote debugging started.

Let me know if you have any questions.

Greg Clayton