[lldb-dev] Remote debugging a docker process

[Mason Kramer via lldb-dev]

Greetings and salutations!

I am trying to remotely debug a process running inside of a Docker 
container. I can connect to lldb-server from my host, but can't launch a 
debugging process.  I can debug the target locally, inside or outside of 
the container.

Container:

lldb-server-4.0 platform --verbose --listen "*:5000"
Connection established.

Host:

$ lldb

(lldb) target create target/debug/hist

(lldb) platform connect connect://localhost:5000
   Platform: remote-linux
     Triple: x86_64-pc-linux-gnu
OS Version: 4.15.0 (4.15.0-45-generic)
     Kernel: #48-Ubuntu SMP Tue Jan 29 16:28:13 UTC 2019
   Hostname: 4ce058c8dba3
  Connected: yes
WorkingDir: /seraphim
(lldb) run
error: connect remote failed (Connection refused)
error: process launch failed: Connection refused

Docker is a containerization system that sandboxes the processes it 
manages in various ways. Processes inside of the container are running 
on a virtualized network stack and do not know the IP address of their 
host and cannot communicate to the outside except on "published" ports, 
for instance.

A mail [1] on this list dating to 2017 suggested the problem is that the 
gdbserver child process spawned by `process launch` can't talk to Docker 
over the firewall. However, I don't believe it's the issue - or at 
least, not the only one.

I isolated this problem by binding 5000 and 5001 in the container to the 
same values in the host. Then, I restricted the acceptable range of 
gdbserver ports to just 5001, using the flags suggested in the email.

lldb-server-4.0 platform --verbose --listen "*:5000" 
--min-gdbserver-port 5001 --max-gdbserver-port 5001

This had no apparent effect.

I also found a pull request [2] from 2018 which suggested that the 
problem is the virtualized IP address in the container. That is a 
promising direction, but unfortunately, that patch was abandoned and 
nothing took its place.

There are traces of this issue all over the net, but none of them that I 
have found were ever resolved. I think that remote-debugging a Docker 
container is an increasingly important use-case for lldb-remote, and if 
anyone is interested in this, I'm happy to work with you to hammer it out.

Unanswered Stack Overflow [3]

A bug filed on Swift's tracker [4]

[1] http://lists.llvm.org/pipermail/lldb-dev/2017-February/012004.html

[2] https://reviews.llvm.org/D42845

[3] 
https://stackoverflow.com/questions/45533026/remote-lldb-debugging-docker-container

[4] https://bugs.swift.org/browse/SR-3596?attachmentViewMode=list