[lldb-dev] Remote connection expansion?

[Greg Clayton via lldb-dev]

> On Nov 11, 2020, at 11:11 AM, Mike Mestnik <cheako+lists_llvm_org at mikemestnik.net> wrote:
> 
> On Mon, Nov 9, 2020 at 5:37 PM Greg Clayton <clayborg at gmail.com> wrote:
>> 
>> 
>> 
>>> On Nov 4, 2020, at 1:28 PM, Mike Mestnik via lldb-dev <lldb-dev at lists.llvm.org> wrote:
>>> 
>>> I'm looking for support running lldb over ssh.  I can forward the
>>> originating connection, but the run command is attempting to use
>>> random ports on localhost to attain another connection.  This fails as
>>> the localhost's are not the same.
>> 
>> When you say you want to run lldb over ssh, do you mean run "lldb-server"
> Is there really an issue with saying these are both lldb?  Seems like
> my statements were unambiguous without noting a distinction.
> 
>> remotely and then have a local LLDB connect to that lldb-server?
> That's the usual way lldb is used remotely, so I don't really get that
> lldb does not, in some cases, mean lldb-server.

I am trying to figure out what exactly you would like to see happen here. There are many people that run ssh to open a terminal to another computer and run lldb on the command line on that remote machine, hence why I was trying to clarify what you are are asking for with regards to ssh. 
> 
>> You are looking to avoid "lldb-server" from having to bind to port 0 and then tell you which port it was actually bound to?
> This is indeed a *feature* of lldb-server and not lldb, so I don't
> really get your confusion when I claim lldb has a feature that's
> entirely implemented with regard to lldb-server.

Again, trying to understand exactly what you are asking for any not sure why the snippy answer.


> 
> 
>>> 
>>> Is there a platform, preferably real time, for lldb support?
>>> 
> This is a really important question of mine!  If we can only go back
> and forth on email I'd like ppl to make a best effort to lessen the
> effects this has.  One tip I have, even for when ppl are chatting, is
> to phrase questions in the form of an answer...  Don't ask yes/no
> questions or even questions with say less than 7 answers.  Instead
> provide bullet points of all the possible answers, one at a time.
> There are many good reasons for everyone doing this, but here the most
> relevant is to avoid round-trips that are expansive.


LLDB does have an IRC channel on the LLVM IRC chatbot named "lldb". Many LLDB developers do hang out on this chat and can provide some realtime answers for LLDB and lldb-server.

> 
> Can you answer your question twice?  Once If I claim that "Yes" I do
> plan on having lldb choose a port to listen on and have ssh forward
> that port and again for if I answer "No".

> 
> I hope you can appreciate that having ssh forward a handful of ports
> for a single service is not optimal.  I'd actually do better by
> writing another set of client-server applications that tunnel the IPC
> over ssh...  Something I'd only do if the program was closed source.
> As lldb is not, the obvious path forward is to re-implement the lldb
> IPC so it's more friendly to ssh.

We are happy to help make remote debugging better and more friendly to ssh. We don't have a lot of ssh expertise in the LLDB core group of developers as far as I know, so if you have expertise here and can add ways for us to be better in that respect, we would love to see any patches. See more detailed solution suggestion below on where I think this would fit the best in LLDB.

> 
>>> One might ask why ssh, the basic answer is I don't want to open
>>> *another port on my remote host...  Even if I did I'd still have the
>>> same problem, a random port would fail to connect(this time because of
>>> a firewall).  The main answer is, without ssh, lldb is limited to
>>> running on local or VPN networks.  I'd rather use ssh than configure a
>>> VPN for this one use case.
>>> 
>>> * lldb doesn't sound like something one would want to host, even if
>>> connections were blocked from everywhere "else."
>>> 
>>> Now I'm attempting forward error correction by guessing where this
>>> topic could lead.  I would be willing to expand the network code to
>>> include domain sockets, to replace the whole idea of using, IMHO
>>> barbaric, port numbers.  This work could potentially include direct
>>> support for ssh.  

lldb-server is not limited to port connections. LLDB has a plug-in mechanism for communications that is URL based. For example, if you are going to launch lldb-server on a remote system and specify a port number of 1234, you would connect to it with:

(lldb) process connect connect://remote.foo.com:1234 <connect://remote.foo.com:1234>

"process connect" uses a communication plug-in that is specified by the URL for the first argument: "connect://". Each communication plug-in built into lldb and lldb-server registers a URL with the LLDB codebase so it can be easily found and used. "connect://" will get you a direct TCP connection using IPV4 of IPV6 depending on the rest of the URL. There isn't a valid URL for direct TCP connections so we made one up ("connect://"). 

The reason I mention this is this is where I would envision any ssh support could easily be added to LLDB. So my theory would be we would add a new "ssh" communication plug-in, if that makes sense as I have no SSH experience other than using a lot of command line tools that can use it, and then when connecting to the remote via ssh we would end up using:

(lldb) process connect ssh://<args> <ssh://remote.foo.com%3Cargs%3E>

where <args> is anything we need to make the connection, like "remote.foo.com <http://remote.foo.com/>". We would probably add a new option to lldb-server to specify it would use ssh as well when it was launched like:

remote$ lldb-server gdbserver --ssh <ssh-args-if-needed> -- /bin/ls -lAF

Does that seam feasible? 

>>> I understand that this would likely be a breaking
>>> change, is there version negotiation?

I don't think this needs to be a breaking change. For lldb-server, we just need a way to move packets over a communication layer, so we should be able to use our communication plug-ins to do this. 

Another example of using different communication plug-ins is when lldb-server is running on the same machine as lldb: lldb calls socketpair() and gets 2 socket file handles that are connected and dupes one of them into the lldb-server using posix_spawn attributes. When we actually spawn the process we send the file descriptor that it was duped to down as an argument: "--fd 12". So when lldb-server is starting up, it will parse the options, and discover it is just needs to use the specified file descriptor.

Greg Clayton

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20201111/ae58ace3/attachment.html>