<div dir="ltr">I have been using both a custom port-forwarding program as well as OpenSSH. The custom solution is a bit slower because it goes through a proxy, so the problem doesn't manifest. I've been trying to switch to OpenSSH which is fast enough to consistent hit the race condition.<div><br></div><div>I would expect other solutions to be similar to OpenSSH because I don't think there's a way to have the client connection fail based on the result of a remote connection. Our own forwarding solution was built using this method because of this.<br><div><div><div><br></div><div>I agree that fixing this on the server seems like a better solution, since the client only gets a port when it has one to connect to. But I don't know the best way to make this happen from lldb-server after launching the gdb instance.</div><div><br><div class="gmail_quote"><div dir="ltr">On Mon, Nov 27, 2017 at 4:48 PM Greg Clayton <<a href="mailto:clayborg@gmail.com">clayborg@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">When I wrote the code I was assuming that when the platform lldb-server responds with the port that the gdb-remote would be ready to receive packets right away, so I can see how and why this is happening. Seems like we have the retry stuff under control when we don't get a connection right away, but we should fix this such that when we hand the port back to LLDB from platform lldb-server, it should be listening and ready to accept a connection right away with no delays needed, though this can wait until later since it currently works.<br>
<br>
What kind of port forwarding are you using? The main issue is I would assume that when someone tries to connect to a port on the port forwarder that it would fail to connect if it isn't able to connect on the other side. So this really comes down to a question of what a standard port forwarder's contract should really be.<br>
<br>
If anyone has extensive experience in port forward tech, please chime in.<br>
<br>
Short answer: not sure what the right solution is as it depends on what proper port forwarding etiquette is.<br>
<br>
Greg<br>
<br>
<br>
<br>
<br>
<br>
> On Nov 27, 2017, at 12:33 PM, Christopher Book via lldb-dev <<a href="mailto:lldb-dev@lists.llvm.org" target="_blank">lldb-dev@lists.llvm.org</a>> wrote:<br>
><br>
> Greetings, I've been using liblldb to remotely debug to a linux server with port forwarding. To do this, I start lldb-server to with --listen specifying a localhost port, as well as with ----min-gdbserver-port and --max-gdbserver-port to specify a specific port for use by 'gdb remote'. Both ports are forwarded to the remote PC, where liblldb connects to localhost.<br>
><br>
> This generally works fine, but there is a race condition. When the client tells lldb-server to start gdb-remote, the port is returned to the client which may try to connect before the gdb-remote process is actually listening. Without port-forwarding, this is okay because the client has retry logic:<br>
><br>
> ProcessGDBRemote::ConnectToDebugserver<br>
> ...<br>
> retry_count++;<br>
> if (retry_count >= max_retry_count)<br>
> break;<br>
> usleep(100000);<br>
><br>
> But with port-forwarding, the initial connection is always accepted by the port-forwarder, and only then does it try to establish a connection to the remote port. It has no way to not accept the incoming local connection until it tries the remote end.<br>
><br>
> lldb has some logic to detect this further in the function, by using a handshake to ensure the connection is actually made:<br>
><br>
> // We always seem to be able to open a connection to a local port<br>
> // so we need to make sure we can then send data to it. If we can't<br>
> // then we aren't actually connected to anything, so try and do the<br>
> // handshake with the remote GDB server and make sure that goes<br>
> // alright.<br>
> if (!m_gdb_comm.HandshakeWithServer(&error)) {<br>
> m_gdb_comm.Disconnect();<br>
> if (error.Success())<br>
> error.SetErrorString("not connected to remote gdb server");<br>
> return error;<br>
> }<br>
><br>
> But the problem here is that no retry is performed on failure. The caller to the 'attach' API also can't retry because the gdb server is terminated on the error.<br>
><br>
> I would like to submit a patch, but first check to see if this solution would be acceptable:<br>
> - Include the handshake within the connection retry loop.<br>
> - This means fully disconnecting the re-establishing the connection in the loop if the handshake fails.<br>
> - Changing the timeout check to be based on a total absolute time instead of 50 iterations with a 100ms sleep.<br>
><br>
> Thoughts?<br>
><br>
> Alternatives could be:<br>
> - Have lldb-server delay responding to the 'start gdb server' request until it could tell (somehow) that the process is listening.<br>
> - A sleep of some kind on the client side after starting the server but before trying to connect.<br>
><br>
> Thanks,<br>
> Chris<br>
><br>
><br>
> _______________________________________________<br>
> lldb-dev mailing list<br>
> <a href="mailto:lldb-dev@lists.llvm.org" target="_blank">lldb-dev@lists.llvm.org</a><br>
> <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev</a><br>
<br>
</blockquote></div></div></div></div></div></div>