[lldb-dev] GDB RSPs non-stop mode capability in v5.0

[Jim Ingham via lldb-dev]

> On Mar 29, 2018, at 10:40 AM, Greg Clayton via lldb-dev <lldb-dev at lists.llvm.org> wrote:
> 
> 
> 
>> On Mar 29, 2018, at 10:36 AM, Frédéric Riss <friss at apple.com> wrote:
>> 
>> 
>> 
>>> On Mar 29, 2018, at 9:27 AM, Greg Clayton <clayborg at gmail.com> wrote:
>>> 
>>> 
>>> 
>>>> On Mar 29, 2018, at 9:10 AM, Frédéric Riss <friss at apple.com> wrote:
>>>> 
>>>> 
>>>> 
>>>>> On Mar 29, 2018, at 7:32 AM, Greg Clayton via lldb-dev <lldb-dev at lists.llvm.org> wrote:
>>>>> 
>>>>> 
>>>>> 
>>>>>> On Mar 29, 2018, at 2:08 AM, Ramana via lldb-dev <lldb-dev at lists.llvm.org> wrote:
>>>>>> 
>>>>>> Hi,
>>>>>> 
>>>>>> It appears that the lldb-server, as of v5.0, did not implement the GDB RSPs non-stop mode (https://sourceware.org/gdb/onlinedocs/gdb/Remote-Non_002dStop.html#Remote-Non_002dStop). Am I wrong?
>>>>>> 
>>>>>> If the support is actually not there, what needs to be changed to enable the same in lldb-server?
>>>>> 
>>>>> As Pavel said, adding support into lldb-server will be easy. Adding support to LLDB will be harder. One downside of enabling this mode will be a performance loss in the GDB remote packet transfer. Why? IIRC this mode requires a read thread where one thread is always reading packets and putting them into a packet buffer. Threads that want to send a packet an get a reply must not send the packet then use a condition variable + mutex to wait for the response. This threading overhead really slows down the packet transfers. Currently we have a mutex on the GDB remote communication where each thread that needs to send a packet will take the mutex and then send the packet and wait for the response on the same thread. I know the performance differences are large on MacOS, not sure how they are on other systems. If you do end up enabling this, please run the "process plugin packet speed-test" command which is available only when debugging with ProcessGDBRemote. It will send an receive various packets of various sizes and report speed statistics back to you.
>>>>>> 
>>>>>> Also, in lldb at least I see some code relevant to non-stop mode, but is non-stop mode fully implemented in lldb or there is only partial support?
>>>>> 
>>>>> Everything in LLDB right now assumes a process centric debugging model where when one thread stops all threads are stopped. There will be quite a large amount of changes needed for a thread centric model. The biggest issue I know about is breakpoints. Any time you need to step over a breakpoint, you must stop all threads, disable the breakpoint, single step the thread and re-enable the breakpoint, then start all threads again. So even the thread centric model would need to start and stop all threads many times. 
>>>> 
>>>> If we work on this, that’s not the way we should approach breakpoints in non-stop mode (and it’s not how GDB does it). I’m not sure why Ramana is interested in it, but I think one of the main motivations to add it to GDB was systems where stopping all some threads for even a small amount of time would just break things. You want a way to step over breakpoints without disrupting the other threads.
>>>> 
>>>> Instead of removing the breakpoint, you can just teach the debugger to execute the code that has been patched in a different context. You can either move the code someplace else and execute it there or emulate it. Sometimes you’ll need to patch it if it is PC-relative. IIRC, GDB calls this displaced stepping. It’s relatively simple and works great.
>>> 
>>> This indeed is one of the changes we would need to do for non-stop mode. We have the EmulateInstruction class in LLDB that is designed just for this kind of thing. You can give the emulator function a read/write memory and read/write register callbacks and a baton and it can execute the instruction and read/write memory and regisrters as needed through the context. It would be very easy to have the read register callback know to take the PC of the original instruction and return it if the PC is requested.
>>> 
>>> We always got push back in the past about adding full instruction emulation support as Chris Lattner wanted it to exist in LLVM in the tablegen tables, but no one ever got around to doing that part. So we added prologue instruction parsing and any instructions that can modify the PC (for single stepping) to the supported emulated instructions. 
>>> 
>>> So yes, emulating instructions without removing them from the code is one of the things required for this feature. Not impossible, just very time consuming to be able to emulate every instruction out of place. I would _love_ to see that go in and would be happy to review patches for anyone wanting to take this on. Though the question still remains: does this happen in LLVM or in LLDB. Emulating instruction in LLVM might provide some great testing that could happen in the LLVM layers.
>> 
>> In my porting experience, emulation is actually rarely needed. Of course, if LLVM had a readily available emulation library we could just use that, but it’s not the case. Most of the time, just copying the instruction to some scratch space and executing it there is enough (you potentially need to patch offsets if the instruction uses PC-relative addressing).
>> 
> 
> That is true but that involves starting and stopping the thread one time which can be time consuming. It is easier to do it this way, but the starting and stopping of a thread is very costly. It would be better to try and emulate all the instructions we can and then fall back to emulating the instruction at another address if needed. Of course, you might be able to emulate the instruction and have a branch that branches to the next real instruction so we just have to start the process again without having to stop it. That would be a nice approach.
  
The really cool trick would be to insert the breakpoint as a branch to a landing pad we insert.  Then the landing pad could look like:

if (thread_is_supposed_to_stop_at_breakpoints() && breakpoint_condition())
  __builtin_trap();

emulate_instructions_you_needed_to_remove();
jump_back_to_next_instruction();

Then we could support some threads that NEVER stop, and also run breakpoint conditions locally which would make them really cheap.  You could even squirrel away some state in the target that told you that a NEVER stop breakpoint hit the breakpoint, so your accounting would still be correct.

> 
>> Fred
>> 
>>>> 
>>>> I’ve been interested in displaced stepping for different reasons. If we had that capability, it would become much easier to patch code. I’d love to use this to have breakpoint conditions injected and evaluated without round tripping to the debugger when the condition returns false.
>>> 
>>> Agreed!
>>> 
>>>> 
>>>> Fred  
>>>> 
>>>>> Be sure to speak with myself, Jim Ingham and Pavel in depth before undertaking this task as there will be many changes required.
>>>>> 
>>>>> Greg 
>>>>>> 
>>>>>> Thanks,
>>>>>> Ramana
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> lldb-dev mailing list
>>>>>> lldb-dev at lists.llvm.org
>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>>> 
>>>>> _______________________________________________
>>>>> lldb-dev mailing list
>>>>> lldb-dev at lists.llvm.org
>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev