[lldb-dev] Simultaneous multiple target debugging

[jingham at apple.com]

Right, it seems to me clear that you need two entities.  

One knows what targets can be created in a given debugging scenario, and how to hook up to them.  Then you need another to manage picking some subset of these targets, and coordinating the processes running in each of them.

The Platform seemed the logical place to do the first job.  However, Matthew is right that at present the Platforms are homogenous, and more deal with OS details.  So maybe it would be better to keep the Platform more about OS details, then we could add a "device" abstraction that represents composite entities with multiple debuggable opportunities, and then each of these "debuggable opportunities" would have a Platform to represent the OS like features of this opportunity (need some good word for this.)  That might be a better way to go.  Note that the "debuggable opportunities" are more general than just different devices on a board.  For instance, you could imagine debugging the kernel, and a user-space process running on that OS, and coordinating those just as you would a main processor and a co-processor...  To make matters a little confusing, a "device" might represent all the processes running on a single OS, since that's not formally different from the more straightforward device scenario.  So in some ways a platform IS also a device in this sense.  Maybe the abstraction is more a target provider, and the Platform is a homogenous target provider, in addition to its OS duties, and a device is a heterogenous target provider?

But in either case, once you've chosen to attach to several debug sockets, there's the whole business that Matthew mentions of coordinating the sessions.  That is clearly a whole different kettle of fish from just "what can I attach to".  

BTW, this coordinating entity should not be restricted to different devices.  At that level, of course, it is really about coordinating targets & their process regardless of where they come from.  For instance you'd want to be able to use the same structure to coordinate debugging message passing or socket traffic, etc on two user space processes on the same or different systems.  It would also be interesting to model this coordination in a way that could also be extended to threads in a single process.  Right now, each thread's behavior is programmed using the ThreadPlans which work only on a per thread basis and don't make any attempt to coordinate threads.  But it would be useful (and more so when you start doing keep alive debugging) to have some way to program "when thread A does X, wait for thread B to do Y..."  That isn't formally different from two processes or several co-processors.  Be interesting to see how much of the coordination we could make very general.

Jim



> On Aug 27, 2014, at 7:55 AM, Todd Fiala <tfiala at google.com> wrote:
> 
> > Greg and Jim both mentioned using the Platform class as the place to implement this kind of thing.
> 
> I think Jim later mentioned a higher-level concept is needed to do some of the orchestration that we'd want to enable, IIRC.
> 
> 
> On Tue, Aug 26, 2014 at 11:18 PM, Matthew Gardiner <mg11 at csr.com> wrote:
> Hi Colin,
> 
> Multiple target debugging is a massive interest to us at CSR. We design chips with various processor types (e.g. kalimba, XAP, 8051, ARM etc) and on several of our chips we have multiple-processors. There are lots of combinations of setups that we have either already done, or are actively experimenting on. Generally, we have heterogenous setups (e.g. XAP+8051, or 4*XAP+kalimba+8051) etc.
> 
> I see that lldb already supports the concept of a target list, an active target and manual switching between current targets. However, as Colin alludes, there are several features associated with multiple-target which require control from a higher-level.
> 
> What we currently have in our existing debuggers is options of the form, "I'm debugging targets A and B, if A stops do I want B stop as well?". The answer to that question is very much specific to that user's current debug scenario. Of course, getting B to stop if A does, is best implemented in the hardware, and typically a register will be available as a mechanism to configure this feature. In our (CSRs) world probably one of the processors will have access to the associated hardware block, and our debugger will talk to this target to access the feature.
> 
> So, of course, if non-active target(s) stops whilst stepping/running the active one, some notification needs to be passed up, informing the debug session controller of this, and determining whether or not to switch active target.
> 
> Greg and Jim both mentioned using the Platform class as the place to implement this kind of thing. However, does the Platform not only deal in homogenous entities? Is it correct to use this concept to control different processor families. With my limited lldb architectural knowledge, I would have thought that the most likely candidate to control this is the Debugger object itself.
> 
> Matt
> 
> 
> 
> Colin Riley wrote:
> Has anybody done any work on integrating features into LLDB to allow for 'meaningful' simultaneous multiple target debugging? There are various scenarios in which this is a very valuable feature:
> 
> 1) coprocessor debugging, in single-process systems (i.e, embedded DSP alongside say a host CPU core)
> 2) graphical debugging, e.g. games: ideally you want to be able to debug the CPU code alongside any GPU workgroups, and have a single interface to any shared resources such as memory.
> 
> We've done work like this in the past to LLDB, it's not been contributed back because we couldn't do so for commercial reasons (and it's not in a state to contribute back, either). However in the future I think this will become a 'killer app' feature for LLDB and we should be planning to support it.
> 
> At the moment we can have multiple targets, processes etc running in an LLDB session. However I am failing to see any system for communication and interpretation of multiple targets as a whole. If we take the DSP/CPU situation, I may be watching a CPU memory location whilst at the same time single-stepping through the DSP. It's currently undefined and a bit unknown as to how this situation would work in LLDB as stands. From what I can see, it's quite hard to use the current independent target framework to achieve a meaningful debugging session.
> 
> It's as though we'd want some sort of session object, that can take multiple targets together and understand how they operate as to achieve some sort of well-defined behaviour in how it's debugged. I.e, in the DSP/CPU scenario, the session object would understand the DSP has access to the CPU memory, and as such, if we're currently on the DSP single stepping, it would allow a CPU watchpoint event through to the DSP session, with an ability to switch target.
> 
> There are many more items we'd need to allow communication between. A quick example, we have an LLDB version here that supports non-stop mode debugging (see https://sourceware.org/gdb/current/onlinedocs/gdb/Non_002dStop-Mode.html - and we _will_ contribute this back). At the moment stepping through one thread and a breakpoint happens in another is a bit nasty: LLDB simply switches to whatever thread id is greater. When this sort of usability issue exists in a single-target fashion, we may need to look at extracting this out into some sort of policy system that targets (and, these theoretical session objects) can use to decide how to handle certain event situations.
> 
> Apologies if this is a bit of a brain dump. It's quite a complex concept, which is why I think dialogue needs to start now as it's something as I've mentioned we are actively doing at Codeplay, but when the time comes to push upstream, want to do so in a way the community thinks is valuable. There may be other viewpoints, like 'super debugservers' that can manage multiple targets and spoof a single target to LLDB, for example.
> 
> Any other opinions or thoughts out there? :)
> 
> Colin
> 
> 
> 
> 
> Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom
> More information can be found at www.csr.com. Keep up to date with CSR on our technical blog, www.csr.com/blog, CSR people blog, www.csr.com/people, YouTube, www.youtube.com/user/CSRplc, Facebook, www.facebook.com/pages/CSR/191038434253534, or follow us on Twitter at www.twitter.com/CSR_plc.
> New for 2014, you can now access the wide range of products powered by aptX at www.aptx.com.
> 
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
> 
> 
> 
> -- 
> Todd Fiala |	 Software Engineer |	 tfiala at google.com |	 650-943-3180
> 
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev