[lldb-dev] Shared Process plugin between Linux and BSD

Thu Dec 15 06:14:14 PST 2016

On 15.12.2016 14:14, Pavel Labath wrote:
> I am communicating with Kamil about the possibility of sharing code
> between the two plugins. We plan to have a common base class for the
> shared functionality between the OSs. I have asked Kamil to perform
> this experiment, so we can identify which parts of the code can be
> shared in the first place. My thoughts on the noticed differences are
> below
> 
>>>
>>> Comments to the disabled code:
>>>
>>> 1. Thread resume/suspend operation - to be implemented on NetBSD with
>>> ptrace(2).
> This is going to be the trickiest part, as thread management works
> quite differently on the two OSs. we'll need to investigate more on
> how much we can reuse here
> 

My plan was to add two ptrace(2) calls like: PT_SUSPEND/PT_RESUME and
specify LWP to be resumed/suspended. It shouldn't be hard to implement
as there is already _lwp_suspend(2) and _lwp_continue(2), just call it
for other process with ptrace(2).

PT_CONTINUE could be kept to resume the whole process (modulo frozen
threads) along with PT_STEP. However, I was holding on with it as I
wanted to reflect it with needs of a real debugger like LLDB.

>>>
>>> 2. PTRACE_SETSIGINFO - equivalent currently unsupported, I will need to
>>> add support for it in ptrace(2).
> This value is currently only used in logging code, so we can just
> remove it from the code base. However, it's a useful thing to have for
> the future, so it may be worth making sure the NetBSD kernel supports
> it.
> 

I was wondering whether waitid(2) or wait6(2) could be used there, as
they return siginfo_t. If so, NetBSD does not need dedicated ptrace(2)
entry.

>>>
>>> 3. PTRACE_O_TRACEEXEC, PTRACE_O_TRACECLONE, PTRACE_O_TRACEEXIT -
>>> equivalent to be implemented.
> Do you mean "implemented in lldb-server" or "implemented in the
> kernel"? Being notified when the inferior exec()s would definitely be
> good. Tracing thread creation and exit is a nice-to-have but low
> priority. The only reason we used it on linux is because that's the
> only way to auto-attach to new threads, but I think that for you that
> will happen automatically.
> 

I mean implemented in the kernel.

Thread (lwp) creation and termination is currently detectable in FreeBSD
with a special event, if this is useful in the NetBSD context -- like a
user setting a break on this event explicitly -- I will add it to TODO
list. On NetBSD there is no need to explicitly start tracing new
threads, tracing is per process and it is composed of a bulk of threads
(lwp). POSIX threads are a layer with extra features built upon LWP.

There is also the the exect(3) call in our libc inherited from BSD4.2,
but it no longer works in a useful way. It used to singlestep execve(2)
call. I was thinking whether/how to make it useful again, like combine
in it execve(2) + PT_TRACE_ME + SIGSTOP/SIGTRAP on exec.

>>>
>>> 4. No tracing of VFORK events implemented.
>>>
>>> 5. Hardware assisted watchpoints (debug registers on amd64) have their
>>> dedicated ptrace(2) API to set/unset watchpoints, they do not export raw
>>> debug registers.
>>>
>>> 6. Other ptrace(2) calls have their equivalents on NetBSD
>>> (PTRACE_PEEKUSER, PTRACE_POKEUSER etc).
> Cool, I guess that means we can share (Read|Write)Memory(WithoutTrap|).
> 

Functionality is the same, however the API is different.

>>>
>>> 7. SIGTRAP has currently only two si_code values (specified by POSIX).
> The code for decoding si_code values is already messy and in need of a
> rewrite. I think the act of decoding si_code values will need to be
> OS- (and maybe event architecture-) specific, but the actions taken
> upon individual events should be quite similar.
> 

I was wondering whether it is useful to return distinct si_code for
hardware assisted traps or PT_STEP. At the moment I ignored it.

>>>
>>> 8. No SI_TKILL available.
> This is used because when a single thread in a process stops, we need
> to go and manually stop all other threads. The whole function around
> this should probably be moved into the linux-specific class, as you
> don't need to do anything special to interrupt a process (IIUC).
> 

In NetBSD the whole process is stopped.

> 
>>>
>>> 9. There is no process_vm_readv/process_vm_writev call available.
> These are used only as a preformance optimization on linux. If
> (PEEK|POKE)DATA work the same way, we can put that in the base class.
> Then, linux can override and do it's fancy thing.
> 

ptrace(2) PT_IO call is our local equivalent call.

>>>
>>> 10. __WALL and __WNOTHREAD are Linux specific, but they have their
>>> counterparts.
> We should probably have a waitpid wrapper, which adds any os-specific
> flags if necessary. We can put the EINTR handling there as well to
> make it's usage easier.
> 

NetBSD ships with wait(2), wait3(2), wait4(2), wait6(2), waitid(2),
waitpid(2).

Perhaps wait4(2) or wait6(2) could be a better choice as they offer more
features, but it's an option for later.

>>>
>>> 11. The cpu_set_t structure and appropriate functions have similar
>>> counterparts on NetBSD.
> This is only used to work around a bug in the linux arm64 kernels. I
> don't think you need to worry yourself with that. It can stay hidden
> in NativeThreadLinux::SingleStep
> 

Acknowledged.

>>>
>>> 12. No <sys/procfs.h> and no dependency of procfs (/proc) is acceptable,
>>> everything shall be accessible through ptrace(2) and sysctl(7).
> We'll need to check whether the appropriate interfaces are in place to
> abstract this (I expect it to be the case), and add new ones if
> necessary.
> 
>>>
>>> 13. personality.h - unsupported as header/function call, compatibility
>>> with other OSes (mostly Linux) implemented on syscall level.
> This was already dead code, and I have removed it.
> 

Acknowledged.

>>>
>>> 14. Process, system thread (lwp) and POSIX (pthread_t) thread are not
>>> the same on NetBSD, unlike some other systems and cannot be mixed.
> I believe the code has already good separation between thread and
> process IDs, and doesn't use pthread_t, so this shouldn't be a
> problem. NetBSD-specific pthread_t debugging code can be contained in
> the NetBSD class.
> 

Some code for NetBSD expect pthread_t which, as discussed, is wrong and
should be lwpid_t. And in case of requesting pthread_t specific data we
want to query with the lwpid_t key.

>>>
>>>
>>> The currently missing features on the NetBSD side don't stop us from
>>> making a functional implementation, lacing interfaces might be added
>>> after getting the core part to work.
> 
> Agreed. The only thing that slightly worries me is the lack of
> execve() tracing capability, if I understood correctly. While it
> probably may be possible to work without it. I'd think about adding it
> nonetheless.
> 

I'm going to research it now and in case of required - as in
implementing a debugger much easier -- I will go for it and add support.

> good luck,
> pl
> 

Thank you for your insight.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20161215/913ece61/attachment-0001.sig>