<div dir="ltr"><div><div>I'm running into some issues trying to debug with lldb on linux.</div><div>It started with errors that debugserver aborted. I ran lldb-server manually and then used process connect, and got this error:</div><div><br></div><div><font face="monospace, monospace">lldb-server: ../tools/lldb/source/Plugins/Process/Linux/NativeThreadLinux.cpp:227: virtual NativeRegisterContextSP lldb_private::process_linux::NativeThreadLinux::GetRegisterContext(): Assertion `reg_interface && "OS or CPU not supported!"' failed.</font></div></div><div><br></div><div>I synced lldb and llvm to head and rebuilt, and this error went away. But then later it showed up again on a different machine when debugging the same binary with the newly built lldb.</div><div><br></div><div>Once I get the binary running under lldb I'm seeing strange things on my OperatingSystem plugin. Sometimes things work normally, sometimes the OS plugin seems to see incorrect register values but they're correct at the lldb prompt, and sometimes the registers just seem completely wrong.</div><div>The behaviour is pretty consistent when I set a breakpoint at the same line, but if I move it to a different line or move to a different machine I'll get a different behavior.</div><div><br></div><div>Here's an example where the plugin sees invalid registers:</div><div><div><font face="monospace, monospace">(lldb) run</font></div><div><font face="monospace, monospace">OperatingSystemGo::UpdateThreadList(0, 1, 1) fetching thread data from Go for pid 678</font></div><div><font face="monospace, monospace">Process 678 launched: 'vc.test' (x86_64)</font></div><div><font face="monospace, monospace">I0528 17:09:01.105108 678 rand.go:86] Seeding pseudo-random number generator with 1432858141105104474</font></div><div><font face="monospace, monospace">OperatingSystemGo::UpdateThreadList(1, 1, 1) fetching thread data from Go for pid 678</font></div><div><font face="monospace, monospace">OperatingSystemGo: RealThread 0 has sp 7fffffffd430</font></div><div><font face="monospace, monospace">Process 678 stopped<br></font></div><div><font face="monospace, monospace">* thread #1: tid = 0x02a6, 0x000000000049f379 vc.test`vc.createVCs(~r0=stream.XVC at 0x000000c208030668, ~r1=stream.XVC at 0x000000c208030678) + 73 at xvc_test.go:23, name =</font></div><div><font face="monospace, monospace">'vc.test', stop reason = breakpoint 1.1</font></div><div><font face="monospace, monospace"> frame #0: 0x000000000049f379 vc.test`vc.createVCs(~r0=stream.XVC at 0x000000c208030668, ~r1=stream.XVC at 0x000000c208030678) + 73 at xvc_test.go:23</font></div><div><font face="monospace, monospace">(lldb) register read</font></div><div><font face="monospace, monospace">General Purpose Registers:</font></div><div><font face="monospace, monospace"> rax = 0x000000c208030580</font></div><div><font face="monospace, monospace"> rbx = 0x0000000000000000</font></div><div><font face="monospace, monospace"> rcx = 0x000000c208001a40</font></div><div><font face="monospace, monospace"> rdx = 0x000000000095e678 vc.test`vc.TestXVC.f</font></div><div><font face="monospace, monospace"> rdi = 0x0000000000000000</font></div><div><font face="monospace, monospace"> rsi = 0x000000c208030768</font></div><div><font face="monospace, monospace"> rbp = 0x0000000000abe578 main.statictmp_0002 + 312</font></div><div><font face="monospace, monospace"> rsp = 0x000000c208030500</font></div><div><font face="monospace, monospace"> r8 = 0x000000005567ae1d</font></div><div><font face="monospace, monospace"> r9 = 0x0000000000000007</font></div><div><font face="monospace, monospace"> r10 = 0x000000000300c146</font></div><div><font face="monospace, monospace"> r11 = 0x000000000000007b</font></div><div><font face="monospace, monospace"> r12 = 0x0000000000000025</font></div><div><font face="monospace, monospace"> r13 = 0x000000000095d840 vc.test`runtime.gcbits.3082880518</font></div><div><font face="monospace, monospace"> r14 = 0x0000000000000008</font></div><div><font face="monospace, monospace"> r15 = 0x0000000000000000</font></div><div><font face="monospace, monospace"> rip = 0x000000000049f379 vc.test`vc.createVCs + 73 at xvc_test.go:23</font></div><div><font face="monospace, monospace"> rflags = 0x0000000000000246</font></div><div><font face="monospace, monospace"> cs = 0x0000000000000033</font></div><div><font face="monospace, monospace"> fs = 0x0000000000000000</font></div><div><font face="monospace, monospace"> gs = 0x0000000000000000</font></div><div><font face="monospace, monospace"> ss = 0x000000000000002b</font></div><div><font face="monospace, monospace"> ds = 0x0000000000000000</font></div><div><font face="monospace, monospace"> es = 0x0000000000000000</font></div><div><br></div></div><div>So my plugin sees one real thread, but the SP is bogus: 7fffffffd430</div><div>When I print registers at the prompt i get the correct value: 0x000000c208030500</div><div><br></div><div>Then I run stepi:</div><div><br></div><div><div><font face="monospace, monospace">(lldb) stepi</font></div><div><font face="monospace, monospace">OperatingSystemGo::UpdateThreadList(8, 3, 3) fetching thread data from Go for pid 678</font></div><div><font face="monospace, monospace">OperatingSystemGo: RealThread 0 has sp c2080304f8</font></div><div><font face="monospace, monospace">OperatingSystemGo: RealThread 1 has sp 7ffff77f5df8</font></div><div><font face="monospace, monospace">OperatingSystemGo: RealThread 2 has sp 7ffff6fb4d70</font></div><div><br></div><div>Now my plugin is getting three threads. But we've only gone one instruction, so where are these other threads coming from? Thread 0 has the correct SP this time, but not the other threads.</div><div>Now that we've got the right SP value, the OS plugin is able to find the memory thread which confuses the threadplan so it keeps running till it hits a breakpoint:</div><div><br></div><div><font face="monospace, monospace">OperatingSystemGo::UpdateThreadList(9, 3, 3) fetching thread data from Go for pid 678</font></div><div><font face="monospace, monospace">OperatingSystemGo: RealThread 0 has sp c208030500</font></div><div><font face="monospace, monospace">OperatingSystemGo: RealThread 1 has sp 7ffff77f5df8</font></div><div><font face="monospace, monospace">OperatingSystemGo: RealThread 2 has sp 7ffff6fb4d70</font></div><div><font face="monospace, monospace">Process 678 stopped</font></div><div><font face="monospace, monospace">* thread #8: tid = 0x0006, 0x000000000049f3b1 vc.test`vc.createVCs(~r0=stream.XVC at 0x000000c208030668, ~r1=stream.XVC at 0x000000c208030678) + 129 at xvc_test.go:24, stop reason = breakpoint 2.1</font></div><div><font face="monospace, monospace"> frame #0: 0x000000000049f3b1 vc.test`vc.createVCs(~r0=stream.XVC at 0x000000c208030668, ~r1=stream.XVC at 0x000000c208030678) + 129 at xvc_test.go:24</font></div><div><font face="monospace, monospace">(lldb)</font></div></div><div><br></div><div>Debugging the same binary on a different machine with the same lldb binary, I get different behavior:</div><div><br></div><div><div><font face="monospace, monospace">(lldb) r</font></div><div><font face="monospace, monospace">There is a running process, kill it and restart?: [Y/n] y</font></div><div><font face="monospace, monospace">Process 127429 exited with status = 9 (0x00000009)</font></div><div><font face="monospace, monospace">OperatingSystemGo::UpdateThreadList(0, 1, 1) fetching thread data from Go for pid 127449</font></div><div><font face="monospace, monospace">Process 127449 launched: 'vc.test' (x86_64)</font></div><div><font face="monospace, monospace">I0528 17:10:43.950815 127449 rand.go:86] Seeding pseudo-random number generator with 1432858243950810731</font></div><div><font face="monospace, monospace">OperatingSystemGo::UpdateThreadList(1, 1, 1) fetching thread data from Go for pid 127449</font></div><div><font face="monospace, monospace">OperatingSystemGo: RealThread 0 has sp 7fffffffde50</font></div><div><font face="monospace, monospace">Process 127449 stopped</font></div><div><font face="monospace, monospace">* thread #1: tid = 0x1f1d9, 0x00007ffff7ddb2d0, name = 'vc.test', stop reason = signal SIGSTOP</font></div><div><font face="monospace, monospace"> frame #0: 0x00007ffff7ddb2d0</font></div><div><font face="monospace, monospace">-> 0x7ffff7ddb2d0: movq %rsp, %rdi</font></div><div><font face="monospace, monospace"> 0x7ffff7ddb2d3: callq 0x7ffff7ddea70</font></div><div><font face="monospace, monospace"> 0x7ffff7ddb2d8: movq %rax, %r12</font></div><div><font face="monospace, monospace"> 0x7ffff7ddb2db: movl 0x221b17(%rip), %eax</font></div><div><font face="monospace, monospace">(lldb) register read</font></div><div><font face="monospace, monospace">General Purpose Registers:</font></div><div><font face="monospace, monospace"> rax = 0x0000000000000000</font></div><div><font face="monospace, monospace"> rbx = 0x0000000000000000</font></div><div><font face="monospace, monospace"> rcx = 0x0000000000000000</font></div><div><font face="monospace, monospace"> rdx = 0x0000000000000000</font></div><div><font face="monospace, monospace"> rdi = 0x0000000000000000</font></div><div><font face="monospace, monospace"> rsi = 0x0000000000000000</font></div><div><font face="monospace, monospace"> rbp = 0x0000000000000000</font></div><div><font face="monospace, monospace"> rsp = 0x00007fffffffde50</font></div><div><font face="monospace, monospace"> r8 = 0x0000000000000000</font></div><div><font face="monospace, monospace"> r9 = 0x0000000000000000</font></div><div><font face="monospace, monospace"> r10 = 0x0000000000000000</font></div><div><font face="monospace, monospace"> r11 = 0x0000000000000000</font></div><div><font face="monospace, monospace"> r12 = 0x0000000000000000</font></div><div><font face="monospace, monospace"> r13 = 0x0000000000000000</font></div><div><font face="monospace, monospace"> r14 = 0x0000000000000000</font></div><div><font face="monospace, monospace"> r15 = 0x0000000000000000</font></div><div><font face="monospace, monospace"> rip = 0x00007ffff7ddb2d0</font></div><div><font face="monospace, monospace"> rflags = 0x0000000000000200</font></div><div><font face="monospace, monospace"> cs = 0x0000000000000033</font></div><div><font face="monospace, monospace"> fs = 0x0000000000000000</font></div><div><font face="monospace, monospace"> gs = 0x0000000000000000</font></div><div><font face="monospace, monospace"> ss = 0x000000000000002b</font></div><div><font face="monospace, monospace"> ds = 0x0000000000000000</font></div><div><font face="monospace, monospace"> es = 0x0000000000000000</font></div></div><div><font face="monospace, monospace"><div><br></div><div>(lldb) bt</div><div>* thread #1: tid = 0x1f1d9, 0x00007ffff7ddb2d0, name = 'vc.test', stop reason = signal SIGSTOP</div><div> * frame #0: 0x00007ffff7ddb2d0</div><div><br></div></font></div><div><font face="monospace, monospace"><br></font></div><div>This time the plugin gets the invalid register values, but when lldb stops at the prompt things are still invalid. The rip is garbage, so lldb doesn't recognize that it's hit a breakpoint and the backtrace is wrong.</div><div><br></div><div>Any ideas what could be going on here, or how I can debug this?</div><br clear="all"><div><div class="gmail_signature">-- Ryan Brown<br></div></div>
</div>