<div dir="ltr"><div>Well, i wrote a very long mail detailing my journey to resolve issue #2 (hanging after setting target.use-fast-stepping=false), only to eventually realize that it doesn't hang but instead just waits for the above loop to complete.</div><div><br></div><div>This means turning off target.use-fast-stepping is not an option and i'm back to square one. I'd be grateful for any pointers on how to fix issue #1 (EXC_BAD_INSTRUCTION). I guess i'll start by investigating the <span style="color:rgb(0,0,0);white-space:pre-wrap">"run to next branch" stepping algorithm</span> in LLDB, though my understanding is likely not sufficient to make a dent.<br></div><div><br></div><div>Thanks,</div><div>Mario</div><div><br></div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Dec 1, 2014 at 11:05 AM, Mario Zechner <span dir="ltr"><<a href="mailto:badlogicgames@gmail.com" target="_blank">badlogicgames@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi,<div><br></div><div>setting target.use-fast-stepping to false did indeed solve this issue, albeit at the cost of increased runtime obviously. However, i ran into another issue right after i stepped out of the previously problematic function: <a href="http://sht.tl/bdAKRC" target="_blank">http://sht.tl/bdAKRC</a></div><div><br></div><div>Trying to source-level step this function (with use-fast-stepping=false) results in 1) the disassembly getting all kinds of messed up and 2) the process not stepping but hanging at the `cmp r1, #0` instruction. The original assembly code around that PC looks like this:</div><div><br></div><div><div>LBB24_1: @ %label0</div><div> @ =>This Inner Loop Header: Depth=1</div><div><span style="white-space:pre-wrap"> </span>@DEBUG_VALUE: [J]java.lang.Thread.<init>(Ljava/lang/Runnable;Ljava/lang/String;)V:__$env <- R5</div><div><span style="white-space:pre-wrap"> </span>ldrexd<span style="white-space:pre-wrap"> </span>r1, r2, [r0]</div><div><span style="white-space:pre-wrap"> </span>strexd<span style="white-space:pre-wrap"> </span>r1, r6, r6, [r0]</div><div><span style="white-space:pre-wrap"> </span>cmp<span style="white-space:pre-wrap"> </span>r1, #0</div><div><span style="white-space:pre-wrap"> </span>bne<span style="white-space:pre-wrap"> </span>LBB24_1</div><div>@ BB#2: @ %label0</div><div><span style="white-space:pre-wrap"> </span>@DEBUG_VALUE: [J]java.lang.Thread.<init>(Ljava/lang/Runnable;Ljava/lang/String;)V:__$env <- R5</div><div><span style="white-space:pre-wrap"> </span>dmb<span style="white-space:pre-wrap"> </span>ish</div><div><span style="white-space:pre-wrap"> </span>movs<span style="white-space:pre-wrap"> </span>r1, #5</div></div><div><br></div><div>A simple loop, which is actually part of an inlined function. We had some issues with inlined functions previously, i assume this issue is related. Interestingly enough, the back trace is also a bit wonky:</div><div><br></div><div>
<p>(lldb) <span>bt</span></p>
<p>* thread #1: tid = 0x18082, 0x0021a9b4 AttachTestIOSDev`[J]java.lang.Thread.<init>(Ljava/lang/Runnable;Ljava/lang/String;)V [inlined] [j]java.lang.Thread.threadPtr(J)[set] + 14 at Thread.java:1, stop reason = trace</p>
<p> * frame #0: 0x0021a9b4 AttachTestIOSDev`[J]java.lang.Thread.<init>(Ljava/lang/Runnable;Ljava/lang/String;)V [inlined] [j]java.lang.Thread.threadPtr(J)[set] + 14 at Thread.java:1</p><p> frame #1: 0x0021a9a6 AttachTestIOSDev`[J]java.lang.Thread.<init>(__$env=0x01662fc8, __$this=0x64da3833, runnable=0xa4f07400, threadName=0x00286000)V + 46 at Thread.java:138</p></div><div>There should be a lot more frame. I'm gonna try to dig up some more details.</div><div><br></div><div>Thanks a lot!</div><span class="HOEnZb"><font color="#888888"><div>Mario</div><div><br></div><div><br></div></font></span></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Nov 30, 2014 at 1:32 AM, Jason Molenda <span dir="ltr"><<a href="mailto:jason@molenda.com" target="_blank">jason@molenda.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">The size of the breakpoint instruction is set by GetSoftwareBreakpointTrapOpcode(). In your case, most likely you're in PlatformDarwin::GetSoftwareBreakpointTrapOpcode() - lldb uses the symbol table (from the binary file) to determine if the code in a given function is arm or thumb. If it's arm, a 4 byte breakpoint is used. If it's thumb, a 2 byte breakpoint. Of course thumbv2 of T32 instructions can be 4 bytes -- the blne instruction is in your program -- but I assume the 2 byte breakpoint instruction still works correctly in these cases; the cpu sees the 2-byte instruction and stops execution.<br>
<br>
I am a little wary about the fact that this comes after an it instruction, I kind of vaguely remember issues with that instruction's behavior.<br>
<br>
It shouldn't make any difference but you might want to try<br>
<br>
(lldb) settings set target.use-fast-stepping false<br>
<br>
which will force lldb to single instruction step through the function. Right now lldb is looking at the instruction stream and putting breakpoints on branch/call/jump instructions to do your high-level "step" command, instead of stopping on every instruction. It is possible there could be a problem with that approach and the it instruction. Please report back if this changes the behavior.<br>
<br>
J<br>
<div><div><br>
<br>
> On Nov 26, 2014, at 9:22 AM, Mario Zechner <<a href="mailto:badlogicgames@gmail.com" target="_blank">badlogicgames@gmail.com</a>> wrote:<br>
><br>
> I dug a little deeper, inspecting the GDB remote packets send by LLDB to perform the stepping. It appears when sending memory breakpoint commands used for stepping, the size of the instruction being replaced isn't taken into account, or writing back the original instruction isn't done properly. The following log shows what happens when stepping into the previously mentioned function:<br>
><br>
> (lldb) s<br>
> Process 166 stopped<br>
> * thread #1: tid = 0x0fd9, 0x002602e0 AttachTestIOSDev`[J]java.lang.Object.<init>(__$env=0x016bffc8, __$this=0x017864b0)V + 12 at Object.java:136, queue = 'com.apple.main-thread', stop reason = step in<br>
> frame #0: 0x002602e0 AttachTestIOSDev`[J]java.lang.Object.<init>(__$env=0x016bffc8, __$this=0x017864b0)V + 12 at Object.java:136<br>
> (lldb) disassemble -p<br>
> AttachTestIOSDev`[J]java.lang.Object.<init>()V + 12 at Object.java:136:<br>
> -> 0x2602e0: ldr r2, [r1]<br>
> 0x2602e2: ldr r2, [r2, #0x30]<br>
> 0x2602e4: tst.w r2, #0x100000<br>
> 0x2602e8: it ne<br>
> (lldb) s<br>
> Process 166 stopped<br>
> * thread #1: tid = 0x0fd9, 0x002602ec AttachTestIOSDev`[J]java.lang.Object.<init>(__$env=0x016bffc8, __$this=0x017864b0)V + 24 at Object.java:136, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=EXC_ARM_UNDEFINED, subcode=0xffd1b001)<br>
> frame #0: 0x002602ec AttachTestIOSDev`[J]java.lang.Object.<init>(__$env=0x016bffc8, __$this=0x017864b0)V + 24 at Object.java:136<br>
> (lldb) disassemble -p<br>
> AttachTestIOSDev`[J]java.lang.Object.<init>()V + 24 at Object.java:136:<br>
> -> 0x2602ec: .long 0xb001ffd1 ; unknown opcode<br>
> 0x2602f0: pop {r7, pc}<br>
><br>
> AttachTestIOSDev`[J]java.lang.Object.<init>()V + 30:<br>
> 0x2602f2: nop<br>
><br>
> AttachTestIOSDev`[J]java.lang.Object.clone()Ljava/lang/Object; at Object.java:154:<br>
> 0x2602f4: push {r4, r5, r7, lr}<br>
> (lldb) disassemble -f<br>
> AttachTestIOSDev`[J]java.lang.Object.<init>()V at Object.java:136:<br>
> 0x2602d4: push {r7, lr}<br>
> 0x2602d6: mov r7, sp<br>
> 0x2602d8: sub sp, #0x4<br>
> 0x2602da: movs r2, #0x0<br>
> 0x2602dc: str r2, [sp]<br>
> 0x2602de: str r1, [sp]<br>
> 0x2602e0: ldr r2, [r1]<br>
> 0x2602e2: ldr r2, [r2, #0x30]<br>
> 0x2602e4: tst.w r2, #0x100000<br>
> 0x2602e8: it ne<br>
> 0x2602ea: blne 0x44b290 ; _bcRegisterFinalizer<br>
> 0x2602ee: add sp, #0x4<br>
> 0x2602f0: pop {r7, pc}<br>
><br>
> AttachTestIOSDev`[J]java.lang.Object.<init>()V + 30:<br>
> 0x2602f2: nop<br>
><br>
> The first step succeeds and ends up right after the prologue, at 0x2602e0: ldr r2, [r1]. The next step ends up at 0x2602ec: .long 0xb001ffd1 which is wrong, it should be 0x2602ea: blne 0x44b290.<br>
><br>
> The GDB remote conversation between lldb and the debugserver on the device (only relevant parts):<br>
><br>
> # First step<br>
> lldb->debugserver: $Z0,2602e0,2#73<br>
> debugserver->lldb: $OK#00<br>
> lldb->debugserver: $vCont;c:0fd9#15<br>
> debugserver->lldb: (320) $T05thread:fd9;qaddr:37ebfad0;threads:fd9,ffa,ffb,ffd,fff,1009,100a,100b;00:c8ff6b01;01:b0647801;02:00000000;03:c87d6a00;04:00000000;05:c8ff6b01;06:fc6a6501;07:0c6a6501;08:90e96b01;09:28000000;0a:74a0ea37;0b:c8ff6b01;0c:b09e5b00;0d:086a6501;0e:d1b22000;0f:<br>
><br>
> # Second step<br>
> lldb->debugserver: $Z0,2602ea,2#a4<br>
> debugserver->lldb: $OK#00<br>
> lldb->debugserver: $vCont;c:0fd9#15<br>
> debugserver->lldb: (324) $T92thread:fd9;qaddr:37ebfad0;threads:fd9,ffa,ffb,ffd,fff,1009,100a,100b;00:c8ff6b01;01:b0647801;02:01004300;03:c87d6a00;04:00000000;05:c8ff6b01;06:fc6a6501;07:0c6a6501;08:90e96b01;09:28000000;0a:74a0ea37;0b:c8ff6b01;0c:b09e5b00;0d:086a6501;0e:d1b22000;0f:<br>
><br>
> For the first step, a 2 byte memory breakpoint is written to 0x2602e0 ($Z0,2602e0,2#73), which is where the first step ended up. The instruction that got replaced is 2 bytes long. The GDB command wrote a 2 bytes memory breakpoint to the address, so all is good.<br>
><br>
> For the second step, a 2 byte memory breakpoint is written to 0x2602ea ($Z0,2602ea,2#a4). But instead of ending up at 0x2602ec, which is in the middle of the 4-byte blne instruction.<br>
><br>
> Is it correct for LLDB to set a 2 byte memory breakpoint instead of a 4-byte memory breakpoint in this case? The PC will be set to an invalid address, which then causes the EXC_BAD_INSTRUCTION.<br>
><br>
> Am i understanding this correctly? Is there a way for me to fix this?<br>
><br>
> On Wed, Nov 26, 2014 at 5:26 PM, Mario Zechner <<a href="mailto:badlogicgames@gmail.com" target="_blank">badlogicgames@gmail.com</a>> wrote:<br>
> Hi,<br>
><br>
> we generate thumbv7 binaries for iOS devices. We deploy, launch and debug those via LLDB. Stepping into functions seems to almost always generate a EXC_BAD_INSTRUCTION signal. The signal is not generated when running the app without the debugger attached. It is also not generated when we attach a debugger, but simply let the app run without breakpoints or any stepping.<br>
><br>
> Here's one of these function's LLVM IR:<br>
><br>
> =======================<br>
> define external void @"[J]java.lang.Object.<init>()V"(%Env* %p0, %Object* %p1) nounwind noinline optsize {<br>
> label0:<br>
> call void @"llvm.dbg.declare"(metadata !{%Env* %p0}, metadata !19), !dbg !{i32 136, i32 0, metadata !{i32 786478, metadata !0, metadata !1, metadata !"[J]java.lang.Object.<init>()V", metadata !"[J]java.lang.Object.<init>()V", metadata !"", i32 136, metadata !15, i1 false, i1 true, i32 0, i32 0, null, i32 256, i1 false, void (%Env*, %Object*)* @"[J]java.lang.Object.<init>()V", null, null, metadata !17, i32 136}, null}<br>
> %r0 = alloca %Object*<br>
> store %Object* null, %Object** %r0<br>
> call void @"llvm.dbg.declare"(metadata !{%Object** %r0}, metadata !21), !dbg !{i32 136, i32 0, metadata !14, null}<br>
> store %Object* %p1, %Object** %r0<br>
> call void @"register_finalizable"(%Env* %p0, %Object* %p1), !dbg !{i32 136, i32 0, metadata !18, null}<br>
> ret void, !dbg !{i32 136, i32 0, metadata !18, null}<br>
> }<br>
> =======================<br>
><br>
> The corresponding thumbv7 assembler code as generated by LLVM:<br>
><br>
> =======================<br>
> .globl "_[J]java.lang.Object.<init>()V"<br>
> .align 2<br>
> .code 16 @ @"[J]java.lang.Object.<init>()V"<br>
> .thumb_func "_[J]java.lang.Object.<init>()V"<br>
> "_[J]java.lang.Object.<init>()V":<br>
> .cfi_startproc<br>
> Lfunc_begin18:<br>
> .loc 1 136 0 @ Object.java:136:0<br>
> @ BB#0: @ %label0<br>
> .loc 1 136 0 @ Object.java:136:0<br>
> push {r7, lr}<br>
> mov r7, sp<br>
> sub sp, #4<br>
> @DEBUG_VALUE: [J]java.lang.Object.<init>()V:__$env <- R0<br>
> movs r2, #0<br>
> str r2, [sp]<br>
> str r1, [sp]<br>
> .loc 1 136 0 prologue_end @ Object.java:136:0<br>
> Ltmp6:<br>
> ldr r2, [r1]<br>
> ldr r2, [r2, #48]<br>
> tst.w r2, #1048576<br>
> Ltmp7:<br>
> @DEBUG_VALUE: [J]java.lang.Object.<init>()V:__$env <- R0<br>
> it ne<br>
> blxne __bcRegisterFinalizer<br>
> add sp, #4<br>
> pop {r7, pc}<br>
> Ltmp8:<br>
> Lfunc_end18:<br>
> "L_[J]java.lang.Object.<init>()V_end":<br>
><br>
> .cfi_endproc<br>
> =======================<br>
><br>
> Now, when stepping into this function, LLDB receives a signal from the debug server:<br>
><br>
> =======================<br>
> (lldb) s<br>
> Process 176 stopped<br>
> * thread #1: tid = 0x11f5, 0x0023e2ec AttachTestIOSDev`[J]java.lang.Object.<init>(__$env=0x0169efc8, __$this=0x0174cd10)V + 24 at Object.java:136, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=EXC_ARM_UNDEFINED, subcode=0xffd1b001)<br>
> frame #0: 0x0023e2ec AttachTestIOSDev`[J]java.lang.Object.<init>(__$env=0x0169efc8, __$this=0x0174cd10)V + 24 at Object.java:136<br>
> =======================<br>
><br>
> Disassembling around the PC gives:<br>
><br>
> =======================<br>
> (lldb) disassemble --pc<br>
> AttachTestIOSDev`[J]java.lang.Object.<init>()V + 24 at Object.java:136:<br>
> -> 0x23e2ec: .long 0xb001ffd1 ; unknown opcode<br>
> 0x23e2f0: pop {r7, pc}<br>
><br>
> AttachTestIOSDev`[J]java.lang.Object.<init>()V + 30:<br>
> 0x23e2f2: nop<br>
><br>
> Disassembling until the beginning of the frame gives:<br>
><br>
> (lldb) disassemble -f<br>
> AttachTestIOSDev`[J]java.lang.Object.<init>()V at Object.java:136:<br>
> 0x23e2d4: push {r7, lr}<br>
> 0x23e2d6: mov r7, sp<br>
> 0x23e2d8: sub sp, #0x4<br>
> 0x23e2da: movs r2, #0x0<br>
> 0x23e2dc: str r2, [sp]<br>
> 0x23e2de: str r1, [sp]<br>
> 0x23e2e0: ldr r2, [r1]<br>
> 0x23e2e2: ldr r2, [r2, #0x30]<br>
> 0x23e2e4: tst.w r2, #0x100000<br>
> 0x23e2e8: it ne<br>
> 0x23e2ea: blne 0x429290 ; _bcRegisterFinalizer<br>
> 0x23e2ee: add sp, #0x4<br>
> 0x23e2f0: pop {r7, pc}<br>
><br>
> Accprding to this, execution should never end up at address 0x23e2ec. That's right in the middle of the blne and add instructions in the second disassembly. I have a hunch that the debugserver on the device may interfere here, e.g. add a trap instruction to implement the stepping. I'm not quite sure what to make of it.<br>
><br>
> I'd appreciate any hints. If you require more information, i got plenty of logs :)<br>
><br>
> Thanks,<br>
> Mario<br>
><br>
</div></div>> _______________________________________________<br>
> lldb-dev mailing list<br>
> <a href="mailto:lldb-dev@cs.uiuc.edu" target="_blank">lldb-dev@cs.uiuc.edu</a><br>
> <a href="http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev</a><br>
<br>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>