<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class="">Ding! That’s it. This program does use SIGALRM. I wouldn’t have thought it would be enabled at this point, but apparently it is because I got lots of sigalarm halts during the .so loading.</div><div class=""><br class=""></div><div class="">Further, if I run in lldb with </div><div class=""><br class=""></div><div class="">process handle -n false -p false -s false SIGALRM</div><div class=""><br class=""></div><div class="">then the debugger seems to run fine without stopping during dlopen(). The SIGALRM itself isn’t important to the operation of the program.</div><div class=""><br class=""></div><div class="">With this knowledge I can probably create a simple testcase. Would this be considered an lldb bug? If so should I submit a testcase in some way?</div><div class=""><br class=""></div><div class="">thanks!</div><div class="">--steve</div><br class=""><div class="">
<div dir="auto" style="color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div dir="auto" style="color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div dir="auto" style="color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div style="margin: 0px; font-size: 14px; font-family: "Apple Garamond"; -webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial;" class=""><span style="color: rgb(237, 113, 158); font-family: "Lucida Grande";" class=""></span><span style="color: rgb(55, 69, 245); -webkit-text-stroke-color: rgb(55, 69, 245);" class=""><i class=""> </i></span><i class="">The lover of nature is he whose inward and outward senses are still truly adjusted to each other; who has retained the spirit of infancy even into the era of manhood.</i></div><div class=""><br class=""></div></div></div></div></div></div></div></div><br class="Apple-interchange-newline">
</div>
<div><br class=""><blockquote type="cite" class=""><div class="">On Apr 17, 2018, at 1:12 PM, Pavel Labath <<a href="mailto:labath@google.com" class="">labath@google.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="">It's a bit of a wild guess, but is it possible that you (or one of the<br class="">libraries you use) are doing anything with signals (SIGALRM or such?). I<br class="">think I remember looking at the code handling the server-side ignored<br class="">signal handling and thinking that it could go wrong if you get a signal<br class="">while doing a instruction-step. I am not sure it fully applies here as the<br class="">last command that lldb client did was a "continue", but i think it has to<br class="">have something to do with signals, as you end up stopped in a signal<br class="">handler.<br class=""><br class="">Could you try the following sequence of commands?<br class="">(lldb) process launch --stop-at-entry-point<br class="">(lldb) process handle --notify true --stop true #Stop on all signals<br class="">(lldb) continue<br class=""><br class="">and let us know if you see any extra stops due to signals. If that doesn't<br class="">find anything then I think we'll have start pulling logs from the<br class="">lldb-server side, as there doesn't seem to be anything wrong with the<br class="">client. The easiest way to achieve that is to do a<br class="">export LLDB_SERVER_LOG_CHANNELS="posix all:gdb-remote packets"<br class="">export LLDB_DEBUGSERVER_LOG_FILE=/tmp/server.log<br class="">before launching lldb.<br class="">On Tue, 17 Apr 2018 at 18:28, Steve Ravet <<a href="mailto:steve.ravet@apple.com" class="">steve.ravet@apple.com</a>> wrote:<br class=""><br class=""><blockquote type="cite" class="">Pavel asked for a dump of gdb-remote commands. I got that and ran it<br class=""></blockquote>through the gdbremote decoder, and trimmed to include what looks like the<br class="">last successful continue after breakpoint and then the halt on dlopen.<br class="">Both cases stop on signal 5.<br class=""><br class=""><blockquote type="cite" class="">After the stop message the debugger issues two binary reads and then<br class=""></blockquote>apparently makes the decision that it should stop rather than continue.<br class="">The stopping case is missing the equivalent of "Element 1: Single stepping<br class="">past breakpoint site 2 at 0x2aaaaaab9eb0” which is in the continuing case.<br class="">I’ve attached the file here:<br class=""><br class=""><br class=""><blockquote type="cite" class="">thanks,<br class="">--steve<br class=""></blockquote><br class=""><br class=""><blockquote type="cite" class=""> From the ashes of disaster grow the roses of success.<br class=""></blockquote><br class=""><br class=""><br class=""><blockquote type="cite" class="">On Apr 17, 2018, at 11:27 AM, Jim Ingham <<a href="mailto:jingham@apple.com" class="">jingham@apple.com</a>> wrote:<br class=""></blockquote><br class=""><blockquote type="cite" class="">It is interesting that the stop reason on the thread that stopped is<br class=""></blockquote>"trace". That's what you would expect returning from the single-step to<br class="">step over the breakpoint. But it looks like we got a signal while<br class="">single-stepping, but the stop reason was misreported by somebody.<br class=""><br class=""><blockquote type="cite" class="">Jim<br class=""></blockquote><br class=""><br class=""><blockquote type="cite" class="">On Apr 17, 2018, at 6:00 AM, Pavel Labath via lldb-dev <<br class=""></blockquote><a href="mailto:lldb-dev@lists.llvm.org" class="">lldb-dev@lists.llvm.org</a>> wrote:<br class=""><br class=""><blockquote type="cite" class="">[+lldb-dev]<br class=""></blockquote><br class=""><blockquote type="cite" class="">Hello Steve,<br class=""></blockquote><br class=""><blockquote type="cite" class="">thanks for the report.<br class=""></blockquote><br class=""><blockquote type="cite" class="">The fact that you see the rendezvous breakpoint being hit many times is<br class=""></blockquote>not<br class=""><blockquote type="cite" class="">surprising. We get those every time the library is loaded (we need that to<br class="">load relevant debug info and set potential breakpoints). However, they<br class="">should generally not be surfaced to the user (unless you have the<br class="">stop-on-sharedlibrary-events setting set, which you don't).<br class=""></blockquote><br class=""><blockquote type="cite" class="">The part that is suspicious to me is that __restore_rt shows up on the top<br class="">of the backtrace. This is a trampoline used to return from signal<br class=""></blockquote>handlers,<br class=""><blockquote type="cite" class="">and it would seem to indicate that you got some sort of a signal while<br class="">loading the libraries. I don't know why this would happen, but it could be<br class="">that this is confusing lldb's auto-resume logic.<br class=""></blockquote><br class=""><blockquote type="cite" class="">The interesting part to see here is what lldb thinks are the stop reasons<br class="">for individual threads in the process (is the process multi-threaded?) for<br class="">the last couple of stops. The "lldb step" and "gdb-remote packets" log<br class="">categories are the most interesting to observe here. If you are able to<br class="">send me the log traces, I can help you interpret them.<br class=""></blockquote><br class=""><blockquote type="cite" class="">regards,<br class="">pavel<br class=""></blockquote><br class=""><br class=""><br class=""><br class=""><br class=""><blockquote type="cite" class="">On Tue, 17 Apr 2018 at 02:27, Steve Ravet via llvm-dev <<br class=""><a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> wrote:<br class=""></blockquote><br class=""><blockquote type="cite" class="">Hello lldb developers, I am running into a problem with lldb on Linux. I<br class=""></blockquote><br class=""><blockquote type="cite" class="">am currently running llvm 6.0.0.<br class=""></blockquote><br class=""><blockquote type="cite" class="">I have an executable that dynamically loads a large number of shared<br class=""></blockquote><br class=""><blockquote type="cite" class="">libraries at runtime. These are explicitly loaded via dlopen (they are<br class="">specified in a configuration file), and after loading a few (typically a<br class="">dozen or so, but the number varies) lldb will halt during dlopen. If I<br class="">continue, it will load a few more then halt again, which makes debugging<br class="">from startup impractical since there are so many libraries to be loaded<br class="">(more than a hundred of them).<br class=""></blockquote><br class=""><blockquote type="cite" class="">When I build and debug this same C++ on macOS, the debugger works fine.<br class=""></blockquote><br class=""><blockquote type="cite" class="">I have verified that target.process.stop-on-sharedlibrary-events is false.<br class="">I turned on dyld logging and I see lots of log messages about<br class="">RendezvousBreakpoint being hit, but I don’t see anything that sheds light<br class="">on why some libraries load without stopping but others don’t.<br class=""></blockquote><br class=""><blockquote type="cite" class="">I have tried to recreate this in a trivial program that calls dlopen in a<br class=""></blockquote><br class=""><blockquote type="cite" class="">loop, but haven’t been able to reproduce.<br class=""></blockquote><br class=""><blockquote type="cite" class="">Can your offer any suggestions for further debugging this? More<br class=""></blockquote><br class=""><blockquote type="cite" class="">supporting evidence follows.<br class=""></blockquote><br class=""><blockquote type="cite" class="">Here is the message when the debugger stops:<br class=""></blockquote><br class=""><br class=""><blockquote type="cite" class="">Process 120004 stopped<br class="">* thread #1, name = ‘xxxxxxxx', stop reason = trace<br class=""> frame #0: 0x00002aaaacfca6a0 libc.so.6`__restore_rt<br class="">libc.so.6`__restore_rt:<br class="">-> 0x2aaaacfca6a0 <+0>: movq $0xf, %rax<br class=""> 0x2aaaacfca6a7 <+7>: syscall<br class=""> 0x2aaaacfca6a9 <+9>: nopl (%rax)<br class=""></blockquote><br class=""><br class=""><blockquote type="cite" class="">libc.so.6`__libc_sigaction:<br class=""> 0x2aaaacfca6b0 <+0>: subq $0xd0, %rsp<br class=""></blockquote><br class=""><br class=""><blockquote type="cite" class="">I do not have the stop on shared library events setting enabled:<br class=""></blockquote><br class=""><br class=""><blockquote type="cite" class="">(lldb) settings show target.process.stop-on-sharedlibrary-events<br class="">target.process.stop-on-sharedlibrary-events (boolean) = false<br class=""></blockquote><br class=""><br class=""><br class=""><br class=""><blockquote type="cite" class="">The backtrace goes back to dlopen:<br class=""></blockquote><br class=""><br class=""><blockquote type="cite" class="">(lldb) bt<br class="">* thread #1, name = ‘xxxxx', stop reason = trace<br class=""> * frame #0: 0x00002aaaacfca6a0 libc.so.6`__restore_rt<br class=""> frame #1: 0x00002aaaaaab9eb0 ld-linux-x86-64.so.2<br class=""> frame #2: 0x00002aaaaaabdc53 ld-linux-x86-64.so.2`dl_open_worker + 499<br class=""> frame #3: 0x00002aaaaaab9286 ld-linux-x86-64.so.2`_dl_catch_error +<br class=""></blockquote><br class=""><blockquote type="cite" class="">102<br class=""></blockquote><br class=""><blockquote type="cite" class=""> frame #4: 0x00002aaaaaabd63a ld-linux-x86-64.so.2`_dl_open + 186<br class=""> frame #5: 0x00002aaaac39df66 libdl.so.2`dlopen_doit + 102<br class=""> frame #6: 0x00002aaaaaab9286 ld-linux-x86-64.so.2`_dl_catch_error +<br class=""></blockquote><br class=""><blockquote type="cite" class="">102<br class=""></blockquote><br class=""><blockquote type="cite" class=""> frame #7: 0x00002aaaac39e29c libdl.so.2`_dlerror_run + 124<br class=""> frame #8: 0x00002aaaac39dee1 libdl.so.2`__dlopen_check + 49<br class=""></blockquote><br class=""><br class=""><blockquote type="cite" class="">the dyld debug log has a lot of this:<br class="">209 intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid<br class=""></blockquote><br class=""><blockquote type="cite" class="">153501 stop_when_images_change=false<br class=""></blockquote><br class=""><blockquote type="cite" class="">210 intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit<br class=""></blockquote><br class=""><blockquote type="cite" class="">called for pid 153501<br class=""></blockquote><br class=""><blockquote type="cite" class="">211 intern-state DYLDRendezvous::Resolve address size: 8, padding 4<br class="">212 intern-state DYLDRendezvous::Resolve cursor = 0x2aaaaaccc160<br class="">213 intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid<br class=""></blockquote><br class=""><blockquote type="cite" class="">153501 stop_when_images_change=false<br class=""></blockquote><br class=""><blockquote type="cite" class="">214 intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit<br class=""></blockquote><br class=""><blockquote type="cite" class="">called for pid 153501<br class=""></blockquote><br class=""><blockquote type="cite" class="">215 intern-state DYLDRendezvous::Resolve address size: 8, padding 4<br class="">216 intern-state DYLDRendezvous::Resolve cursor = 0x2aaaaaccc160<br class=""></blockquote><br class=""><br class=""><br class=""><br class=""><blockquote type="cite" class="">thanks,<br class="">--steve<br class=""></blockquote><br class=""><br class=""><br class=""><blockquote type="cite" class=""> In the woods too, a man casts off his years, as the snake his slough,<br class=""></blockquote><br class=""><blockquote type="cite" class="">and at what period soever of life, is always a child.<br class=""></blockquote><br class=""><br class=""><br class=""><blockquote type="cite" class="">_______________________________________________<br class="">LLVM Developers mailing list<br class=""><a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a><br class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev<br class=""></blockquote><br class=""><blockquote type="cite" class="">_______________________________________________<br class="">lldb-dev mailing list<br class=""><a href="mailto:lldb-dev@lists.llvm.org" class="">lldb-dev@lists.llvm.org</a><br class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev<br class=""></blockquote></div></div></blockquote></div><br class=""></body></html>