[lldb-dev] Race condition crashes during launching LLDB
Jeffrey Tan via lldb-dev
lldb-dev at lists.llvm.org
Thu Feb 4 21:09:21 PST 2016
After adding some logging I figured out that the race condition is caused
by process.Continue() did not guarantee process has been really resumed yet
in async mode, so the second wait_for_process_stop() is skipped immediately
to kill listener thread and destroying debugger. I know I have a race
condition bug here because of polling for process state, but why is lldb
crashing when listener thread has exited and SBDebugger.Destroy() is
called? What is the situation that SBDebugger.Destroy() can be called
safely?
==================do_test==================
Launch result: success
<Listener> Listening Thread ID: 4660334592
WaitForEvent...
Target event: ModulesLoaded
WaitForEvent...
Process event: StateChanged, Stopped
Stop reason: 5
WaitForEvent...
Breakpoint event: [Added] SBBreakpoint: id = 1, name = 'main', locations = 1
WaitForEvent...
[main] killing listener thread
Process event: StateChanged, Running
Stop reason: 0
<Listener> Exiting listener thread
[main] destroy debugger
Segmentation fault: 11
On Thu, Feb 4, 2016 at 8:22 PM, Jeffrey Tan <jeffrey.fudan at gmail.com> wrote:
> Sorry, I have actually tried to exit the event listening thread before
> debugger destroy, but still go the crash randomly(1 out of 5 runs). Here is
> the code:
>
>
> ==========Main Thread==========
> def wait_for_process_stop(process):
> while not process.is_stopped:
> time.sleep(0.1)
>
> def launch_debugging(debugger, stop_at_entry):
> error = lldb.SBError()
> listener = lldb.SBListener('Chrome Dev Tools Listener')
> target = debugger.GetSelectedTarget()
> process = target.Launch (listener,
> None, # argv
> None, # envp
> None, # stdin_path
> None, # stdout_path
> None, # stderr_path
> None, # working directory
> 0, # launch flags
> stop_at_entry, # Stop at entry
> error) # error
> print 'Launch result: %s' % str(error)
> listener_thread = LLDBListenerThread(debugger)
> listener_thread.start()
> return listener_thread
>
> def do_test():
> debugger = lldb.SBDebugger.Create()
> debugger.SetAsync(True)
> executable_path =
> '~/Personal/compiler/CompilerConstruction/code/compiler'
> target = debugger.CreateTargetWithFileAndArch(executable_path,
> lldb.LLDB_ARCH_DEFAULT)
>
> listener_thread = launch_debugging(debugger, stop_at_entry=True)
> process = debugger.GetSelectedTarget().process
>
> wait_for_process_stop(process) # wait for entry breakpoint.
> target.BreakpointCreateByName('main')
> process.Continue()
> wait_for_process_stop(process) # wait for main breakpoint.
>
> listener_thread.should_quit = True
> listener_thread.join()
>
> lldb.SBDebugger.Destroy(debugger)
>
> def main():
> do_test()
> do_test()
>
> ==========Listening Thread==========
> class LLDBListenerThread(Thread):
> should_quit = False
>
> def __init__(self, debugger):
> Thread.__init__(self)
> process = debugger.GetSelectedTarget().process
> self.listener = debugger.GetListener()
> self._add_listener_to_process(process)
> self._add_listener_to_target(process.target)
>
> def _add_listener_to_target(self, target):
> # Listen for breakpoint/watchpoint events
> (Added/Removed/Disabled/etc).
> broadcaster = target.GetBroadcaster()
> mask = lldb.SBTarget.eBroadcastBitBreakpointChanged |
> lldb.SBTarget.eBroadcastBitWatchpointChanged |
> lldb.SBTarget.eBroadcastBitModulesLoaded
> broadcaster.AddListener(self.listener, mask)
>
> def _add_listener_to_process(self, process):
> # Listen for process events (Start/Stop/Interrupt/etc).
> broadcaster = process.GetBroadcaster()
> mask = lldb.SBProcess.eBroadcastBitStateChanged |
> lldb.SBProcess.eBroadcastBitSTDOUT | lldb.SBProcess.eBroadcastBitSTDERR |
> lldb.SBProcess.eBroadcastBitInterrupt
> broadcaster.AddListener(self.listener, mask)
>
> def run(self):
> while not self.should_quit:
> event = lldb.SBEvent()
> print 'WaitForEvent...'
> if self.listener.WaitForEvent(1, event):
> if lldb.SBTarget.EventIsTargetEvent(event):
> self._handle_target_event(event)
> elif lldb.SBProcess.EventIsProcessEvent(event):
> self._handle_process_event(event)
> elif lldb.SBBreakpoint.EventIsBreakpointEvent(event):
> self._handle_breakpoint_event(event)
> elif lldb.SBThread.EventIsThreadEvent(event):
> self._handle_thread_event(event)
> else:
> self._handle_unknown_event(event)
>
> _hand_XXX_event() methods just dump some logging information of the debug
> events.
>
>
> Crashed Thread: 0 Dispatch queue: com.apple.main-thread
>
> Exception Type: EXC_BAD_ACCESS (SIGSEGV)
> Exception Codes: EXC_I386_GPFLT
>
> Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
> 0 _lldb.so 0x0000000111528179
> EventMatcher::operator()(std::__1::shared_ptr<lldb_private::Event> const&)
> const + 21
> 1 _lldb.so 0x00000001115275d2
> lldb_private::Listener::FindNextEventInternal(lldb_private::Broadcaster*,
> lldb_private::ConstString const*, unsigned int, unsigned int,
> std::__1::shared_ptr<lldb_private::Event>&, bool) + 176
> 2 _lldb.so 0x0000000111527952
> lldb_private::Listener::WaitForEventsInternal(lldb_private::TimeValue
> const*, lldb_private::Broadcaster*, lldb_private::ConstString const*,
> unsigned int, unsigned int, std::__1::shared_ptr<lldb_private::Event>&) +
> 134
> 3 _lldb.so 0x0000000111527ae9
> lldb_private::Listener::WaitForEventForBroadcasterWithType(lldb_private::TimeValue
> const*, lldb_private::Broadcaster*, unsigned int,
> std::__1::shared_ptr<lldb_private::Event>&) + 27
> 4 _lldb.so 0x000000011171de6c
> lldb_private::Process::WaitForStateChangedEvents(lldb_private::TimeValue
> const*, std::__1::shared_ptr<lldb_private::Event>&,
> lldb_private::Listener*) + 112
> 5 _lldb.so 0x000000011171dc95
> lldb_private::Process::WaitForProcessToStop(lldb_private::TimeValue const*,
> std::__1::shared_ptr<lldb_private::Event>*, bool, lldb_private::Listener*,
> lldb_private::Stream*) + 377
> 6 _lldb.so 0x000000011172616a
> lldb_private::Process::HaltForDestroyOrDetach(std::__1::shared_ptr<lldb_private::Event>&)
> + 216
> 7 _lldb.so 0x000000011171d8b0
> lldb_private::Process::Destroy(bool) + 146
> 8 _lldb.so 0x000000011171d56d
> lldb_private::Process::Finalize() + 91
> 9 _lldb.so 0x00000001115173c4
> lldb_private::Debugger::Clear() + 148
> 10 _lldb.so 0x00000001115171fd
> lldb_private::Debugger::Destroy(std::__1::shared_ptr<lldb_private::Debugger>&)
> + 37
> 11 _lldb.so 0x000000010f83c144
> lldb::SBDebugger::Destroy(lldb::SBDebugger&) + 116
> 12 _lldb.so 0x000000010f884daf
> _wrap_SBDebugger_Destroy(_object*, _object*) + 120
> 13 org.python.python 0x000000010e53e75f PyEval_EvalFrameEx +
> 12761
>
>
> On Thu, Feb 4, 2016 at 5:37 PM, Jim Ingham <jingham at apple.com> wrote:
>
>> I don't know what:
>>
>> event_thread = LLDBListenerThread(debugger)
>>
>> does, but from your little sketch it looks like you are starting up a
>> thread listening on this debugger, and so far as I can see you destroy the
>> debugger out from under it without ever closing down that thread. That
>> doesn't seem like a good idea.
>>
>> Jim
>>
>>
>>
>>
>> > On Feb 4, 2016, at 5:27 PM, Jeffrey Tan via lldb-dev <
>> lldb-dev at lists.llvm.org> wrote:
>> >
>> > Hi,
>> >
>> > I am revising our lldb automation tests into async mode. However, I
>> found it randomly crashes depends on timing. And the crash happens mostly
>> while launching lldb twice in a row. I have narrowed down the code into a
>> simple repro below. Any assumption I made wrong with the LLDB API here?
>> >
>> > The crash stack seems to be not consistently. In the small repro below,
>> the crash stack is:
>> > Crashed Thread: 0 Dispatch queue: com.apple.main-thread
>> >
>> > Exception Type: EXC_BAD_ACCESS (SIGSEGV)
>> > Exception Codes: EXC_I386_GPFLT
>> >
>> > Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
>> > 0 _lldb.so 0x00000001088c7179
>> EventMatcher::operator()(std::__1::shared_ptr<lldb_private::Event> const&)
>> const + 21
>> > 1 _lldb.so 0x00000001088c65d2
>> lldb_private::Listener::FindNextEventInternal(lldb_private::Broadcaster*,
>> lldb_private::ConstString const*, unsigned int, unsigned int,
>> std::__1::shared_ptr<lldb_private::Event>&, bool) + 176
>> > 2 _lldb.so 0x00000001088c6952
>> lldb_private::Listener::WaitForEventsInternal(lldb_private::TimeValue
>> const*, lldb_private::Broadcaster*, lldb_private::ConstString const*,
>> unsigned int, unsigned int, std::__1::shared_ptr<lldb_private::Event>&) +
>> 134
>> > 3 _lldb.so 0x00000001088c6ae9
>> lldb_private::Listener::WaitForEventForBroadcasterWithType(lldb_private::TimeValue
>> const*, lldb_private::Broadcaster*, unsigned int,
>> std::__1::shared_ptr<lldb_private::Event>&) + 27
>> > 4 _lldb.so 0x0000000108abce6c
>> lldb_private::Process::WaitForStateChangedEvents(lldb_private::TimeValue
>> const*, std::__1::shared_ptr<lldb_private::Event>&,
>> lldb_private::Listener*) + 112
>> > 5 _lldb.so 0x0000000108abcc95
>> lldb_private::Process::WaitForProcessToStop(lldb_private::TimeValue const*,
>> std::__1::shared_ptr<lldb_private::Event>*, bool, lldb_private::Listener*,
>> lldb_private::Stream*) + 377
>> > 6 _lldb.so 0x0000000108ac516a
>> lldb_private::Process::HaltForDestroyOrDetach(std::__1::shared_ptr<lldb_private::Event>&)
>> + 216
>> > 7 _lldb.so 0x0000000108abc8b0
>> lldb_private::Process::Destroy(bool) + 146
>> > 8 _lldb.so 0x0000000108abc56d
>> lldb_private::Process::Finalize() + 91
>> > 9 _lldb.so 0x00000001088b63c4
>> lldb_private::Debugger::Clear() + 148
>> > 10 _lldb.so 0x00000001088b61fd
>> lldb_private::Debugger::Destroy(std::__1::shared_ptr<lldb_private::Debugger>&)
>> + 37
>> > 11 _lldb.so 0x0000000106bdb144
>> lldb::SBDebugger::Destroy(lldb::SBDebugger&) + 116
>> > 12 _lldb.so 0x0000000106c23daf
>> _wrap_SBDebugger_Destroy(_object*, _object*) + 120
>> > 13 org.python.python 0x00000001058dd75f
>> PyEval_EvalFrameEx + 12761
>> >
>> > while in the real unit test it is crashing at:
>> > Thread 12 Crashed:
>> > 0 libsystem_kernel.dylib 0x00007fff8635a286 __pthread_kill
>> + 10
>> > 1 libsystem_c.dylib 0x00007fff919409b3 abort + 129
>> > 2 libc++abi.dylib 0x00007fff8a94ea21 abort_message
>> + 257
>> > 3 libc++abi.dylib 0x00007fff8a9769d1
>> default_terminate_handler() + 267
>> > 4 libobjc.A.dylib 0x00007fff935e77eb
>> _objc_terminate() + 124
>> > 5 libc++abi.dylib 0x00007fff8a9740a1
>> std::__terminate(void (*)()) + 8
>> > 6 libc++abi.dylib 0x00007fff8a973b30 __cxa_throw +
>> 121
>> > 7 com.apple.LLDB.framework 0x000000010b994c6b
>> std::__1::shared_ptr<lldb_private::Process>::shared_ptr<lldb_private::Process>(std::__1::weak_ptr<lldb_private::Process>
>> const&, std::__1::enable_if<is_convertible<lldb_private::Process*,
>> lldb_private::Process*>::value,
>> std::__1::shared_ptr<lldb_private::Process>::__nat>::type) + 99
>> > 8 com.apple.LLDB.framework 0x000000010b8ac762
>> lldb_private::Process::AppendSTDOUT(char const*, unsigned long) + 86
>> > 9 com.apple.LLDB.framework 0x000000010b6951d7
>> lldb_private::Communication::ReadThread(void*) + 287
>> > 10 libsystem_pthread.dylib 0x00007fff8d92c05a _pthread_body
>> + 131
>> > 11 libsystem_pthread.dylib 0x00007fff8d92bfd7 _pthread_start
>> + 176
>> >
>> >
>> > ================Repro Code====================
>> >
>> > def wait_for_process_stop(process):
>> > while not process.is_stopped:
>> > time.sleep(0.1)
>> >
>> > def launch_debugging(debugger, stop_at_entry):
>> > error = lldb.SBError()
>> > listener = lldb.SBListener('Chrome Dev Tools Listener')
>> > target = debugger.GetSelectedTarget()
>> > process = target.Launch (listener,
>> > None, # argv
>> > None, # envp
>> > None, # stdin_path
>> > None, # stdout_path
>> > None, # stderr_path
>> > None, # working directory
>> > 0, # launch flags
>> > stop_at_entry, # Stop at entry
>> > error) # error
>> > print 'Launch result: %s' % str(error)
>> > event_thread = LLDBListenerThread(debugger)
>> > event_thread.start()
>> > return process
>> >
>> > def do_test():
>> > debugger = lldb.SBDebugger.Create()
>> > debugger.SetAsync(True)
>> > target = debugger.CreateTargetWithFileAndArch(executable_path,
>> lldb.LLDB_ARCH_DEFAULT)
>> >
>> > process = launch_debugging(debugger, stop_at_entry=True)
>> >
>> > wait_for_process_stop(process) # wait for entry breakpoint.
>> > target.BreakpointCreateByName('main')
>> > process.Continue()
>> > wait_for_process_stop(process) # wait for main breakpoint.
>> > lldb.SBDebugger.Destroy(debugger)
>> >
>> > def main():
>> > do_test()
>> > do_test()
>> > _______________________________________________
>> > lldb-dev mailing list
>> > lldb-dev at lists.llvm.org
>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20160204/009fe378/attachment-0001.html>
More information about the lldb-dev
mailing list