[lldb-dev] Synchronous execution with process plugin
Ilia K
ki.stfu at gmail.com
Mon Mar 23 08:16:35 PDT 2015
Hello,
Look at it: http://reviews.llvm.org/D8541
Perhaps it will fix your test cases too.
Thanks,
Ilia
On Mon, Mar 23, 2015 at 5:58 PM, Ted Woodward <ted.woodward at codeaurora.org>
wrote:
> We've seen the same behavior when running an lldb command file at startup.
>
> Foo contains:
> b main
> run
>
> lldb -s foo test.exe
> (this hangs)
>
> Take out the run from foo, and
> lldb -s foo -o run test.exe
> (this works)
>
> --
> Qualcomm Innovation Center, Inc.
> The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a
> Linux Foundation Collaborative Project
>
> > -----Original Message-----
> > From: lldb-dev-bounces at cs.uiuc.edu [mailto:lldb-dev-bounces at cs.uiuc.edu]
> On
> > Behalf Of Pavel Labath
> > Sent: Monday, March 23, 2015 4:22 AM
> > To: Zachary Turner
> > Cc: lldb-dev at cs.uiuc.edu
> > Subject: Re: [lldb-dev] Synchronous execution with process plugin
> >
> > This looks very much like the issue I was experiencing on linux, where
> lldb
> > would just lock up if I executed the process in synchronous mode.
> > This was fixed with <http://reviews.llvm.org/D8079>, and I suspect you
> need to
> > do something similar. The trick here is that you need to set up process
> event
> > hijacking in your host launch code. Otherwise, Target::Launch will
> attempt
> to
> > use the global process listener to wait for the process to stop. However,
> this will
> > race with the event handler thread, since it uses the same listener to
> process
> > events. So, depending on which thread processes the event, it will either
> work
> > fine, or you will end up locked in WaitForProcessToStop forever, because
> the
> > event will never come.
> >
> > Also, note that there is another issue with launching in synchronous
> mode,
> > where stdio is not forwarded to the target process
> > <http://lists.cs.uiuc.edu/pipermail/lldb-dev/2015-March/006853.html>.
> > I was planning on dealing with this, but I don't think I will get around
> to it for a
> > while yet...
> >
> > pl
> >
> > On 21 March 2015 at 01:35, Zachary Turner <zturner at google.com> wrote:
> > > Hmm, on Monday I'll try to see if i can figure out where to put a
> > > breakpoint to get hit when someone removes something from the queue.
> > > Thanks On Fri, Mar 20, 2015 at 5:52 PM <jingham at apple.com> wrote:
> > >>
> > >>
> > >> > On Mar 20, 2015, at 5:21 PM, Zachary Turner <zturner at google.com>
> > wrote:
> > >> >
> > >> > If I do that my process does stop at main, but I think I'm hit with
> > >> > a race condition. 3 times out of 5 I saw the same results as you,
> > >> > where it doesn't print the backtrace but if I do bt it works fine.
> > >> > The other 2 times out of 5 It does print the backtrace, but then
> > >> > I'm deadlocked. I added a bunch of tracepoints so we can see the
> > >> > order of events. here's what it looks like when it deadlocks.
> > >> >
> > >> > Target::Launch about to call ProcessWindows::Launch
> > >> > ProcessWindows::DoLaunch entering ProcessWindows::DoLaunch about to
> > >> > call DebugLaunch ProcessWindows::DoLaunch succeeded, waiting for
> > >> > initial stop Application
> > >> > "\??\D:\src\llvm\tools\lldb\test\expression_command\formatters\a.ou
> > >> > t" found in cache ProcessWindows::OnDebuggerConnected
> > >> > ProcessWindows::OnDebugException
> > >> > ProcessWindows got EXCEPTION_BREAKPOINT ProcessWindows got initial
> > >> > stop, setting initial stop event ProcessWindows::DoLaunch received
> > >> > initial stop, returning ProcessWindows::DoLaunch finished launching
> > >> > process, exiting Target::Launch returned from
> > >> > ProcessWindows::DoLaunch, eLaunchFlagStopAtEntry == false, about to
> > >> > call WaitForProcessToStop Target::Launch, WaitForProcessToStop
> > >> > returned, new state = eStateStopped Target::Launch about to call
> > >> > ProcessWindows::PrivateResume ProcessWindows::DoResume resuming
> > >> > from active exception ProcessWindows::DoResume couldn't find an
> > >> > active exception, setting state to eStateRunning
> > >> > ProcessWindows::OnDebugException ProcessWindows got
> > >> > EXCEPTION_BREAKPOINT Target::Launch synchronous_execution is true,
> > >> > calling WaitForProcessToStop again
> > >> >
> > >> > The interesting stuff is the last 2 lines. It hits the breakpoint
> > >> > in main *before* calling WaitForProcessToStop, so
> > >> > WaitForProcessToStop never sees a "change".
> > >>
> > >> > Here's what it looks like when it works.
> > >> >
> > >> > Target::Launch about to call ProcessWindows::Launch
> > >> > ProcessWindows::DoLaunch entering ProcessWindows::DoLaunch about to
> > >> > call DebugLaunch ProcessWindows::DoLaunch succeeded, waiting for
> > >> > initial stop Application
> > >> > "\??\D:\src\llvm\tools\lldb\test\expression_command\formatters\a.ou
> > >> > t" found in cache ProcessWindows::OnDebuggerConnected
> > >> > ProcessWindows::OnDebugException
> > >> > ProcessWindows got EXCEPTION_BREAKPOINT ProcessWindows got initial
> > >> > stop, setting initial stop event ProcessWindows::DoLaunch received
> > >> > initial stop, returning ProcessWindows::DoLaunch finished launching
> > >> > process, exiting ProcessWindows breakpoint handler set private
> > >> > state to eStateStopped Target::Launch returned from
> > >> > ProcessWindows::DoLaunch, eLaunchFlagStopAtEntry == false, about to
> > >> > call WaitForProcessToStop Target::Launch, WaitForProcessToStop
> > >> > returned, new state = eStateStopped Target::Launch about to call
> > >> > ProcessWindows::PrivateResume ProcessWindows::DoResume resuming
> > >> > from active exception ProcessWindows::DoResume couldn't find an
> > >> > active exception, nothing to resume
> > >> > ProcessWindows::OnDebugException ProcessWindows got
> > >> > EXCEPTION_BREAKPOINT Target::Launch synchronous_execution is true,
> > >> > calling WaitForProcessToStop again ProcessWindows breakpoint
> > >> > handler set private state to eStateStopped Target::Launch
> > >> > WaitForProcessToStop returned, yay!
> > >> >
> > >> >
> > >> > The difference here is that "calling WaitForProcessToStop again"
> happens
> > >> > before the process plugin sets the private state. Can you try
> repeating
> > >> > your test multiple times and seeing if you can get the deadlock to
> occur?
> > >>
> > >> I can't get this to fail on OS X but with timing related things I'm
> > >> not sure that's all that significant...
> > >>
> > >> Anyway, what you are describing doesn't make sense to me yet. After
> > >> calling PrivateResume, we call WaitForProcessToStop, passing
> > "wait_always"
> > >> set to true. When wait_always is true, WaitForProcessToStop won't
> > >> return till it receives a stop event. It isn't comparing current
> > >> state against desired state, it is actually waiting for an event. So
> > >> it should not matter when the breakpoint hit arrives relative to
> calling
> > "WaitForProcessToStop"
> > >> since the breakpoint hit is just going to generate a stop event,
> > >> which will sit in the event queue till somebody fetches it. It's
> > >> fine for that to happen before you call WaitForProcessToStop, because
> > >> we aren't comparing current to desired state, we are waiting for that
> event.
> > >>
> > >> Maybe somebody else is grabbing the breakpoint stop event and taking
> > >> it off of the queue? That would explain why the second
> > >> WaitForProcessToStop - the one with wait_always set to true - is
> missing the
> > event?
> > >>
> > >> Jim
> > >>
> > >>
> > >> >
> > >> >
> > >> > On Fri, Mar 20, 2015 at 4:49 PM <jingham at apple.com> wrote:
> > >> > I'm not sure it is deadlocking the debugger. lldb is just waiting
> > >> > for a stop. For instance ^C should interrupt it, or sending a
> > >> > signal externally to the process, or triggering a breakpoint or
> crash, etc.
> > >> >
> > >> > Actually, Greg must have fixed the bug I was remembering, because
> > >> > this works correctly for me with TOT lldb.
> > >> >
> > >> > What happens for you if your .lldbinit has:
> > >> >
> > >> > file a.out
> > >> > break set -n main
> > >> > run
> > >> >
> > >> > For me this stops at the breakpoint at main. We still have a
> > >> > little clean up to do here, because I don't see the stop
> notification
> in this
> > case.
> > >> > I see:
> > >> >
> > >> > > lldb -S cmds.lldb
> > >> > (lldb) command source -s 1 'cmds.lldb'
> > >> > 4 locations added to breakpoint 1
> > >> > (lldb)
> > >> >
> > >> > but if I then do "bt" I'm sitting at main:
> > >> >
> > >> > (lldb) bt
> > >> > * thread #1: tid = 0x1bf8a4, function: main , stop reason =
> > >> > breakpoint
> > >> > 1.1
> > >> > * frame #0: 0x0000000100018e87 Sketch`main at SKTMain.m:17
> > >> > frame #1: 0x00007fff94f445ad libdyld.dylib`start
> > >> > frame #2: 0x00007fff94f445ad libdyld.dylib`start
> > >> >
> > >> > Not sure what's up with the stutter at start either. But that's a
> > >> > different rabbit to chase...
> > >> >
> > >> > Jim
> > >> >
> > >> >
> > >> > > On Mar 20, 2015, at 4:40 PM, Zachary Turner <zturner at google.com>
> > >> > > wrote:
> > >> > >
> > >> > > If that's the case, then a .lldbinit file like this:
> > >> > >
> > >> > > file a.out
> > >> > > run
> > >> > >
> > >> > > Will deadlock the debugger, because the real stop never comes?
> > >> > >
> > >> > > On Fri, Mar 20, 2015 at 4:35 PM <jingham at apple.com> wrote:
> > >> > > That's the stop at entry stop. The code you quoted is in a block
> > >> > > that starts with:
> > >> > >
> > >> > > if (launch_info.GetFlags().Test(eLaunchFlagStopAtEntry)
> > >> > > ==
> > >> > > false)
> > >> > > {
> > >> > >
> > >> > > So we've stopped at the entry point, but the user didn't want to
> > >> > > know about that, so we resume and wait for a "real" stop.
> > >> > >
> > >> > > Jim
> > >> > >
> > >> > >
> > >> > > > On Mar 20, 2015, at 4:30 PM, Zachary Turner
> > >> > > > <zturner at google.com>
> > >> > > > wrote:
> > >> > > >
> > >> > > > I'm a little confused. You said that in synchronous execution,
> > >> > > > Launch won't return until the process has stopped. That makes
> > >> > > > sense, but it already checks that the process has stopped once
> > >> > > > regardless of whether synchronous execution is set. Then, it
> > >> > > > calls PrivateResume() (even if synchronous_execution is set),
> and
> then
> > waits for the process to stop again?
> > >> > > > What would trigger this second stop? Target::Launch already
> > >> > > > asked it to resume, so now it's happily running while
> > >> > > > Target::Launch is waiting for it to stop a second time.
> > >> > > >
> > >> > > > On Fri, Mar 20, 2015 at 4:23 PM <jingham at apple.com> wrote:
> > >> > > > In synchronous execution, the "Launch" command won't return
> > >> > > > till the process has stopped. The point of synchronous
> execution
> is that
> > you can do:
> > >> > > >
> > >> > > > break set -n foo
> > >> > > > run
> > >> > > > bt
> > >> > > >
> > >> > > > So "run" can't return till the breakpoint has been hit. That
> > >> > > > is why it waits for the process to stop. I'm not quite sure
> > >> > > > why this is done in Target::Launch, in other cases (e.g. in for
> > >> > > > "step" and "continue" the command object is the one that takes
> > >> > > > care of waiting for the stop. Launch is a little funny
> > >> > > > however, because it can't use the normal process wait mechanism
> > >> > > > to do its job since the real process isn't alive when it has to
> start
> > waiting...
> > >> > > >
> > >> > > > I think the reason you are hanging here is that the code that
> > >> > > > reads in all the init statements runs an event loop temporarily
> > >> > > > while it is reading them in, and the kills that and hands off
> > >> > > > the the real command execution loop, and this continuation gets
> > >> > > > lost in the handoff. I thought Greg had already fixed that, but
> maybe
> > it's still sitting in his queue.
> > >> > > >
> > >> > > > Jim
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > > On Mar 20, 2015, at 3:57 PM, Zachary Turner
> > >> > > > > <zturner at google.com>
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > I ran into an issue earlier where I tried to make a .lldbinit
> > >> > > > > file with some lines like this:
> > >> > > > >
> > >> > > > > file a.out
> > >> > > > > run
> > >> > > > >
> > >> > > > >
> > >> > > > > When this happens the process runs, the breakpoint gets hit
> > >> > > > > and I see the source listing, it returns to the lldb prompt,
> > >> > > > > but then I can't type anything. It appears LLDB is
> > >> > > > > deadlocked inside of Target::Launch() at the following
> location:
> > >> > > > >
> > >> > > > > if (!synchronous_execution)
> > >> > > > > m_process_sp->RestoreProcessEvents ();
> > >> > > > >
> > >> > > > > error = m_process_sp->PrivateResume();
> > >> > > > >
> > >> > > > > if (error.Success())
> > >> > > > > {
> > >> > > > > // there is a race condition where this
> > >> > > > > thread will return up the call stack to the main command
> > >> > > > > // handler and show an (lldb) prompt
> > >> > > > > before HandlePrivateEvent (from PrivateStateThread) has
> > >> > > > > // a chance to call PushProcessIOHandler()
> > >> > > > > m_process_sp->SyncIOHandler(2000);
> > >> > > > >
> > >> > > > > if (synchronous_execution)
> > >> > > > > {
> > >> > > > >
> > >> > > > > state = m_process_sp->WaitForProcessToStop (NULL, NULL, true,
> > >> > > > > hijack_listener_sp.get(), stream);
> > >> > > > > const bool must_be_alive = false; //
> > >> > > > > eStateExited is ok, so this must be false
> > >> > > > > if (!StateIsStoppedState(state,
> > >> > > > > must_be_alive))
> > >> > > > > {
> > >> > > > >
> > >> > > > > error.SetErrorStringWithFormat("process isn't stopped: %s",
> > >> > > > > StateAsCString(state));
> > >> > > > > }
> > >> > > > > }
> > >> > > > > }
> > >> > > > >
> > >> > > > > Normally when I'm using LLDB and entering the commands
> > >> > > > > myself, this synchronous_execution value is not set, and
> > >> > > > > everything works as expected. How is this supposed to work?
> > >> > > > > What does my plugin need to do differently in order to handle
> > >> > > > > this case? The process has already stopped once and resumed,
> > >> > > > > so I'm not sure why it would need to stop again? I see that
> > >> > > > > it's not restoring process events in the case of synchronous
> > execution, so maybe it should have never resumed in the first place?
> > >> > > > > _______________________________________________
> > >> > > > > lldb-dev mailing list
> > >> > > > > lldb-dev at cs.uiuc.edu
> > >> > > > > http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > > _______________________________________________
> > > lldb-dev mailing list
> > > lldb-dev at cs.uiuc.edu
> > > http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
> > >
> > _______________________________________________
> > lldb-dev mailing list
> > lldb-dev at cs.uiuc.edu
> > http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
>
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20150323/91b7b984/attachment.html>
More information about the lldb-dev
mailing list