[lldb-dev] pexpect and getting tests working on Windows

[jingham at apple.com]

Generally lldb commands can be synchronous or not depending on context.  For instance if I attach to a program, then "continue" will return control to the user as soon as the process continues and won't wait till it stops.  But I don't think many tests use this feature.

Anyway, that's not the real issue.  The times that the testsuite is actually using pexpect, it isn't directly "running a command" in a way that you could know when it is done.  Rather, it just funnels some text at an externally spawned lldb, and reads output till it see the "stop condition" which in our case is generally the prompt.  That tells you you're done.  Then you take the text before the prompt, and that's what you compare to.  That's a very simple use of expect, but that's pretty much all we do.  Anyway, so "wait till you see the stop condition" and all that that entails is what expect is doing for you.

OTOH, by my (admittedly quick) reading, the lldb test suite doesn't use pexpect very much at all.  For instance, lldbtest.runCmd and lldbtest.expect don't actually use pexpect, they just run a command with SBCommandInterpreter::HandleCommand and grab output from the command result object.  The benchmarks tests do use pexpect and just a few others.  You could probably switch around the "import pexpect" statements in the main lldbtest.py if you are on windows, and then skip the few tests that actually import it, and you'd be losing only 7 or 8 tests.

Jim



> On Jul 9, 2014, at 1:48 PM, Zachary Turner <zturner at google.com> wrote:
> 
> Aren't most (all?) of the commands in LLDB synchronous?  If I call a method to insert a breakpoint, the functino doesn't return until the breakpoint is inserted and the output has been printed.  So if there were a way to simply redirect the output to a buffer which could be read in python, just calling the methods you need to call should be sufficient to ensure that the operations have completed.
> 
> I might be overlooking something though.
> 
> 
> On Wed, Jul 9, 2014 at 11:31 AM, <jingham at apple.com> wrote:
> 
> > On Jul 9, 2014, at 11:14 AM, Zachary Turner <zturner at google.com> wrote:
> >
> > The current issue with regards to getting tests working on Windows is that the pexpect module doesn't exist on Windows.  I've looked over a number of the tests, and as far as I can tell, the pexpect module is used for two primary reasons.  The first is spawning lldb and waiting for it to terminate, and the second is sending text commands to lldb, and reading the textual results of these commands, comparing them against expected values.
> >
> > #1 can be solved without pexpect (e.g. by using the subprocess module), but #2 is more difficult.
> >
> > It seems to me like there are two paths forward for getting the test suite working on windows:
> >
> > 1) Port pexpect to Windows.  I expect this to be difficult for the sole reason that it hasn't been done yet, so despite the fact Windows does provide a sufficiently rich API to do everything that pexpect does, something tells me that there are some subtle issues that make this a difficult problem.
> 
> According to the pexpect sourceforge page the port depends on a working pty module.  The pexpect author doesn't comment on how hard that is, he just says he doesn't know how to do it.  OTOH the "real" expect has worked on Windows for quite some time now, so it can't be impossible...  Anyway, it sounds like porting the "pty" module and not all of pexpect itself is the blocker.  Maybe since we aren't trying to interact with real pty's for the testsuite, we could hack up a "good enough" pty module for the testsuite's purposes?
> 
> >
> > 2) Change LLDB, and the test suite, to not rely on pexpect.  The entire problem seems to boil down to the fact that LLDB writes all of its output to stdout and reads all of its input to stdin. When a command completes, In CommandInterpreter::IOHandlerInputComplete, it calls this line:
> >
> >                 io_handler.GetOutputStreamFile()->PutCString(output);
> >
> > which ultimately resolves to a FILE* referring to stdout.  What would it take to make this output stream configurable?  I'm imagining, for example, a function that's exposed through the Python API that lets me write something like
> >
> > lldb.SBDebugger.SetOutputStream(foo)
> >
> > "foo" here could be a python object that simply collects output, and then the "expect" command on the Python side could just compare the expected value against this buffer.
> 
> It's not quite as simple as this, since expect also does the job of specifying the conditions under which an operation is considered complete, and then triggering the matches at that point (and timing out appropriately if the completion conditions don't occur, etc...)  So if you want to replace pexpect you will have to duplicate that logic as well.  We don't do a lot of fancy things with expect, but it is not entirely trivial.
> 
> Jim
> 
>