[lldb-dev] [RFC]The future of pexpect

[Davide Italiano via lldb-dev]

On Mon, Feb 25, 2019 at 1:57 PM Adrian Prantl <aprantl at apple.com> wrote:
>
>
>
> > On Feb 25, 2019, at 1:40 PM, Davide Italiano <dccitaliano at gmail.com> wrote:
> >
> > On Mon, Feb 25, 2019 at 1:35 PM Adrian Prantl <aprantl at apple.com> wrote:
> >>
> >>
> >>
> >>> On Feb 25, 2019, at 1:15 PM, Davide Italiano <dccitaliano at gmail.com> wrote:
> >>>
> >>> On Fri, Feb 22, 2019 at 6:32 AM Pavel Labath <pavel at labath.sk> wrote:
> >>>>
> >>>> On 21/02/2019 19:48, Ted Woodward wrote:
> >>>>>
> >>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: lldb-dev <lldb-dev-bounces at lists.llvm.org> On Behalf Of Pavel Labath
> >>>>>> via lldb-dev
> >>>>>> Sent: Thursday, February 21, 2019 8:35 AM
> >>>>>> To: Davide Italiano <dccitaliano at gmail.com>
> >>>>>> Cc: LLDB <lldb-dev at lists.llvm.org>
> >>>>>> Subject: [EXT] Re: [lldb-dev] [RFC]The future of pexpect
> >>>>>>
> >>>>>> On 21/02/2019 00:03, Davide Italiano wrote:
> >>>>>>> I found out that there are tests that effectively require
> >>>>>>> interactivity. Some of the lldb-mi ones are an example.
> >>>>>>> A common use-case is that of sending SIGTERM in a loop to make sure
> >>>>>>> `lldb-mi` doesn't crash and handle the signal correctly.
> >>>>>>>
> >>>>>>> This functionality is really hard to replicate in lit_as is_.
> >>>>>>> Any ideas on how we could handle this case?
> >>>>>>
> >>>>>> How hard is it to import a new version of pexpect which supports python3 and
> >>>>>> stuff?
> >>>>>>
> >>>>>> I'm not sure how the situation is on darwin, but I'd expect (:P) that most linux
> >>>>>> systems either already have it installed, or have an easy way to do so. So we
> >>>>>> may not even be able to get away with just using the system one and skipping
> >>>>>> tests when it's not present.
> >>>>>>
> >>>>>> BTW, for lldb-mi I would actually argue that it should *not* use pexpect :D.
> >>>>>> Interactivity is one thing, and I'm very much in favour of keeping that ability,
> >>>>>> but pexpect is not a prerequisite for that. For me, the main advantage of
> >>>>>> pexpect is that it emulates a real terminal. However, lldb-mi does not need
> >>>>>> that stuff. It doesn't have any command line editing capabilities or similar. It's
> >>>>>> expecting to communicate with an IDE over a pipe, and that's it.
> >>>>>>
> >>>>>> Given that, it should be fairly easy to rewrite the lldb-mi tests to work on top
> >>>>>> of the standard python "subprocess" library. While we're doing that, we might
> >>>>>> actually fix some of the issues that have been bugging everyone in the lldb-mi
> >>>>>> tests. At least for me, the most annoying thing was that when lldb-mi fails to
> >>>>>> produce the expected output, the test does not fail immediately, but instead
> >>>>>> the implementation of self.expect("^whatever") waits until the timeout
> >>>>>> expires, optimistically hoping that it will find some output that match the
> >>>>>> pattern.
> >>>>>>
> >>>
> >>> Pavel, I think yours is a really nice idea.
> >>> I'm no python expert, but I found out making the conversion is
> >>> relatively simple.
> >>> I propose a proof-of-concept API and implementation here:
> >>>
> >>> https://gist.github.com/dcci/94a4936a227d9c7627b91ae9575b7b68
> >>>
> >>> Comments appreciated! Once we agree on how this should look like, I do
> >>> recommend to have a new lldbMITest base class and incrementally start
> >>> moving the tests to it.
> >>> Once we're done, we can delete the old class.
> >>>
> >>> Does this sound reasonable?
> >>
> >> What you are saying is that we would write the tests as Python tests in a way similar to how lldbtest.expect() look in the dotest.py tests, banking on synchronous mode taking care of all the synchronization? Are you thinking of doing this for all the remaining tests or only the ones where a command input depends on the result of a previous command?
> >>
> >
> > I'm thinking to do this for all the remaining tests. Do you have any
> > concerns about this? (I'm aware your GSoC student introduced the `lit
> > lldb-mi` tests for a reason, I just don't know exactly what the reason
> > was).
>
> I think the reason was that for tests that don't need synchronization and have a static command input, writing a lit/FileCheck test is straightforward and easy. At the time I thought we could just rewrite *all* lldb-mi tests as FileCheck tests. If we do need a python mechanism anyway, however, I don't actually think that having two ways of writing tests is better; I'd rather have all tests in one place. Thankfully there shouldn't be that many tests, so we should be able to just convert all of them over to whatever format we eventually settle on.
> -- Adrian
>

Sounds good to me. Do you have a preference? I wanted to move away
from pexpect, so my preference was that of using `lit`.
I then realized that there are some functionalities that are not
really that easy to replicate in lit (e.g. sending signals, see my
previous mail), hence my vote for Python.subprocess (as Pavel
suggested).
What do you think?

--
Davide