[Lldb-commits] [PATCH] D42656: [testsuite] Remove flakey test relying on `pexpect`

[Zachary Turner via lldb-commits]

Also, I can think of at least 3 different companies/people who are
investing in LLDB for their downstream needs (who haven't publicly
announced this, so this isn't widely known), which involves bringing LLDB
up on currently unsupported platforms.  It's easy to lose sight of what
that entails when you've had a supported platform for 10+ years, but
suffice it to say that the less a test does, the better.  For these people,
when a test fails, you want as close to an absolute guarantee as possible
that the failure points immediately to the underlying cause as possible.
This drastically reduces the amount of work people have to do.

We can have another bi-monthly centithread about this if you want, but at
the end of the day if we want the test situation to improve, this is the
way to go and I believe there's pretty wide concensus in the larger LLVM
community about this.

On Mon, Jan 29, 2018 at 5:51 PM Zachary Turner <zturner at google.com> wrote:

> We’ve had many instances of flakiness in non pexpect tests (on all
> platforms). There’s no obvious pattern to when a test will be flaky.
> Whether those are due to dotest or liblldb is an open question, but one
> good way of answering those types of questions is to replace one source of
> unknown-flakiness with a source of known-not-flakiness and seeing if the
> flakiness goes away.
>
> The new-and-not-tested code you’re referring to would be about 5 lines of
> c++ that also directly calls the api, just like your dotest example. So
> that aspect doesn’t feel like a convincing argument
> On Mon, Jan 29, 2018 at 5:28 PM Jim Ingham via Phabricator <
> reviews at reviews.llvm.org> wrote:
>
>> jingham added a comment.
>>
>> lldb testcases are know not to be flakey if they don't use pexpect, which
>> these wouldn't.  The setup machinery for running a dotest based test is
>> pretty well tested at this point.
>>
>> And the lldb-test test would not just magically come into being by
>> writing the lit-form text you suggested.  You will have to write a
>> lldb-test func that marshals the input (both --complete-string and you'll
>> also need a cursor position to test completion inside a line).  That's new
>> and not tested code.  Whereas the dotest test relies on the API it is
>> directly testing, and trust that the basic machinery of dotest is going to
>> continue to function.
>>
>>
>> https://reviews.llvm.org/D42656
>>
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-commits/attachments/20180130/dc409c1b/attachment-0001.html>