[Lldb-commits] [PATCH] D32930: New framework for lldb client-server communication tests.

[Zachary Turner via lldb-commits]

Writing tests that are

A) as easy as possible to write and understand
B) built on a multiprocessing infrastructure that is battle tested and
known to not be flaky
C) Familiar to other llvm developers, so as to not discourage subject
matter experts from other areas to make relevant improvements to LLDB


For example, I assume you are on board at least to some extent with
lldbinline tests. After all, they're simpler than the other style of test.
Now, suppose there were some hypothetical DSL that allowed every test to be
an inline test but still test equally complicated scenarios in half the
code. Then suppose it also ran on a more robust multiprocessing
infrastructure than dotest.py. That's all we're really talking about
On Wed, May 31, 2017 at 12:06 PM Jim Ingham via lldb-commits <
lldb-commits at lists.llvm.org> wrote:

> Before we get past "it's hard" to "just to do it", it would help me to be
> clear first on what you are actually trying to achieve with this proposal.
> It's not clear to me what problem are people trying to solve here?  If it
> is writing tests for the decomposable parts of lldb - like the tests Jason
> wrote for the unwinder recently - why was the gtest path not a good way to
> do this?  If it is rewriting the parts of the testsuite that exercise the
> debugger on live targets what would a lit-based suite do that we can't do
> with the current testsuite?  Or maybe you are thinking of some other good
> I'm missing?
>
> Jim
>
>
> > On May 31, 2017, at 10:37 AM, Zachary Turner via Phabricator via
> lldb-commits <lldb-commits at lists.llvm.org> wrote:
> >
> > zturner added a comment.
> >
> > In https://reviews.llvm.org/D32930#767820, @beanz wrote:
> >
> >> One small comment below. In general I agree with the thoughts here, and
> I think that this is a huge step forward for testing the debug server
> components.
> >>
> >> I also agree with Zachary in principal that it would be nice to come up
> with lit-based test harnesses for more parts of LLDB, although I'm
> skeptical about whether or not that is actually the best way to test the
> debug server pieces. Either way, this is a huge step forward from what we
> have today, so we should go with it.
> >
> >
> > It would be nice if, at some point, we could move past "It's hard" and
> start getting into the details of what's hard about it.  (Note this goes
> for LLDB client as well as lldb server).  I see a lot of general hand-wavy
> comments about how conditionals are needed, or variables, etc, but that
> doesn't really do anything to convince me that it's hard.  After all, we
> wrote a C++ compiler!  And I'm pretty sure that the compiler-rt and
> sanitizer test suite is just as complicated as, if not more complicated
> than any hypothetical lldb test suite.  And those have been solved.
> >
> > What *would* help would be to ignore how difficult it may or may not be,
> and just take a couple of tests and re-write them in some DSL that you
> invent specifically for this purpose that is as concise as possible yet as
> expressive as you need, and we go from there.  I did this with a couple of
> fairly hairy tests a few months ago and it didn't seem that bad to me.
> >
> > The thing is, the set of people who are experts on the client side of
> LLDB and the set of people who are experts on the client side of
> LLVM/lit/etc are mostly disjoint, so nothing is ever going to happen
> without some sort of collaboration.  For example, I'm more than willing to
> help out writing the lit bits of it, but I would need a specification of
> what the test language needs to look like to support all of the use cases.
> And someone else has to provide that since we want to get the design
> right.  Even if implementing the language is hard, deciding what it needs
> to look like is supposed to be the easy part!
> >
> >
> > https://reviews.llvm.org/D32930
> >
> >
> >
> > _______________________________________________
> > lldb-commits mailing list
> > lldb-commits at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
>
> _______________________________________________
> lldb-commits mailing list
> lldb-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-commits/attachments/20170531/6d6aa933/attachment.html>