[Lldb-commits] [PATCH] D24591: [LIT] First pass of LLDB LIT support

[Zachary Turner via lldb-commits]

I'm only saying that we should have an open mind.  Obviously there are
(valid!) concerns.  If we can't solve them then we can't solve them.  The
goal (my goal anyway) is always to make things better, not to use X because
it's X.  There's value in consistency, but that doesn't mean that the value
from consistency always outweighs the value you get from doing a custom
thing.

So what I'm saying is: IF we can find a way to have one test suite across
all of LLVM and it's subprojects, AND it is sufficiently powerful to test
all the things we need to test while remaining maintainable in the long
term, we should absolutely jump on the opportunity.

But this is one of those things that requires a lot of upfront time
investment before you can actually know if it can work for 100% of things.
Obviously some people don't want to invest their time that way when they're
already satisfied, and I don't blame them.

But for the people who do, and who think they can solve the problem, what's
the harm?  Obviously the burden is on those people to prove that their
vision can be realized.

But if it is successful, then there's no denying the benefits.  1) Tests
become easier to write.  2) Tests become easier to debug.  3) Consistency
encourages people who have traditionally stayed away from LLDB to
contribute.  4) All the people pouring their effort into the custom thing
can now pour it into the shared thing, so everybody benefits.

I don't blame you for being scared of command tests.  I don't support their
use in the current LLDB test suite either, for exactly the same reasons you
and Jason have expressed.  But I do think it's possible to come up with
something that a) doesn't suffer from the same problems, b) allows testing
a ton of extra functionality that is not currently testable through the
api, and c) doesn't rely on python at all.  If I'm wrong I'll eat crow :)

On Wed, Sep 14, 2016 at 5:00 PM Jim Ingham <jingham at apple.com> wrote:

> Also, w.r.t:
>
> >  Aside from write imperative control flow constructs, which I see as a
> positive rather than a negative.
>
> I wrote a bunch of tests to test that stepping behavior for swift and C
> was reasonable.  When stepping through source code, there is not one
> correct way to write the line tables, and in fact clang & swiftc change how
> they describe the source through the line tables all the time.  So you have
> to do: I stepped, and sometimes I'll get to A, sometimes to B, both are
> "right" but I have to do different things in either case.  If A, step again
> before the next test, if B go to the next test.
>
> You could "fix" that by only doing one step per test, and taking each of
> these as a success.  But then you wouldn't test that series of steps don't
> accumulate errors, you'd only test "run to a breakpoint and step once."
> That would not be good.  So your positive would be very much a negative for
> this kind of test.
>
> Traditionally the answer to this has been: we know we have to keep the
> current testsuite around but we're just adding other new different ways to
> write tests.  Now you are saying something very different.  Do you really
> mean that?
>
> Jim
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-commits/attachments/20160915/3d4d497b/attachment-0001.html>