[Lldb-commits] [PATCH] D24591: [LIT] First pass of LLDB LIT support

[Zachary Turner via lldb-commits]

Also, you mentioned that this is very similar to the lldbinline tests.  For
that reason, I would actually propose dropping lldbinline tests in favor of
this.  If they are essentially the same, then it seems better to have 2
types of tests rather than 3, and it seems better for those 2 to be lit +
python rather than lldbinline + python, solely for the reason that lit is
shared and receives many performance and stability improvements from non
LLDB developers.

On Wed, Sep 14, 2016 at 5:13 PM Zachary Turner <zturner at google.com> wrote:

> I'm only saying that we should have an open mind.  Obviously there are
> (valid!) concerns.  If we can't solve them then we can't solve them.  The
> goal (my goal anyway) is always to make things better, not to use X because
> it's X.  There's value in consistency, but that doesn't mean that the value
> from consistency always outweighs the value you get from doing a custom
> thing.
>
> So what I'm saying is: IF we can find a way to have one test suite across
> all of LLVM and it's subprojects, AND it is sufficiently powerful to test
> all the things we need to test while remaining maintainable in the long
> term, we should absolutely jump on the opportunity.
>
> But this is one of those things that requires a lot of upfront time
> investment before you can actually know if it can work for 100% of things.
> Obviously some people don't want to invest their time that way when they're
> already satisfied, and I don't blame them.
>
> But for the people who do, and who think they can solve the problem,
> what's the harm?  Obviously the burden is on those people to prove that
> their vision can be realized.
>
> But if it is successful, then there's no denying the benefits.  1) Tests
> become easier to write.  2) Tests become easier to debug.  3) Consistency
> encourages people who have traditionally stayed away from LLDB to
> contribute.  4) All the people pouring their effort into the custom thing
> can now pour it into the shared thing, so everybody benefits.
>
> I don't blame you for being scared of command tests.  I don't support
> their use in the current LLDB test suite either, for exactly the same
> reasons you and Jason have expressed.  But I do think it's possible to come
> up with something that a) doesn't suffer from the same problems, b) allows
> testing a ton of extra functionality that is not currently testable through
> the api, and c) doesn't rely on python at all.  If I'm wrong I'll eat crow
> :)
>
> On Wed, Sep 14, 2016 at 5:00 PM Jim Ingham <jingham at apple.com> wrote:
>
> Also, w.r.t:
>
> >  Aside from write imperative control flow constructs, which I see as a
> positive rather than a negative.
>
> I wrote a bunch of tests to test that stepping behavior for swift and C
> was reasonable.  When stepping through source code, there is not one
> correct way to write the line tables, and in fact clang & swiftc change how
> they describe the source through the line tables all the time.  So you have
> to do: I stepped, and sometimes I'll get to A, sometimes to B, both are
> "right" but I have to do different things in either case.  If A, step again
> before the next test, if B go to the next test.
>
> You could "fix" that by only doing one step per test, and taking each of
> these as a success.  But then you wouldn't test that series of steps don't
> accumulate errors, you'd only test "run to a breakpoint and step once."
> That would not be good.  So your positive would be very much a negative for
> this kind of test.
>
> Traditionally the answer to this has been: we know we have to keep the
> current testsuite around but we're just adding other new different ways to
> write tests.  Now you are saying something very different.  Do you really
> mean that?
>
> Jim
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-commits/attachments/20160915/7923aecc/attachment.html>