[Lldb-commits] [PATCH] D24591: [LIT] First pass of LLDB LIT support

[Zachary Turner via lldb-commits]

One thing I wonder about. It seems like everyone is mostly on the same page
about command line output .

What about input? Would anyone have objections to a test which ran a few
commands to get the debugger into a particular state before doing something
to verify the output? Let's assume I'm waving my hands about this last step
but that it doesn't involve scraping the output of a debugger command

Maybe this will never even be an issue, but I just want to make sure
everyone is on the same page and that the objections are:

a) specific to command OUTPUT, not input (i.e. it's ok to have a test run
"break set -n main")
b) specific to *non trivial* output (checking that "p 5" displays 5 is ok)
c) specific to the the output of the *user* command line interface, so that
some hypothetical other command line interface (which again I'm being hand
wavy about) would not be subject to the same objections.


On Thu, Sep 15, 2016 at 7:28 PM Jason Molenda <jmolenda at apple.com> wrote:

>
> > On Sep 15, 2016, at 8:02 AM, Zachary Turner <zturner at google.com> wrote:
> >
> >
> > It sounds like your goal is also "tests have to use the SB API and no
> other API", which if so I think that's counterproductive.   More
> productive, IMO, would be being open to any alternative that addresses the
> concerns you have with command-line tests.  There are more than 2 ways to
> skin a cat, so to speak.
>
> Thinking about this a bit, another approach would be to do lit tests on
> top of lldb-mi.  MI is a structured format for the debugger and a UI to
> communicate back and forth with a simple text markup language (it would be
> JSON if it were being done today, but it was added to gdb eighteen years
> ago, so it's not).  The commands correspond to the commands a debugger user
> would think to use -- no need to understand the structure of how lldb is
> implemented, like with the SB API.  The format is a little unusual for a
> human to type, but back when we supported gdb at Apple we worked in MI all
> the time (it was used to communicate between Xcode, our IDE, and gdb) by
> hand when testing and it was fine. "-exec-run" instead of run, that kind of
> thing.  I think there are four dozens different commands.
>
> lldb-mi itself uses SB API.  And the concerns about hardcoding command
> line UI don't apply, it's a key-value format intended for computers, no one
> is going to add an extra space character to anything -- the most it changes
> is that new key-value pairs are added to responses.
>
>
> I agree there are many acceptable ways to write lit tests that don't
> involve lldb command line scraping, and I'm all in favor of using lit with
> tests that don't do that.  Of course the patch we're discussing has lit
> test examples that contradict our own policy on writing tests.  Once lit is
> supported in lldb, are we going to reject any testsuite additions that
> depend on the text output from the command line lldb?  If we're all on the
> same page here, then I have no reservations.
>
> Just to say out loud the future I can easily see:  We add lit, then we
> have people helpfully write a few dozen tests in lit that depend on the
> command line debugger output.  Those patches have to be rejected.
>
> J
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-commits/attachments/20160916/8654e807/attachment.html>