[lldb-dev] Testing through api vs. commands

[Zachary Turner via lldb-dev]

I agree that we should test the command interface, but

a) they should be explicitly marked as interface tests.
b) There should be MUCH fewer.
c) It should only verify that typing a particular command maps to the right
core sequence of public / private API calls.  Not that the debugger
functionality works as expected (since that is already tested through API
tests).
d) Results of these interface tests should also not be *verified* by the
use of self.expect, but itself through the API.  (Relying on the text to be
formatted a specific way is obviously problematic, as opposed to just
verifying the actual state of the debugger)

c is probably the hardest, because it's hard to verify that a command does
what you think it does without actually having it do the thing.  It's
possible with some refactoring, but not somethign that can be whipped up in
a day or two though.

On Tue, Sep 15, 2015 at 4:23 PM <dawn at burble.org> wrote:

> > > > > I do still think we need some tests that verify commands run, but
> I think those tests should focus not on doing complicated interactions with
> the debugger, and instead just verifying that things parse correctly and
> the command is configured correctly, with the underlying functionality
> being tested by the api tests.
> > > > >
> > > > > Thoughts?
>
> I agree that we should have both testing methods - SB API *and* commands,
> because we should be testing the user command interface too!
>
> Instead, to fix the Windows vs. other issue, I would suggest writing a
> sub-class that won't expect the missing params based on platform.
>
> In any case, there's a lot I never could figure out how to do in the SB
> API that I could only do via commands.  For example, how do you test
> that a trailing space at the end of the expr --language option's argument
> is trimmed in the SB API?
>
> -Dawn
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20150915/e0e08916/attachment.html>