[lldb-dev] Writing test cases...

[Chaoren Lin]

Whenever I debug a test with runCmd/expect, I could just grep for runCmd in
the test trace and execute the same steps manually (or just paste
everything in). It's especially useful if I need another debugger attached
to lldb. AFAICT, with the SB API, I'd need to look at the test script and
translate everything myself. Is there a way to do this automatically, or is
there a better workflow that I don't know of?

On Wed, Mar 18, 2015 at 4:19 PM, <jingham at apple.com> wrote:

> I don't mean to pick on Chaoren here, his latest patch just reminded me of
> this, so I thought I'd repeat it...
>
> We strongly prefer writing test cases using the SB API's rather than the
> runCmd & expect.  Unless you are actually testing some feature of the
> command line, please don't write command based tests.  I realize that there
> are plenty of examples of tests in the test suite that use runCmd where
> they shouldn't, but don't copy them, copy the plenty that do use the SB
> API's instead.
>
> The reason for this is that our policy is that we will maintain
> compatibility with the SB API's - at least till we decide to do an
> API-2.0.  But we don't make any similar guarantee about the details of
> command result format.  If your test is using the command line, it is going
> to have to check against the command result, and you either end up writing
> your check pattern by checking as little as possible so you won't be
> exposed to random changes, in which case you can end up missing some
> failure, or you test too much and it means irrelevant changes break your
> tests.
>
> If you use the Python API's it is possible to check all the results you
> want to check in a very explicit way, which makes the tests much more
> robust.
>
> Even if you are testing that a command-line command does some specific
> thing, it is still better in general to use the SB API's to drive to the
> point where you want to run the test, then use SBInterpreter::HandleCommand
> to run the command.  You get the full result text from the command in the
> command return object, and all the part where you are driving the debugger
> to the point you want to test will be more robust.
>
> BTW, I don't think we've said this anywhere, I just added a note to this
> effect to the Testsuite README.
>
> Jim
>
>
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20150318/63cb353f/attachment.html>