[lldb-dev] [RFC] Testsuite in lldb & possible future directions

[Zachary Turner via lldb-dev]

On Mon, Feb 5, 2018 at 8:12 PM Davide Italiano via lldb-dev <
lldb-dev at lists.llvm.org> wrote:

>
>
> Conclusions:
> The reliability of the suite (and the health of the codebase) is very
> important to us. If you have issues, let us know.
> In general, I'm looking for any kind of feedback, feel free to speak!
>
>
I think that the path forward is to massively expand test coverage in all
areas.  We need roughly 20x - 30x the amount of tests that we currently
have.  25,000 ~ 30,000 tests that run equally well on all platforms is a
good target to shoot for.

The goal of tests is, obviously, to increase test coverage by increasing
the amount of code that is tested.  Another way to increase test coverage
is to reduce the amount of code that isn't tested.  If you can delete an
untested branch then even if you don't add a test, you've increased test
coverage.  To that end, we should be looking to assert more liberally and
end the dubious practice of defensive programming.

On the subject of lldb-test.  I think the existing test suite serves its
purpose as an integration test suite well, and I would even say that it has
a reasonable amount of test coverage from what you could expect of an
integration test suite.  But what we need is a regression test suite.  I
don't think we should spend a significant amount of time adding new tests
to the integration test suite.  It's coverage is already decent.  I think
almost all new tests going forward should be very lightweight, targeted,
regression tests that do not depend on the host platform at all.  lldb-test
is the perfect vehicle for this kind of test.  It's still early and doesn't
do much, so we will need to continually add more functionality to lldb-test
as well to realize this goal, but I think we can rapidly expand test
coverage going this route.

Finally, I think we should get buildbots running sanitized builds of LLDB
under the test suite.  For LLDB specifically I think TSan and UBSan would
add the most value, but long term I think we should get all sanitizers
enabled.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20180206/5fdb41e0/attachment.html>