[Lldb-commits] [PATCH] D16334: Fix TestSignedTypes.py by removing a bogus step-over

[Tamas Berghammer via lldb-commits]

I think to get a very good coverage for the basic functionalities we need
some sort of stress testing or fuzzing infrastructure because the most
issues I hit come from compilers generating something a bit different then
we are expecting. For stack unwinding I added a stress test
(TestStandardUnwind.py, working on arm and aarch64) what catches a lot of
bugs but running it takes so long that it can't be added to the everyday
test suit. Should we try to design a fuzz testing infrastructure what runs
on a build bot so we don't have to depend on the fuzz testing behavior of
the normal test suit?

On Wed, Jan 20, 2016 at 7:22 PM Zachary Turner <zturner at google.com> wrote:

> zturner added a comment.
>
> In http://reviews.llvm.org/D16334#331338, @jingham wrote:
>
> > I sort of agree with this and sort of don't.  Formally, I agree with the
> notion of limited focused tests.  But in practice it is often the noise in
> tests that catches bugs that we don't yet have tests for.  And especially
> when the "noise" is doing things like step over that 100% should work in
> any functional debugger...  So I am also a little leery of cleaning up the
> tests too much so that they only test the things we've thought to test and
> miss other things.
> >
> > Jim
>
>
> I think we should solve that by adding more tests though.  I mean I don't
> want to do a 1 step forward, 2 steps back kind of thing.  If someone knows
> of an area where we're missing coverage, then stop what you're doing and go
> add the coverage.  If it's the type of thing where we don't know we're
> missing coverage and we're just hoping an unrelated test catches it
> someday, I don't agree with that.  Just because we know we have a problem
> doesn't mean we should solve it the wrong way, because then we can never
> possibly reach the desired end state since we're actively moving farther
> away from it.
>
>
> http://reviews.llvm.org/D16334
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-commits/attachments/20160120/57fedb51/attachment.html>