[Lldb-commits] [PATCH] D16334: Fix TestSignedTypes.py by removing a bogus step-over

[Zachary Turner via lldb-commits]

Right, but I don't agree in this case that "different" has to mean "we
discourage the use of reduced test cases".  I have a hard time imagining a
scenario where not having reduced test cases is an advantage.  It's also
easy to explain to people.  "Write reduced test cases".  It's easy to
understand.  Moreso than "Reduced test cases are good, but not always
because fuzziness, but we can't really put into words exactly what amount
of fuzziness we want, or what it should look like, so I guess anything goes
- don't ask don't tell".

If we want fuzziness, we should do fuzz testing.  Feature tests and
regression tests should be reduced.  The whole reason the tests in
lldb/tests were written is because people were testing a specific feature
or bugfix.  I don't see a reason to add fuzziness to this kind of test.
All it does is pollute failure logs.  Fuzziness is better tested by fuzz
tests.

On Thu, Jan 21, 2016 at 11:46 AM Jim Ingham <jingham at apple.com> wrote:

>
> > On Jan 21, 2016, at 10:17 AM, Zachary Turner <zturner at google.com> wrote:
> >
> > zturner added a comment.
> >
> > Sure, an interface change to Process might break the mock, but it would
> break at compile time, you just fix it up.  It's not something that would
> happen frequently, this is the same situation going on in LLVM where there
> are unit tests, sometimes they break, and people fix them.  But it's never
> been an issue because it doesn't happen often.
> >
> > I know there are different opinions about what type of tests to write
> and how reduced they should be, but in this case I'm simply following the
> LLVM guidelines for writing test cases, which we supposed to adhere to.
> http://llvm.org/docs/DeveloperPolicy.html#test-cases (except for obvious
> differences about how it uses lit, etc)
>
> I do not think that we should be restricted to guidelines for testing that
> are meant for a very different kind of program.  You don't stop a compiler
> midway through a compile, drop into the "compiler command line", add a few
> more lines of code and a couple of functions, change some compiler options
> and continue.  You give it input, and collect the output or errors.  But
> you do the functionally equivalent thing in the debugger all the time.  So
> the way you test things is going to be very different.
>
> Jim
>
>
> >
> >
> > http://reviews.llvm.org/D16334
> >
> >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-commits/attachments/20160121/da6d46d0/attachment.html>