[lldb-dev] lldb tests and tear down hooks

[Zachary Turner via lldb-dev]

Yea, that's what I think too.  I think this mechanism was probably invented
to just to save some code and promote reusability, but in practice leads to
these kinds of problems.

On Wed, Oct 21, 2015 at 2:07 AM Pavel Labath <labath at google.com> wrote:

> I think we can remove these, provided there is a way to mimic the
> functionality they are used for now, which I think shouldn't be hard.
> Anything which was set up in the setUp() method should be undone in
> tearDown(). Anything which was set up in the test method, can be
> undone using a try-finally block. Is there a use case not covered by
> this?
>
> pl
>
> On 21 October 2015 at 04:47, Zachary Turner via lldb-dev
> <lldb-dev at lists.llvm.org> wrote:
> > There's a subtle bug that is pervasive throughout the test suite.
> Consider
> > the following seemingly innocent test class.
> >
> > class MyTest(TestBase);
> >     def setUp():
> >         TestBase.setUp()    #1
> >
> >         # Do some stuff      #2
> >         self.addTearDownHook(lambda: self.foo())   #3
> >
> >     def test_interesting_stuff():
> >         pass
> >
> > Here's the problem.  As a general principle, cleanup needs to happen in
> > reverse order from initialization.  That's why, if we had a tearDown()
> > method, it would probably look something like this:
> >
> >     def tearDown():
> >         # Clean up some stuff  #2
> >
> >         TestBase.tearDown()    #1
> >
> > This follows the pattern in other languages like C++, for example, where
> > construction goes from base -> derived, but destruction goes from
> derived ->
> > base.
> >
> > But if you add these tear down hooks into the mix, it violates that.
> tear
> > down hooks get invoked as part of TestBase.tearDown(), so in the above
> > example the initialization order is 1 -> 2 -> 3 but the teardown order
> is 2
> > -> 1 -> 3  (or 2 -> 3 -> 1, or none of the above depending on where
> inside
> > of TestBase.tearDown() hook the hooks get invoked).
> >
> > To make matters worse, tear down hooks can be added from arbitrary
> points in
> > a test's run, not just during setup.
> >
> > The only way I can see to fix this is to delete this tearDownHook
> mechanism
> > entirely.  Anyone who wants it can easily reimplement this in the
> individual
> > test by just keeping their own list of lambdas in the derived class,
> > overriding tearDown(), and running through their own list in reverse
> order
> > before calling TestBase.tearDown().
> >
> > I don't intend to do this work right now, but I would like to do it in
> the
> > future, so I want to throw this out there and see if anyone has thoughts
> on
> > it.
> >
> > _______________________________________________
> > lldb-dev mailing list
> > lldb-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20151021/91d8da84/attachment.html>