[lldb-dev] lldb tests and tear down hooks

[Zachary Turner via lldb-dev]

There's a subtle bug that is pervasive throughout the test suite.  Consider
the following seemingly innocent test class.

class MyTest(TestBase);
    def setUp():
        TestBase.setUp()    #1

        # Do some stuff      #2
        self.addTearDownHook(lambda: self.foo())   #3

    def test_interesting_stuff():
        pass

Here's the problem.  As a general principle, cleanup needs to happen in
reverse order from initialization.  That's why, if we had a tearDown()
method, it would probably look something like this:

    def tearDown():
        # Clean up some stuff  #2

        TestBase.tearDown()    #1

This follows the pattern in other languages like C++, for example, where
construction goes from base -> derived, but destruction goes from derived
-> base.

But if you add these tear down hooks into the mix, it violates that.  tear
down hooks get invoked as part of TestBase.tearDown(), so in the above
example the initialization order is 1 -> 2 -> 3 but the teardown order is 2
-> 1 -> 3  (or 2 -> 3 -> 1, or none of the above depending on where inside
of TestBase.tearDown() hook the hooks get invoked).

To make matters worse, tear down hooks can be added from arbitrary points
in a test's run, not just during setup.

The only way I can see to fix this is to delete this tearDownHook mechanism
entirely.  Anyone who wants it can easily reimplement this in the
individual test by just keeping their own list of lambdas in the derived
class, overriding tearDown(), and running through their own list in reverse
order before calling TestBase.tearDown().

I don't intend to do this work right now, but I would like to do it in the
future, so I want to throw this out there and see if anyone has thoughts on
it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20151021/37faac9e/attachment.html>