[lldb-dev] Python object lifetimes affect the reliability of tests

[Adrian McCarthy via lldb-dev]

I've tracked down a source of flakiness in tests on Windows to Python
object lifetimes and the SB interface, and I'm wondering how best to handle
it.

Consider this portion of a test from TestTargetAPI:

    def find_functions(self, exe_name):
        """Exercise SBTaget.FindFunctions() API."""
        exe = os.path.join(os.getcwd(), exe_name)

        # Create a target by the debugger.
        target = self.dbg.CreateTarget(exe)
        self.assertTrue(target, VALID_TARGET)
        list = target.FindFunctions('c', lldb.eFunctionNameTypeAuto)
        self.assertTrue(list.GetSize() == 1)

        for sc in list:
            self.assertTrue(sc.GetModule().GetFileSpec().GetFilename() ==
exe_name)
            self.assertTrue(sc.GetSymbol().GetName() == 'c')

The local variables go out of scope when the function exits, but the SB
(C++) objects they represent aren't (always) immediately destroyed.  At
least some of these objects keep references to the executable module in the
shared module list, so when the test framework cleans up and calls
`SBDebugger::DeleteTarget`, the module isn't orphaned, so LLDB maintains an
open handle to the executable.

The result of the lingering handle is that, when the next test case in the
test suite tries to re-build the executable, it fails because the file is
not writable.  (This is problematic on Windows because the file system
works differently in this regard than Unix derivatives.)  Every subsequent
case in the test suite fails.

I managed to make the test work reliably by rewriting it like this:

    def find_functions(self, exe_name):
        """Exercise SBTaget.FindFunctions() API."""
        exe = os.path.join(os.getcwd(), exe_name)

        # Create a target by the debugger.
        target = self.dbg.CreateTarget(exe)
        self.assertTrue(target, VALID_TARGET)

        try:
            list = target.FindFunctions('c', lldb.eFunctionNameTypeAuto)
            self.assertTrue(list.GetSize() == 1)

            for sc in list:
                try:

self.assertTrue(sc.GetModule().GetFileSpec().GetFilename() == exe_name)
                    self.assertTrue(sc.GetSymbol().GetName() == 'c')
                finally:
                    del sc

        finally:
            del list

The finally blocks ensure that the corresponding C++ objects are destroyed,
even if the function exits as a result of a Python exception (e.g., if one
of the assertion expressions is false and the code throws an exception).
Since the objects are destroyed, the reference counts are back to where
they should be, and the orphaned module is closed when the target is
deleted.

But this is ugly and maintaining it would be error prone.  Is there a
better way to address this?

In general, it seems bad that our tests aren't well-isolated.  I sympathize
with the concern that re-starting LLDB for each test case would slow down
testing, but I'm also concerned that the state of LLDB for any given test
case can depend on what happened in the earlier cases.

Adrian.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20151015/562e0909/attachment.html>