[lldb-dev] Python object lifetimes affect the reliability of tests

[Nico Weber via lldb-dev]

One possible approach is to never run a built executable directly, but to
copy it to some other place (e.g. my-exe.exe -> my-exe-run-13.exe) and only
run the copied executable. Then rebuilding the original won't fail.

On Thu, Oct 15, 2015 at 9:10 AM, Oleksiy Vyalov via lldb-dev <
lldb-dev at lists.llvm.org> wrote:

> I stumbled upon similar problem when was looking into why SBDebugger
> wasn't unloaded upon app's exit.
> The problem was in Python global objects like lldb.debugger, lldb.target
> sitting around.
> So, my guess is to try to call ScriptInterpreterPython::Clear  within
> test's tearDown call - e.g., expose Clear method as part of
> SBCommandInterpreter and call it via SBDebugger::GetCommandInterpreter
>
> On Thu, Oct 15, 2015 at 8:50 AM, Adrian McCarthy via lldb-dev <
> lldb-dev at lists.llvm.org> wrote:
>
>> I've tracked down a source of flakiness in tests on Windows to Python
>> object lifetimes and the SB interface, and I'm wondering how best to handle
>> it.
>>
>> Consider this portion of a test from TestTargetAPI:
>>
>>     def find_functions(self, exe_name):
>>         """Exercise SBTaget.FindFunctions() API."""
>>         exe = os.path.join(os.getcwd(), exe_name)
>>
>>         # Create a target by the debugger.
>>         target = self.dbg.CreateTarget(exe)
>>         self.assertTrue(target, VALID_TARGET)
>>         list = target.FindFunctions('c', lldb.eFunctionNameTypeAuto)
>>         self.assertTrue(list.GetSize() == 1)
>>
>>         for sc in list:
>>             self.assertTrue(sc.GetModule().GetFileSpec().GetFilename() ==
>> exe_name)
>>             self.assertTrue(sc.GetSymbol().GetName() == 'c')
>>
>> The local variables go out of scope when the function exits, but the SB
>> (C++) objects they represent aren't (always) immediately destroyed.  At
>> least some of these objects keep references to the executable module in the
>> shared module list, so when the test framework cleans up and calls
>> `SBDebugger::DeleteTarget`, the module isn't orphaned, so LLDB maintains an
>> open handle to the executable.
>>
>> The result of the lingering handle is that, when the next test case in
>> the test suite tries to re-build the executable, it fails because the file
>> is not writable.  (This is problematic on Windows because the file system
>> works differently in this regard than Unix derivatives.)  Every subsequent
>> case in the test suite fails.
>>
>> I managed to make the test work reliably by rewriting it like this:
>>
>>     def find_functions(self, exe_name):
>>         """Exercise SBTaget.FindFunctions() API."""
>>         exe = os.path.join(os.getcwd(), exe_name)
>>
>>         # Create a target by the debugger.
>>         target = self.dbg.CreateTarget(exe)
>>         self.assertTrue(target, VALID_TARGET)
>>
>>         try:
>>             list = target.FindFunctions('c', lldb.eFunctionNameTypeAuto)
>>             self.assertTrue(list.GetSize() == 1)
>>
>>             for sc in list:
>>                 try:
>>
>> self.assertTrue(sc.GetModule().GetFileSpec().GetFilename() == exe_name)
>>                     self.assertTrue(sc.GetSymbol().GetName() == 'c')
>>                 finally:
>>                     del sc
>>
>>         finally:
>>             del list
>>
>> The finally blocks ensure that the corresponding C++ objects are
>> destroyed, even if the function exits as a result of a Python exception
>> (e.g., if one of the assertion expressions is false and the code throws an
>> exception).  Since the objects are destroyed, the reference counts are back
>> to where they should be, and the orphaned module is closed when the target
>> is deleted.
>>
>> But this is ugly and maintaining it would be error prone.  Is there a
>> better way to address this?
>>
>> In general, it seems bad that our tests aren't well-isolated.  I
>> sympathize with the concern that re-starting LLDB for each test case would
>> slow down testing, but I'm also concerned that the state of LLDB for any
>> given test case can depend on what happened in the earlier cases.
>>
>> Adrian.
>>
>> _______________________________________________
>> lldb-dev mailing list
>> lldb-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>
>>
>
>
> --
> Oleksiy Vyalov | Software Engineer | ovyalov at google.com
>
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20151015/b29fdc48/attachment.html>