[lldb-dev] lldb-dev] Patches for python bindings

Wed Mar 14 16:18:15 PDT 2012

I forgot to mention that I just tested the difference with this last patch (without the -threads arg to SWIG) and the tests had the same result as the trunk version.

Regards,  

  Filipe

On Thursday, March 15, 2012 at 12:15 AM, Filipe Cabecinhas wrote:

> Hi Enrico,
>  
> That was my problem. I'm using thread-enabled SWIG bindings for an app (actually an app plugin) I'm writing using lldb's Python bindings.
>  
> But for ScriptInterpreterPython to work, SWIG can't have threads enabled yet (I want to correct that in the future), since it doesn't manage Python's GIL like it should.
>  
> With this patch an lldb user can use the python bindings and lldb's asynchronous mode (without the Python script interpreter). Otherwise, threads would get locked up because a python thread was waiting for events without releasing the GIL, and other Python threads couldn't execute any Python code.
>  
> Attached is a new patch, that doesn't enable SWIG's thread stuff. That way, the code is already split and allows asynchronous lldb usage in Python when disabling the script interpreter but not the bindings.
>  
> In the future, when the script interpreter's bugs are fixed, we would be able to add the "-threads" flags to SWIG (and maybe enable/disable if desired).
>  
> Thanks,  
>  
> Filipe
>  
>  
> On Wednesday, March 14, 2012 at 8:31 PM, Enrico Granata wrote:
>  
> > Hi Filipe,
> > I have looked at the lldb-disable-python-interpreter.patch.
> >  
> > I am unsure as to why we need to have this finer-grained control over disabling the ScriptInterpreter AND/OR the SWIG-stuff. Would we ever want one without the other?
> >  
> > Also, the patch as-is causes crashes on my setup. Did you notice anything unusual when testing it?
> >  
> > This is what I get when testing the data formatters (which heavily use Python, so make for a good candidate to detect issues)
> >  
> > egranata:test egranata$ ./dotest.py functionalities/data-formatter/
> > LLDB build dir: /Volumes/work/egranata/lldb.patching/build/Debug
> > LLDB-131
> > Path: /Volumes/work/egranata/lldb.patching
> > URL: https://enrico@llvm.org/svn/llvm-project/lldb/trunk
> > Repository Root: https://enrico@llvm.org/svn/llvm-project
> > Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8
> > Revision: 152732
> > Node Kind: directory
> > Schedule: normal
> > Last Changed Author: johnny
> > Last Changed Rev: 152732
> > Last Changed Date: 2012-03-14 11:35:38 -0700 (Wed, 14 Mar 2012)
> >  
> >  
> >  
> > Session logs for test failures/errors/unexpected successes will go into directory '2012-03-14-12_20_37'
> > Command invoked: python ./dotest.py functionalities/data-formatter/
> > compilers=['clang']
> >  
> > Configuration: arch=x86_64 compiler=clang
> > ----------------------------------------------------------------------
> > Collected 54 tests
> >  
> > ..Fatal Python error: PyThreadState_Get: no current thread
> > Abort trap: 6
> >  
> >  
> > crash log:
> > Application Specific Information:
> > abort() called
> > objc[6529]: garbage collection is OFF
> > HandleCommand(command = "type summary add --python-script "return 'Area = ' + str( int(valobj.GetChildMemberWithName('w').GetValue()) * int(valobj.GetChildMemberWithName('h').GetValue()) );" Rectangle -w Category2")
> >  
> > Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
> > 0 libsystem_kernel.dylib 0x00007fff935b623a __pthread_kill + 10
> > 1 libsystem_c.dylib 0x00007fff8db08830 pthread_kill + 90
> > 2 libsystem_c.dylib 0x00007fff8db52631 abort + 143
> > 3 org.python.python 0x000000010ad93e9a Py_FatalError + 49
> > 4 org.python.python 0x000000010ad92360 PyThreadState_Get + 28
> > 5 org.python.python 0x000000010ad8df06 Py_InitModule4_64 + 58
> > 6 _lldb.so 0x000000010bdcae32 init_lldb + 82 (LLDBWrapPython.cpp:47759)
> > 7 _lldb.so 0x000000010cf8724f lldb_private::ScriptInterpreterPython::InitializePrivate() + 239 (ScriptInterpreterPython.cpp:1897)
> > 8 _lldb.so 0x000000010cf86cda lldb_private::ScriptInterpreterPython::ScriptInterpreterPython(lldb_private::CommandInterpreter&) + 426 (ScriptInterpreterPython.cpp:257)
> > 9 _lldb.so 0x000000010cf86b1d lldb_private::ScriptInterpreterPython::ScriptInterpreterPython(lldb_private::CommandInterpreter&) + 29 (ScriptInterpreterPython.cpp:305)
> > 10 _lldb.so 0x000000010cf79fc9 lldb_private::CommandInterpreter::GetScriptInterpreter() + 185 (CommandInterpreter.cpp:2366)
> > 11 _lldb.so 0x000000010d0ecd2c CommandObjectTypeSummaryAdd::Execute_ScriptSummary(lldb_private::Args&, lldb_private::CommandReturnObject&) + 892 (CommandObjectType.cpp:1055)
> > 12 _lldb.so 0x000000010d0ee5ba CommandObjectTypeSummaryAdd::Execute(lldb_private::Args&, lldb_private::CommandReturnObject&) + 58 (CommandObjectType.cpp:1320)
> > 13 _lldb.so 0x000000010cec6330 lldb_private::CommandObject::ExecuteWithOptions(lldb_private::Args&, lldb_private::CommandReturnObject&) + 656 (CommandObject.cpp:281)
> > 14 _lldb.so 0x000000010cf75a59 lldb_private::CommandInterpreter::HandleCommand(char const*, bool, lldb_private::CommandReturnObject&, lldb_private::ExecutionContext*, bool, bool) + 7561 (CommandInterpreter.cpp:1540)
> > 15 _lldb.so 0x000000010bcfc23e lldb::SBCommandInterpreter::HandleCommand(char const*, lldb::SBCommandReturnObject&, bool) + 670 (SBCommandInterpreter.cpp:97)
> > 16 _lldb.so 0x000000010bdf28f2 _wrap_SBCommandInterpreter_HandleCommand__SWIG_1 + 626 (LLDBWrapPython.cpp:8667)
> > 17 _lldb.so 0x000000010bd487c1 _wrap_SBCommandInterpreter_HandleCommand + 465 (LLDBWrapPython.cpp:8702)
> > 18 org.python.python 0x000000010ad59f62 PyEval_EvalFrameEx + 3557
> > 19 org.python.python 0x000000010ad59137 PyEval_EvalCodeEx + 1934
> > 20 org.python.python 0x000000010ad5f8cf 0x10ad42000 + 121039
> > 21 org.python.python 0x000000010ad5b62a PyEval_EvalFrameEx + 9389
> > 22 org.python.python 0x000000010ad59137 PyEval_EvalCodeEx + 1934
> > 23 org.python.python 0x000000010ad5f8cf 0x10ad42000 + 121039
> > 24 org.python.python 0x000000010ad5b62a PyEval_EvalFrameEx + 9389
> > 25 org.python.python 0x000000010ad59137 PyEval_EvalCodeEx + 1934
> > 26 org.python.python 0x000000010ad5f8cf 0x10ad42000 + 121039
> > 27 org.python.python 0x000000010ad5b62a PyEval_EvalFrameEx + 9389
> > 28 org.python.python 0x000000010ad5f859 0x10ad42000 + 120921
> > 29 org.python.python 0x000000010ad5b62a PyEval_EvalFrameEx + 9389
> > 30 org.python.python 0x000000010ad59137 PyEval_EvalCodeEx + 1934
> > 31 org.python.python 0x000000010ad92d6a 0x10ad42000 + 331114
> > 32 org.python.python 0x000000010ad516b6 PyObject_Call + 97
> > 33 org.python.python 0x000000010ad59f72 PyEval_EvalFrameEx + 3573
> > 34 org.python.python 0x000000010ad59137 PyEval_EvalCodeEx + 1934
> > 35 org.python.python 0x000000010ad92d6a 0x10ad42000 + 331114
> > 36 org.python.python 0x000000010ad516b6 PyObject_Call + 97
> > 37 org.python.python 0x000000010ad6e9af 0x10ad42000 + 182703
> > 38 org.python.python 0x000000010ad516b6 PyObject_Call + 97
> > 39 org.python.python 0x000000010addf19a 0x10ad42000 + 643482
> > 40 org.python.python 0x000000010ad516b6 PyObject_Call + 97
> > 41 org.python.python 0x000000010ad5b77d PyEval_EvalFrameEx + 9728
> > 42 org.python.python 0x000000010ad59137 PyEval_EvalCodeEx + 1934
> > 43 org.python.python 0x000000010ad5f8cf 0x10ad42000 + 121039
> > 44 org.python.python 0x000000010ad5b62a PyEval_EvalFrameEx + 9389
> > 45 org.python.python 0x000000010ad59137 PyEval_EvalCodeEx + 1934
> > 46 org.python.python 0x000000010ad5f8cf 0x10ad42000 + 121039
> > 47 org.python.python 0x000000010ad5b62a PyEval_EvalFrameEx + 9389
> > 48 org.python.python 0x000000010ad59137 PyEval_EvalCodeEx + 1934
> > 49 org.python.python 0x000000010ad92d6a 0x10ad42000 + 331114
> > 50 org.python.python 0x000000010ad516b6 PyObject_Call + 97
> > 51 org.python.python 0x000000010ad59f72 PyEval_EvalFrameEx + 3573
> > 52 org.python.python 0x000000010ad59137 PyEval_EvalCodeEx + 1934
> > 53 org.python.python 0x000000010ad92d6a 0x10ad42000 + 331114
> > 54 org.python.python 0x000000010ad516b6 PyObject_Call + 97
> > 55 org.python.python 0x000000010ad6e9af 0x10ad42000 + 182703
> > 56 org.python.python 0x000000010ad516b6 PyObject_Call + 97
> > 57 org.python.python 0x000000010addf19a 0x10ad42000 + 643482
> > 58 org.python.python 0x000000010ad516b6 PyObject_Call + 97
> > 59 org.python.python 0x000000010ad5b77d PyEval_EvalFrameEx + 9728
> > 60 org.python.python 0x000000010ad5f859 0x10ad42000 + 120921
> > 61 org.python.python 0x000000010ad5b62a PyEval_EvalFrameEx + 9389
> > 62 org.python.python 0x000000010ad59137 PyEval_EvalCodeEx + 1934
> > 63 org.python.python 0x000000010ad589a3 PyEval_EvalCode + 54
> > 64 org.python.python 0x000000010ad94c60 0x10ad42000 + 339040
> > 65 org.python.python 0x000000010ad94d2c PyRun_FileExFlags + 165
> > 66 org.python.python 0x000000010ad94716 PyRun_SimpleFileExFlags + 410
> > 67 org.python.python 0x000000010adb8e1f Py_Main + 2715
> > 68 libdyld.dylib 0x00007fff95f497e1 start + 1
> >  
> > Does your patch depend on having SWIG's thread support? The way we are currently handling the locking issues in Python is by having a Locker class. You can look for samples of its usage throughout ScriptInterpreterPython, but it is basically follows the RIIA concept, and you can pass parameters to the constructor telling it exactly what behavior you need. If there are any specific combinations you think are missing, feel free to suggest and/or code a patch yourself.
> > In the simplest case, all you need to do to lock other threads out is:
> > {
> > Locker py_lock(this);
> > //your code here
> > }
> >  
> > Thanks,
> > Enrico Granata
> > ✉ egranata@.com
> > ✆ (408) 972-7683
>  
>  
>  
> Attachments:  
> - lldb-disable-python-interpreter-no-threads.patch
>  