[Lldb-commits] [PATCH] The great Python split

[Zachary Turner]

Hi vharron, ovyalov, clayborg, granata.enrico,

(This CL looks scarier than it really is.  It's honestly not that great of a split.)

After the discussion on list, I took it up on myself to make my ideas concrete.  Along the way I learned some interesting things.

1) There are definitely some assumptions baked into places where they shouldn't be about the existence of Python.  See some of the Type summary code and data formatters for examples.  Or grep for LLDB_DISABLE_PYTHON.  This isn't anyone's fault, it's just a consequence of the weak barrier between Python and the rest of LLDB.  

2) It is pretty much impossible right now to link Python into some binaries and not others.  A concrete example of what I'm talking about is LLGS.  All of our decisions are based on conditionally compiling code based on the value of a global flag, LLDB_DISABLE_PYTHON.  If you want python for the host but not the server, this can't really be done without compiling LLDB core libraries twice.  If you try not to link Python into lldb-server you will get linker errors because it links against some .a file which has already been compiled with LLDB_DISABLE_PYTHON.

3) There were some hidden race conditions involving Python initialization.  Because python code was split between lldbAPI, Interpreter, and LLDBWrapPython.cpp (which was linked into liblldb), there were some complexity introduced to make sure Python was initialized at the right time.

#2 and #3 are largely solved with this patch.  With a few minor exceptions, all python code is now compiled into a single library, lldbPythonInterpreter.  The end goal is to get *all* Python code into this library.  The overall structure of script interpreter is now as follows:

lldb
|_source
  |_Interpreter
  |
  |_ScriptInterpreter
  | |_ScriptInterpreter.cpp
  | |_ScriptInterpreterNone.cpp
  |
  |_PythonInterpreter
    |_ScriptInterpreterPython.cpp
    |_PythonDataObjects.cpp
    |_LLDBWrapPython.cpp
    |_PythonPointer.h

The idea is that the rest of LLDB should be shielded from the particular language of the interpreter.  It seems like that was the original design goal anyway, this just makes it explicit.  The advantage of this is that now someone who doesn't want Python can easily just not link against lldbPythonInterpreter.  Of course they could do this already by using LLDB_DISABLE_PYTHON, but that doesn't solve the issue of wanting Python sometimes and not others.  All uses of LLDB_DISABLE_PYTHON will eventually end up hidden inside of PythonInterpreter.

I know LLGS wants to not link against Python.  I believe this patch is a necessary first step.  Importantly, it removes the dependency from the CommandInterpreter on Python.  But it is not yet sufficient.  There are still some references to Python in the TypeSummary stuff and Data formatter code which will cause an implit dependency on Python.

Note: There is no Xcode project here.  I know this probably makes it hard to review for people at Apple.  I plan to work on updates to the Xcode project today, I just wanted to get this up in the meantime.

Note 2: If you have a diff utility that works across renames it makes looking at this patch much less scary.  Most changes were purely mechanical (changing incldue paths).  There were only a few that were not, mostly surrounding factory creation of the ScriptInterpreter, and a complete removal of the callback-initialization-via-function-pointer that was used (callbacks are now initialized automatically through static linkage).

http://reviews.llvm.org/D7946

Files:
  CMakeLists.txt
  cmake/LLDBDependencies.cmake
  include/lldb/API/SBBreakpointLocation.h
  include/lldb/API/SBCommandInterpreter.h
  include/lldb/API/SBFrame.h
  include/lldb/DataFormatters/TypeSummary.h
  include/lldb/Interpreter/CommandInterpreter.h
  include/lldb/Interpreter/PythonDataObjects.h
  include/lldb/Interpreter/ScriptInterpreter.h
  include/lldb/Interpreter/ScriptInterpreterNone.h
  include/lldb/Interpreter/ScriptInterpreterPython.h
  include/lldb/ScriptInterpreter/Python/PythonDataObjects.h
  include/lldb/ScriptInterpreter/Python/PythonPointer.h
  include/lldb/ScriptInterpreter/Python/ScriptInterpreterPython.h
  include/lldb/ScriptInterpreter/ScriptInterpreter.h
  include/lldb/ScriptInterpreter/ScriptInterpreterNone.h
  include/lldb/Utility/PythonPointer.h
  include/lldb/lldb-forward.h
  source/API/SBBreakpoint.cpp
  source/API/SBBreakpointLocation.cpp
  source/API/SBCommandInterpreter.cpp
  source/API/SBDebugger.cpp
  source/API/SBTypeCategory.cpp
  source/CMakeLists.txt
  source/Commands/CommandObjectCommands.cpp
  source/Core/Module.cpp
  source/Core/ValueObject.cpp
  source/DataFormatters/FormatManager.cpp
  source/DataFormatters/TypeSynthetic.cpp
  source/Interpreter/CMakeLists.txt
  source/Interpreter/CommandInterpreter.cpp
  source/Interpreter/CommandObject.cpp
  source/Interpreter/CommandObjectScript.cpp
  source/Interpreter/PythonDataObjects.cpp
  source/Interpreter/ScriptInterpreter.cpp
  source/Interpreter/ScriptInterpreterNone.cpp
  source/Interpreter/ScriptInterpreterPython.cpp
  source/Interpreter/embedded_interpreter.py
  source/Plugins/OperatingSystem/Python/OperatingSystemPython.cpp
  source/Plugins/OperatingSystem/Python/OperatingSystemPython.h
  source/Plugins/Process/Utility/DynamicRegisterInfo.cpp
  source/Plugins/Process/gdb-remote/GDBRemoteRegisterContext.cpp
  source/Plugins/Process/gdb-remote/ProcessGDBRemote.cpp
  source/ScriptInterpreter/CMakeLists.txt
  source/ScriptInterpreter/Python/CMakeLists.txt
  source/ScriptInterpreter/Python/PythonDataObjects.cpp
  source/ScriptInterpreter/Python/ScriptInterpreterPython.cpp
  source/ScriptInterpreter/Python/embedded_interpreter.py
  source/ScriptInterpreter/ScriptInterpreter.cpp
  source/ScriptInterpreter/ScriptInterpreterNone.cpp
  source/Target/ThreadPlanPython.cpp
  source/lldb.cpp
  tools/driver/CMakeLists.txt
  tools/lldb-server/CMakeLists.txt

EMAIL PREFERENCES
  http://reviews.llvm.org/settings/panel/emailpreferences/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: D7946.20862.patch
Type: text/x-patch
Size: 372545 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/lldb-commits/attachments/20150227/5ff4f766/attachment.bin>