[Lldb-commits] [lldb] r249886 - Port native Python-API to 3.x
Zachary Turner via lldb-commits
lldb-commits at lists.llvm.org
Fri Oct 9 13:35:04 PDT 2015
This breaks the build on Linux. Working on a fix right now.
On Fri, Oct 9, 2015 at 12:47 PM Zachary Turner via lldb-commits <
lldb-commits at lists.llvm.org> wrote:
> Author: zturner
> Date: Fri Oct 9 14:45:41 2015
> New Revision: 249886
>
> URL: http://llvm.org/viewvc/llvm-project?rev=249886&view=rev
> Log:
> Port native Python-API to 3.x
>
> With this change, liblldb is 95% of the way towards being able
> to work under both Python 2.x and Python 3.x. This should
> introduce no functional change for Python 2.x, but for Python
> 3.x there are some important changes. Primarily, these are:
>
> 1) PyString doesn't exist in Python 3. Everything is a PyUnicode.
> To account for this, PythonString now stores a PyBytes instead
> of a PyString. In Python 2, this is equivalent to a PyUnicode,
> and in Python 3, we do a conversion from PyUnicode to PyBytes
> and store the PyBytes.
> 2) PyInt doesn't exist in Python 3. Everything is a PyLong. To
> account for this, PythonInteger stores a PyLong instead of a
> PyInt. In Python 2.x, this requires doing a conversion to
> PyLong when creating a PythonInteger from a PyInt. In 3.x,
> there is no PyInt anyway, so we can assume everything is a
> PyLong.
> 3) PyFile_FromFile doesn't exist in Python 3. Instead there is a
> PyFile_FromFd. This is not addressed in this patch because it
> will require quite a large change to plumb fd's all the way
> through the system into the ScriptInterpreter. This is the only
> remaining piece of the puzzle to get LLDB supporting Python 3.x.
>
> Being able to run the test suite is not addressed in this patch.
> After the extension module can compile and you can enter an embedded
> 3.x interpreter, the test suite will be addressed in a followup.
>
> Added:
> lldb/trunk/unittests/ScriptInterpreter/
> lldb/trunk/unittests/ScriptInterpreter/CMakeLists.txt
> lldb/trunk/unittests/ScriptInterpreter/Python/
> lldb/trunk/unittests/ScriptInterpreter/Python/CMakeLists.txt
>
> lldb/trunk/unittests/ScriptInterpreter/Python/PythonDataObjectsTests.cpp
> Modified:
>
> lldb/trunk/source/Plugins/ScriptInterpreter/Python/PythonDataObjects.cpp
> lldb/trunk/source/Plugins/ScriptInterpreter/Python/PythonDataObjects.h
>
> lldb/trunk/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp
> lldb/trunk/unittests/CMakeLists.txt
>
> Modified:
> lldb/trunk/source/Plugins/ScriptInterpreter/Python/PythonDataObjects.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/lldb/trunk/source/Plugins/ScriptInterpreter/Python/PythonDataObjects.cpp?rev=249886&r1=249885&r2=249886&view=diff
>
> ==============================================================================
> ---
> lldb/trunk/source/Plugins/ScriptInterpreter/Python/PythonDataObjects.cpp
> (original)
> +++
> lldb/trunk/source/Plugins/ScriptInterpreter/Python/PythonDataObjects.cpp
> Fri Oct 9 14:45:41 2015
> @@ -71,10 +71,18 @@ PythonObject::GetObjectType() const
> return PyObjectType::List;
> if (PyDict_Check(m_py_obj))
> return PyObjectType::Dictionary;
> + if (PyUnicode_Check(m_py_obj))
> + return PyObjectType::String;
> + if (PyLong_Check(m_py_obj))
> + return PyObjectType::Integer;
> +#if PY_MAJOR_VERSION < 3
> + // These functions don't exist in Python 3.x. PyString is PyUnicode
> + // and PyInt is PyLong.
> if (PyString_Check(m_py_obj))
> return PyObjectType::String;
> - if (PyInt_Check(m_py_obj) || PyLong_Check(m_py_obj))
> + if (PyInt_Check(m_py_obj))
> return PyObjectType::Integer;
> +#endif
> return PyObjectType::Unknown;
> }
>
> @@ -142,14 +150,16 @@ PythonString::PythonString (const Python
> Reset(object.get()); // Use "Reset()" to ensure that py_obj is a
> string
> }
>
> -PythonString::PythonString (llvm::StringRef string) :
> - PythonObject(PyString_FromStringAndSize(string.data(), string.size()))
> +PythonString::PythonString(llvm::StringRef string)
> + : PythonObject()
> {
> + SetString(string);
> }
>
> -PythonString::PythonString(const char *string) :
> - PythonObject(PyString_FromString(string))
> +PythonString::PythonString(const char *string)
> + : PythonObject()
> {
> + SetString(llvm::StringRef(string));
> }
>
> PythonString::PythonString () :
> @@ -162,20 +172,53 @@ PythonString::~PythonString ()
> }
>
> bool
> -PythonString::Reset (PyObject *py_obj)
> +PythonString::Check(PyObject *py_obj)
> +{
> + if (!py_obj)
> + return false;
> +#if PY_MAJOR_VERSION >= 3
> + // Python 3 does not have PyString objects, only PyUnicode.
> + return PyUnicode_Check(py_obj);
> +#else
> + return PyUnicode_Check(py_obj) || PyString_Check(py_obj);
> +#endif
> +}
> +
> +bool
> +PythonString::Reset(PyObject *py_obj)
> {
> - if (py_obj && PyString_Check(py_obj))
> - return PythonObject::Reset(py_obj);
> -
> - PythonObject::Reset(nullptr);
> - return py_obj == nullptr;
> + if (!PythonString::Check(py_obj))
> + {
> + PythonObject::Reset(nullptr);
> + return false;
> + }
> +
> +// Convert this to a PyBytes object, and only store the PyBytes. Note
> that in
> +// Python 2.x, PyString and PyUnicode are interchangeable, and PyBytes is
> an alias
> +// of PyString. So on 2.x, if we get into this branch, we already have a
> PyBytes.
> + //#if PY_MAJOR_VERSION >= 3
> + if (PyUnicode_Check(py_obj))
> + {
> + PyObject *unicode = py_obj;
> + py_obj = PyUnicode_AsUTF8String(py_obj);
> + Py_XDECREF(unicode);
> + }
> + //#endif
> +
> + assert(PyBytes_Check(py_obj) && "PythonString::Reset received a
> non-string");
> + return PythonObject::Reset(py_obj);
> }
>
> llvm::StringRef
> PythonString::GetString() const
> {
> if (m_py_obj)
> - return llvm::StringRef(PyString_AsString(m_py_obj), GetSize());
> + {
> + Py_ssize_t size;
> + char *c;
> + PyBytes_AsStringAndSize(m_py_obj, &c, &size);
> + return llvm::StringRef(c, size);
> + }
> return llvm::StringRef();
> }
>
> @@ -183,14 +226,21 @@ size_t
> PythonString::GetSize() const
> {
> if (m_py_obj)
> - return PyString_Size(m_py_obj);
> + return PyBytes_Size(m_py_obj);
> return 0;
> }
>
> void
> PythonString::SetString (llvm::StringRef string)
> {
> +#if PY_MAJOR_VERSION >= 3
> + PyObject *unicode = PyUnicode_FromStringAndSize(string.data(),
> string.size());
> + PyObject *bytes = PyUnicode_AsUTF8String(unicode);
> + PythonObject::Reset(bytes);
> + Py_XDECREF(unicode);
> +#else
> PythonObject::Reset(PyString_FromStringAndSize(string.data(),
> string.size()));
> +#endif
> }
>
> StructuredData::StringSP
> @@ -229,16 +279,44 @@ PythonInteger::~PythonInteger ()
> }
>
> bool
> -PythonInteger::Reset (PyObject *py_obj)
> +PythonInteger::Check(PyObject *py_obj)
> {
> - if (py_obj)
> + if (!py_obj)
> + return false;
> +
> +#if PY_MAJOR_VERSION >= 3
> + // Python 3 does not have PyInt_Check. There is only one type of
> + // integral value, long.
> + return PyLong_Check(py_obj);
> +#else
> + return PyLong_Check(py_obj) || PyInt_Check(py_obj);
> +#endif
> +}
> +
> +bool
> +PythonInteger::Reset(PyObject *py_obj)
> +{
> + if (!PythonInteger::Check(py_obj))
> {
> - if (PyInt_Check (py_obj) || PyLong_Check(py_obj))
> - return PythonObject::Reset(py_obj);
> + PythonObject::Reset(nullptr);
> + return false;
> }
> -
> - PythonObject::Reset(nullptr);
> - return py_obj == nullptr;
> +
> +#if PY_MAJOR_VERSION < 3
> + // Always store this as a PyLong, which makes interoperability between
> + // Python 2.x and Python 3.x easier. This is only necessary in 2.x,
> + // since 3.x doesn't even have a PyInt.
> + if (PyInt_Check(py_obj))
> + {
> + PyObject *py_long = PyLong_FromLongLong(PyInt_AsLong(py_obj));
> + Py_XDECREF(py_obj);
> + py_obj = py_long;
> + }
> +#endif
> +
> + assert(PyLong_Check(py_obj) && "Couldn't get a PyLong from this
> PyObject");
> +
> + return PythonObject::Reset(py_obj);
> }
>
> int64_t
> @@ -246,10 +324,9 @@ PythonInteger::GetInteger() const
> {
> if (m_py_obj)
> {
> - if (PyInt_Check(m_py_obj))
> - return PyInt_AsLong(m_py_obj);
> - else if (PyLong_Check(m_py_obj))
> - return PyLong_AsLongLong(m_py_obj);
> + assert(PyLong_Check(m_py_obj) && "PythonInteger::GetInteger has a
> PyObject that isn't a PyLong");
> +
> + return PyLong_AsLongLong(m_py_obj);
> }
> return UINT64_MAX;
> }
> @@ -272,13 +349,8 @@ PythonInteger::CreateStructuredInteger()
> // PythonList
> //----------------------------------------------------------------------
>
> -PythonList::PythonList (bool create_empty) :
> - PythonObject(create_empty ? PyList_New(0) : nullptr)
> -{
> -}
> -
> -PythonList::PythonList (uint32_t count) :
> - PythonObject(PyList_New(count))
> +PythonList::PythonList()
> + : PythonObject(PyList_New(0))
> {
> }
>
> @@ -300,13 +372,23 @@ PythonList::~PythonList ()
> }
>
> bool
> -PythonList::Reset (PyObject *py_obj)
> +PythonList::Check(PyObject *py_obj)
> +{
> + if (!py_obj)
> + return false;
> + return PyList_Check(py_obj);
> +}
> +
> +bool
> +PythonList::Reset(PyObject *py_obj)
> {
> - if (py_obj && PyList_Check(py_obj))
> - return PythonObject::Reset(py_obj);
> -
> - PythonObject::Reset(nullptr);
> - return py_obj == nullptr;
> + if (!PythonList::Check(py_obj))
> + {
> + PythonObject::Reset(nullptr);
> + return false;
> + }
> +
> + return PythonObject::Reset(py_obj);
> }
>
> uint32_t
> @@ -356,8 +438,8 @@ PythonList::CreateStructuredArray() cons
> // PythonDictionary
> //----------------------------------------------------------------------
>
> -PythonDictionary::PythonDictionary (bool create_empty) :
> -PythonObject(create_empty ? PyDict_New() : nullptr)
> +PythonDictionary::PythonDictionary()
> + : PythonObject(PyDict_New())
> {
> }
>
> @@ -379,13 +461,24 @@ PythonDictionary::~PythonDictionary ()
> }
>
> bool
> -PythonDictionary::Reset (PyObject *py_obj)
> +PythonDictionary::Check(PyObject *py_obj)
> {
> - if (py_obj && PyDict_Check(py_obj))
> - return PythonObject::Reset(py_obj);
> -
> - PythonObject::Reset(nullptr);
> - return py_obj == nullptr;
> + if (!py_obj)
> + return false;
> +
> + return PyDict_Check(py_obj);
> +}
> +
> +bool
> +PythonDictionary::Reset(PyObject *py_obj)
> +{
> + if (!PythonDictionary::Check(py_obj))
> + {
> + PythonObject::Reset(nullptr);
> + return false;
> + }
> +
> + return PythonObject::Reset(py_obj);
> }
>
> uint32_t
> @@ -423,8 +516,11 @@ PythonDictionary::GetItemForKeyAsString
> if (m_py_obj && key)
> {
> PyObject *py_obj = PyDict_GetItem(m_py_obj, key.get());
> - if (py_obj && PyString_Check(py_obj))
> - return PyString_AsString(py_obj);
> + if (py_obj && PythonString::Check(py_obj))
> + {
> + PythonString str(py_obj);
> + return str.GetString().data();
> + }
> }
> return fail_value;
> }
> @@ -435,13 +531,10 @@ PythonDictionary::GetItemForKeyAsInteger
> if (m_py_obj && key)
> {
> PyObject *py_obj = PyDict_GetItem(m_py_obj, key.get());
> - if (py_obj)
> + if (PythonInteger::Check(py_obj))
> {
> - if (PyInt_Check(py_obj))
> - return PyInt_AsLong(py_obj);
> -
> - if (PyLong_Check(py_obj))
> - return PyLong_AsLong(py_obj);
> + PythonInteger int_obj(py_obj);
> + return int_obj.GetInteger();
> }
> }
> return fail_value;
> @@ -452,7 +545,7 @@ PythonDictionary::GetKeys () const
> {
> if (m_py_obj)
> return PythonList(PyDict_Keys(m_py_obj));
> - return PythonList(true);
> + return PythonList();
> }
>
> PythonString
>
> Modified:
> lldb/trunk/source/Plugins/ScriptInterpreter/Python/PythonDataObjects.h
> URL:
> http://llvm.org/viewvc/llvm-project/lldb/trunk/source/Plugins/ScriptInterpreter/Python/PythonDataObjects.h?rev=249886&r1=249885&r2=249886&view=diff
>
> ==============================================================================
> --- lldb/trunk/source/Plugins/ScriptInterpreter/Python/PythonDataObjects.h
> (original)
> +++ lldb/trunk/source/Plugins/ScriptInterpreter/Python/PythonDataObjects.h
> Fri Oct 9 14:45:41 2015
> @@ -156,7 +156,7 @@ enum class PyObjectType
> protected:
> PyObject* m_py_obj;
> };
> -
> +
> class PythonString: public PythonObject
> {
> public:
> @@ -167,6 +167,8 @@ enum class PyObjectType
> PythonString (const char *string);
> virtual ~PythonString ();
>
> + static bool Check(PyObject *py_obj);
> +
> virtual bool
> Reset (PyObject* py_obj = NULL);
>
> @@ -180,7 +182,7 @@ enum class PyObjectType
>
> StructuredData::StringSP CreateStructuredString() const;
> };
> -
> +
> class PythonInteger: public PythonObject
> {
> public:
> @@ -190,7 +192,9 @@ enum class PyObjectType
> PythonInteger (const PythonObject &object);
> PythonInteger (int64_t value);
> virtual ~PythonInteger ();
> -
> +
> + static bool Check(PyObject *py_obj);
> +
> virtual bool
> Reset (PyObject* py_obj = NULL);
>
> @@ -205,13 +209,13 @@ enum class PyObjectType
> class PythonList: public PythonObject
> {
> public:
> -
> - PythonList (bool create_empty);
> + PythonList();
> PythonList (PyObject* py_obj);
> PythonList (const PythonObject &object);
> - PythonList (uint32_t count);
> virtual ~PythonList ();
> -
> +
> + static bool Check(PyObject *py_obj);
> +
> virtual bool
> Reset (PyObject* py_obj = NULL);
>
> @@ -231,12 +235,13 @@ enum class PyObjectType
> class PythonDictionary: public PythonObject
> {
> public:
> -
> - explicit PythonDictionary (bool create_empty);
> + PythonDictionary();
> PythonDictionary (PyObject* object);
> PythonDictionary (const PythonObject &object);
> virtual ~PythonDictionary ();
> -
> +
> + static bool Check(PyObject *py_obj);
> +
> virtual bool
> Reset (PyObject* object = NULL);
>
>
> Modified:
> lldb/trunk/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/lldb/trunk/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp?rev=249886&r1=249885&r2=249886&view=diff
>
> ==============================================================================
> ---
> lldb/trunk/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp
> (original)
> +++
> lldb/trunk/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp
> Fri Oct 9 14:45:41 2015
> @@ -46,6 +46,7 @@
> #endif
>
> #include "llvm/ADT/StringRef.h"
> +#include "llvm/ADT/STLExtras.h"
>
> using namespace lldb;
> using namespace lldb_private;
> @@ -79,6 +80,29 @@ static ScriptInterpreterPython::SWIGPyth
>
> static bool g_initialized = false;
>
> +#if PY_MAJOR_VERSION >= 3 && defined(LLDB_PYTHON_HOME)
> +typedef wchar_t PythonHomeChar;
> +#else
> +typedef char PythonHomeChar;
> +#endif
> +
> +PythonHomeChar *
> +GetDesiredPythonHome()
> +{
> +#if defined(LLDB_PYTHON_HOME)
> +#if PY_MAJOR_VERSION >= 3
> + size_t size = 0;
> + static PythonHomeChar *g_python_home =
> Py_DecodeLocale(LLDB_PYTHON_HOME, &size);
> + return g_python_home;
> +#else
> + static PythonHomeChar *g_python_home = LLDB_PYTHON_HOME;
> + return g_python_home;
> +#endif
> +#else
> + return nullptr;
> +#endif
> +}
> +
> static std::string
> ReadPythonBacktrace (PyObject* py_backtrace);
>
> @@ -116,7 +140,7 @@ ScriptInterpreterPython::Locker::DoAcqui
> // place outside of Python (e.g. printing to screen, waiting for the
> network, ...)
> // in that case, _PyThreadState_Current will be NULL - and we would
> be unable
> // to set the asynchronous exception - not a desirable situation
> - m_python_interpreter->SetThreadState (_PyThreadState_Current);
> + m_python_interpreter->SetThreadState(PyThreadState_Get());
> m_python_interpreter->IncrementLockCount();
> return true;
> }
> @@ -908,13 +932,13 @@ ScriptInterpreterPython::Interrupt()
>
> if (IsExecutingPython())
> {
> - PyThreadState* state = _PyThreadState_Current;
> + PyThreadState *state = PyThreadState_Get();
> if (!state)
> state = GetThreadState();
> if (state)
> {
> long tid = state->thread_id;
> - _PyThreadState_Current = state;
> + PyThreadState_Swap(state);
> int num_threads = PyThreadState_SetAsyncExc(tid,
> PyExc_KeyboardInterrupt);
> if (log)
> log->Printf("ScriptInterpreterPython::Interrupt() sending
> PyExc_KeyboardInterrupt (tid = %li, num_threads = %i)...", tid,
> num_threads);
> @@ -1124,15 +1148,16 @@ ScriptInterpreterPython::ExecuteMultiple
>
> if (in_string != nullptr)
> {
> - struct _node *compiled_node = PyParser_SimpleParseString
> (in_string, Py_file_input);
> +#if PY_MAJOR_VERSION >= 3
> + PyObject *code_object = Py_CompileString(in_string, "temp.py",
> Py_file_input);
> +#else
> + PyCodeObject *code_object = nullptr;
> + struct _node *compiled_node =
> PyParser_SimpleParseString(in_string, Py_file_input);
> if (compiled_node)
> - {
> - PyCodeObject *compiled_code = PyNode_Compile (compiled_node,
> "temp.py");
> - if (compiled_code)
> - {
> - return_value.Reset(PyEval_EvalCode (compiled_code,
> globals.get(), locals.get()));
> - }
> - }
> + code_object = PyNode_Compile(compiled_node, "temp.py");
> +#endif
> + if (code_object)
> + return_value.Reset(PyEval_EvalCode(code_object,
> globals.get(), locals.get()));
> }
>
> py_error = PyErr_Occurred ();
> @@ -1152,7 +1177,11 @@ ScriptInterpreterPython::ExecuteMultiple
> std::string bt = ReadPythonBacktrace(traceback);
>
> if (value && value != Py_None)
> - error.SetErrorStringWithFormat("%s\n%s",
> PyString_AsString(PyObject_Str(value)),bt.c_str());
> + {
> + PythonString str(value);
> + llvm::StringRef value_str(str.GetString());
> + error.SetErrorStringWithFormat("%s\n%s",
> value_str.str().c_str(), bt.c_str());
> + }
> else
> error.SetErrorStringWithFormat("%s",bt.c_str());
> Py_XDECREF(type);
> @@ -2342,8 +2371,12 @@ ReadPythonBacktrace (PyObject* py_backtr
> if (stringIO_getvalue && stringIO_getvalue !=
> Py_None)
> {
> printTB_string = PyObject_CallObject
> (stringIO_getvalue,nullptr);
> - if (printTB_string && printTB_string !=
> Py_None && PyString_Check(printTB_string))
> -
> retval.assign(PyString_AsString(printTB_string));
> + if (printTB_string &&
> PythonString::Check(printTB_string))
> + {
> + PythonString str(printTB_string);
> + llvm::StringRef
> string_data(str.GetString());
> + retval.assign(string_data.data(),
> string_data.size());
> + }
> }
> }
> }
> @@ -2924,14 +2957,13 @@ ScriptInterpreterPython::GetShortHelpFor
> PyErr_Print();
> PyErr_Clear();
> }
> -
> - if (py_return != nullptr && py_return != Py_None)
> +
> + if (py_return != Py_None && PythonString::Check(py_return))
> {
> - if (PyString_Check(py_return))
> - {
> - dest.assign(PyString_AsString(py_return));
> - got_string = true;
> - }
> + PythonString py_string(py_return);
> + llvm::StringRef return_data(py_string.GetString());
> + dest.assign(return_data.data(), return_data.size());
> + got_string = true;
> }
> Py_XDECREF(py_return);
>
> @@ -2997,13 +3029,11 @@ ScriptInterpreterPython::GetFlagsForComm
> PyErr_Print();
> PyErr_Clear();
> }
> -
> - if (py_return != nullptr && py_return != Py_None)
> +
> + if (py_return != Py_None && PythonInteger::Check(py_return))
> {
> - if (PyInt_Check(py_return))
> - result = (uint32_t)PyInt_AsLong(py_return);
> - else if (PyLong_Check(py_return))
> - result = (uint32_t)PyLong_AsLong(py_return);
> + PythonInteger int_value(py_return);
> + result = int_value.GetInteger();
> }
> Py_XDECREF(py_return);
>
> @@ -3071,14 +3101,13 @@ ScriptInterpreterPython::GetLongHelpForC
> PyErr_Print();
> PyErr_Clear();
> }
> -
> - if (py_return != nullptr && py_return != Py_None)
> +
> + if (py_return != Py_None && PythonString::Check(py_return))
> {
> - if (PyString_Check(py_return))
> - {
> - dest.assign(PyString_AsString(py_return));
> - got_string = true;
> - }
> + PythonString str(py_return);
> + llvm::StringRef str_data(str.GetString());
> + dest.assign(str_data.data(), str_data.size());
> + got_string = true;
> }
> Py_XDECREF(py_return);
>
> @@ -3164,8 +3193,9 @@ ScriptInterpreterPython::InitializePriva
> stdin_tty_state.Save(STDIN_FILENO, false);
>
> #if defined(LLDB_PYTHON_HOME)
> - Py_SetPythonHome(LLDB_PYTHON_HOME);
> + Py_SetPythonHome(GetDesiredPythonHome());
> #endif
> +
> PyGILState_STATE gstate;
> Log *log (lldb_private::GetLogIfAllCategoriesSet (LIBLLDB_LOG_SCRIPT
> | LIBLLDB_LOG_VERBOSE));
> bool threads_already_initialized = false;
>
> Modified: lldb/trunk/unittests/CMakeLists.txt
> URL:
> http://llvm.org/viewvc/llvm-project/lldb/trunk/unittests/CMakeLists.txt?rev=249886&r1=249885&r2=249886&view=diff
>
> ==============================================================================
> --- lldb/trunk/unittests/CMakeLists.txt (original)
> +++ lldb/trunk/unittests/CMakeLists.txt Fri Oct 9 14:45:41 2015
> @@ -25,4 +25,5 @@ endfunction()
>
> add_subdirectory(Host)
> add_subdirectory(Interpreter)
> +add_subdirectory(ScriptInterpreter)
> add_subdirectory(Utility)
>
> Added: lldb/trunk/unittests/ScriptInterpreter/CMakeLists.txt
> URL:
> http://llvm.org/viewvc/llvm-project/lldb/trunk/unittests/ScriptInterpreter/CMakeLists.txt?rev=249886&view=auto
>
> ==============================================================================
> --- lldb/trunk/unittests/ScriptInterpreter/CMakeLists.txt (added)
> +++ lldb/trunk/unittests/ScriptInterpreter/CMakeLists.txt Fri Oct 9
> 14:45:41 2015
> @@ -0,0 +1,3 @@
> +if (NOT LLDB_DISABLE_PYTHON)
> + add_subdirectory(Python)
> +endif()
>
> Added: lldb/trunk/unittests/ScriptInterpreter/Python/CMakeLists.txt
> URL:
> http://llvm.org/viewvc/llvm-project/lldb/trunk/unittests/ScriptInterpreter/Python/CMakeLists.txt?rev=249886&view=auto
>
> ==============================================================================
> --- lldb/trunk/unittests/ScriptInterpreter/Python/CMakeLists.txt (added)
> +++ lldb/trunk/unittests/ScriptInterpreter/Python/CMakeLists.txt Fri Oct
> 9 14:45:41 2015
> @@ -0,0 +1,6 @@
> +add_lldb_unittest(ScriptInterpreterPythonTests
> + PythonDataObjectsTests.cpp
> + )
> +
> + target_link_libraries(ScriptInterpreterPythonTests
> lldbPluginScriptInterpreterPython ${PYTHON_LIBRARY})
> +
> \ No newline at end of file
>
> Added:
> lldb/trunk/unittests/ScriptInterpreter/Python/PythonDataObjectsTests.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/lldb/trunk/unittests/ScriptInterpreter/Python/PythonDataObjectsTests.cpp?rev=249886&view=auto
>
> ==============================================================================
> ---
> lldb/trunk/unittests/ScriptInterpreter/Python/PythonDataObjectsTests.cpp
> (added)
> +++
> lldb/trunk/unittests/ScriptInterpreter/Python/PythonDataObjectsTests.cpp
> Fri Oct 9 14:45:41 2015
> @@ -0,0 +1,411 @@
> +//===-- PythonDataObjectsTests.cpp ------------------------------*- C++
> -*-===//
> +//
> +// The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
>
> +//===----------------------------------------------------------------------===//
> +
> +#include "gtest/gtest.h"
> +
> +#include "lldb/Host/HostInfo.h"
> +#include "Plugins/ScriptInterpreter/Python/lldb-python.h"
> +#include "Plugins/ScriptInterpreter/Python/PythonDataObjects.h"
> +#include "Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.h"
> +
> +using namespace lldb_private;
> +
> +class PythonDataObjectsTest : public testing::Test
> +{
> + public:
> + void
> + SetUp() override
> + {
> + HostInfoBase::Initialize();
> + // ScriptInterpreterPython::Initialize() depends on things like
> HostInfo being initialized
> + // so it can compute the python directory etc, so we need to do
> this after
> + // SystemInitializerCommon::Initialize().
> + ScriptInterpreterPython::Initialize();
> + }
> +
> + void
> + TearDown() override
> + {
> + ScriptInterpreterPython::Terminate();
> + }
> +};
> +
> +TEST_F(PythonDataObjectsTest, TestPythonInteger)
> +{
> +// Test that integers behave correctly when wrapped by a PythonInteger.
> +
> +#if PY_MAJOR_VERSION < 3
> + // Verify that `PythonInt` works correctly when given a PyInt object.
> + // Note that PyInt doesn't exist in Python 3.x, so this is only for
> 2.x
> + PyObject *py_int = PyInt_FromLong(12);
> + EXPECT_TRUE(PythonInteger::Check(py_int));
> + PythonInteger python_int(py_int);
> +
> + EXPECT_EQ(PyObjectType::Integer, python_int.GetObjectType());
> + EXPECT_EQ(12, python_int.GetInteger());
> +#endif
> +
> + // Verify that `PythonInt` works correctly when given a PyLong object.
> + PyObject *py_long = PyLong_FromLong(12);
> + EXPECT_TRUE(PythonInteger::Check(py_long));
> + PythonInteger python_long(py_long);
> + EXPECT_EQ(PyObjectType::Integer, python_long.GetObjectType());
> +
> + // Verify that you can reset the value and that it is reflected
> properly.
> + python_long.SetInteger(40);
> + EXPECT_EQ(40, python_long.GetInteger());
> +}
> +
> +TEST_F(PythonDataObjectsTest, TestPythonString)
> +{
> + // Test that strings behave correctly when wrapped by a PythonString.
> +
> + static const char *test_string =
> "PythonDataObjectsTest::TestPythonString";
> + static const char *test_string2 =
> "PythonDataObjectsTest::TestPythonString";
> +
> +#if PY_MAJOR_VERSION < 3
> + // Verify that `PythonString` works correctly when given a PyString
> object.
> + // Note that PyString doesn't exist in Python 3.x, so this is only
> for 2.x
> + PyObject *py_string = PyString_FromString(test_string);
> + EXPECT_TRUE(PythonString::Check(py_string));
> + PythonString python_string(py_string);
> +
> + EXPECT_EQ(PyObjectType::String, python_string.GetObjectType());
> + EXPECT_STREQ(test_string, python_string.GetString().data());
> +#endif
> +
> + // Verify that `PythonString` works correctly when given a PyUnicode
> object.
> + PyObject *py_unicode = PyUnicode_FromString(test_string);
> + EXPECT_TRUE(PythonString::Check(py_unicode));
> + PythonString python_unicode(py_unicode);
> +
> + EXPECT_EQ(PyObjectType::String, python_unicode.GetObjectType());
> + EXPECT_STREQ(test_string, python_unicode.GetString().data());
> +
> + // Verify that you can reset the value and that it is reflected
> properly.
> + python_unicode.SetString(test_string2);
> + EXPECT_STREQ(test_string2, python_unicode.GetString().data());
> +}
> +
> +TEST_F(PythonDataObjectsTest, TestPythonListPrebuilt)
> +{
> + // Test that a list which is built through the native
> + // Python API behaves correctly when wrapped by a PythonList.
> + static const int list_size = 2;
> + static const long long_idx0 = 5;
> + static const char *const string_idx1 = "String Index 1";
> +
> + PyObject *list_items[list_size];
> +
> + PyObject *py_list = PyList_New(2);
> + list_items[0] = PyLong_FromLong(long_idx0);
> + list_items[1] = PyString_FromString(string_idx1);
> +
> + for (int i = 0; i < list_size; ++i)
> + PyList_SetItem(py_list, i, list_items[i]);
> +
> + EXPECT_TRUE(PythonList::Check(py_list));
> +
> + PythonList list(py_list);
> + EXPECT_EQ(list_size, list.GetSize());
> + EXPECT_EQ(PyObjectType::List, list.GetObjectType());
> +
> + // PythonList doesn't yet support getting objects by type.
> + // For now, we have to call CreateStructuredArray and use
> + // those objects. That will be in a different test.
> + // TODO: Add the ability for GetItemByIndex() to return a
> + // typed object.
> +}
> +
> +TEST_F(PythonDataObjectsTest, TestPythonDictionaryPrebuilt)
> +{
> + // Test that a dictionary which is built through the native
> + // Python API behaves correctly when wrapped by a PythonDictionary.
> + static const int dict_entries = 2;
> +
> + PyObject *keys[dict_entries];
> + PyObject *values[dict_entries];
> +
> + keys[0] = PyString_FromString("Key 0");
> + keys[1] = PyLong_FromLong(1);
> + values[0] = PyLong_FromLong(0);
> + values[1] = PyString_FromString("Value 1");
> +
> + PyObject *py_dict = PyDict_New();
> + for (int i = 0; i < dict_entries; ++i)
> + PyDict_SetItem(py_dict, keys[i], values[i]);
> +
> + EXPECT_TRUE(PythonDictionary::Check(py_dict));
> +
> + PythonDictionary dict(py_dict);
> + EXPECT_EQ(dict.GetSize(), dict_entries);
> + EXPECT_EQ(PyObjectType::Dictionary, dict.GetObjectType());
> +
> + // PythonDictionary doesn't yet support getting objects by type.
> + // For now, we have to call CreateStructuredDictionary and use
> + // those objects. That will be in a different test.
> + // TODO: Add the ability for GetItemByKey() to return a
> + // typed object.
> +}
> +
> +TEST_F(PythonDataObjectsTest, TestPythonListManipulation)
> +{
> + // Test that manipulation of a PythonList behaves correctly when
> + // wrapped by a PythonDictionary.
> +
> + static const long long_idx0 = 5;
> + static const char *const string_idx1 = "String Index 1";
> +
> + PyObject *py_list = PyList_New(0);
> + PythonList list(py_list);
> + PythonInteger integer(long_idx0);
> + PythonString string(string_idx1);
> +
> + list.AppendItem(integer);
> + list.AppendItem(string);
> + EXPECT_EQ(2, list.GetSize());
> +
> + // PythonList doesn't yet support getting typed objects out, so we
> + // can't easily test that the first item is an integer with the
> correct
> + // value, etc.
> + // TODO: Add the ability for GetItemByIndex() to return a
> + // typed object.
> +}
> +
> +TEST_F(PythonDataObjectsTest, TestPythonDictionaryManipulation)
> +{
> + // Test that manipulation of a dictionary behaves correctly when
> wrapped
> + // by a PythonDictionary.
> + static const int dict_entries = 2;
> +
> + PyObject *keys[dict_entries];
> + PyObject *values[dict_entries];
> +
> + keys[0] = PyString_FromString("Key 0");
> + keys[1] = PyString_FromString("Key 1");
> + values[0] = PyLong_FromLong(1);
> + values[1] = PyString_FromString("Value 1");
> +
> + PyObject *py_dict = PyDict_New();
> +
> + PythonDictionary dict(py_dict);
> + for (int i = 0; i < 2; ++i)
> + dict.SetItemForKey(PythonString(keys[i]), values[i]);
> +
> + EXPECT_EQ(dict_entries, dict.GetSize());
> +
> + // PythonDictionary doesn't yet support getting objects by type.
> + // For now, we have to call CreateStructuredDictionary and use
> + // those objects. That will be in a different test.
> + // TODO: Add the ability for GetItemByKey() to return a
> + // typed object.
> +}
> +
> +TEST_F(PythonDataObjectsTest, TestPythonListToStructuredObject)
> +{
> + // Test that a PythonList is properly converted to a StructuredArray.
> + // This includes verifying that a list can contain a nested list as
> + // well as a nested dictionary.
> +
> + static const int item_count = 4;
> + static const long long_idx0 = 5;
> + static const char *const string_idx1 = "String Index 1";
> +
> + static const long nested_list_long_idx0 = 6;
> + static const char *const nested_list_str_idx1 = "Nested String Index
> 1";
> +
> + static const char *const nested_dict_key0 = "Nested Key 0";
> + static const char *const nested_dict_value0 = "Nested Value 0";
> + static const char *const nested_dict_key1 = "Nested Key 1";
> + static const long nested_dict_value1 = 2;
> +
> + PythonList list;
> + PythonList nested_list;
> + PythonDictionary nested_dict;
> +
> + nested_list.AppendItem(PythonInteger(nested_list_long_idx0));
> + nested_list.AppendItem(PythonString(nested_list_str_idx1));
> + nested_dict.SetItemForKey(PythonString(nested_dict_key0),
> PythonString(nested_dict_value0));
> + nested_dict.SetItemForKey(PythonString(nested_dict_key1),
> PythonInteger(nested_dict_value1));
> +
> + list.AppendItem(PythonInteger(long_idx0));
> + list.AppendItem(PythonString(string_idx1));
> + list.AppendItem(nested_list);
> + list.AppendItem(nested_dict);
> +
> + EXPECT_EQ(item_count, list.GetSize());
> +
> + StructuredData::ArraySP array_sp = list.CreateStructuredArray();
> + EXPECT_EQ(list.GetSize(), array_sp->GetSize());
> + EXPECT_EQ(StructuredData::Type::eTypeInteger,
> array_sp->GetItemAtIndex(0)->GetType());
> + EXPECT_EQ(StructuredData::Type::eTypeString,
> array_sp->GetItemAtIndex(1)->GetType());
> + EXPECT_EQ(StructuredData::Type::eTypeArray,
> array_sp->GetItemAtIndex(2)->GetType());
> + EXPECT_EQ(StructuredData::Type::eTypeDictionary,
> array_sp->GetItemAtIndex(3)->GetType());
> +
> + auto list_int_sp =
> std::static_pointer_cast<StructuredData::Integer>(array_sp->GetItemAtIndex(0));
> + auto list_str_sp =
> std::static_pointer_cast<StructuredData::String>(array_sp->GetItemAtIndex(1));
> + auto list_list_sp =
> std::static_pointer_cast<StructuredData::Array>(array_sp->GetItemAtIndex(2));
> + auto list_dict_sp =
> std::static_pointer_cast<StructuredData::Dictionary>(array_sp->GetItemAtIndex(3));
> +
> + // Verify that the first item (long) has the correct value
> + EXPECT_EQ(long_idx0, list_int_sp->GetValue());
> +
> + // Verify that the second item (string) has the correct value
> + EXPECT_STREQ(string_idx1, list_str_sp->GetValue().c_str());
> +
> + // Verify that the third item is a list with the correct length and
> element types
> + EXPECT_EQ(nested_list.GetSize(), list_list_sp->GetSize());
> + EXPECT_EQ(StructuredData::Type::eTypeInteger,
> list_list_sp->GetItemAtIndex(0)->GetType());
> + EXPECT_EQ(StructuredData::Type::eTypeString,
> list_list_sp->GetItemAtIndex(1)->GetType());
> + // Verify that the values of each element in the list are correct
> + auto nested_list_value_0 =
> std::static_pointer_cast<StructuredData::Integer>(list_list_sp->GetItemAtIndex(0));
> + auto nested_list_value_1 =
> std::static_pointer_cast<StructuredData::String>(list_list_sp->GetItemAtIndex(1));
> + EXPECT_EQ(nested_list_long_idx0, nested_list_value_0->GetValue());
> + EXPECT_STREQ(nested_list_str_idx1,
> nested_list_value_1->GetValue().c_str());
> +
> + // Verify that the fourth item is a dictionary with the correct length
> + EXPECT_EQ(nested_dict.GetSize(), list_dict_sp->GetSize());
> + auto dict_keys =
> std::static_pointer_cast<StructuredData::Array>(list_dict_sp->GetKeys());
> +
> + // Verify that all of the keys match the values and types of keys we
> inserted
> + EXPECT_EQ(StructuredData::Type::eTypeString,
> dict_keys->GetItemAtIndex(0)->GetType());
> + EXPECT_EQ(StructuredData::Type::eTypeString,
> dict_keys->GetItemAtIndex(1)->GetType());
> + auto nested_key_0 =
> std::static_pointer_cast<StructuredData::String>(dict_keys->GetItemAtIndex(0));
> + auto nested_key_1 =
> std::static_pointer_cast<StructuredData::String>(dict_keys->GetItemAtIndex(1));
> + EXPECT_STREQ(nested_dict_key0, nested_key_0->GetValue().c_str());
> + EXPECT_STREQ(nested_dict_key1, nested_key_1->GetValue().c_str());
> +
> + // Verify that for each key, the value has the correct type and value
> as what we inserted.
> + auto nested_dict_value_0 =
> list_dict_sp->GetValueForKey(nested_key_0->GetValue());
> + auto nested_dict_value_1 =
> list_dict_sp->GetValueForKey(nested_key_1->GetValue());
> + EXPECT_EQ(StructuredData::Type::eTypeString,
> nested_dict_value_0->GetType());
> + EXPECT_EQ(StructuredData::Type::eTypeInteger,
> nested_dict_value_1->GetType());
> + auto nested_dict_str_value_0 =
> std::static_pointer_cast<StructuredData::String>(nested_dict_value_0);
> + auto nested_dict_int_value_1 =
> std::static_pointer_cast<StructuredData::Integer>(nested_dict_value_1);
> + EXPECT_STREQ(nested_dict_value0,
> nested_dict_str_value_0->GetValue().c_str());
> + EXPECT_EQ(nested_dict_value1, nested_dict_int_value_1->GetValue());
> +}
> +
> +TEST_F(PythonDataObjectsTest, TestPythonDictionaryToStructuredObject)
> +{
> + // Test that a PythonDictionary is properly converted to a
> + // StructuredDictionary. This includes verifying that a dictionary
> + // can contain a nested dictionary as well as a nested list.
> +
> + static const int dict_item_count = 4;
> + static const char *const dict_keys[dict_item_count] = {"Key 0 (str)",
> "Key 1 (long)", "Key 2 (dict)",
> + "Key 3
> (list)"};
> +
> + static const StructuredData::Type dict_value_types[dict_item_count] =
> {
> + StructuredData::Type::eTypeString,
> StructuredData::Type::eTypeInteger, StructuredData::Type::eTypeDictionary,
> + StructuredData::Type::eTypeArray};
> +
> + static const char *const nested_dict_keys[2] = {"Nested Key 0 (str)",
> "Nested Key 1 (long)"};
> +
> + static const StructuredData::Type nested_dict_value_types[2] = {
> + StructuredData::Type::eTypeString,
> StructuredData::Type::eTypeInteger,
> + };
> +
> + static const StructuredData::Type nested_list_value_types[2] =
> {StructuredData::Type::eTypeInteger,
> +
> StructuredData::Type::eTypeString};
> +
> + static const char *const dict_value0 = "Value 0";
> + static const long dict_value1 = 2;
> +
> + static const long nested_list_value0 = 5;
> + static const char *const nested_list_value1 = "Nested list string";
> +
> + static const char *const nested_dict_value0 = "Nested Dict Value 0";
> + static const long nested_dict_value1 = 7;
> +
> + PythonDictionary dict;
> + PythonDictionary nested_dict;
> + PythonList nested_list;
> +
> + nested_dict.SetItemForKey(PythonString(nested_dict_keys[0]),
> PythonString(nested_dict_value0));
> + nested_dict.SetItemForKey(PythonString(nested_dict_keys[1]),
> PythonInteger(nested_dict_value1));
> +
> + nested_list.AppendItem(PythonInteger(nested_list_value0));
> + nested_list.AppendItem(PythonString(nested_list_value1));
> +
> + dict.SetItemForKey(PythonString(dict_keys[0]),
> PythonString(dict_value0));
> + dict.SetItemForKey(PythonString(dict_keys[1]),
> PythonInteger(dict_value1));
> + dict.SetItemForKey(PythonString(dict_keys[2]), nested_dict);
> + dict.SetItemForKey(PythonString(dict_keys[3]), nested_list);
> +
> + StructuredData::DictionarySP dict_sp =
> dict.CreateStructuredDictionary();
> + EXPECT_EQ(dict_item_count, dict_sp->GetSize());
> + auto dict_keys_array =
> std::static_pointer_cast<StructuredData::Array>(dict_sp->GetKeys());
> +
> + std::vector<StructuredData::StringSP> converted_keys;
> + std::vector<StructuredData::ObjectSP> converted_values;
> + // Verify that all of the keys match the values and types of keys we
> inserted
> + // (Keys are always strings, so this is easy)
> + for (int i = 0; i < dict_sp->GetSize(); ++i)
> + {
> + EXPECT_EQ(StructuredData::Type::eTypeString,
> dict_keys_array->GetItemAtIndex(i)->GetType());
> + auto converted_key =
> std::static_pointer_cast<StructuredData::String>(dict_keys_array->GetItemAtIndex(i));
> + converted_keys.push_back(converted_key);
> +
> converted_values.push_back(dict_sp->GetValueForKey(converted_key->GetValue().c_str()));
> +
> + EXPECT_STREQ(dict_keys[i], converted_key->GetValue().c_str());
> + EXPECT_EQ(dict_value_types[i], converted_values[i]->GetType());
> + }
> +
> + auto dict_string_value =
> std::static_pointer_cast<StructuredData::String>(converted_values[0]);
> + auto dict_int_value =
> std::static_pointer_cast<StructuredData::Integer>(converted_values[1]);
> + auto dict_dict_value =
> std::static_pointer_cast<StructuredData::Dictionary>(converted_values[2]);
> + auto dict_list_value =
> std::static_pointer_cast<StructuredData::Array>(converted_values[3]);
> +
> + // The first two dictionary values are easy to test, because they are
> just a string and an integer.
> + EXPECT_STREQ(dict_value0, dict_string_value->GetValue().c_str());
> + EXPECT_EQ(dict_value1, dict_int_value->GetValue());
> +
> + // For the nested dictionary, repeat the same process as before.
> + EXPECT_EQ(2, dict_dict_value->GetSize());
> + auto nested_dict_keys_array =
> std::static_pointer_cast<StructuredData::Array>(dict_dict_value->GetKeys());
> +
> + std::vector<StructuredData::StringSP> nested_converted_keys;
> + std::vector<StructuredData::ObjectSP> nested_converted_values;
> + // Verify that all of the keys match the values and types of keys we
> inserted
> + // (Keys are always strings, so this is easy)
> + for (int i = 0; i < dict_dict_value->GetSize(); ++i)
> + {
> + EXPECT_EQ(StructuredData::Type::eTypeString,
> nested_dict_keys_array->GetItemAtIndex(i)->GetType());
> + auto converted_key =
> +
> std::static_pointer_cast<StructuredData::String>(nested_dict_keys_array->GetItemAtIndex(i));
> + nested_converted_keys.push_back(converted_key);
> +
> nested_converted_values.push_back(dict_dict_value->GetValueForKey(converted_key->GetValue().c_str()));
> +
> + EXPECT_STREQ(nested_dict_keys[i],
> converted_key->GetValue().c_str());
> + EXPECT_EQ(nested_dict_value_types[i],
> converted_values[i]->GetType());
> + }
> +
> + auto converted_nested_dict_value_0 =
> std::static_pointer_cast<StructuredData::String>(nested_converted_values[0]);
> + auto converted_nested_dict_value_1 =
> std::static_pointer_cast<StructuredData::Integer>(nested_converted_values[1]);
> +
> + // The first two dictionary values are easy to test, because they are
> just a string and an integer.
> + EXPECT_STREQ(nested_dict_value0,
> converted_nested_dict_value_0->GetValue().c_str());
> + EXPECT_EQ(nested_dict_value1,
> converted_nested_dict_value_1->GetValue());
> +
> + // For the nested list, just verify the size, type and value of each
> item
> + nested_converted_values.clear();
> + EXPECT_EQ(2, dict_list_value->GetSize());
> + for (int i = 0; i < dict_list_value->GetSize(); ++i)
> + {
> + auto converted_value = dict_list_value->GetItemAtIndex(i);
> + EXPECT_EQ(nested_list_value_types[i], converted_value->GetType());
> + nested_converted_values.push_back(converted_value);
> + }
> +
> + auto converted_nested_list_value_0 =
> std::static_pointer_cast<StructuredData::Integer>(nested_converted_values[0]);
> + auto converted_nested_list_value_1 =
> std::static_pointer_cast<StructuredData::String>(nested_converted_values[1]);
> + EXPECT_EQ(nested_list_value0,
> converted_nested_list_value_0->GetValue());
> + EXPECT_STREQ(nested_list_value1,
> converted_nested_list_value_1->GetValue().c_str());
> +}
>
>
> _______________________________________________
> lldb-commits mailing list
> lldb-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-commits/attachments/20151009/6f0cbff0/attachment-0001.html>
More information about the lldb-commits
mailing list