[all-commits] [llvm/llvm-project] a65985: [LLDB] Fix Python GIL-not-held issues

[Ralf W. Grosse-Kunstleve via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: a6598575f4bc20f9a01c2bced2d0b1ff14d7576f
      https://github.com/llvm/llvm-project/commit/a6598575f4bc20f9a01c2bced2d0b1ff14d7576f
  Author: Ralf Grosse-Kunstleve <rwgk at google.com>
  Date:   2022-01-17 (Mon, 17 Jan 2022)

  Changed paths:
    M lldb/source/Plugins/ScriptInterpreter/Python/PythonDataObjects.cpp
    M lldb/source/Plugins/ScriptInterpreter/Python/PythonDataObjects.h
    M lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp

  Log Message:
  -----------
  [LLDB] Fix Python GIL-not-held issues

The GIL must be held when calling any Python C API functions. In multithreaded applications that use callbacks this requirement can easily be violated by accident. A general tool to ensure GIL health is not available, but patching Python Py_INCREF to add an assert provides a basic health check:

```
+int PyGILState_Check(void); /* Include/internal/pystate.h */
+
 #define Py_INCREF(op) (                         \
+    assert(PyGILState_Check()),                 \
     _Py_INC_REFTOTAL  _Py_REF_DEBUG_COMMA       \
     ((PyObject *)(op))->ob_refcnt++)

 #define Py_DECREF(op)                                   \
     do {                                                \
+        assert(PyGILState_Check());                     \
         PyObject *_py_decref_tmp = (PyObject *)(op);    \
         if (_Py_DEC_REFTOTAL  _Py_REF_DEBUG_COMMA       \
         --(_py_decref_tmp)->ob_refcnt != 0)             \
```

Adding this assertion causes around 50 test failures in LLDB. Adjusting the scope of things guarded by `py_lock` fixes them.

More background: https://docs.python.org/3/glossary.html#term-global-interpreter-lock

Patch by Ralf Grosse-Kunstleve

Differential Revision: https://reviews.llvm.org/D114722