[lldb-dev] Segfault using the lldb python module with a non-Xcode python binary

Thu Nov 14 09:17:47 PST 2019

Hey António,

If I understand correctly, you're trying to mix between two versions
of the Python interpreter and library. That's not something that's
supported and has always been an issue. Internally we get the
occasional bug report where somebody install python via homebrew or
python.org and the corresponding interpreter ends up first in their
path because /usr/local/bin comes before /usr/bin. The same is true if
you build your own LLDB, if you link against Homebrew's Python3 dylib,
you need to use the Homebrew interpreter. In CMake we try really hard
to ensure those two are in sync for when we run the (Python) test
suite.

On macOS Catalina, there's a shim for python3 in /usr/bin/ that will
launch the interpreter from Xcode, which matches what the LLDB from
Xcode is linked against.

Given that this is an issue that comes up frequently, I'm going to add
a bit of documentation about this on the LLDB website.

Best,
Jonas

On Wed, Nov 13, 2019 at 10:53 PM António Afonso via lldb-dev
<lldb-dev at lists.llvm.org> wrote:
>
> I'm building lldb with python3 support by using the framework that is shipped with the latest versions of Xcode.
>
> I'm able to build and run lldb just fine but if I try to use the lldb python module on a python binary that is not the one from Xcode it segfaults when creating the module. I then tried with the stock lldb from Xcode and found the exact same issue ☹. I don’t think this was a problem before?
>
>
>
> I'm not sure why this happens and I wasn't able to debug the issue. I've already tried with a binary that has the exact same version of python but still the same problem:
>
>
>
> Works fine with the Xcode binary:
>
> $ PYTHONPATH=`lldb -P` /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/bin/python3
>
> Python 3.7.3 (default, Sep 18 2019, 14:29:06)
>
> [Clang 11.0.0 (clang-1100.0.33.8)] on darwin
>
> Type "help", "copyright", "credits" or "license" for more information.
>
> >>> import lldb
>
> >>>
>
>
>
> Fails with any other:
>
> $ PYTHONPATH=`lldb -P` /Users/aadsm/.pyenv/versions/3.7.3/bin/python
>
> Python 3.7.3 (default, Nov 12 2019, 23:19:54)
>
> [Clang 11.0.0 (clang-1100.0.33.8)] on darwin
>
> Type "help", "copyright", "credits" or "license" for more information.
>
> >>> import lldb
>
> Segmentation fault: 11
>
>
>
> I attached lldb to see where it was failing and it's right after liblldb is loaded and python is trying to create the module itself, in the PyModule_Create2 function (https://github.com/python/cpython/blob/master/Objects/moduleobject.c#L173-L179).
>
> The disassembly shows:
>
>
>
> Process 89097 stopped
>
> * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x10)
>
>     frame #0: 0x000000010f4cae5b Python3`PyModule_Create2 + 27
>
> Python3`PyModule_Create2:
>
> ->  0x10f4cae5b <+27>: movq   0x10(%rax), %rdi
>
>     0x10f4cae5f <+31>: callq  0x10f5823b0               ; _PyImport_IsInitialized
>
>     0x10f4cae64 <+36>: testl  %eax, %eax
>
>     0x10f4cae66 <+38>: je     0x10f4cae77               ; <+55>
>
> Target 0: (Python) stopped.
>
> (lldb) dis
>
> Python3`PyModule_Create2:
>
>     0x10f4cae40 <+0>:  pushq  %rbp
>
>     0x10f4cae41 <+1>:  movq   %rsp, %rbp
>
>     0x10f4cae44 <+4>:  pushq  %r14
>
>     0x10f4cae46 <+6>:  pushq  %rbx
>
>     0x10f4cae47 <+7>:  movl   %esi, %r14d
>
>     0x10f4cae4a <+10>: movq   %rdi, %rbx
>
>     0x10f4cae4d <+13>: leaq   0x2226ac(%rip), %rax      ; _PyRuntime
>
>     0x10f4cae54 <+20>: movq   0x5a0(%rax), %rax
>
> ->  0x10f4cae5b <+27>: movq   0x10(%rax), %rdi
>
>     0x10f4cae5f <+31>: callq  0x10f5823b0               ; _PyImport_IsInitialized
>
>     0x10f4cae64 <+36>: testl  %eax, %eax
>
>     0x10f4cae66 <+38>: je     0x10f4cae77               ; <+55>
>
>     0x10f4cae68 <+40>: movq   %rbx, %rdi
>
>     0x10f4cae6b <+43>: movl   %r14d, %esi
>
>     0x10f4cae6e <+46>: popq   %rbx
>
>     0x10f4cae6f <+47>: popq   %r14
>
>     0x10f4cae71 <+49>: popq   %rbp
>
>     0x10f4cae72 <+50>: jmp    0x10f4cae90               ; _PyModule_CreateInitialized
>
>     0x10f4cae77 <+55>: leaq   0x14f111(%rip), %rdi      ; "Python import machinery not initialized"
>
>     0x10f4cae7e <+62>: callq  0x10f593d40               ; Py_FatalError
>
>     0x10f4cae83 <+67>: nopw   %cs:(%rax,%rax)
>
>     0x10f4cae8d <+77>: nopl   (%rax)
>
>
>
> Not really sure how to debug this besides trying to build my own version of python and see if I can repro (I don't have this issue on linux). I’ve also checked the sys.abiflags and both binaries have the same ones.
>
> Has anyone experienced this before or has any pointers to debug it?
>
> - Afonso
>
> --
> Best regards,
> António Afonso
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev