[lldb-dev] Segfault using the lldb python module with a non-Xcode python binary

Thu Nov 14 10:00:59 PST 2019

I've put up a patch to extend the documentation: https://reviews.llvm.org/D70252

Please have a look let me know if you have any comments or suggestions!

On Thu, Nov 14, 2019 at 9:17 AM Jonas Devlieghere <jonas at devlieghere.com> wrote:
>
> Hey António,
>
> If I understand correctly, you're trying to mix between two versions
> of the Python interpreter and library. That's not something that's
> supported and has always been an issue. Internally we get the
> occasional bug report where somebody install python via homebrew or
> python.org and the corresponding interpreter ends up first in their
> path because /usr/local/bin comes before /usr/bin. The same is true if
> you build your own LLDB, if you link against Homebrew's Python3 dylib,
> you need to use the Homebrew interpreter. In CMake we try really hard
> to ensure those two are in sync for when we run the (Python) test
> suite.
>
> On macOS Catalina, there's a shim for python3 in /usr/bin/ that will
> launch the interpreter from Xcode, which matches what the LLDB from
> Xcode is linked against.
>
> Given that this is an issue that comes up frequently, I'm going to add
> a bit of documentation about this on the LLDB website.
>
> Best,
> Jonas
>
> On Wed, Nov 13, 2019 at 10:53 PM António Afonso via lldb-dev
> <lldb-dev at lists.llvm.org> wrote:
> >
> > I'm building lldb with python3 support by using the framework that is shipped with the latest versions of Xcode.
> >
> > I'm able to build and run lldb just fine but if I try to use the lldb python module on a python binary that is not the one from Xcode it segfaults when creating the module. I then tried with the stock lldb from Xcode and found the exact same issue ☹. I don’t think this was a problem before?
> >
> >
> >
> > I'm not sure why this happens and I wasn't able to debug the issue. I've already tried with a binary that has the exact same version of python but still the same problem:
> >
> >
> >
> > Works fine with the Xcode binary:
> >
> > $ PYTHONPATH=`lldb -P` /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/bin/python3
> >
> > Python 3.7.3 (default, Sep 18 2019, 14:29:06)
> >
> > [Clang 11.0.0 (clang-1100.0.33.8)] on darwin
> >
> > Type "help", "copyright", "credits" or "license" for more information.
> >
> > >>> import lldb
> >
> > >>>
> >
> >
> >
> > Fails with any other:
> >
> > $ PYTHONPATH=`lldb -P` /Users/aadsm/.pyenv/versions/3.7.3/bin/python
> >
> > Python 3.7.3 (default, Nov 12 2019, 23:19:54)
> >
> > [Clang 11.0.0 (clang-1100.0.33.8)] on darwin
> >
> > Type "help", "copyright", "credits" or "license" for more information.
> >
> > >>> import lldb
> >
> > Segmentation fault: 11
> >
> >
> >
> > I attached lldb to see where it was failing and it's right after liblldb is loaded and python is trying to create the module itself, in the PyModule_Create2 function (https://github.com/python/cpython/blob/master/Objects/moduleobject.c#L173-L179).
> >
> > The disassembly shows:
> >
> >
> >
> > Process 89097 stopped
> >
> > * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x10)
> >
> >     frame #0: 0x000000010f4cae5b Python3`PyModule_Create2 + 27
> >
> > Python3`PyModule_Create2:
> >
> > ->  0x10f4cae5b <+27>: movq   0x10(%rax), %rdi
> >
> >     0x10f4cae5f <+31>: callq  0x10f5823b0               ; _PyImport_IsInitialized
> >
> >     0x10f4cae64 <+36>: testl  %eax, %eax
> >
> >     0x10f4cae66 <+38>: je     0x10f4cae77               ; <+55>
> >
> > Target 0: (Python) stopped.
> >
> > (lldb) dis
> >
> > Python3`PyModule_Create2:
> >
> >     0x10f4cae40 <+0>:  pushq  %rbp
> >
> >     0x10f4cae41 <+1>:  movq   %rsp, %rbp
> >
> >     0x10f4cae44 <+4>:  pushq  %r14
> >
> >     0x10f4cae46 <+6>:  pushq  %rbx
> >
> >     0x10f4cae47 <+7>:  movl   %esi, %r14d
> >
> >     0x10f4cae4a <+10>: movq   %rdi, %rbx
> >
> >     0x10f4cae4d <+13>: leaq   0x2226ac(%rip), %rax      ; _PyRuntime
> >
> >     0x10f4cae54 <+20>: movq   0x5a0(%rax), %rax
> >
> > ->  0x10f4cae5b <+27>: movq   0x10(%rax), %rdi
> >
> >     0x10f4cae5f <+31>: callq  0x10f5823b0               ; _PyImport_IsInitialized
> >
> >     0x10f4cae64 <+36>: testl  %eax, %eax
> >
> >     0x10f4cae66 <+38>: je     0x10f4cae77               ; <+55>
> >
> >     0x10f4cae68 <+40>: movq   %rbx, %rdi
> >
> >     0x10f4cae6b <+43>: movl   %r14d, %esi
> >
> >     0x10f4cae6e <+46>: popq   %rbx
> >
> >     0x10f4cae6f <+47>: popq   %r14
> >
> >     0x10f4cae71 <+49>: popq   %rbp
> >
> >     0x10f4cae72 <+50>: jmp    0x10f4cae90               ; _PyModule_CreateInitialized
> >
> >     0x10f4cae77 <+55>: leaq   0x14f111(%rip), %rdi      ; "Python import machinery not initialized"
> >
> >     0x10f4cae7e <+62>: callq  0x10f593d40               ; Py_FatalError
> >
> >     0x10f4cae83 <+67>: nopw   %cs:(%rax,%rax)
> >
> >     0x10f4cae8d <+77>: nopl   (%rax)
> >
> >
> >
> > Not really sure how to debug this besides trying to build my own version of python and see if I can repro (I don't have this issue on linux). I’ve also checked the sys.abiflags and both binaries have the same ones.
> >
> > Has anyone experienced this before or has any pointers to debug it?
> >
> > - Afonso
> >
> > --
> > Best regards,
> > António Afonso
> > _______________________________________________
> > lldb-dev mailing list
> > lldb-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev