[lldb-dev] Problems running the lldb tests on Fedora Core 20 - 64bit (email originally titled "Add kalimba as a platform.")

Zachary Turner zturner at google.com
Mon Jul 21 07:14:53 PDT 2014 

My guess is that Fedora's python build reports a different location for
where to install the libraries to.  You can debug this by changing the
CMake to run sh -x when it calls the finalize swig wrapper classes shell
scripts.  It might also be worth seeing what happens if you define the
CMake variable to use the new python-based SWIG API generation.  AFAIK
nobody's ever tested this on non-Windows, but might be worth checking out.


On Mon, Jul 21, 2014 at 7:03 AM, Todd Fiala <tfiala at google.com> wrote:

> Hey Matthew,
>
> Great investigating!  Glad you were able to at least find the core of the
> issue, which does look like at least on Fedora Core 20 x86_64 we are doing
> something funky with configure/make.  I'm not sure why we're choosing lib64
> there and lib on Ubuntu 14.04 x86_64, might have to do with how the python
> is configured.  I think that directory may be coming from a helper library
> in python that determines the correct location for build output and has to
> do with some Python finalizer scripts during the build process.  I won't be
> totally surprised if this might just be broken on Fedora Core x86_64
> regardless of cmake or configure-based builds.
>
> I'll do a configure/make build on Ubuntu 14.04 x86_64 later today to see
> if I'm getting the same results on configure/make - there has been some
> changes in python handling more recently, generally to support Windows
> builds, but we may have unintentionally introduced that via those changes.
>  At least that way we can see if its consistent with configure/make or if
> it might just be specific to Fedora Core.
>
> Thanks for identifying that!
>
> -Todd
>
>
>
> On Mon, Jul 21, 2014 at 6:51 AM, Matthew Gardiner <mg11 at csr.com> wrote:
>
>> Todd,
>>
>> I've since discovered that the cmake/ninja build system also results in a
>> lib and lib64 directory, with the liblldb.so and other .sos in the lib
>> folder but the python2.7/site-packages/lldb under lib64. (In fact with
>> cmake build I see no python2.7 folder at all under lib, it's all under
>> lib64).
>>
>> So Todd, focusing back on autoconf/make builds, after doing the
>> ./configure && make in the sibling build directory, do you see for your
>> linux 64-bit builds the python2.7/site-packages/lldb under the lib64
>> sub-folder and the lldb/llvm .sos under the lib64 sub-folder?
>>
>> How do you get this to work:
>>
>>
>> tfiala at tfiala2:/mnt/ssd/work/macosx.sync/mbp-svn/build-debug/lib$
>> PYTHONPATH=`pwd`/python2.7/site-packages python
>> Python 2.7.6 (default, Mar 22 2014, 22:59:56)
>> [GCC 4.8.2] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>> >>> import lldb
>> >>> dir(lldb)
>>
>> Do you run another script or make target after doing the basic llvm/lldb
>> make?  That is to move python2.7/site-packages/lldb underneath lib. I think
>> that's what I'd have to do on my system, since after the basic make my lib
>> and lib64 directories have the following contents:
>>
>> ~/src/p4play/build/Debug+Asserts/lib/python2.7/site-packages
>>
>> $ ls -al
>> total 32
>> drwxrwxr-x 2 mg11 mg11  4096 Jul 21 14:40 .
>> drwxrwxr-x 3 mg11 mg11  4096 Jul 21 14:40 ..
>> -rwxr-xr-x 1 mg11 mg11 21795 Jul 21 14:40 readline.so
>>
>> $ cd ../../../lib64/python2.7/site-packages/
>>
>> ~/src/p4play/build/Debug+Asserts/lib64/python2.7/site-packages
>>
>> $ ls -al
>> total 12
>> drwxrwxr-x 3 mg11 mg11 4096 Jul 21 14:42 .
>> drwxrwxr-x 3 mg11 mg11 4096 Jul 21 14:42 ..
>> drwxrwxr-x 5 mg11 mg11 4096 Jul 21 14:42 lldb
>>
>>
>> Any ideas?
>> Matt
>>
>>
>> Matthew Gardiner wrote:
>>
>>> Hi Todd,
>>>
>>> Thanks for taking the time to come up with these tips. First off my
>>> distro is Fedora Core 20 64-bit. So to further my investigation I blew away
>>> all the sources, and did svn co llvm, (lldb clang into llvm/tools), made
>>> the sibling build dir, then did:
>>>
>>> ~/src/p4play/build
>>> $ ../llvm/configure --enable-targets=x86_64 --enable-shared
>>> --enable-cxx11 --prefix=`pwd`/../install && make -j 8
>>>
>>> After the successful build I do ldd. Basically lld finds all
>>> dependencies for liblldb.so...
>>>
>>> ~/src/p4play/build/Debug+Asserts/lib
>>> $ ldd liblldb.so
>>>     linux-vdso.so.1 =>  (0x00007fffa61fe000)
>>>     libLLVM-3.5svn.so => /home/mg11/src/p4play/build/
>>> Debug+Asserts/lib/./libLLVM-3.5svn.so (0x00007f2e53405000)
>>>     libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f2e531cc000)
>>>     libdl.so.2 => /lib64/libdl.so.2 (0x00007f2e52fc8000)
>>>     libutil.so.1 => /lib64/libutil.so.1 (0x00007f2e52dc5000)
>>>     libpython2.7.so.1.0 => /lib64/libpython2.7.so.1.0
>>> (0x00007f2e529fe000)
>>>     librt.so.1 => /lib64/librt.so.1 (0x00007f2e527f6000)
>>>     libedit.so.0 => /lib64/libedit.so.0 (0x00007f2e525ba000)
>>>     libncurses.so.5 => /lib64/libncurses.so.5 (0x00007f2e52392000)
>>>     libtinfo.so.5 => /lib64/libtinfo.so.5 (0x00007f2e52168000)
>>>     libpanel.so.5 => /lib64/libpanel.so.5 (0x00007f2e51f64000)
>>>     libz.so.1 => /lib64/libz.so.1 (0x00007f2e51d4d000)
>>>     libm.so.6 => /lib64/libm.so.6 (0x00007f2e51a46000)
>>>     libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f2e5173e000)
>>>     libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f2e51527000)
>>>     libc.so.6 => /lib64/libc.so.6 (0x00007f2e51168000)
>>>     /lib64/ld-linux-x86-64.so.2 (0x0000003675a00000)
>>>
>>> However, regarding python loading the lldb bindings I see my
>>> build/Debug+Asserts has a lib and a lib64 folder.
>>>
>>> ~/src/p4play/build/Debug+Asserts
>>> $ ls
>>> bin  lib  lib64
>>>
>>> If I try to import lldb when I have "lib" as my PYTHONPATH i.e.
>>>
>>> ~/src/main/build/Debug+Asserts/lib
>>> $ PYTHONPATH=`pwd`/python2.7/site-packages/ python
>>> >>> import lldb
>>> Traceback (most recent call last):
>>>   File "<stdin>", line 1, in <module>
>>> ImportError: No module named lldb
>>> >>>
>>>
>>> This is not surprising, since ~/src/main/build/Debug+
>>> Asserts/lib/python2.7/site-packages/ only contains readline.so.
>>> So I cd to lib64. This looks more promising since there is a lldb folder
>>> below site-packages. So now I do "import lldb" with "lib64" instead:
>>>
>>> ~/src/main/build/Debug+Asserts/lib64
>>> $ PYTHONPATH=`pwd`/lib64/python2.7/site-packages/ python
>>> Python 2.7.5 (default, Feb 19 2014, 13:47:28)
>>> [GCC 4.8.2 20131212 (Red Hat 4.8.2-7)] on linux2
>>> Type "help", "copyright", "credits" or "license" for more information.
>>> >>> import lldb
>>> Traceback (most recent call last):
>>>   File "<stdin>", line 1, in <module>
>>>   File "/home/mg11/src/main/build/Debug+Asserts/lib64/python2.7/
>>> site-packages/lldb/__init__.py", line 52, in <module>
>>>     _lldb = swig_import_helper()
>>>   File "/home/mg11/src/main/build/Debug+Asserts/lib64/python2.7/
>>> site-packages/lldb/__init__.py", line 44, in swig_import_helper
>>>     import _lldb
>>> ImportError: No module named _lldb
>>> >>>
>>>
>>> So cd to ~/src/p4play/build/Debug+Asserts/lib64/python2.7/site-packages/lldb,
>>> and list:
>>>
>>> ~/src/p4play/build/Debug+Asserts/lib64/python2.7/site-packages/lldb
>>> $ ls -l
>>> total 1196
>>> -rw-rw-r-- 1 mg11 mg11   3905 Jul 21 08:25 embedded_interpreter.py
>>> drwxrwxr-x 3 mg11 mg11   4096 Jul 21 08:25 formatters
>>> -rw-rw-r-- 1 mg11 mg11 495675 Jul 21 08:25 __init__.py
>>> -rw-rw-r-- 1 mg11 mg11 708200 Jul 21 09:09 __init__.pyc
>>> lrwxrwxrwx 1 mg11 mg11     19 Jul 21 08:25 _lldb.so ->
>>> ../../../liblldb.so   !BROKEN SYMLINK!
>>> drwxrwxr-x 2 mg11 mg11   4096 Jul 21 08:25 runtime
>>> drwxrwxr-x 2 mg11 mg11   4096 Jul 21 08:25 utils
>>>
>>> _lldb.so is broken above as the link above assumes liblldb.so is just 3
>>> dirs back. However, it isn't - because it's not in lib64, *it's in lib*. So
>>> it looks like things are getting a bit mashed up due to 32-bit/64-bit
>>> directory issues. If I copy the .so files from lib to lib64, then it fixes
>>> my broken sym-link issue. And I progress a little further.
>>>
>>> ~/src/p4play/build/Debug+Asserts/lib64
>>> $ PYTHONPATH=`pwd`/python2.7/site-packages python
>>> Python 2.7.5 (default, Feb 19 2014, 13:47:28)
>>> [GCC 4.8.2 20131212 (Red Hat 4.8.2-7)] on linux2
>>> Type "help", "copyright", "credits" or "license" for more information.
>>> >>> import lldb
>>> Traceback (most recent call last):
>>>   File "<stdin>", line 1, in <module>
>>>   File "/home/mg11/src/p4play/build/Debug+Asserts/lib64/python2.7/
>>> site-packages/lldb/__init__.py", line 52, in <module>
>>>     _lldb = swig_import_helper()
>>>   File "/home/mg11/src/p4play/build/Debug+Asserts/lib64/python2.7/
>>> site-packages/lldb/__init__.py", line 48, in swig_import_helper
>>>     _mod = imp.load_module('_lldb', fp, pathname, description)
>>> ImportError: libLLVM-3.5svn.so: cannot open shared object file: No such
>>> file or directory
>>> >>>
>>>
>>> So now I push the location of these .sos into my environment with
>>> LD_LIBRARY_PATH. Finally lldb imports!
>>>
>>> ~/src/p4play/build/Debug+Asserts/lib64
>>> $ LD_LIBRARY_PATH=. PYTHONPATH=`pwd`/python2.7/site-packages python
>>> Python 2.7.5 (default, Feb 19 2014, 13:47:28)
>>> [GCC 4.8.2 20131212 (Red Hat 4.8.2-7)] on linux2
>>> Type "help", "copyright", "credits" or "license" for more information.
>>> >>> import lldb
>>> >>> dir(lldb)
>>> ['INT32_MAX', 'LLDB_ARCH_DEFAULT',
>>> ... lots more ...
>>> >>>
>>>
>>> So, I think the crux of the problem I'm seeing here is that on a 64-bit
>>> linux build (using autoconf/make) we get the binaries distributed around
>>> the lib and lib64 sub-folders without adequate coordination with the test
>>> tools. I'll retry now with cmake/ninja and see if my experience differs.
>>>
>>> Matt
>>>
>>>
>>>
>>> Todd Fiala wrote:
>>>
>>>>
>>>> Hey Matt,
>>>>
>>>> (Adding back the list since these are general troubleshooting tips for
>>>> getting tests running).
>>>>
>>>> Sorry you’re having so much trouble with this!
>>>>
>>>> Ok so a few things to troubleshoot:
>>>>
>>>> Regardless of the build system used, go to the build output’s lib dir.
>>>> Do this command:
>>>> ldd liblldb.so
>>>>
>>>> That will spit out all the shared libraries that your liblldb.so is
>>>> trying to link against. The python message your seeing will happen if
>>>> python can’t find the lldb module (as the message suggests and you were
>>>> tracing down), /or/ if it can find it but fails to load it (which is often
>>>> the case - something cannot be found when the liblldb.so tries to load as a
>>>> consequence of the lldb python module trying to load).
>>>>
>>>> You should see something that looks roughly like this:
>>>>
>>>> |tfiala at tfiala2:/mnt/ssd/work/macosx.sync/mbp-svn/build-debug/lib$ ldd
>>>> liblldb.so
>>>>          linux-vdso.so.1 =>  (0x00007fffe9dfe000)
>>>>          libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
>>>> (0x00007f6357e71000)
>>>>          libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6
>>>> (0x00007f6357b6b000)
>>>>          libedit.so.2 => /usr/lib/x86_64-linux-gnu/libedit.so.2
>>>> (0x00007f635793a000)
>>>>          libpanel.so.5 => /usr/lib/x86_64-linux-gnu/libpanel.so.5
>>>> (0x00007f6357736000)
>>>>          libncurses.so.5 => /lib/x86_64-linux-gnu/libncurses.so.5
>>>> (0x00007f6357513000)
>>>>          libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5
>>>> (0x00007f63572e9000)
>>>>          libpython2.7.so.1.0 => /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
>>>> (0x00007f6356d82000)
>>>>          librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1
>>>> (0x00007f6356b7a000)
>>>>          libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2
>>>> (0x00007f6356975000)
>>>>          libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
>>>> (0x00007f6356757000)
>>>>          libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1
>>>> (0x00007f635653e000)
>>>>          libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
>>>> (0x00007f6356327000)
>>>>          libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
>>>> (0x00007f6355f61000)
>>>>          /lib64/ld-linux-x86-64.so.2 (0x00007f635d584000)
>>>>          libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1
>>>> (0x00007f6355d5d000)
>>>> tfiala at tfiala2:/mnt/ssd/work/macosx.sync/mbp-svn/build-debug/lib$site-packages
>>>>
>>>> |
>>>>
>>>> The left side shows the short name that the .so is referencing. The
>>>> right side shows what the loader actually mapped it to with the current
>>>> environment. If there is a problem loading liblldb.so, this will show as (I
>>>> think) question marks on the right side indicating that the shared library
>>>> linkage was not found. Let me know if you see something like that.
>>>>
>>>> If you get past this part and you don’t see any issues, the next thing
>>>> to try is:
>>>> PYTHONPATH=/path/to/your/lldb/lib/python2.7/site-packages python
>>>> Then do an ‘import lldb’, then a ‘dir (lldb)’, like so:
>>>>
>>>> |tfiala at tfiala2:/mnt/ssd/work/macosx.sync/mbp-svn/build-debug/lib$
>>>> PYTHONPATH=`pwd`/python2.7/site-packages python
>>>> Python 2.7.6 (default, Mar 22 2014, 22:59:56)
>>>> [GCC 4.8.2] on linux2
>>>> Type "help", "copyright", "credits" or "license" for more information.
>>>> >>> import lldb
>>>> >>> dir(lldb)
>>>> ['INT32_MAX', 'LLDB_ARCH_DEFAULT', 'LLDB_ARCH_DEFAULT_32BIT',
>>>> 'LLDB_ARCH_DEFAULT_64BIT', 'LLDB_DEFAULT_BREAK_SIZE', 'LLDB_DEFAULT_SHELL',
>>>> 'LLDB_GENERIC_ERROR', 'LLDB_INVALID_ADDRESS', 'LLDB_INVALID_BREAK_ID',
>>>> 'LLDB_INVALID_CPUTYPE', 'LLDB_INVALID_FRAME_ID',
>>>> 'LLDB_INVALID_IMAGE_TOKEN', 'LLDB_INVALID_INDEX32',
>>>> 'LLDB_INVALID_IVAR_OFFSET', 'LLDB_INVALID_LINE_NUMBER',
>>>> 'LLDB_INVALID_MODULE_VERSION', 'LLDB_INVALID_OFFSET',
>>>> 'LLDB_INVALID_PROCESS_ID', 'LLDB_INVALID_QUEUE_ID', 'LLDB_INVALID_REGNUM',
>>>> 'LLDB_INVALID_SIGNAL_NUMBER', 'LLDB_INVALID_THREAD_ID', 'LLDB_INVALID_UID',
>>>> 'LLDB_INVALID_WATCH_ID', 'LLDB_MAX_NUM_OPTION_SETS', 'LLDB_OPT_SET_1',
>>>>
>>>> ... (lots of stuff removed)
>>>> |
>>>>
>>>> That will tell you if you can load the lldb module when you specify
>>>> exactly where it should be. If you can do this, the tests basically should
>>>> not have trouble loading lldb. (Obviously somewhere we’ll find an issue
>>>> here). If this does work, then we need to trace why the python test runner
>>>> calls are mixing up the python path. In general you don’t need to specify
>>>> any options to the ninja/make commands to get the tests to run from the
>>>> build dir.
>>>>
>>>> Can you refresh my memory on what Linux distro you’re using and whether
>>>> it’s 64 or 32 bit? (I don’t get over to the 32-bit distros much these days).
>>>>
>>>> I hope that gets us closer to finding out what’s up!  Keep me posted.
>>>>
>>>> -Todd
>>>>
>>>> ​
>>>>
>>>>
>>>> On Fri, Jul 18, 2014 at 4:56 AM, Matthew Gardiner <mg11 at csr.com
>>>> <mailto:mg11 at csr.com>> wrote:
>>>>
>>>>     Hi Todd,
>>>>
>>>>     Well I managed to get cmake and ninja working! I needed (it would
>>>>     appear) to download/build/install the latest versions of both from
>>>>     source. Then I do:
>>>>
>>>>     build $  cmake ../llvm/ -G Ninja -DLLVM_TARGETS_TO_BUILD="X86"
>>>>     -DBUILD_SHARED_LIBS=ON -DLLVM_ENABLE_CXX1Y=ON
>>>>     ...
>>>>     build $ ninja
>>>>     ...
>>>>
>>>>     However "ninja check-lldb" still fails to find lldb.py. It is does
>>>>     get built, but the path which dotest.py uses to find it is wrong
>>>>     on my system. Anyway, I'm going to continue trying to analyse it
>>>>     myself for now. I'll probably post a new thread to lldb-dev if I
>>>>     continue to struggle.
>>>>
>>>>     Anyway, have a good weekend,
>>>>     Matt
>>>>
>>>>
>>>>
>>>>
>>>>     Member of the CSR plc group of companies. CSR plc registered in
>>>>     England and Wales, registered number 4187346, registered office
>>>>     Churchill House, Cambridge Business Park, Cowley Road, Cambridge,
>>>>     CB4 0WZ, United Kingdom
>>>>     More information can be found at www.csr.com <http://www.csr.com>.
>>>>     Keep up to date with CSR on our technical blog, www.csr.com/blog
>>>>     <http://www.csr.com/blog>, CSR people blog, www.csr.com/people
>>>>     <http://www.csr.com/people>, YouTube, www.youtube.com/user/CSRplc
>>>>     <http://www.youtube.com/user/CSRplc>, Facebook,
>>>>     www.facebook.com/pages/CSR/191038434253534
>>>>     <http://www.facebook.com/pages/CSR/191038434253534>, or follow us
>>>>     on Twitter at www.twitter.com/CSR_plc
>>>>     <http://www.twitter.com/CSR_plc>.
>>>>     New for 2014, you can now access the wide range of products
>>>>     powered by aptX at www.aptx.com <http://www.aptx.com>.
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Todd Fiala |      Software Engineer |     tfiala at google.com <mailto:
>>>> tfiala at google.com> |     650-943-3180
>>>>
>>>>
>>>>
>>>>
>>>> To report this email as spam click here <https://www.mailcontrol.com/
>>>> sr/ik8jrXaVfBTGX2PQPOmvUj%21GOBh06pKK3+K9eZ%212Dx6F%
>>>> 21Jeeftdx9RBpik88jjuoRisi26jPZSzJaydNu2tBwg==>.
>>>>
>>>>
>>> _______________________________________________
>>> lldb-dev mailing list
>>> lldb-dev at cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
>>>
>>
>>
>
>
> --
> Todd Fiala | Software Engineer |  tfiala at google.com |  650-943-3180
>
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20140721/e6847a1f/attachment.html>

More information about the lldb-dev mailing list