[lldb-dev] Problems running the lldb tests on Fedora Core 20 - 64bit (email originally titled "Add kalimba as a platform.")
Todd Fiala
tfiala at google.com
Mon Jul 21 07:03:11 PDT 2014
Hey Matthew,
Great investigating! Glad you were able to at least find the core of the
issue, which does look like at least on Fedora Core 20 x86_64 we are doing
something funky with configure/make. I'm not sure why we're choosing lib64
there and lib on Ubuntu 14.04 x86_64, might have to do with how the python
is configured. I think that directory may be coming from a helper library
in python that determines the correct location for build output and has to
do with some Python finalizer scripts during the build process. I won't be
totally surprised if this might just be broken on Fedora Core x86_64
regardless of cmake or configure-based builds.
I'll do a configure/make build on Ubuntu 14.04 x86_64 later today to see if
I'm getting the same results on configure/make - there has been some
changes in python handling more recently, generally to support Windows
builds, but we may have unintentionally introduced that via those changes.
At least that way we can see if its consistent with configure/make or if
it might just be specific to Fedora Core.
Thanks for identifying that!
-Todd
On Mon, Jul 21, 2014 at 6:51 AM, Matthew Gardiner <mg11 at csr.com> wrote:
> Todd,
>
> I've since discovered that the cmake/ninja build system also results in a
> lib and lib64 directory, with the liblldb.so and other .sos in the lib
> folder but the python2.7/site-packages/lldb under lib64. (In fact with
> cmake build I see no python2.7 folder at all under lib, it's all under
> lib64).
>
> So Todd, focusing back on autoconf/make builds, after doing the
> ./configure && make in the sibling build directory, do you see for your
> linux 64-bit builds the python2.7/site-packages/lldb under the lib64
> sub-folder and the lldb/llvm .sos under the lib64 sub-folder?
>
> How do you get this to work:
>
>
> tfiala at tfiala2:/mnt/ssd/work/macosx.sync/mbp-svn/build-debug/lib$
> PYTHONPATH=`pwd`/python2.7/site-packages python
> Python 2.7.6 (default, Mar 22 2014, 22:59:56)
> [GCC 4.8.2] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import lldb
> >>> dir(lldb)
>
> Do you run another script or make target after doing the basic llvm/lldb
> make? That is to move python2.7/site-packages/lldb underneath lib. I think
> that's what I'd have to do on my system, since after the basic make my lib
> and lib64 directories have the following contents:
>
> ~/src/p4play/build/Debug+Asserts/lib/python2.7/site-packages
>
> $ ls -al
> total 32
> drwxrwxr-x 2 mg11 mg11 4096 Jul 21 14:40 .
> drwxrwxr-x 3 mg11 mg11 4096 Jul 21 14:40 ..
> -rwxr-xr-x 1 mg11 mg11 21795 Jul 21 14:40 readline.so
>
> $ cd ../../../lib64/python2.7/site-packages/
>
> ~/src/p4play/build/Debug+Asserts/lib64/python2.7/site-packages
>
> $ ls -al
> total 12
> drwxrwxr-x 3 mg11 mg11 4096 Jul 21 14:42 .
> drwxrwxr-x 3 mg11 mg11 4096 Jul 21 14:42 ..
> drwxrwxr-x 5 mg11 mg11 4096 Jul 21 14:42 lldb
>
>
> Any ideas?
> Matt
>
>
> Matthew Gardiner wrote:
>
>> Hi Todd,
>>
>> Thanks for taking the time to come up with these tips. First off my
>> distro is Fedora Core 20 64-bit. So to further my investigation I blew away
>> all the sources, and did svn co llvm, (lldb clang into llvm/tools), made
>> the sibling build dir, then did:
>>
>> ~/src/p4play/build
>> $ ../llvm/configure --enable-targets=x86_64 --enable-shared
>> --enable-cxx11 --prefix=`pwd`/../install && make -j 8
>>
>> After the successful build I do ldd. Basically lld finds all dependencies
>> for liblldb.so...
>>
>> ~/src/p4play/build/Debug+Asserts/lib
>> $ ldd liblldb.so
>> linux-vdso.so.1 => (0x00007fffa61fe000)
>> libLLVM-3.5svn.so => /home/mg11/src/p4play/build/Debug+Asserts/lib/./
>> libLLVM-3.5svn.so (0x00007f2e53405000)
>> libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f2e531cc000)
>> libdl.so.2 => /lib64/libdl.so.2 (0x00007f2e52fc8000)
>> libutil.so.1 => /lib64/libutil.so.1 (0x00007f2e52dc5000)
>> libpython2.7.so.1.0 => /lib64/libpython2.7.so.1.0 (0x00007f2e529fe000)
>> librt.so.1 => /lib64/librt.so.1 (0x00007f2e527f6000)
>> libedit.so.0 => /lib64/libedit.so.0 (0x00007f2e525ba000)
>> libncurses.so.5 => /lib64/libncurses.so.5 (0x00007f2e52392000)
>> libtinfo.so.5 => /lib64/libtinfo.so.5 (0x00007f2e52168000)
>> libpanel.so.5 => /lib64/libpanel.so.5 (0x00007f2e51f64000)
>> libz.so.1 => /lib64/libz.so.1 (0x00007f2e51d4d000)
>> libm.so.6 => /lib64/libm.so.6 (0x00007f2e51a46000)
>> libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f2e5173e000)
>> libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f2e51527000)
>> libc.so.6 => /lib64/libc.so.6 (0x00007f2e51168000)
>> /lib64/ld-linux-x86-64.so.2 (0x0000003675a00000)
>>
>> However, regarding python loading the lldb bindings I see my
>> build/Debug+Asserts has a lib and a lib64 folder.
>>
>> ~/src/p4play/build/Debug+Asserts
>> $ ls
>> bin lib lib64
>>
>> If I try to import lldb when I have "lib" as my PYTHONPATH i.e.
>>
>> ~/src/main/build/Debug+Asserts/lib
>> $ PYTHONPATH=`pwd`/python2.7/site-packages/ python
>> >>> import lldb
>> Traceback (most recent call last):
>> File "<stdin>", line 1, in <module>
>> ImportError: No module named lldb
>> >>>
>>
>> This is not surprising, since ~/src/main/build/Debug+
>> Asserts/lib/python2.7/site-packages/ only contains readline.so.
>> So I cd to lib64. This looks more promising since there is a lldb folder
>> below site-packages. So now I do "import lldb" with "lib64" instead:
>>
>> ~/src/main/build/Debug+Asserts/lib64
>> $ PYTHONPATH=`pwd`/lib64/python2.7/site-packages/ python
>> Python 2.7.5 (default, Feb 19 2014, 13:47:28)
>> [GCC 4.8.2 20131212 (Red Hat 4.8.2-7)] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>> >>> import lldb
>> Traceback (most recent call last):
>> File "<stdin>", line 1, in <module>
>> File "/home/mg11/src/main/build/Debug+Asserts/lib64/python2.7/
>> site-packages/lldb/__init__.py", line 52, in <module>
>> _lldb = swig_import_helper()
>> File "/home/mg11/src/main/build/Debug+Asserts/lib64/python2.7/
>> site-packages/lldb/__init__.py", line 44, in swig_import_helper
>> import _lldb
>> ImportError: No module named _lldb
>> >>>
>>
>> So cd to ~/src/p4play/build/Debug+Asserts/lib64/python2.7/site-packages/lldb,
>> and list:
>>
>> ~/src/p4play/build/Debug+Asserts/lib64/python2.7/site-packages/lldb
>> $ ls -l
>> total 1196
>> -rw-rw-r-- 1 mg11 mg11 3905 Jul 21 08:25 embedded_interpreter.py
>> drwxrwxr-x 3 mg11 mg11 4096 Jul 21 08:25 formatters
>> -rw-rw-r-- 1 mg11 mg11 495675 Jul 21 08:25 __init__.py
>> -rw-rw-r-- 1 mg11 mg11 708200 Jul 21 09:09 __init__.pyc
>> lrwxrwxrwx 1 mg11 mg11 19 Jul 21 08:25 _lldb.so ->
>> ../../../liblldb.so !BROKEN SYMLINK!
>> drwxrwxr-x 2 mg11 mg11 4096 Jul 21 08:25 runtime
>> drwxrwxr-x 2 mg11 mg11 4096 Jul 21 08:25 utils
>>
>> _lldb.so is broken above as the link above assumes liblldb.so is just 3
>> dirs back. However, it isn't - because it's not in lib64, *it's in lib*. So
>> it looks like things are getting a bit mashed up due to 32-bit/64-bit
>> directory issues. If I copy the .so files from lib to lib64, then it fixes
>> my broken sym-link issue. And I progress a little further.
>>
>> ~/src/p4play/build/Debug+Asserts/lib64
>> $ PYTHONPATH=`pwd`/python2.7/site-packages python
>> Python 2.7.5 (default, Feb 19 2014, 13:47:28)
>> [GCC 4.8.2 20131212 (Red Hat 4.8.2-7)] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>> >>> import lldb
>> Traceback (most recent call last):
>> File "<stdin>", line 1, in <module>
>> File "/home/mg11/src/p4play/build/Debug+Asserts/lib64/python2.7/
>> site-packages/lldb/__init__.py", line 52, in <module>
>> _lldb = swig_import_helper()
>> File "/home/mg11/src/p4play/build/Debug+Asserts/lib64/python2.7/
>> site-packages/lldb/__init__.py", line 48, in swig_import_helper
>> _mod = imp.load_module('_lldb', fp, pathname, description)
>> ImportError: libLLVM-3.5svn.so: cannot open shared object file: No such
>> file or directory
>> >>>
>>
>> So now I push the location of these .sos into my environment with
>> LD_LIBRARY_PATH. Finally lldb imports!
>>
>> ~/src/p4play/build/Debug+Asserts/lib64
>> $ LD_LIBRARY_PATH=. PYTHONPATH=`pwd`/python2.7/site-packages python
>> Python 2.7.5 (default, Feb 19 2014, 13:47:28)
>> [GCC 4.8.2 20131212 (Red Hat 4.8.2-7)] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>> >>> import lldb
>> >>> dir(lldb)
>> ['INT32_MAX', 'LLDB_ARCH_DEFAULT',
>> ... lots more ...
>> >>>
>>
>> So, I think the crux of the problem I'm seeing here is that on a 64-bit
>> linux build (using autoconf/make) we get the binaries distributed around
>> the lib and lib64 sub-folders without adequate coordination with the test
>> tools. I'll retry now with cmake/ninja and see if my experience differs.
>>
>> Matt
>>
>>
>>
>> Todd Fiala wrote:
>>
>>>
>>> Hey Matt,
>>>
>>> (Adding back the list since these are general troubleshooting tips for
>>> getting tests running).
>>>
>>> Sorry you’re having so much trouble with this!
>>>
>>> Ok so a few things to troubleshoot:
>>>
>>> Regardless of the build system used, go to the build output’s lib dir.
>>> Do this command:
>>> ldd liblldb.so
>>>
>>> That will spit out all the shared libraries that your liblldb.so is
>>> trying to link against. The python message your seeing will happen if
>>> python can’t find the lldb module (as the message suggests and you were
>>> tracing down), /or/ if it can find it but fails to load it (which is often
>>> the case - something cannot be found when the liblldb.so tries to load as a
>>> consequence of the lldb python module trying to load).
>>>
>>> You should see something that looks roughly like this:
>>>
>>> |tfiala at tfiala2:/mnt/ssd/work/macosx.sync/mbp-svn/build-debug/lib$ ldd
>>> liblldb.so
>>> linux-vdso.so.1 => (0x00007fffe9dfe000)
>>> libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
>>> (0x00007f6357e71000)
>>> libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6
>>> (0x00007f6357b6b000)
>>> libedit.so.2 => /usr/lib/x86_64-linux-gnu/libedit.so.2
>>> (0x00007f635793a000)
>>> libpanel.so.5 => /usr/lib/x86_64-linux-gnu/libpanel.so.5
>>> (0x00007f6357736000)
>>> libncurses.so.5 => /lib/x86_64-linux-gnu/libncurses.so.5
>>> (0x00007f6357513000)
>>> libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5
>>> (0x00007f63572e9000)
>>> libpython2.7.so.1.0 => /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
>>> (0x00007f6356d82000)
>>> librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1
>>> (0x00007f6356b7a000)
>>> libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2
>>> (0x00007f6356975000)
>>> libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
>>> (0x00007f6356757000)
>>> libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1
>>> (0x00007f635653e000)
>>> libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
>>> (0x00007f6356327000)
>>> libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
>>> (0x00007f6355f61000)
>>> /lib64/ld-linux-x86-64.so.2 (0x00007f635d584000)
>>> libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1
>>> (0x00007f6355d5d000)
>>> tfiala at tfiala2:/mnt/ssd/work/macosx.sync/mbp-svn/build-debug/lib$site-packages
>>>
>>> |
>>>
>>> The left side shows the short name that the .so is referencing. The
>>> right side shows what the loader actually mapped it to with the current
>>> environment. If there is a problem loading liblldb.so, this will show as (I
>>> think) question marks on the right side indicating that the shared library
>>> linkage was not found. Let me know if you see something like that.
>>>
>>> If you get past this part and you don’t see any issues, the next thing
>>> to try is:
>>> PYTHONPATH=/path/to/your/lldb/lib/python2.7/site-packages python
>>> Then do an ‘import lldb’, then a ‘dir (lldb)’, like so:
>>>
>>> |tfiala at tfiala2:/mnt/ssd/work/macosx.sync/mbp-svn/build-debug/lib$
>>> PYTHONPATH=`pwd`/python2.7/site-packages python
>>> Python 2.7.6 (default, Mar 22 2014, 22:59:56)
>>> [GCC 4.8.2] on linux2
>>> Type "help", "copyright", "credits" or "license" for more information.
>>> >>> import lldb
>>> >>> dir(lldb)
>>> ['INT32_MAX', 'LLDB_ARCH_DEFAULT', 'LLDB_ARCH_DEFAULT_32BIT',
>>> 'LLDB_ARCH_DEFAULT_64BIT', 'LLDB_DEFAULT_BREAK_SIZE', 'LLDB_DEFAULT_SHELL',
>>> 'LLDB_GENERIC_ERROR', 'LLDB_INVALID_ADDRESS', 'LLDB_INVALID_BREAK_ID',
>>> 'LLDB_INVALID_CPUTYPE', 'LLDB_INVALID_FRAME_ID',
>>> 'LLDB_INVALID_IMAGE_TOKEN', 'LLDB_INVALID_INDEX32',
>>> 'LLDB_INVALID_IVAR_OFFSET', 'LLDB_INVALID_LINE_NUMBER',
>>> 'LLDB_INVALID_MODULE_VERSION', 'LLDB_INVALID_OFFSET',
>>> 'LLDB_INVALID_PROCESS_ID', 'LLDB_INVALID_QUEUE_ID', 'LLDB_INVALID_REGNUM',
>>> 'LLDB_INVALID_SIGNAL_NUMBER', 'LLDB_INVALID_THREAD_ID', 'LLDB_INVALID_UID',
>>> 'LLDB_INVALID_WATCH_ID', 'LLDB_MAX_NUM_OPTION_SETS', 'LLDB_OPT_SET_1',
>>>
>>> ... (lots of stuff removed)
>>> |
>>>
>>> That will tell you if you can load the lldb module when you specify
>>> exactly where it should be. If you can do this, the tests basically should
>>> not have trouble loading lldb. (Obviously somewhere we’ll find an issue
>>> here). If this does work, then we need to trace why the python test runner
>>> calls are mixing up the python path. In general you don’t need to specify
>>> any options to the ninja/make commands to get the tests to run from the
>>> build dir.
>>>
>>> Can you refresh my memory on what Linux distro you’re using and whether
>>> it’s 64 or 32 bit? (I don’t get over to the 32-bit distros much these days).
>>>
>>> I hope that gets us closer to finding out what’s up! Keep me posted.
>>>
>>> -Todd
>>>
>>>
>>>
>>>
>>> On Fri, Jul 18, 2014 at 4:56 AM, Matthew Gardiner <mg11 at csr.com <mailto:
>>> mg11 at csr.com>> wrote:
>>>
>>> Hi Todd,
>>>
>>> Well I managed to get cmake and ninja working! I needed (it would
>>> appear) to download/build/install the latest versions of both from
>>> source. Then I do:
>>>
>>> build $ cmake ../llvm/ -G Ninja -DLLVM_TARGETS_TO_BUILD="X86"
>>> -DBUILD_SHARED_LIBS=ON -DLLVM_ENABLE_CXX1Y=ON
>>> ...
>>> build $ ninja
>>> ...
>>>
>>> However "ninja check-lldb" still fails to find lldb.py. It is does
>>> get built, but the path which dotest.py uses to find it is wrong
>>> on my system. Anyway, I'm going to continue trying to analyse it
>>> myself for now. I'll probably post a new thread to lldb-dev if I
>>> continue to struggle.
>>>
>>> Anyway, have a good weekend,
>>> Matt
>>>
>>>
>>>
>>>
>>> Member of the CSR plc group of companies. CSR plc registered in
>>> England and Wales, registered number 4187346, registered office
>>> Churchill House, Cambridge Business Park, Cowley Road, Cambridge,
>>> CB4 0WZ, United Kingdom
>>> More information can be found at www.csr.com <http://www.csr.com>.
>>> Keep up to date with CSR on our technical blog, www.csr.com/blog
>>> <http://www.csr.com/blog>, CSR people blog, www.csr.com/people
>>> <http://www.csr.com/people>, YouTube, www.youtube.com/user/CSRplc
>>> <http://www.youtube.com/user/CSRplc>, Facebook,
>>> www.facebook.com/pages/CSR/191038434253534
>>> <http://www.facebook.com/pages/CSR/191038434253534>, or follow us
>>> on Twitter at www.twitter.com/CSR_plc
>>> <http://www.twitter.com/CSR_plc>.
>>> New for 2014, you can now access the wide range of products
>>> powered by aptX at www.aptx.com <http://www.aptx.com>.
>>>
>>>
>>>
>>>
>>> --
>>> Todd Fiala | Software Engineer | tfiala at google.com <mailto:
>>> tfiala at google.com> | 650-943-3180
>>>
>>>
>>>
>>>
>>> To report this email as spam click here <https://www.mailcontrol.com/
>>> sr/ik8jrXaVfBTGX2PQPOmvUj%21GOBh06pKK3+K9eZ%212Dx6F%
>>> 21Jeeftdx9RBpik88jjuoRisi26jPZSzJaydNu2tBwg==>.
>>>
>>>
>> _______________________________________________
>> lldb-dev mailing list
>> lldb-dev at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
>>
>
>
--
Todd Fiala | Software Engineer | tfiala at google.com | 650-943-3180
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20140721/35bb22d6/attachment.html>
More information about the lldb-dev
mailing list