[lldb-dev] Problems running the lldb tests on Fedora Core 20 - 64bit (email originally titled "Add kalimba as a platform.")
Matthew Gardiner
mg11 at csr.com
Mon Jul 21 23:46:04 PDT 2014
Guys,
This little tweak fixes things for me. I.e. both the llvm/lldb .sos and
the python2.7 contents get written under build/lib. (Not partially under
lib64.)
I'm not sure whether it's the right fix, though. Or whether it has the
potential to break other systems.
Matt
Index: scripts/Python/finish-swig-Python-LLDB.sh
===================================================================
--- scripts/Python/finish-swig-Python-LLDB.sh (revision 213502)
+++ scripts/Python/finish-swig-Python-LLDB.sh (working copy)
@@ -101,9 +101,9 @@
if [ -n "${PYTHON_INSTALL_DIR}" ]
then
- framework_python_dir=`${PYTHON} -c "from distutils.sysconfig
import get_python_lib; print get_python_lib(True, False,
\"${PYTHON_INSTALL_DIR}\");"`/lldb
+ framework_python_dir=`${PYTHON} -c "from distutils.sysconfig
import get_python_lib; print get_python_lib(False, False,
\"${PYTHON_INSTALL_DIR}\");"`/lldb
else
- framework_python_dir=`${PYTHON} -c "from distutils.sysconfig
import get_python_lib; print get_python_lib(True, False);"`/lldb
+ framework_python_dir=`${PYTHON} -c "from distutils.sysconfig
import get_python_lib; print get_python_lib(False, False);"`/lldb
fi
fi
Matthew Gardiner wrote:
> More investigating reveals that it is finish-swig-Python-LLDB.sh which
> is lldb's responsible script. (Thanks for the -x reminder, Zachary!).
> The line in the script is
>
> framework_python_dir=`${PYTHON} -c "from distutils.sysconfig import
> get_python_lib; print get_python_lib(True, False,
> \"${PYTHON_INSTALL_DIR}\");"`/lldb
>
> When I run the raw invocation at my shell:
>
> $ /usr/bin/python -c 'from distutils.sysconfig import get_python_lib;
> print get_python_lib(True, False, "/home/mg11/src/p4play/build");'
> /home/mg11/src/p4play/build/lib64/python2.7/site-packages
>
> So, we might suspect that FC20s python install is wrong.
>
> However, I downloaded python-2.7.7 and built from source, with a
> pretty much a raw config (only configure tweaks being enable shared
> and threads), and I get a similar output:
>
> $ /usr/local/Python-2.7.7/bin/python -c 'from distutils.sysconfig
> import get_python_lib; print get_python_lib(True, False,
> "/home/mg11/src/p4play/build");'
> /home/mg11/src/p4play/build/lib64/python2.7/site-packages
>
> I may well be sticking my neck out a bit here... but I'm wondering if
> lib64 is actually the __right__ answer here (given my 64-bit host). Is
> python trying to support 32-bit binaries in lib and 64-bit ones in
> lib64 or something?
>
> (For *NIX 64-bit builds should our llvm/lldb binaries go in lib64? not
> lib)
>
> I'm curious as to what people think here... so we can engineer as nice
> a fix as possible.
>
> Matt
>
>
>
> Todd Fiala wrote:
>> Hey Matthew,
>>
>> Great investigating! Glad you were able to at least find the core of
>> the issue, which does look like at least on Fedora Core 20 x86_64 we
>> are doing something funky with configure/make. I'm not sure why
>> we're choosing lib64 there and lib on Ubuntu 14.04 x86_64, might have
>> to do with how the python is configured. I think that directory may
>> be coming from a helper library in python that determines the correct
>> location for build output and has to do with some Python finalizer
>> scripts during the build process. I won't be totally surprised if
>> this might just be broken on Fedora Core x86_64 regardless of cmake
>> or configure-based builds.
>>
>> I'll do a configure/make build on Ubuntu 14.04 x86_64 later today to
>> see if I'm getting the same results on configure/make - there has
>> been some changes in python handling more recently, generally to
>> support Windows builds, but we may have unintentionally introduced
>> that via those changes. At least that way we can see if its
>> consistent with configure/make or if it might just be specific to
>> Fedora Core.
>>
>> Thanks for identifying that!
>>
>> -Todd
>>
>>
>>
>> On Mon, Jul 21, 2014 at 6:51 AM, Matthew Gardiner <mg11 at csr.com
>> <mailto:mg11 at csr.com>> wrote:
>>
>> Todd,
>>
>> I've since discovered that the cmake/ninja build system also
>> results in a lib and lib64 directory, with the liblldb.so and
>> other .sos in the lib folder but the python2.7/site-packages/lldb
>> under lib64. (In fact with cmake build I see no python2.7 folder
>> at all under lib, it's all under lib64).
>>
>> So Todd, focusing back on autoconf/make builds, after doing the
>> ./configure && make in the sibling build directory, do you see for
>> your linux 64-bit builds the python2.7/site-packages/lldb under
>> the lib64 sub-folder and the lldb/llvm .sos under the lib64
>> sub-folder?
>>
>> How do you get this to work:
>>
>>
>> tfiala at tfiala2:/mnt/ssd/work/macosx.sync/mbp-svn/build-debug/lib$
>> PYTHONPATH=`pwd`/python2.7/site-packages python
>> Python 2.7.6 (default, Mar 22 2014, 22:59:56)
>> [GCC 4.8.2] on linux2
>> Type "help", "copyright", "credits" or "license" for more
>> information.
>> >>> import lldb
>> >>> dir(lldb)
>>
>> Do you run another script or make target after doing the basic
>> llvm/lldb make? That is to move python2.7/site-packages/lldb
>> underneath lib. I think that's what I'd have to do on my system,
>> since after the basic make my lib and lib64 directories have the
>> following contents:
>>
>> ~/src/p4play/build/Debug+Asserts/lib/python2.7/site-packages
>>
>> $ ls -al
>> total 32
>> drwxrwxr-x 2 mg11 mg11 4096 Jul 21 14:40 .
>> drwxrwxr-x 3 mg11 mg11 4096 Jul 21 14:40 ..
>> -rwxr-xr-x 1 mg11 mg11 21795 Jul 21 14:40 readline.so
>>
>> $ cd ../../../lib64/python2.7/site-packages/
>>
>> ~/src/p4play/build/Debug+Asserts/lib64/python2.7/site-packages
>>
>> $ ls -al
>> total 12
>> drwxrwxr-x 3 mg11 mg11 4096 Jul 21 14:42 .
>> drwxrwxr-x 3 mg11 mg11 4096 Jul 21 14:42 ..
>> drwxrwxr-x 5 mg11 mg11 4096 Jul 21 14:42 lldb
>>
>>
>> Any ideas?
>> Matt
>>
>>
>> Matthew Gardiner wrote:
>>
>> Hi Todd,
>>
>> Thanks for taking the time to come up with these tips. First
>> off my distro is Fedora Core 20 64-bit. So to further my
>> investigation I blew away all the sources, and did svn co
>> llvm, (lldb clang into llvm/tools), made the sibling build
>> dir, then did:
>>
>> ~/src/p4play/build
>> $ ../llvm/configure --enable-targets=x86_64 --enable-shared
>> --enable-cxx11 --prefix=`pwd`/../install && make -j 8
>>
>> After the successful build I do ldd. Basically lld finds all
>> dependencies for liblldb.so...
>>
>> ~/src/p4play/build/Debug+Asserts/lib
>> $ ldd liblldb.so
>> linux-vdso.so.1 => (0x00007fffa61fe000)
>> libLLVM-3.5svn.so <http://libLLVM-3.5svn.so> =>
>> /home/mg11/src/p4play/build/Debug+Asserts/lib/./libLLVM-3.5svn.so
>> <http://libLLVM-3.5svn.so> (0x00007f2e53405000)
>> libpthread.so.0 => /lib64/libpthread.so.0
>> (0x00007f2e531cc000)
>> libdl.so.2 => /lib64/libdl.so.2 (0x00007f2e52fc8000)
>> libutil.so.1 => /lib64/libutil.so.1 (0x00007f2e52dc5000)
>> libpython2.7.so.1.0 => /lib64/libpython2.7.so.1.0
>> (0x00007f2e529fe000)
>> librt.so.1 => /lib64/librt.so.1 (0x00007f2e527f6000)
>> libedit.so.0 => /lib64/libedit.so.0 (0x00007f2e525ba000)
>> libncurses.so.5 => /lib64/libncurses.so.5
>> (0x00007f2e52392000)
>> libtinfo.so.5 => /lib64/libtinfo.so.5 (0x00007f2e52168000)
>> libpanel.so.5 => /lib64/libpanel.so.5 (0x00007f2e51f64000)
>> libz.so.1 => /lib64/libz.so.1 (0x00007f2e51d4d000)
>> libm.so.6 => /lib64/libm.so.6 (0x00007f2e51a46000)
>> libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f2e5173e000)
>> libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f2e51527000)
>> libc.so.6 => /lib64/libc.so.6 (0x00007f2e51168000)
>> /lib64/ld-linux-x86-64.so.2 (0x0000003675a00000)
>>
>> However, regarding python loading the lldb bindings I see my
>> build/Debug+Asserts has a lib and a lib64 folder.
>>
>> ~/src/p4play/build/Debug+Asserts
>> $ ls
>> bin lib lib64
>>
>> If I try to import lldb when I have "lib" as my PYTHONPATH i.e.
>>
>> ~/src/main/build/Debug+Asserts/lib
>> $ PYTHONPATH=`pwd`/python2.7/site-packages/ python
>> >>> import lldb
>> Traceback (most recent call last):
>> File "<stdin>", line 1, in <module>
>> ImportError: No module named lldb
>> >>>
>>
>> This is not surprising, since
>> ~/src/main/build/Debug+Asserts/lib/python2.7/site-packages/
>> only contains readline.so.
>> So I cd to lib64. This looks more promising since there is a
>> lldb folder below site-packages. So now I do "import lldb"
>> with "lib64" instead:
>>
>> ~/src/main/build/Debug+Asserts/lib64
>> $ PYTHONPATH=`pwd`/lib64/python2.7/site-packages/ python
>> Python 2.7.5 (default, Feb 19 2014, 13:47:28)
>> [GCC 4.8.2 20131212 (Red Hat 4.8.2-7)] on linux2
>> Type "help", "copyright", "credits" or "license" for more
>> information.
>> >>> import lldb
>> Traceback (most recent call last):
>> File "<stdin>", line 1, in <module>
>> File
>> "/home/mg11/src/main/build/Debug+Asserts/lib64/python2.7/site-packages/lldb/__init__.py",
>> line 52, in <module>
>> _lldb = swig_import_helper()
>> File
>> "/home/mg11/src/main/build/Debug+Asserts/lib64/python2.7/site-packages/lldb/__init__.py",
>> line 44, in swig_import_helper
>> import _lldb
>> ImportError: No module named _lldb
>> >>>
>>
>> So cd to
>> ~/src/p4play/build/Debug+Asserts/lib64/python2.7/site-packages/lldb,
>> and list:
>>
>> ~/src/p4play/build/Debug+Asserts/lib64/python2.7/site-packages/lldb
>> $ ls -l
>> total 1196
>> -rw-rw-r-- 1 mg11 mg11 3905 Jul 21 08:25
>> embedded_interpreter.py
>> drwxrwxr-x 3 mg11 mg11 4096 Jul 21 08:25 formatters
>> -rw-rw-r-- 1 mg11 mg11 495675 Jul 21 08:25 __init__.py
>> -rw-rw-r-- 1 mg11 mg11 708200 Jul 21 09:09 __init__.pyc
>> lrwxrwxrwx 1 mg11 mg11 19 Jul 21 08:25 _lldb.so ->
>> ../../../liblldb.so !BROKEN SYMLINK!
>> drwxrwxr-x 2 mg11 mg11 4096 Jul 21 08:25 runtime
>> drwxrwxr-x 2 mg11 mg11 4096 Jul 21 08:25 utils
>>
>> _lldb.so is broken above as the link above assumes liblldb.so
>> is just 3 dirs back. However, it isn't - because it's not in
>> lib64, *it's in lib*. So it looks like things are getting a
>> bit mashed up due to 32-bit/64-bit directory issues. If I copy
>> the .so files from lib to lib64, then it fixes my broken
>> sym-link issue. And I progress a little further.
>>
>> ~/src/p4play/build/Debug+Asserts/lib64
>> $ PYTHONPATH=`pwd`/python2.7/site-packages python
>> Python 2.7.5 (default, Feb 19 2014, 13:47:28)
>> [GCC 4.8.2 20131212 (Red Hat 4.8.2-7)] on linux2
>> Type "help", "copyright", "credits" or "license" for more
>> information.
>> >>> import lldb
>> Traceback (most recent call last):
>> File "<stdin>", line 1, in <module>
>> File
>> "/home/mg11/src/p4play/build/Debug+Asserts/lib64/python2.7/site-packages/lldb/__init__.py",
>> line 52, in <module>
>> _lldb = swig_import_helper()
>> File
>> "/home/mg11/src/p4play/build/Debug+Asserts/lib64/python2.7/site-packages/lldb/__init__.py",
>> line 48, in swig_import_helper
>> _mod = imp.load_module('_lldb', fp, pathname, description)
>> ImportError: libLLVM-3.5svn.so <http://libLLVM-3.5svn.so>:
>> cannot open shared object file: No such file or directory
>> >>>
>>
>> So now I push the location of these .sos into my environment
>> with LD_LIBRARY_PATH. Finally lldb imports!
>>
>> ~/src/p4play/build/Debug+Asserts/lib64
>> $ LD_LIBRARY_PATH=. PYTHONPATH=`pwd`/python2.7/site-packages
>> python
>> Python 2.7.5 (default, Feb 19 2014, 13:47:28)
>> [GCC 4.8.2 20131212 (Red Hat 4.8.2-7)] on linux2
>> Type "help", "copyright", "credits" or "license" for more
>> information.
>> >>> import lldb
>> >>> dir(lldb)
>> ['INT32_MAX', 'LLDB_ARCH_DEFAULT',
>> ... lots more ...
>> >>>
>>
>> So, I think the crux of the problem I'm seeing here is that on
>> a 64-bit linux build (using autoconf/make) we get the binaries
>> distributed around the lib and lib64 sub-folders without
>> adequate coordination with the test tools. I'll retry now with
>> cmake/ninja and see if my experience differs.
>>
>> Matt
>>
>>
>>
>> Todd Fiala wrote:
>>
>>
>> Hey Matt,
>>
>> (Adding back the list since these are general
>> troubleshooting tips for getting tests running).
>>
>> Sorry you’re having so much trouble with this!
>>
>> Ok so a few things to troubleshoot:
>>
>> Regardless of the build system used, go to the build
>> output’s lib dir. Do this command:
>> ldd liblldb.so
>>
>> That will spit out all the shared libraries that your
>> liblldb.so is trying to link against. The python message
>> your seeing will happen if python can’t find the lldb
>> module (as the message suggests and you were tracing
>> down), /or/ if it can find it but fails to load it (which
>> is often the case - something cannot be found when the
>> liblldb.so tries to load as a consequence of the lldb
>> python module trying to load).
>>
>> You should see something that looks roughly like this:
>>
>> |tfiala at tfiala2:/mnt/ssd/work/macosx.sync/mbp-svn/build-debug/lib$
>> ldd liblldb.so
>> linux-vdso.so.1 => (0x00007fffe9dfe000)
>> libstdc++.so.6 =>
>> /usr/lib/x86_64-linux-gnu/libstdc++.so.6
>> (0x00007f6357e71000)
>> libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6
>> (0x00007f6357b6b000)
>> libedit.so.2 =>
>> /usr/lib/x86_64-linux-gnu/libedit.so.2 (0x00007f635793a000)
>> libpanel.so.5 =>
>> /usr/lib/x86_64-linux-gnu/libpanel.so.5 (0x00007f6357736000)
>> libncurses.so.5 =>
>> /lib/x86_64-linux-gnu/libncurses.so.5 (0x00007f6357513000)
>> libtinfo.so.5 =>
>> /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007f63572e9000)
>> libpython2.7.so.1.0 =>
>> /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
>> (0x00007f6356d82000)
>> librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1
>> (0x00007f6356b7a000)
>> libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2
>> (0x00007f6356975000)
>> libpthread.so.0 =>
>> /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f6356757000)
>> libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1
>> (0x00007f635653e000)
>> libgcc_s.so.1 =>
>> /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f6356327000)
>> libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
>> (0x00007f6355f61000)
>> /lib64/ld-linux-x86-64.so.2 (0x00007f635d584000)
>> libutil.so.1 =>
>> /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f6355d5d000)
>> tfiala at tfiala2:/mnt/ssd/work/macosx.sync/mbp-svn/build-debug/lib$site-packages
>>
>> |
>>
>> The left side shows the short name that the .so is
>> referencing. The right side shows what the loader actually
>> mapped it to with the current environment. If there is a
>> problem loading liblldb.so, this will show as (I think)
>> question marks on the right side indicating that the
>> shared library linkage was not found. Let me know if you
>> see something like that.
>>
>> If you get past this part and you don’t see any issues,
>> the next thing to try is:
>> PYTHONPATH=/path/to/your/lldb/lib/python2.7/site-packages
>> python
>> Then do an ‘import lldb’, then a ‘dir (lldb)’, like so:
>>
>> |tfiala at tfiala2:/mnt/ssd/work/macosx.sync/mbp-svn/build-debug/lib$
>> PYTHONPATH=`pwd`/python2.7/site-packages python
>> Python 2.7.6 (default, Mar 22 2014, 22:59:56)
>> [GCC 4.8.2] on linux2
>> Type "help", "copyright", "credits" or "license" for more
>> information.
>> >>> import lldb
>> >>> dir(lldb)
>> ['INT32_MAX', 'LLDB_ARCH_DEFAULT',
>> 'LLDB_ARCH_DEFAULT_32BIT', 'LLDB_ARCH_DEFAULT_64BIT',
>> 'LLDB_DEFAULT_BREAK_SIZE', 'LLDB_DEFAULT_SHELL',
>> 'LLDB_GENERIC_ERROR', 'LLDB_INVALID_ADDRESS',
>> 'LLDB_INVALID_BREAK_ID', 'LLDB_INVALID_CPUTYPE',
>> 'LLDB_INVALID_FRAME_ID', 'LLDB_INVALID_IMAGE_TOKEN',
>> 'LLDB_INVALID_INDEX32', 'LLDB_INVALID_IVAR_OFFSET',
>> 'LLDB_INVALID_LINE_NUMBER', 'LLDB_INVALID_MODULE_VERSION',
>> 'LLDB_INVALID_OFFSET', 'LLDB_INVALID_PROCESS_ID',
>> 'LLDB_INVALID_QUEUE_ID', 'LLDB_INVALID_REGNUM',
>> 'LLDB_INVALID_SIGNAL_NUMBER', 'LLDB_INVALID_THREAD_ID',
>> 'LLDB_INVALID_UID', 'LLDB_INVALID_WATCH_ID',
>> 'LLDB_MAX_NUM_OPTION_SETS', 'LLDB_OPT_SET_1',
>>
>> ... (lots of stuff removed)
>> |
>>
>> That will tell you if you can load the lldb module when
>> you specify exactly where it should be. If you can do
>> this, the tests basically should not have trouble loading
>> lldb. (Obviously somewhere we’ll find an issue here). If
>> this does work, then we need to trace why the python test
>> runner calls are mixing up the python path. In general you
>> don’t need to specify any options to the ninja/make
>> commands to get the tests to run from the build dir.
>>
>> Can you refresh my memory on what Linux distro you’re
>> using and whether it’s 64 or 32 bit? (I don’t get over to
>> the 32-bit distros much these days).
>>
>> I hope that gets us closer to finding out what’s up! Keep
>> me posted.
>>
>> -Todd
>>
>>
>>
>>
>> On Fri, Jul 18, 2014 at 4:56 AM, Matthew Gardiner
>> <mg11 at csr.com <mailto:mg11 at csr.com> <mailto:mg11 at csr.com
>> <mailto:mg11 at csr.com>>> wrote:
>>
>> Hi Todd,
>>
>> Well I managed to get cmake and ninja working! I
>> needed (it would
>> appear) to download/build/install the latest versions
>> of both from
>> source. Then I do:
>>
>> build $ cmake ../llvm/ -G Ninja
>> -DLLVM_TARGETS_TO_BUILD="X86"
>> -DBUILD_SHARED_LIBS=ON -DLLVM_ENABLE_CXX1Y=ON
>> ...
>> build $ ninja
>> ...
>>
>> However "ninja check-lldb" still fails to find
>> lldb.py. It is does
>> get built, but the path which dotest.py uses to find
>> it is wrong
>> on my system. Anyway, I'm going to continue trying to
>> analyse it
>> myself for now. I'll probably post a new thread to
>> lldb-dev if I
>> continue to struggle.
>>
>> Anyway, have a good weekend,
>> Matt
>>
>>
>>
>>
>> Member of the CSR plc group of companies. CSR plc
>> registered in
>> England and Wales, registered number 4187346,
>> registered office
>> Churchill House, Cambridge Business Park, Cowley Road,
>> Cambridge,
>> CB4 0WZ, United Kingdom
>> More information can be found at www.csr.com
>> <http://www.csr.com> <http://www.csr.com>.
>> Keep up to date with CSR on our technical blog,
>> www.csr.com/blog <http://www.csr.com/blog>
>> <http://www.csr.com/blog>, CSR people blog,
>> www.csr.com/people <http://www.csr.com/people>
>> <http://www.csr.com/people>, YouTube,
>> www.youtube.com/user/CSRplc
>> <http://www.youtube.com/user/CSRplc>
>> <http://www.youtube.com/user/CSRplc>, Facebook,
>> www.facebook.com/pages/CSR/191038434253534
>> <http://www.facebook.com/pages/CSR/191038434253534>
>> <http://www.facebook.com/pages/CSR/191038434253534>,
>> or follow us
>> on Twitter at www.twitter.com/CSR_plc
>> <http://www.twitter.com/CSR_plc>
>> <http://www.twitter.com/CSR_plc>.
>> New for 2014, you can now access the wide range of
>> products
>> powered by aptX at www.aptx.com <http://www.aptx.com>
>> <http://www.aptx.com>.
>>
>>
>>
>>
>> -- Todd Fiala | Software Engineer |
>> tfiala at google.com
>> <mailto:tfiala at google.com> <mailto:tfiala at google.com
>> <mailto:tfiala at google.com>> | 650-943-3180
>> <tel:650-943-3180>
>>
>>
>>
>>
>> To report this email as spam click here
>> <https://www.mailcontrol.com/sr/ik8jrXaVfBTGX2PQPOmvUj%21GOBh06pKK3+K9eZ%212Dx6F%21Jeeftdx9RBpik88jjuoRisi26jPZSzJaydNu2tBwg==>.
>>
>>
>> _______________________________________________
>> lldb-dev mailing list
>> lldb-dev at cs.uiuc.edu <mailto:lldb-dev at cs.uiuc.edu>
>> http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
>>
>>
>>
>>
>>
>> --
>> Todd Fiala | Software Engineer | tfiala at google.com
>> <mailto:tfiala at google.com> | 650-943-3180
>>
>>
>
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
More information about the lldb-dev
mailing list