[lldb-dev] cross compile setup
Todd Fiala
todd.fiala at gmail.com
Tue Jul 8 16:54:52 PDT 2014
Hey Paul,
Thanks for all the details!
On Fri, Jul 4, 2014 at 2:41 PM, Paul Osmialowski <pawelo at king.net.pl> wrote:
>
> Hi Todd,
>
> As I promissed:
>
> Host: Gentoo Linux x86_64 (a.k.a. amd64)
> Target: FoundationV8 ARM64 emulator running minimal Linux rootfs image
> Objective: cross-compile as much as possible of llvm + clang + lldb +
> compiler_rt from master branch of their repositories for ARM64/AArch64
> architecture then try to execute lldb and lldb-gdbserver on the Target; try
> to fulfil ncurses and Python dependency
>
>
Okay, good to know what you're shooting for. I'll be working in this same
area very shortly since I want to get Linux ARM64 support working, later
for Android. I had worked briefly with ARM on getting the FoundationV8
simulator running a Linaro build back when they started packaging the ARMv8
simulator with DS-5. I had to jump away from it, but coming back to it
very soon now that llgs is starting to get some traction.
> I'm using CMake throughout all the process. Note that I'm building
> everything in home directory and (with all required development tools
> installed), I don't need superuser privileges at any stage.
>
>
Great.
> Gentoo specific issues:
>
I used to use Gentoo for years - on both x86 and PowerPC (old Mac box)
setups. Glad you're testing things out there!
> 1. Currently, default C/C++ compiler for Gentoo systems is gcc 4.7.x.
> Latest llvm/clang/lldb code requires at least gcc 4.8.x for proper
> compilation. Gcc 4.8.3 must be installed in a separate slot
> (Gentoo-specific approach for holding multiple versions of the software);
> CC and CXX shell variables must be set for CMake to indicate which compiler
> should be used
>
We'll need to fiddle maybe a bit with this. I *think* we should be able to
get gcc 4.7.1+ working, but might require slightly different flags. If we
don't accept gcc 4.7, I think (depending on the work needed) we may want to
get that working anyway to stay at parity with minimum llvm/clang
toolchains.
> 2. get_python_lib Python function from distutils.sysconfig returns path
> which contains '/lib64/' part no matter how first two boolean parameters
> are set (see revision D4058 for fix proposed by me; also patch attached to
> bug report 20206 touches crucial lines of one of affected files); this is
> important only when you plan to do 'make install' (and my approach relys on
> this)
> 3. Python 2.7 in Gentoo is compiled with unicode ucs4, while Python 2.7
> installed on the Target is compiled with unicode ucs2 - separate local
> Python installation must be used during cross-compilation in order to make
> swig happy (installation procedure will be described later).
>
4. Python 2.7 is not the default Python interpreter in Gentoo (currently it
> is Python 3.x) while lldb requires version 2.7; fortunately, many different
> versions of Python interpreter can co-exist in one system - whenever
> required, exact version can be specified to use.
>
> Ok.
Realistically I'll be setting up my environment on Ubuntu 14.04 x86_64 (our
current dev platform over here). However, I will see what happens when I
use gcc 4.7.{latest}. If that's not too hard to fix up, I'll see if I can
get some cycles on that.
> Target specific issues:
> 1. The target architecture is recognised as aarch64 not ARM64, I prepared
> two revisions that try address this issue: D4381 and D4379.
>
Yeah, I think on the lldb side we're going to want to normalize those to
arm64, and deal with ABI differences based on target triples. I'll have a
look at your patches when I get there.
> 2. Ncurses library installed on target is not complete, i.e. libpanel part
> is missing; also two other required libraries aren't there: libedit and
> libxml2.
>
Hmm. If all you want on the ARM64 target is the lldb-gdbserver, then this
issue might go away when we do the static linking of lldb-gdbserver and
lldb-platform in the near future (some cmake/configure/make changes).
That's a little ways off but we're starting to work on it. The goal there
is to remove the llgs <=> liblldb.so linkage, so we can do tight static
linking and dead-code eliminate the majority of the core lldb code that
isn't needed for lldb-gdbserver. We may also do the same for
lldb-platform, although for our direct application this is not critical
path.
> 3. Python 2.7 installed on the Target is compiled with unicode ucs2.
>
> Two stages process:
> 1. Building for PC
> 2. Building for Target
>
> This process is split into two stages since the second stage requires
> llvm-tblgen and clang-tblgen built at the first stage (and since we're
> working with the most recent code, older versions of those two cannot be
> used).
>
> Getting repositories
>
> In my home directory I'm having following directory structure:
>
> $HOME/llvm.git - llvm repository cloned
> $HOME/llvm.git/projects/compiler-rt - compiler-rt repository cloned
> $HOME/llvm.git/tools/clang - clang repository cloned
> $HOME/llvm.git/tools/lldb - lldb repository cloned, that's where the magic
> happens
>
> Building for PC
> ===============
>
> I'm building the stuff for PC in $HOME/llvm/build directory (contrary to
> ARM64 stuff which I'm building in $HOME/llvm-aarch64/build directory).
>
> Being in previously created $HOME/llvm/build directory I'm invoking what
> follows:
>
> $ CC=gcc-4.8.3 \
> CXX=g++-4.8.3 \
> CFLAGS=-pipe \
> CXXFLAGS=-pipe \
> cmake \
> -DPYTHON_EXECUTABLE=/usr/bin/python2.7 \
> -DCMAKE_INSTALL_PREFIX=$HOME/llvm \
> ../../llvm.git
>
> Note that you can select different CMAKE_INSTALL_PREFIX, or you can skip
> installation prefix selection at all so the outcome is to be installed
> (with sudo) in /usr/local by default.
> PYTHON_EXECUTABLE specifies which Python interpreter should be used.
>
> After running cmake (by invoking command above), now make can be invoked,
> e.g.:
>
> $ VERBOSE=1 make -j8
>
> (make sure you have a lot of RAM, 12GB or more, or use -j2 instead of -j8).
>
> Then, when build is successfully completed:
>
> $ make install
>
> And that's it.
>
> Building for ARM64
> ==================
>
> The Linaro toolchain with gcc 4.8.2 for AArch64
> (gcc-linaro-aarch64-linux-gnu-4.8-2013.08_linux.tar.xz) I'm having
> unpacked into $HOME/aarch64/gcc-linaro-aarch64-linux-gnu directory, i.e.
> gcc executable binary is:
>
> $HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc-4.8.2
>
> Due to target specific issues listed above, some dependencies must be
> cross-compiled first. In my home directory I'm initially having this:
>
> $HOME/llvm-aarch64/deps/src/crosscompileconf.sh
> $HOME/llvm-aarch64/deps/src/libedit-20130712-3.1.tar.gz
> $HOME/llvm-aarch64/deps/src/libxml2-2.9.1.tar.gz
> $HOME/llvm-aarch64/deps/libpython2.7.so.1.0
> $HOME/llvm-aarch64/deps/libpython2.7.so.1
> $HOME/llvm-aarch64/deps/libpython2.7.so
> $HOME/llvm-aarch64/deps-x86_64/src/confpython.sh
> $HOME/llvm-aarch64/deps-x86_64/src/Python-2.7.6.tar.xz
>
> The $HOME/llvm-aarch64/deps/libpython2.7.so.1.0 file was obtained form
> Target's rootfs. $HOME/llvm-aarch64/deps/libpython2.7.so.1 and
> $HOME/llvm-aarch64/deps/libpython2.7.so are just symlinks:
>
> libpython2.7.so -> libpython2.7.so.1.0
> libpython2.7.so.1 -> libpython2.7.so.1.0
>
> $ file libpython2.7.so.1.0
> libpython2.7.so.1.0: ELF 64-bit LSB shared object, ARM aarch64, version 1
> (SYSV), dynamically linked, BuildID[sha1]=c6b21fd1aa4225ed9cd979a0dd7b59d1cbd5f281,
> stripped
> $ $HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-objdump
> -T libpython2.7.so |grep -i ucs | head -1
> 00000000000bbba0 g DF .text 0000000000000054 Base _PyUnicodeUCS2_
> AsDefaultEncodedString
>
> It's clear it was built with unicode ucs2.
>
> It turned out that in order to build ncurses for AArch64, the very latest
> snapshot of ncurses from git is required. To have it, go to
> $HOME/llvm-aarch64/deps/src and type:
>
> $ git clone https://github.com/chitranshi/ncurses.git ncurses.git
>
> This leaves you with following directory:
>
> $HOME/llvm-aarch64/deps/src/ncurses.git
>
> In my case git SHA was:
>
> $ cd ncurses.git
> $ git rev-parse HEAD
> 79bb76ec205843adec46225f64aaa1f8e730b466
>
> My $HOME/llvm-aarch64/deps/src/crosscompileconf.sh file looks like this
> (forgive me poor mail formatting, I can send you that file separately if
> needed):
>
> PATH=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin:$PATH \
> PKG_CONFIG_PATH=$HOME/llvm-aarch64/deps/lib/pkgconfig:$PKG_CONFIG_PATH \
> CC=aarch64-linux-gnu-gcc-4.8.2 \
> CXX=aarch64-linux-gnu-g++ \
> CFLAGS="-fPIC -DPIC --sysroot=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/aarch64-linux-gnu/libc
> -isystem $HOME/aarch64/gcc-linaro-aarch64-linux-gnu/aarch64-linux-gnu/libc/usr/include
> -I$HOME/llvm-aarch64/deps/include/ncurses" \
> CXXFLAGS="-fPIC -DPIC --sysroot=$HOME/aarch64/gcc-
> linaro-aarch64-linux-gnu/aarch64-linux-gnu/libc -isystem
> $HOME/aarch64/gcc-linaro-aarch64-linux-gnu/aarch64-linux-gnu/libc/usr/include
> -I$HOME/llvm-aarch64/deps/include/ncurses" \
> LDFLAGS="--sysroot=$HOME/aarch64/gcc-linaro-aarch64-
> linux-gnu/aarch64-linux-gnu/libc -L$HOME/llvm-aarch64/deps/lib" \
> LIBS="--sysroot=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/aarch64-linux-gnu/libc
> -L$HOME/llvm-aarch64/deps/lib" \
> ./configure --prefix=$HOME/llvm-aarch64/deps --host=aarch64-linux-gnu
> --without-python --without-ada --disable-shared --enable-static
> --without-shared --without-cxx-shared --enable-safe-sprintf
>
>
> Note that all the dependencies I'm having as static libraries.
>
> $HOME/llvm-aarch64/deps-x86_64/src/confpython.sh looks like this:
>
> ./configure --prefix=$HOME/llvm-aarch64/deps-x86_64 --enable-unicode=ucs2
>
>
> To build Python with unicode ucs2, unpack Python-2.7.6.tar.xz in
> $HOME/llvm-aarch64/deps-x86_64/src and go to unpacked
> $HOME/llvm-aarch64/deps-x86_64/src/Python-2.7.6 directory.
>
> Invoke configuration script:
>
> $ source ../confpython.sh
>
> When finished type:
>
> $ make
>
> and when it's done type:
>
> $ make install
>
> This should leave you with one more Python 2.7 interpreter:
> $HOME/llvm-aarch64/deps-x86_64/bin/python2.7 and its companion files.
>
>
> To build libmxl2, go to $HOME/llvm-aarch64/deps/src and unpack, configure,
> build and install it in $HOME/llvm-aarch64/deps:
>
> $ tar -xzf libxml2-2.9.1.tar.gz
> $ cd libxml2-2.9.1
> $ source ../crosscompileconf.sh
> $ make
> $ make install
>
> To build ncurses go to $HOME/llvm-aarch64/deps/src/ncurses.git and type:
>
> $ source ../crosscompileconf.sh
> $ make
> $ make install
>
> Now with ncurses ready, you can build libedit, go to
> $HOME/llvm-aarch64/deps/src and do:
>
> $ tar -xzf libedit-20130712-3.1.tar.gz
> $ cd libedit-20130712-3.1
> $ source ../crosscompileconf.sh
> $ make
> $ make install
>
> After this, build environment is ready to use. Examine
> $HOME/llvm-aarch64/deps directory in order to make sure things are there in
> bin, include, lib and share directories.
>
> Create and go to $HOME/llvm-aarch64/build directory and type:
>
> $ PATH=$HOME/llvm-aarch64/deps/bin:$HOME/aarch64/gcc-linaro-
> aarch64-linux-gnu/bin:$HOME/llvm-aarch64/deps-x86_64/bin:$PATH \
> CC=aarch64-linux-gnu-gcc-4.8.2 \
> CXX=aarch64-linux-gnu-g++ \
> CFLAGS="--sysroot=$HOME/aarch64/gcc-linaro-aarch64-
> linux-gnu/aarch64-linux-gnu/libc -isystem $HOME/aarch64/gcc-linaro-
> aarch64-linux-gnu/aarch64-linux-gnu/libc/usr/include
> -I$HOME/llvm-aarch64/deps/include -I$HOME/llvm-aarch64/deps/include/ncurses
> -I$HOME/llvm-aarch64/deps-x86_64/include/python2.7" \
> CXXFLAGS="--sysroot=$HOME/aarch64/gcc-linaro-aarch64-
> linux-gnu/aarch64-linux-gnu/libc -isystem $HOME/aarch64/gcc-linaro-
> aarch64-linux-gnu/aarch64-linux-gnu/libc/usr/include
> -I$HOME/llvm-aarch64/deps/include -I$HOME/llvm-aarch64/deps/include/ncurses
> -I$HOME/llvm-aarch64/deps-x86_64/include/python2.7" \
> LDFLAGS="--sysroot=$HOME/aarch64/gcc-linaro-aarch64-
> linux-gnu/aarch64-linux-gnu/libc -L$HOME/llvm-aarch64/deps/lib
> -L$HOME/llvm-aarch64/deps" \
> LIBS="--sysroot=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/aarch64-linux-gnu/libc
> -L$HOME/llvm-aarch64/deps/lib -L$HOME/llvm-aarch64/deps" \
> PKG_CONFIG_PATH=$HOME/llvm-aarch64/deps/lib/pkgconfig:$
> HOME/llvm-aarch64/deps-x86_64/lib/pkgconfig \
> cmake \
> -DCMAKE_CROSSCOMPILING=True \
> -DPYTHON_EXECUTABLE=$HOME/llvm-aarch64/deps-x86_64/bin/python2.7 \
> -DCMAKE_LINKER=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-ld
> \
> -DCMAKE_AR=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-ar
> \
> -DCMAKE_NM=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-nm
> \
> -DCMAKE_OBJCOPY=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-objcopy
> \
> -DCMAKE_OBJDUMP=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-objdump
> \
> -DCMAKE_STRIP=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-strip
> \
> -DCMAKE_RANLIB=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-ranlib
> \
> -DCMAKE_INSTALL_PREFIX=$HOME/llvm-aarch64 \
> -DLLVM_TABLEGEN=$HOME/llvm/bin/llvm-tblgen \
> -DCLANG_TABLEGEN=$HOME/llvm/build/bin/clang-tblgen \
> -DLLVM_DEFAULT_TARGET_TRIPLE=aarch64-linux-gnu \
> -DLLVM_TARGET_ARCH=AArch64 \
> -DLLDB_DISABLE_CURSES=OFF \
> -DLLDB_DISABLE_PYTHON=OFF \
> -DPYTHON_LIBRARY=$HOME/llvm-aarch64/deps/libpython2.7.so.1.0 \
> ../../llvm.git
>
> Note that CLANG_TABLEGEN points to a binary file held in build directory.
> I've already posted a bug 19908 about that.
>
> After running cmake (by invoking command above), now make can be invoked,
> e.g.:
>
> $ VERBOSE=1 make -j8
>
> (make sure you have a lot of RAM, 12GB or more, or use -j2 instead of -j8).
>
> Then, when build is successfully completed:
>
> $ make install
>
> And that's it. Copy $HOME/llvm-aarch64/lib/liblldb.so.3.5.0,
> $HOME/llvm-aarch64/bin/lldb-3.5.0 (as lldb) and
> $HOME/llvm-aarch64/bin/lldb-gdbserver-3.5.0 (as lldb-gdbserver or llgs
> for short) to your Target.
> Note that these binaries are huge. Don't repeat my mistake and execute
> them from NFS exported directory. Copy to local filesystem before executing.
>
> # TERMINFO=/etc/terminfo LD_LIBRARY_PATH=. ./lldb
> (lldb) gui
>
> The ncurses gui looks bit funny when started on serial terminal emulator.
>
> Have fun,
>
>
Will do, thanks for the details!
Sincerely,
Todd Fiala
> Paul
>
> On Thu, 3 Jul 2014, Todd Fiala wrote:
>
> Thanks, Paul! No rush, just would love to see what that looks like so I
>> can replicate it over here at some point.
>>
>>
>> On Thu, Jul 3, 2014 at 2:14 PM, Paul Osmialowski <pawelo at king.net.pl>
>> wrote:
>> Hi Todd,
>>
>> I'll describe it soon (I guess tomorrow).
>>
>> Cheers,
>> Paul
>>
>> On Thu, 3 Jul 2014, Todd Fiala wrote:
>>
>> Hey Paul,
>> I was wondering if you could post your cross compile Linaro
>> setup. That would be great as a starting point as I look into
>> testing/reworking some of the Linux constants and start
>> gearing up on some alternate
>> architecture support.
>>
>> Much appreciated!
>>
>> Sincerely,
>> Todd Fiala
>>
>>
>>
>>
>>
>> --
>> -Todd
>>
>>
--
-Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20140708/d4ea05dd/attachment.html>
More information about the lldb-dev
mailing list