[lldb-dev] cross compile setup
Paul Osmialowski
pawelo at king.net.pl
Fri Jul 4 14:41:21 PDT 2014
Hi Todd,
As I promissed:
Host: Gentoo Linux x86_64 (a.k.a. amd64)
Target: FoundationV8 ARM64 emulator running minimal Linux rootfs image
Objective: cross-compile as much as possible of llvm + clang + lldb +
compiler_rt from master branch of their repositories for ARM64/AArch64
architecture then try to execute lldb and lldb-gdbserver on the Target;
try to fulfil ncurses and Python dependency
I'm using CMake throughout all the process. Note that I'm building
everything in home directory and (with all required development tools
installed), I don't need superuser privileges at any stage.
Gentoo specific issues:
1. Currently, default C/C++ compiler for Gentoo systems is gcc 4.7.x.
Latest llvm/clang/lldb code requires at least gcc 4.8.x for proper
compilation. Gcc 4.8.3 must be installed in a separate slot
(Gentoo-specific approach for holding multiple versions of the software);
CC and CXX shell variables must be set for CMake to indicate which
compiler should be used
2. get_python_lib Python function from distutils.sysconfig returns path
which contains '/lib64/' part no matter how first two boolean parameters
are set (see revision D4058 for fix proposed by me; also patch attached to
bug report 20206 touches crucial lines of one of affected files); this is
important only when you plan to do 'make install' (and my approach relys
on this)
3. Python 2.7 in Gentoo is compiled with unicode ucs4, while Python 2.7
installed on the Target is compiled with unicode ucs2 - separate local
Python installation must be used during cross-compilation in order to make
swig happy (installation procedure will be described later).
4. Python 2.7 is not the default Python interpreter in Gentoo (currently
it is Python 3.x) while lldb requires version 2.7; fortunately, many
different versions of Python interpreter can co-exist in one system -
whenever required, exact version can be specified to use.
Target specific issues:
1. The target architecture is recognised as aarch64 not ARM64, I prepared
two revisions that try address this issue: D4381 and D4379.
2. Ncurses library installed on target is not complete, i.e. libpanel part
is missing; also two other required libraries aren't there: libedit and
libxml2.
3. Python 2.7 installed on the Target is compiled with unicode ucs2.
Two stages process:
1. Building for PC
2. Building for Target
This process is split into two stages since the second stage requires
llvm-tblgen and clang-tblgen built at the first stage (and since we're
working with the most recent code, older versions of those two cannot be
used).
Getting repositories
In my home directory I'm having following directory structure:
$HOME/llvm.git - llvm repository cloned
$HOME/llvm.git/projects/compiler-rt - compiler-rt repository cloned
$HOME/llvm.git/tools/clang - clang repository cloned
$HOME/llvm.git/tools/lldb - lldb repository cloned, that's where the magic
happens
Building for PC
===============
I'm building the stuff for PC in $HOME/llvm/build directory (contrary to
ARM64 stuff which I'm building in $HOME/llvm-aarch64/build directory).
Being in previously created $HOME/llvm/build directory I'm invoking what
follows:
$ CC=gcc-4.8.3 \
CXX=g++-4.8.3 \
CFLAGS=-pipe \
CXXFLAGS=-pipe \
cmake \
-DPYTHON_EXECUTABLE=/usr/bin/python2.7 \
-DCMAKE_INSTALL_PREFIX=$HOME/llvm \
../../llvm.git
Note that you can select different CMAKE_INSTALL_PREFIX, or you can skip
installation prefix selection at all so the outcome is to be installed
(with sudo) in /usr/local by default.
PYTHON_EXECUTABLE specifies which Python interpreter should be used.
After running cmake (by invoking command above), now make can be invoked,
e.g.:
$ VERBOSE=1 make -j8
(make sure you have a lot of RAM, 12GB or more, or use -j2 instead of
-j8).
Then, when build is successfully completed:
$ make install
And that's it.
Building for ARM64
==================
The Linaro toolchain with gcc 4.8.2 for AArch64
(gcc-linaro-aarch64-linux-gnu-4.8-2013.08_linux.tar.xz) I'm having
unpacked into $HOME/aarch64/gcc-linaro-aarch64-linux-gnu directory, i.e.
gcc executable binary is:
$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc-4.8.2
Due to target specific issues listed above, some dependencies must be
cross-compiled first. In my home directory I'm initially having this:
$HOME/llvm-aarch64/deps/src/crosscompileconf.sh
$HOME/llvm-aarch64/deps/src/libedit-20130712-3.1.tar.gz
$HOME/llvm-aarch64/deps/src/libxml2-2.9.1.tar.gz
$HOME/llvm-aarch64/deps/libpython2.7.so.1.0
$HOME/llvm-aarch64/deps/libpython2.7.so.1
$HOME/llvm-aarch64/deps/libpython2.7.so
$HOME/llvm-aarch64/deps-x86_64/src/confpython.sh
$HOME/llvm-aarch64/deps-x86_64/src/Python-2.7.6.tar.xz
The $HOME/llvm-aarch64/deps/libpython2.7.so.1.0 file was obtained form
Target's rootfs. $HOME/llvm-aarch64/deps/libpython2.7.so.1 and
$HOME/llvm-aarch64/deps/libpython2.7.so are just symlinks:
libpython2.7.so -> libpython2.7.so.1.0
libpython2.7.so.1 -> libpython2.7.so.1.0
$ file libpython2.7.so.1.0
libpython2.7.so.1.0: ELF 64-bit LSB shared object, ARM aarch64, version 1
(SYSV), dynamically linked,
BuildID[sha1]=c6b21fd1aa4225ed9cd979a0dd7b59d1cbd5f281, stripped
$ $HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-objdump
-T libpython2.7.so |grep -i ucs | head -1
00000000000bbba0 g DF .text 0000000000000054 Base
_PyUnicodeUCS2_AsDefaultEncodedString
It's clear it was built with unicode ucs2.
It turned out that in order to build ncurses for AArch64, the very latest
snapshot of ncurses from git is required. To have it, go to
$HOME/llvm-aarch64/deps/src and type:
$ git clone https://github.com/chitranshi/ncurses.git ncurses.git
This leaves you with following directory:
$HOME/llvm-aarch64/deps/src/ncurses.git
In my case git SHA was:
$ cd ncurses.git
$ git rev-parse HEAD
79bb76ec205843adec46225f64aaa1f8e730b466
My $HOME/llvm-aarch64/deps/src/crosscompileconf.sh file looks like this
(forgive me poor mail formatting, I can send you that file separately if
needed):
PATH=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin:$PATH \
PKG_CONFIG_PATH=$HOME/llvm-aarch64/deps/lib/pkgconfig:$PKG_CONFIG_PATH \
CC=aarch64-linux-gnu-gcc-4.8.2 \
CXX=aarch64-linux-gnu-g++ \
CFLAGS="-fPIC -DPIC
--sysroot=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/aarch64-linux-gnu/libc
-isystem
$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/aarch64-linux-gnu/libc/usr/include
-I$HOME/llvm-aarch64/deps/include/ncurses" \
CXXFLAGS="-fPIC -DPIC
--sysroot=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/aarch64-linux-gnu/libc
-isystem
$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/aarch64-linux-gnu/libc/usr/include
-I$HOME/llvm-aarch64/deps/include/ncurses" \
LDFLAGS="--sysroot=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/aarch64-linux-gnu/libc
-L$HOME/llvm-aarch64/deps/lib" \
LIBS="--sysroot=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/aarch64-linux-gnu/libc
-L$HOME/llvm-aarch64/deps/lib" \
./configure --prefix=$HOME/llvm-aarch64/deps --host=aarch64-linux-gnu
--without-python --without-ada --disable-shared --enable-static
--without-shared --without-cxx-shared --enable-safe-sprintf
Note that all the dependencies I'm having as static libraries.
$HOME/llvm-aarch64/deps-x86_64/src/confpython.sh looks like this:
./configure --prefix=$HOME/llvm-aarch64/deps-x86_64 --enable-unicode=ucs2
To build Python with unicode ucs2, unpack Python-2.7.6.tar.xz in
$HOME/llvm-aarch64/deps-x86_64/src and go to unpacked
$HOME/llvm-aarch64/deps-x86_64/src/Python-2.7.6 directory.
Invoke configuration script:
$ source ../confpython.sh
When finished type:
$ make
and when it's done type:
$ make install
This should leave you with one more Python 2.7 interpreter:
$HOME/llvm-aarch64/deps-x86_64/bin/python2.7 and its companion files.
To build libmxl2, go to $HOME/llvm-aarch64/deps/src and unpack, configure,
build and install it in $HOME/llvm-aarch64/deps:
$ tar -xzf libxml2-2.9.1.tar.gz
$ cd libxml2-2.9.1
$ source ../crosscompileconf.sh
$ make
$ make install
To build ncurses go to $HOME/llvm-aarch64/deps/src/ncurses.git and type:
$ source ../crosscompileconf.sh
$ make
$ make install
Now with ncurses ready, you can build libedit, go to
$HOME/llvm-aarch64/deps/src and do:
$ tar -xzf libedit-20130712-3.1.tar.gz
$ cd libedit-20130712-3.1
$ source ../crosscompileconf.sh
$ make
$ make install
After this, build environment is ready to use. Examine
$HOME/llvm-aarch64/deps directory in order to make sure things are there
in bin, include, lib and share directories.
Create and go to $HOME/llvm-aarch64/build directory and type:
$
PATH=$HOME/llvm-aarch64/deps/bin:$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin:$HOME/llvm-aarch64/deps-x86_64/bin:$PATH
\
CC=aarch64-linux-gnu-gcc-4.8.2 \
CXX=aarch64-linux-gnu-g++ \
CFLAGS="--sysroot=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/aarch64-linux-gnu/libc
-isystem
$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/aarch64-linux-gnu/libc/usr/include
-I$HOME/llvm-aarch64/deps/include
-I$HOME/llvm-aarch64/deps/include/ncurses
-I$HOME/llvm-aarch64/deps-x86_64/include/python2.7" \
CXXFLAGS="--sysroot=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/aarch64-linux-gnu/libc
-isystem
$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/aarch64-linux-gnu/libc/usr/include
-I$HOME/llvm-aarch64/deps/include
-I$HOME/llvm-aarch64/deps/include/ncurses
-I$HOME/llvm-aarch64/deps-x86_64/include/python2.7" \
LDFLAGS="--sysroot=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/aarch64-linux-gnu/libc
-L$HOME/llvm-aarch64/deps/lib -L$HOME/llvm-aarch64/deps" \
LIBS="--sysroot=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/aarch64-linux-gnu/libc
-L$HOME/llvm-aarch64/deps/lib -L$HOME/llvm-aarch64/deps" \
PKG_CONFIG_PATH=$HOME/llvm-aarch64/deps/lib/pkgconfig:$HOME/llvm-aarch64/deps-x86_64/lib/pkgconfig
\
cmake \
-DCMAKE_CROSSCOMPILING=True \
-DPYTHON_EXECUTABLE=$HOME/llvm-aarch64/deps-x86_64/bin/python2.7 \
-DCMAKE_LINKER=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-ld
\
-DCMAKE_AR=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-ar
\
-DCMAKE_NM=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-nm
\
-DCMAKE_OBJCOPY=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-objcopy
\
-DCMAKE_OBJDUMP=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-objdump
\
-DCMAKE_STRIP=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-strip
\
-DCMAKE_RANLIB=$HOME/aarch64/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-ranlib
\
-DCMAKE_INSTALL_PREFIX=$HOME/llvm-aarch64 \
-DLLVM_TABLEGEN=$HOME/llvm/bin/llvm-tblgen \
-DCLANG_TABLEGEN=$HOME/llvm/build/bin/clang-tblgen \
-DLLVM_DEFAULT_TARGET_TRIPLE=aarch64-linux-gnu \
-DLLVM_TARGET_ARCH=AArch64 \
-DLLDB_DISABLE_CURSES=OFF \
-DLLDB_DISABLE_PYTHON=OFF \
-DPYTHON_LIBRARY=$HOME/llvm-aarch64/deps/libpython2.7.so.1.0 \
../../llvm.git
Note that CLANG_TABLEGEN points to a binary file held in build directory.
I've already posted a bug 19908 about that.
After running cmake (by invoking command above), now make can be invoked,
e.g.:
$ VERBOSE=1 make -j8
(make sure you have a lot of RAM, 12GB or more, or use -j2 instead of
-j8).
Then, when build is successfully completed:
$ make install
And that's it. Copy $HOME/llvm-aarch64/lib/liblldb.so.3.5.0,
$HOME/llvm-aarch64/bin/lldb-3.5.0 (as lldb) and
$HOME/llvm-aarch64/bin/lldb-gdbserver-3.5.0 (as lldb-gdbserver or llgs for
short) to your Target.
Note that these binaries are huge. Don't repeat my mistake and execute
them from NFS exported directory. Copy to local filesystem before
executing.
# TERMINFO=/etc/terminfo LD_LIBRARY_PATH=. ./lldb
(lldb) gui
The ncurses gui looks bit funny when started on serial terminal emulator.
Have fun,
Paul
On Thu, 3 Jul 2014, Todd Fiala wrote:
> Thanks, Paul! No rush, just would love to see what that looks like so I can replicate it over here at some point.
>
>
> On Thu, Jul 3, 2014 at 2:14 PM, Paul Osmialowski <pawelo at king.net.pl> wrote:
> Hi Todd,
>
> I'll describe it soon (I guess tomorrow).
>
> Cheers,
> Paul
>
> On Thu, 3 Jul 2014, Todd Fiala wrote:
>
> Hey Paul,
> I was wondering if you could post your cross compile Linaro setup. That would be great as a starting point as I look into testing/reworking some of the Linux constants and start
> gearing up on some alternate
> architecture support.
>
> Much appreciated!
>
> Sincerely,
> Todd Fiala
>
>
>
>
>
> --
> -Todd
>
>
More information about the lldb-dev
mailing list