[llvm] [docs] Rewrite HowToCrossCompileLLVM (PR #129451)

[Fangrui Song via llvm-commits]

================
@@ -1,215 +1,238 @@
 ===================================================================
-How To Cross-Compile Clang/LLVM using Clang/LLVM
+How to cross-compile Clang/LLVM using Clang/LLVM
 ===================================================================
 
 Introduction
-============
+------------
 
 This document contains information about building LLVM and
-Clang on host machine, targeting another platform.
+Clang on a host machine, targeting another platform.
 
 For more information on how to use Clang as a cross-compiler,
 please check https://clang.llvm.org/docs/CrossCompilation.html.
 
-TODO: Add MIPS and other platforms to this document.
+This document describes cross-building a compiler in a single stage, using an
+existing ``clang`` install as the host compiler.
 
-Cross-Compiling from x86_64 to ARM
-==================================
+.. note::
+  These instructions have been tested for targeting 32-bit ARM, AArch64, or
+  64-bit RISC-V from an x86_64 Linux host. But should be equally applicable to
+  any other target.
 
-In this use case, we'll be using CMake and Ninja, on a Debian-based Linux
-system, cross-compiling from an x86_64 host (most Intel and AMD chips
-nowadays) to a hard-float ARM target (most ARM targets nowadays).
-
-The packages you'll need are:
-
- * ``cmake``
- * ``ninja-build`` (from backports in Ubuntu)
- * ``gcc-4.7-arm-linux-gnueabihf``
- * ``gcc-4.7-multilib-arm-linux-gnueabihf``
- * ``binutils-arm-linux-gnueabihf``
- * ``libgcc1-armhf-cross``
- * ``libsfgcc1-armhf-cross``
- * ``libstdc++6-armhf-cross``
- * ``libstdc++6-4.7-dev-armhf-cross``
-
-Configuring CMake
------------------
-
-For more information on how to configure CMake for LLVM/Clang,
-see :doc:`CMake`.
-
-The CMake options you need to add are:
-
- * ``-DCMAKE_SYSTEM_NAME=<target-system>``
- * ``-DCMAKE_INSTALL_PREFIX=<install-dir>``
- * ``-DLLVM_HOST_TRIPLE=arm-linux-gnueabihf``
- * ``-DLLVM_TARGETS_TO_BUILD=ARM``
-
-Note: ``CMAKE_CROSSCOMPILING`` is always set automatically when ``CMAKE_SYSTEM_NAME`` is set. Don't put ``-DCMAKE_CROSSCOMPILING=TRUE`` in your options.
-
-Also note that ``LLVM_HOST_TRIPLE`` specifies the triple of the system
-that the cross built LLVM is going to run on - the flag is named based
-on the autoconf build/host/target nomenclature. (This flag implicitly sets
-other defaults, such as ``LLVM_DEFAULT_TARGET_TRIPLE``.)
+Setting up a sysroot
+--------------------
 
-If you're compiling with GCC, you can use architecture options for your target,
-and the compiler driver will detect everything that it needs:
+You will need a sysroot that contains essential build dependencies compiled
+for the target architecture. In this case, we will be using CMake and Ninja on
+a Linux host and compiling against a Debian sysroot. Detailed instructions on
+producing sysroots are outside of the scope of this documentation, but the
+following instructions should work on any Linux distribution with these
+pre-requisites:
 
- * ``-DCMAKE_CXX_FLAGS='-march=armv7-a -mcpu=cortex-a9 -mfloat-abi=hard'``
+ * ``binfmt_misc`` configured to execute ``qemu-user`` for binaries of the
+   target architecture. This is done by installing the ``qemu-user-static``
+   and ``binfmt-support`` packages on Debian-derived distributions.
+ * Root access (setups involving ``proot`` or other tools to avoid this
+   requirement may be possible, but aren't described here).
+ * The ``debootstrap`` tool. This is available in most distributions.
 
-However, if you're using Clang, the driver might not be up-to-date with your
-specific Linux distribution, version or GCC layout, so you'll need to fudge.
+The following snippet will initialise sysroots for 32-bit Arm, AArch64, and
+64-bit RISC-V (just pick the target(s) you are interested in):
 
-In addition to the ones above, you'll also need:
+   .. code-block:: bash
 
- * ``--target=arm-linux-gnueabihf`` or whatever is the triple of your cross GCC.
- * ``'--sysroot=/usr/arm-linux-gnueabihf'``, ``'--sysroot=/opt/gcc/arm-linux-gnueabihf'``
-   or whatever is the location of your GCC's sysroot (where /lib, /bin etc are).
- * Appropriate use of ``-I`` and ``-L``, depending on how the cross GCC is installed,
-   and where are the libraries and headers.
+    sudo debootstrap --arch=armhf --variant=minbase --include=build-essential,symlinks stable sysroot-deb-armhf-stable
+    sudo debootstrap --arch=arm64 --variant=minbase --include=build-essential,symlinks stable sysroot-deb-arm64-stable
+    sudo debootstrap --arch=riscv64 --variant=minbase --include=build-essential,symlinks unstable sysroot-deb-riscv64-unstable
 
-You may also want to set the ``LLVM_NATIVE_TOOL_DIR`` option - pointing
-at a directory with prebuilt LLVM tools (``llvm-tblgen``, ``clang-tblgen``
-etc) for the build host, allowing you to them reuse them if available.
-E.g. ``-DLLVM_NATIVE_TOOL_DIR=<path-to-native-llvm-build>/bin``.
-If the option isn't set (or the directory doesn't contain all needed tools),
-the LLVM cross build will automatically launch a nested build to build the
-tools that are required.
+The created sysroot may contain absolute symlinks, which will resolve to a
+location within the host when accessed during compilation, so we must convert
+any absolute symlinks to relative ones:
 
-The CXX flags define the target, cpu (which in this case
-defaults to ``fpu=VFP3`` with NEON), and forcing the hard-float ABI. If you're
-using Clang as a cross-compiler, you will *also* have to set ``--sysroot``
-to make sure it picks the correct linker.
+   .. code-block:: bash
 
-When using Clang, it's important that you choose the triple to be *identical*
-to the GCC triple and the sysroot. This will make it easier for Clang to
-find the correct tools and include headers. But that won't mean all headers and
-libraries will be found. You'll still need to use ``-I`` and ``-L`` to locate
-those extra ones, depending on your distribution.
+    sudo chroot sysroot-of-your-choice symlinks -cr .
 
-Most of the time, what you want is to have a native compiler to the
-platform itself, but not others. So there's rarely a point in compiling
-all back-ends. For that reason, you should also set the
-``TARGETS_TO_BUILD`` to only build the back-end you're targeting to.
 
-You must set the ``CMAKE_INSTALL_PREFIX``, otherwise a ``ninja install``
-will copy ARM binaries to your root filesystem, which is not what you
-want.
+Configuring CMake and building
+------------------------------
 
-Hacks
------
+For more information on how to configure CMake for LLVM/Clang,
+see :doc:`CMake`. Following CMake's recommended practice, we will create a
+`toolchain file
+<https://cmake.org/cmake/help/book/mastering-cmake/chapter/Cross%20Compiling%20With%20CMake.html#toolchain-files>`_. 
 
-There are some bugs in current LLVM, which require some fiddling before
-running CMake:
+The following assumes you have a system install of ``clang`` and ``lld`` that
+will be used for cross compiling and that the listed commands are executed
+from within the root of a checkout of the ``llvm-project`` git repository.
 
-#. If you're using Clang as the cross-compiler, there is a problem in
-   the LLVM ARM back-end that is producing absolute relocations on
-   position-independent code (``R_ARM_THM_MOVW_ABS_NC``), so for now, you
-   should disable PIC:
+First, set variables in your shell session that will be used throughout the
+build instructions:
 
    .. code-block:: bash
 
-      -DLLVM_ENABLE_PIC=False
+    SYSROOT=$HOME/sysroot-deb-arm64-stable
+    TARGET=aarch64-linux-gnu
+    CFLAGS=""
 
-   This is not a problem, since Clang/LLVM libraries are statically
-   linked anyway, it shouldn't affect much.
+To customise details of the compilation target or choose a different
+architecture altogether, change the ``SYSROOT``,
+``TARGET``, and ``CFLAGS`` variables to something matching your target. For
+example, for 64-bit RISC-V you might set
+``SYSROOT=$HOME/sysroot-deb-riscv64-unstable``, ``TARGET=riscv64-linux-gnu``
+and ``CFLAGS="-march=rva20u64"``. Refer to documentation such as your target's
+compiler documentation or processor manual for guidance on which ``CFLAGS``
+settings may be appropriate. The specified ``TARGET`` should match the triple
+used within the sysroot (i.e. ``$SYSROOT/usr/lib/$TARGET`` should exist).
 
-#. The ARM libraries won't be installed in your system.
-   But the CMake prepare step, which checks for
-   dependencies, will check the *host* libraries, not the *target*
-   ones. Below there's a list of some dependencies, but your project could
-   have more, or this document could be outdated. You'll see the errors
-   while linking as an indication of that.
+Then execute the following snippet to create a toolchain file:
 
-   Debian based distros have a way to add ``multiarch``, which adds
-   a new architecture and allows you to install packages for those
-   systems. See https://wiki.debian.org/Multiarch/HOWTO for more info.
+   .. code-block:: bash
 
-   But not all distros will have that, and possibly not an easy way to
-   install them in any anyway, so you'll have to build/download
-   them separately.
+    cat - <<EOF > $TARGET-clang.cmake
+    set(CMAKE_SYSTEM_NAME Linux)
+    set(CMAKE_SYSROOT "$SYSROOT")
+    set(CMAKE_C_COMPILER_TARGET $TARGET)
+    set(CMAKE_CXX_COMPILER_TARGET $TARGET)
+    set(CMAKE_C_FLAGS_INIT "$CFLAGS")
+    set(CMAKE_CXX_FLAGS_INIT "$CFLAGS")
+    set(CMAKE_LINKER_TYPE LLD)
+    set(CMAKE_C_COMPILER clang)
+    set(CMAKE_CXX_COMPILER clang++)
+    set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
+    set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
+    set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
+    set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
+    EOF
+
+
+Then configure and build by invoking ``cmake``:
 
-   A quick way of getting the libraries is to download them from
-   a distribution repository, like Debian (http://packages.debian.org/jessie/),
-   and download the missing libraries. Note that the ``libXXX``
-   will have the shared objects (``.so``) and the ``libXXX-dev`` will
-   give you the headers and the static (``.a``) library. Just in
-   case, download both.
+   .. code-block:: bash
 
-   The ones you need for ARM are: ``libtinfo``, ``zlib1g``,
-   ``libxml2`` and ``liblzma``. In the Debian repository you'll
-   find downloads for all architectures.
+    cmake -G Ninja \
+      -DCMAKE_BUILD_TYPE=Release \
+      -DLLVM_ENABLE_PROJECTS="lld;clang" \
----------------
MaskRay wrote:

worth mentioning compiler-rt (and libc++)? it could be challenging to set up and readers might want to know how to do it.

https://github.com/llvm/llvm-project/pull/129451