[llvm-dev] Trying to create a pure LLVM toolchain on musl based distribution

[David Demelier via llvm-dev]

Hello,

I'm trying to create a pure LLVM toolchain (that will not depend on GNU 
and produce GNU-free code too) on a musl based distribution.

For now, I use gcc to bootstrap and build all LLVM components. I do it 
individually because I was running out of space and memory trying to 
build all using LLVM_ENABLE_PROJECTS. Also, I don't want to create a 
all-in-one package. Then, once I'm able to build program with clang, 
I'll rebuild all using clang instead of gcc.

# LLVM

I've built LLVM using the following configuration:

     -DCMAKE_BUILD_TYPE=Release
     -DBUILD_SHARED_LIBS=On
     -DCMAKE_INSTALL_PREFIX=/usr
     -DLLVM_HOST_TRIPLE=x86_64-linux-musl
     -DLLVM_ENABLE_RTTI=On

# compiler-rt

I've needed to disable xray/sanitizers otherwise it didn't build on musl.

     -DCMAKE_BUILD_TYPE=Release
     -DCMAKE_INSTALL_PREFIX=/usr
     -DCOMPILER_RT_BUILD_BUILTINS=On
     -DCOMPILER_RT_BUILD_SANITIZERS=Off
     -DCOMPILER_RT_BUILD_XRAY=Off
     -DCOMPILER_RT_USE_BUILTINS_LIBRARY=On

# libc++

     -DBUILD_SHARED_LIBS=On
     -DCMAKE_BUILD_TYPE=Release
     -DCMAKE_INSTALL_PREFIX=/usr
     -DLIBCXX_HAS_MUSL_LIBC=On
     -DLIBCXX_HAS_GCC_S_LIB=Off
     -DLIBCXX_ENABLE_STATIC=Off

# libc++abi

     -DBUILD_SHARED_LIBS=On
     -DCMAKE_BUILD_TYPE=Release
     -DCMAKE_INSTALL_PREFIX=/usr
     -DLIBCXXABI_ENABLE_STATIC=Off

# libunwind

     -DBUILD_SHARED_LIBS=On
     -DCMAKE_BUILD_TYPE=Release
     -DCMAKE_INSTALL_PREFIX=/usr
     -DLIBUNWIND_ENABLE_STATIC=Off
     -DLIBUNWIND_USE_COMPILER_RT=On

# clang

And finally clang:

     -DBUILD_SHARED_LIBS=On
     -DCMAKE_BUILD_TYPE=Release
     -DCMAKE_INSTALL_PREFIX=/usr
     -DCLANG_DEFAULT_CXX_STDLIB=libc++
     -DCLANG_DEFAULT_LINKER=lld
     -DCLANG_DEFAULT_RTLIB=compiler-rt
     -DCLANG_VENDOR="Vanilla Linux"

Everything built fine, now the problem is the location of compiler-rt 
libraries which are not the correct place where clang/ld.lld searches. 
As the following output shows:

clang -v /tmp/test.c
Vanilla Linux clang version 8.0.0 (tags/RELEASE_800/final) (based on 
LLVM 8.0.0)
Target: x86_64-unknown-linux-musl
Thread model: posix
InstalledDir: /usr/bin
  "/usr/bin/clang-8" -cc1 -triple x86_64-unknown-linux-musl -emit-obj 
-mrelax-all -disable-free -disable-llvm-verifier -discard-value-names 
-main-file-name test.c -mrelocation-model pic -pic-level 2 -pic-is-pie 
-mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose 
-mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu 
x86-64 -dwarf-column-info -debugger-tuning=gdb -v -resource-dir 
/usr/lib/clang/8.0.0 -internal-isystem /usr/local/include 
-internal-isystem /usr/lib/clang/8.0.0/include -internal-externc-isystem 
/include -internal-externc-isystem /usr/include -fdebug-compilation-dir 
/usr/src/vanilla -ferror-limit 19 -fmessage-length 184 
-fobjc-runtime=gcc -fdiagnostics-show-option -o /tmp/test-1d99ce.o -x c 
/tmp/test.c -faddrsig
clang -cc1 version 8.0.0 based upon LLVM 8.0.0 default target 
x86_64-linux-musl
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
  /usr/local/include
  /usr/lib/clang/8.0.0/include
  /usr/include
End of search list.
  "/usr/bin/ld.lld" -pie --eh-frame-hdr -m elf_x86_64 -dynamic-linker 
/lib/ld-musl-x86_64.so.1 -o a.out /usr/bin/../lib/Scrt1.o 
/usr/bin/../lib/crti.o crtbeginS.o -L/usr/bin/../lib -L/lib -L/usr/lib 
/tmp/test-1d99ce.o 
/usr/lib/clang/8.0.0/lib/linux/libclang_rt.builtins-x86_64.a -lc 
/usr/lib/clang/8.0.0/lib/linux/libclang_rt.builtins-x86_64.a crtendS.o 
/usr/bin/../lib/crtn.o
ld.lld: error: cannot open crtbeginS.o: No such file or directory
ld.lld: error: cannot open 
/usr/lib/clang/8.0.0/lib/linux/libclang_rt.builtins-x86_64.a: No such 
file or directory
ld.lld: error: cannot open 
/usr/lib/clang/8.0.0/lib/linux/libclang_rt.builtins-x86_64.a: No such 
file or directory
ld.lld: error: cannot open crtendS.o: No such file or directory
clang-8: error: linker command failed with exit code 1 (use -v to see 
invocation)

compiler-rt installed files under the following directories hierarchy 
instead:

usr/lib/linux/libclang_rt.builtins-x86_64.a
usr/lib/linux/libclang_rt.fuzzer-x86_64.a
usr/lib/linux/libclang_rt.fuzzer_no_main-x86_64.a
usr/lib/linux/libclang_rt.profile-x86_64.a

What did I misconfigured? Is it clang/lld or compiler-rt? Also do I need 
to do something else regarding crtbeginS.o and crtendS.o that are not 
provided by musl?

Regards,

-- 
David