[llvm-dev] Trying to create a pure LLVM toolchain on musl based distribution

[Peter Smith via llvm-dev]

Hello David,

I don't know much about the specifics of Musl, so I'm responding generally.

As I understand it, clang expects to find the compiler-rt libraries
relative to the resource directory, which you can find out the
location of with clang --print-resource-dir . By default it is
lib/clang/9.0.0 assuming you are building from master. I think that
-DCMAKE_INSTALL_PREFIX=/usr has broken that assumption. I think that
you would either need to take out the CMAKE_INSTALL_PREFIX or change
the location of the resource directory, which I think that you can
alter at build time.

I don't think that there is a way of preventing clang adding
crtbeginS.o and crtendS.o without --nostdlib or --nostartfiles but
using these options will also not add the other crt*.o files that you
may be expecting.

There is a line in lib/Driver/Toolchains/Gnu.cpp which the Linux driver uses:
const bool HasCRTBeginEndFiles =
      ToolChain.getTriple().hasEnvironment() ||
      (ToolChain.getTriple().getVendor() != llvm::Triple::MipsTechnologies);
It looks like Musl may need to be included there?

Peter

On Mon, 25 Mar 2019 at 12:52, David Demelier via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>
> Hello,
>
> I'm trying to create a pure LLVM toolchain (that will not depend on GNU
> and produce GNU-free code too) on a musl based distribution.
>
> For now, I use gcc to bootstrap and build all LLVM components. I do it
> individually because I was running out of space and memory trying to
> build all using LLVM_ENABLE_PROJECTS. Also, I don't want to create a
> all-in-one package. Then, once I'm able to build program with clang,
> I'll rebuild all using clang instead of gcc.
>
> # LLVM
>
> I've built LLVM using the following configuration:
>
>      -DCMAKE_BUILD_TYPE=Release
>      -DBUILD_SHARED_LIBS=On
>      -DCMAKE_INSTALL_PREFIX=/usr
>      -DLLVM_HOST_TRIPLE=x86_64-linux-musl
>      -DLLVM_ENABLE_RTTI=On
>
> # compiler-rt
>
> I've needed to disable xray/sanitizers otherwise it didn't build on musl.
>
>      -DCMAKE_BUILD_TYPE=Release
>      -DCMAKE_INSTALL_PREFIX=/usr
>      -DCOMPILER_RT_BUILD_BUILTINS=On
>      -DCOMPILER_RT_BUILD_SANITIZERS=Off
>      -DCOMPILER_RT_BUILD_XRAY=Off
>      -DCOMPILER_RT_USE_BUILTINS_LIBRARY=On
>
> # libc++
>
>      -DBUILD_SHARED_LIBS=On
>      -DCMAKE_BUILD_TYPE=Release
>      -DCMAKE_INSTALL_PREFIX=/usr
>      -DLIBCXX_HAS_MUSL_LIBC=On
>      -DLIBCXX_HAS_GCC_S_LIB=Off
>      -DLIBCXX_ENABLE_STATIC=Off
>
> # libc++abi
>
>      -DBUILD_SHARED_LIBS=On
>      -DCMAKE_BUILD_TYPE=Release
>      -DCMAKE_INSTALL_PREFIX=/usr
>      -DLIBCXXABI_ENABLE_STATIC=Off
>
> # libunwind
>
>      -DBUILD_SHARED_LIBS=On
>      -DCMAKE_BUILD_TYPE=Release
>      -DCMAKE_INSTALL_PREFIX=/usr
>      -DLIBUNWIND_ENABLE_STATIC=Off
>      -DLIBUNWIND_USE_COMPILER_RT=On
>
> # clang
>
> And finally clang:
>
>      -DBUILD_SHARED_LIBS=On
>      -DCMAKE_BUILD_TYPE=Release
>      -DCMAKE_INSTALL_PREFIX=/usr
>      -DCLANG_DEFAULT_CXX_STDLIB=libc++
>      -DCLANG_DEFAULT_LINKER=lld
>      -DCLANG_DEFAULT_RTLIB=compiler-rt
>      -DCLANG_VENDOR="Vanilla Linux"
>
> Everything built fine, now the problem is the location of compiler-rt
> libraries which are not the correct place where clang/ld.lld searches.
> As the following output shows:
>
> clang -v /tmp/test.c
> Vanilla Linux clang version 8.0.0 (tags/RELEASE_800/final) (based on
> LLVM 8.0.0)
> Target: x86_64-unknown-linux-musl
> Thread model: posix
> InstalledDir: /usr/bin
>   "/usr/bin/clang-8" -cc1 -triple x86_64-unknown-linux-musl -emit-obj
> -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names
> -main-file-name test.c -mrelocation-model pic -pic-level 2 -pic-is-pie
> -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose
> -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu
> x86-64 -dwarf-column-info -debugger-tuning=gdb -v -resource-dir
> /usr/lib/clang/8.0.0 -internal-isystem /usr/local/include
> -internal-isystem /usr/lib/clang/8.0.0/include -internal-externc-isystem
> /include -internal-externc-isystem /usr/include -fdebug-compilation-dir
> /usr/src/vanilla -ferror-limit 19 -fmessage-length 184
> -fobjc-runtime=gcc -fdiagnostics-show-option -o /tmp/test-1d99ce.o -x c
> /tmp/test.c -faddrsig
> clang -cc1 version 8.0.0 based upon LLVM 8.0.0 default target
> x86_64-linux-musl
> ignoring nonexistent directory "/include"
> #include "..." search starts here:
> #include <...> search starts here:
>   /usr/local/include
>   /usr/lib/clang/8.0.0/include
>   /usr/include
> End of search list.
>   "/usr/bin/ld.lld" -pie --eh-frame-hdr -m elf_x86_64 -dynamic-linker
> /lib/ld-musl-x86_64.so.1 -o a.out /usr/bin/../lib/Scrt1.o
> /usr/bin/../lib/crti.o crtbeginS.o -L/usr/bin/../lib -L/lib -L/usr/lib
> /tmp/test-1d99ce.o
> /usr/lib/clang/8.0.0/lib/linux/libclang_rt.builtins-x86_64.a -lc
> /usr/lib/clang/8.0.0/lib/linux/libclang_rt.builtins-x86_64.a crtendS.o
> /usr/bin/../lib/crtn.o
> ld.lld: error: cannot open crtbeginS.o: No such file or directory
> ld.lld: error: cannot open
> /usr/lib/clang/8.0.0/lib/linux/libclang_rt.builtins-x86_64.a: No such
> file or directory
> ld.lld: error: cannot open
> /usr/lib/clang/8.0.0/lib/linux/libclang_rt.builtins-x86_64.a: No such
> file or directory
> ld.lld: error: cannot open crtendS.o: No such file or directory
> clang-8: error: linker command failed with exit code 1 (use -v to see
> invocation)
>
> compiler-rt installed files under the following directories hierarchy
> instead:
>
> usr/lib/linux/libclang_rt.builtins-x86_64.a
> usr/lib/linux/libclang_rt.fuzzer-x86_64.a
> usr/lib/linux/libclang_rt.fuzzer_no_main-x86_64.a
> usr/lib/linux/libclang_rt.profile-x86_64.a
>
> What did I misconfigured? Is it clang/lld or compiler-rt? Also do I need
> to do something else regarding crtbeginS.o and crtendS.o that are not
> provided by musl?
>
> Regards,
>
> --
> David
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev