[PATCH] D110663: [Driver] Support Debian multiarch style lib/clang/14.0.0/x86_64-linux-gnu runtime path and include/x86_64-linux-gnu/c++/v1 libc++ path

Tue Jan 25 16:27:31 PST 2022

MaskRay added a comment.

No worries! Thanks for the reply.

I use Debian and want it to work well and support the `-DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=on` direction.
Currently, `-DLLVM_DEFAULT_TARGET_TRIPLE=aarch64-linux-gnu` (Debian style multiarch triple, so no `-unknown` or `-pc`) does not work for libc++ and some runtime libraries.
This led to the revert of D107799 <https://reviews.llvm.org/D107799> which I think really unfortunate because runtime builds encourage `LLVM_ENABLE_PER_TARGET_RUNTIME_DIR`, which de facto deviate from the regular builds and llvm-project has to support two hierarchies.

---

Here is `-DLLVM_DEFAULT_TARGET_TRIPLE=aarch64-unknown-linux-gnu`:

  cmake -GNinja -Hllvm -B/tmp/out/custom1 -DCMAKE_BUILD_TYPE=Release -DCMAKE_CROSSCOMPILING=on -DCMAKE_INSTALL_PREFIX=/tmp/opt/aarch64 -DLLVM_TARGETS_TO_BUILD=AArch64 -DLLVM_DEFAULT_TARGET_TRIPLE=aarch64-unknown-linux-gnu -DLLVM_TARGET_ARCH=AArch64 -DLLVM_ENABLE_PROJECTS='clang;lld' -DLLVM_ENABLE_RUNTIMES='compiler-rt;l
  ibcxx;libcxxabi;libunwind'
  ninja -C /tmp/out/custom1 lld cxx cxxabi unwind builtins

  /tmp/out/custom1/bin/clang++ -c -stdlib=libc++ a.cc; /tmp/out/custom1/bin/clang++ -fuse-ld=lld --dyld-prefix=/usr/aarch64-linux-gnu --unwindlib=libunwind --rtlib=compiler-rt -nostdlib++ -pthread -static-libgcc a.o -Wl,--push-state,-Bstatic,-lc++,-lc++abi,--pop-state,-rpath=/usr/aarch64-linux-gnu/lib -ldl -o a  # works
  ./a  # runs if you have binfmt_misc and qemu-aarch64-static

If I change the CMake line to use `-DLLVM_DEFAULT_TARGET_TRIPLE=aarch64-linux-gnu`, libc++ compiles do not work:

  % /tmp/out/custom1/bin/clang++ -fuse-ld=lld --dyld-prefix=/usr/aarch64-linux-gnu -Wl,-rpath=/usr/aarch64-linux-gnu/lib a.cc -o a
  % ./a  # works

  % /tmp/out/custom1/bin/clang++ -c -stdlib=libc++ a.cc
  In file included from a.cc:1:
  In file included from /tmp/out/custom1/bin/../include/c++/v1/stdio.h:101:
  /tmp/out/custom1/bin/../include/c++/v1/__config:13:10: fatal error: '__config_site' file not found
  #include <__config_site>
           ^~~~~~~~~~~~~~~
  1 error generated.

`/tmp/out/custom1/include/aarch64-linux-gnu/c++/v1/` is not in the search path.

With this change, the command will succeed. Here are the search paths for includes and libraries:

  % /tmp/out/custom1/bin/clang++ -fuse-ld=lld -stdlib=libc++ -v a.cc |& sed -E 's/ "?-[iIL]/\n&/g'
  clang version 14.0.0
  Target: aarch64-unknown-linux-gnu
  Thread model: posix
  InstalledDir: /tmp/out/custom1/bin
  Found candidate GCC installation: /usr/lib/gcc-cross/aarch64-linux-gnu/11
  Selected GCC installation: /usr/lib/gcc-cross/aarch64-linux-gnu/11
  Candidate multilib: .;@m64
  Selected multilib: .;@m64
   "/tmp/out/custom1/bin/clang-14" -cc1 -triple aarch64-unknown-linux-gnu -emit-obj -mrelax-all --mrelax-relocations -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name a.cc -mrelocation-model static -mframe-pointer=non-leaf -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu generic -target-feature +neon -target-feature +v8a -target-abi aapcs -fallow-half-arguments-and-returns -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=gdb -v -fcoverage-compilation-dir=/tmp/c -resource-dir /tmp/out/custom1/lib/clang/14.0.0
   -internal-isystem /tmp/out/custom1/bin/../include/aarch64-linux-gnu/c++/v1
   -internal-isystem /tmp/out/custom1/bin/../include/c++/v1
   -internal-isystem /tmp/out/custom1/lib/clang/14.0.0/include
   -internal-isystem /usr/local/include
   -internal-isystem /usr/lib/gcc-cross/aarch64-linux-gnu/11/../../../../aarch64-linux-gnu/include
   -internal-externc-isystem /include
   -internal-externc-isystem /usr/include -fdeprecated-macro -fdebug-compilation-dir=/tmp/c -ferror-limit 19 -fno-signed-char -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -target-feature +outline-atomics -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/a-ffa089.o -x c++ a.cc
  clang -cc1 version 14.0.0 based upon LLVM 14.0.0git default target aarch64-linux-gnu
  ignoring nonexistent directory "/include"
  #include "..." search starts here:
  #include <...> search starts here:
   /tmp/out/custom1/bin/../include/aarch64-linux-gnu/c++/v1
   /tmp/out/custom1/bin/../include/c++/v1
   /tmp/out/custom1/lib/clang/14.0.0/include
   /usr/local/include
   /usr/lib/gcc-cross/aarch64-linux-gnu/11/../../../../aarch64-linux-gnu/include
   /usr/include
  End of search list.
   "/tmp/out/custom1/bin/ld.lld" -EL --eh-frame-hdr -m aarch64linux -dynamic-linker /lib/ld-linux-aarch64.so.1 -o a.out /usr/lib/gcc-cross/aarch64-linux-gnu/11/../../../../aarch64-linux-gnu/lib/crt1.o /usr/lib/gcc-cross/aarch64-linux-gnu/11/../../../../aarch64-linux-gnu/lib/crti.o /usr/lib/gcc-cross/aarch64-linux-gnu/11/crtbegin.o
   -L/tmp/out/custom1/bin/../lib/aarch64-linux-gnu
   -L/usr/lib/gcc-cross/aarch64-linux-gnu/11
   -L/usr/lib/gcc-cross/aarch64-linux-gnu/11/../../../../lib64
   -L/lib/aarch64-linux-gnu
   -L/lib/../lib64
   -L/usr/lib/aarch64-linux-gnu
   -L/usr/lib/../lib64
   -L/usr/lib/gcc-cross/aarch64-linux-gnu/11/../../../../aarch64-linux-gnu/lib
   -L/tmp/out/custom1/bin/../lib
   -L/lib
   -L/usr/lib /tmp/a-ffa089.o -lc++ -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc /usr/lib/gcc-cross/aarch64-linux-gnu/11/crtend.o /usr/lib/gcc-cross/aarch64-linux-gnu/11/../../../../aarch64-linux-gnu/lib/crtn.o

> In my testing, I found that these users aren't currently using a single uniform target triple spelling, I've seen both ${arch}-linux-gnu and ${arch}-unknown-linux-gnu used pretty liberally. So if we were to land this change today, it would break many of these users. We could clean up and unify all of these uses, but it's going to take some effort because it spans lots of projects. I'm also worried that there might be other Clang users in a similar situation who are not included on this change.

This is not a problem. Both `-DLLVM_DEFAULT_TARGET_TRIPLE=aarch64-linux-gnu` and  `-DLLVM_DEFAULT_TARGET_TRIPLE=aarch64-unknown-linux-gnu` will work.

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D110663/new/

https://reviews.llvm.org/D110663