[llvm-dev] RFC: Default path for cross-compiled runtimes

Tue Dec 19 11:53:10 PST 2017

On Tue, Dec 19, 2017 at 8:33 AM Jonathan Roelofs <jonathan at codesourcery.com>
wrote:

> On 12/19/17 9:15 AM, Petr Hosek via llvm-dev wrote:
> > Today, there're two different locations for runtimes files within
> > Clang's installation:
> >
> > compiler-rt:
> >    headers: $prefix/lib/clang/$version/include(/sanitizer)
> >    libraries:
> > $prefix/lib/clang/$version/lib/$os/libclang_rt.$name-$arch.$ext
> >
> > libc++, libc++abi, libunwind:
> >    headers: $prefix/include/c++/v1
> >    libraries: $prefix/lib/$name.$ext
> >
> > The scheme used by libc++, libc++abi, libunwind doesn't support targets
> > other than the host which is a problem when cross-compiling.
>
> Yes, it does: --sysroot=
>

What if my sysroot doesn't contains C++ library, or even if it does I may
still want to use libc++ shipped with the toolchain e.g. because the one
that's part of the sysroot doesn't support C++17?

I don't like the "build libc++ separately and then put it inside your
sysroot" solution for several reasons, most importantly because the sysroot
typically considered read-only e.g. I cannot modify Xcode's sysroot
replacing whatever libc++ is already there, but I can use libSystem.dylib
from Xcode's sysroot with my own libc++ version. In our case also we ship
the toolchain separately from the sysroot at different frequencies because
they come from different source and system libraries change far less often
than libc++.

What this means in practice is that when cross-compiling you have to do
something like:

clang++ --target=<arch>-<vendor>-<os>
--sysroot=/path/to/sysroot -stdlib=libc++ -nostdinc++
-I<libc++-install-prefix>/include/c++/v1 -L<libc++-install-prefix>/lib

This is even when the C++ library for your target is part of your Clang
toolchain so the driver arguably should know how to find it without you
having to duplicate the driver logic.

> What's currently missing is a standardized naming for a) where the
> sysroots live, and b) what they're named. Host libraries should continue
> to live in $prefix/{include, lib, lib32, lib64} as appropriate, whereas
> target libraries should live in something like:
> $prefix/clang-runtimes/$triple/$multilib/{usr/include, usr/lib, etc.}
>

I don't care too much about what the path is going to be. However, it'd be
nice if we could unify the paths between compiler-rt runtimes and libc++. I
always assumed that $prefix/lib/clang/$version was the path for runtimes
hence suggesting it.

>   In our
> > toolchain, we would like to build runtimes for all host and target
> > platforms we support, e.g. a single toolchain will have runtimes for
> > x86_64 and aarch64 Linux, Fuchsia and Windows. All you need to provide
> > is the target triple and the sysroot. While this is possible with
> > builtins, sanitizers and other compiler-rt runtimes, it's not possible
> > with libc++, libc++abi, libunwind.
> >
> > Our proposal is to move both compiler-rt and libc++, libc++abi,
> > libunwind into a new location that would support cross-compilation and
> > unify the layout:
> >
> > headers: $prefix/lib/clang/$version/include(/$triple)(/c++/v1)
> > libraries: $prefix/lib/clang/$version/$triple/lib/$name.$ext
>
> I don't think it's a good idea to tie all the runtimes to the compiler
> like that. It makes sense for the builtins to live there since they are
> heavily coupled with the specific compiler version, but I'm not
> convinced libc++/libc++abi/libunwind should.
>

They may be less tied than sanitizers because they don't rely on compiler
instrumentation, but there's still some dependency, e.g. in order to use
coroutines in libc++ I need a version of Clang that has the appropriate
intrinsics.

I could build libc++ against an older version of Clang which will disable
features not supported by that Clang version, but that version is again
tied to that version of Clang.

> > This means that for compiler-rt, the main difference would be moving the
> > runtime libraries to an target specific subdirectory rather than
> > including the architecture in the library name; for libc++, libc++abi,
> > libunwind, both headers and libraries will be moved to a new, target
> > specific location.
> >
> > In terms of implementation, we'll need to modify the Clang driver to use
> > this location when looking for runtimes and C++ libraries and headers.
> > This should be a non-intrusive change: we'll modify the driver to look
> > into the new location in addition to the existing ones, so if the new
> > path doesn't exist, the driver will simply fallback to the existing
> > behavior. When this is done, we need to modify the CMake build to
> > install files into the new location.
> >
> > This layout would be only used when runtimes are built as part of the
> > llvm/runtimes tree or using LLVM_ENABLE_RUNTIMES in the monorepo layout
> > (because this setup supports cross-compiling runtimes). When built as
> > part of LLVM or standalone, libc++, libc++abi, libunwind would still
> > install their files to $prefix/include and $prefix/lib as today.
> >
> > Once the overall scheme is agreed upon, we could also consider
> > de-duplicating C++ headers across targets, by moving shared headers into
> > a common directory, with only varying subset in include/$triple.
>
> At the very least the __config_site headers cannot be de-duplicated.
>

I think it should be sufficient to have separate __config headers for
separate targets, that's really the only thing that differs.

> Beyond that, I strongly disagree with de-duplicating them in general
> because it will break --sysroot= unless you add a bunch of symlinks...
> which don't exist on every host platform.

I don't think you need symlinks, all you need is to put both include/c++/v1
and include/$triple/c++/v1 to your include paths, where the former contains
all the common headers while the latter contains the target specific
__config. This is exactly what GNU "multi-arch" layout does.

> >
> > To give an example, for x86_64 and aarch64 Linux, this would look like:
> >
> > $prefix/lib/clang/6.0.0/include/sanitizer
> > $prefix/lib/clang/6.0.0/include/c++/v1
> > $prefix/lib/clang/6.0.0/include/x86_64-linux-gnu/c++/v1/__config
> > ...
> > $prefix/lib/clang/6.0.0/x86_64-linux-gnu/lib/libclang_rt.asan.so
> > <http://libclang_rt.asan.so>
> > $prefix/lib/clang/6.0.0/x86_64-linux-gnu/lib/libc++.so
> > ...
> > $prefix/lib/clang/6.0.0/aarch64-linux-gnu/lib/libclang_rt.asan.so
> > <http://libclang_rt.asan.so>
> > $prefix/lib/clang/6.0.0/aarch64-linux-gnu/lib/libc++.so
> > ...
> >
> >
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >
>
> --
> Jon Roelofs
> jonathan at codesourcery.com
> CodeSourcery / Mentor Embedded / Siemens
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20171219/afb043de/attachment-0001.html>