<div dir="ltr"><div class="gmail_quote"><div dir="ltr">On Tue, Dec 19, 2017 at 12:33 PM Jonathan Roelofs <<a href="mailto:jonathan@codesourcery.com" target="_blank">jonathan@codesourcery.com</a>> wrote:</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF">
<div class="m_687200001658868573m_-1440258751414844925m_3402099528619821175moz-cite-prefix">On 12/19/17 12:53 PM, Petr Hosek wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_quote">
<div dir="ltr">On Tue, Dec 19, 2017 at 8:33 AM Jonathan
Roelofs <<a href="mailto:jonathan@codesourcery.com" target="_blank">jonathan@codesourcery.com</a>>
wrote:</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
On 12/19/17 9:15 AM, Petr Hosek via llvm-dev wrote:<br>
> Today, there're two different locations for runtimes
files within<br>
> Clang's installation:<br>
><br>
> compiler-rt:<br>
> headers:
$prefix/lib/clang/$version/include(/sanitizer)<br>
> libraries:<br>
>
$prefix/lib/clang/$version/lib/$os/libclang_rt.$name-$arch.$ext<br>
><br>
> libc++, libc++abi, libunwind:<br>
> headers: $prefix/include/c++/v1<br>
> libraries: $prefix/lib/$name.$ext<br>
><br>
> The scheme used by libc++, libc++abi, libunwind doesn't
support targets<br>
> other than the host which is a problem when
cross-compiling.<br>
<br>
Yes, it does: --sysroot=<br>
</blockquote>
<div><br>
</div>
<div>What if my sysroot doesn't contains C++ library, or even
if it does I may still want to use libc++ shipped with the
toolchain e.g. because the one that's part of the sysroot
doesn't support C++17? </div>
<div><br>
</div>
<div>I don't like the "build libc++ separately and then put it
inside your sysroot" solution for several reasons, most
importantly because the sysroot typically considered
read-only e.g.</div></div></div></blockquote></div><div text="#000000" bgcolor="#FFFFFF">
Copy on write.</div></blockquote><div> </div></div><div dir="ltr"><div class="gmail_quote"><div>I'm not sure what you mean, can you go into more detail?</div><div><br></div><div>What I'm proposing is basically an overlay, but rather than relying on filesystem support which is not portable, I'd like to simply rely on the compiler driver and include/library paths, i.e. first look here and if you don't don't find headers/libraries there keep looking in other include/library paths.</div><div><br></div><div>When you use -stdlib=libc++ today, Clang driver will always first look into ../include/c++/v1 (see <a href="https://github.com/llvm-mirror/clang/blob/master/lib/Driver/ToolChains/Linux.cpp#L736">https://github.com/llvm-mirror/clang/blob/master/lib/Driver/ToolChains/Linux.cpp#L736</a>) and only then check the sysroot. I don't wan to replace or alter --sysroot=, what I'm proposing is generalizing the existing logic to support multiarch, akin to libstdc++ (see <a href="https://github.com/llvm-mirror/clang/blob/master/lib/Driver/ToolChains/Linux.cpp#L774">https://github.com/llvm-mirror/clang/blob/master/lib/Driver/ToolChains/Linux.cpp#L774</a>).</div></div></div><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF">
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_quote">
<div> I cannot modify Xcode's sysroot replacing whatever
libc++ is already there, but I can use libSystem.dylib from
Xcode's sysroot with my own libc++ version. In our case also
we ship the toolchain separately from the sysroot at
different frequencies because they come from different
source and system libraries change far less often than
libc++.</div>
<div><br>
</div>
<div>What this means in practice is that when cross-compiling
you have to do something like:</div>
<div><br>
</div>
<div>clang++ --target=<arch>-<vendor>-<os>
--sysroot=/path/to/sysroot -stdlib=libc++ -nostdinc++
-I<libc++-install-prefix>/include/c++/v1
-L<libc++-install-prefix>/lib</div></div></div></blockquote></div><div text="#000000" bgcolor="#FFFFFF">
From that perspective, there's merit to splitting out what goes in a
sysroot, vs what goes in an "SDK". A sysroot should be a copy of
what's actually on a user's base system, whereas an SDK would
contain whatever gets layered on top of that.</div></blockquote><div> </div><div>Yes, I agree. In our case, we'd like for runtimes to be a part of the "LLVM SDK". The reason is that they are being built and tested as part of LLVM build, separately from the sysroot.</div><div><br></div><div>This is what we already discussed in <a href="https://reviews.llvm.org/D32816" style="outline:transparent solid 1px">https://reviews.llvm.org/D32816</a> and the solution implemented there and in <a href="https://reviews.llvm.org/D32613">https://reviews.llvm.org/D32613</a> is already being used for Fuchsia. However, we would like to use the same setup on other platforms as well (e.g. in our case Linux and Darwin), so this proposal is an attempt at generalizing that solution.<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><blockquote type="cite">
<div dir="ltr">
<div class="gmail_quote">
<div>This is even when the C++ library for your target is part
of your Clang toolchain so the driver arguably should know
how to find it without you having to duplicate the driver
logic.</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
What's currently missing is a standardized naming for a)
where the<br>
sysroots live, and b) what they're named. Host libraries
should continue<br>
to live in $prefix/{include, lib, lib32, lib64} as
appropriate, whereas<br>
target libraries should live in something like:<br>
$prefix/clang-runtimes/$triple/$multilib/{usr/include,
usr/lib, etc.}<br>
</blockquote>
<div><br>
</div>
</div>
<div dir="ltr">
<div class="gmail_quote">
<div>I don't care too much about what the path is going to
be. However, it'd be nice if we could unify the paths
between compiler-rt runtimes and libc++. I always assumed
that $prefix/lib/clang/$version was the path for runtimes
hence suggesting it.</div>
</div>
</div>
<div dir="ltr">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
In our<br>
> toolchain, we would like to build runtimes for all
host and target<br>
> platforms we support, e.g. a single toolchain will
have runtimes for<br>
> x86_64 and aarch64 Linux, Fuchsia and Windows. All
you need to provide<br>
> is the target triple and the sysroot. While this is
possible with<br>
> builtins, sanitizers and other compiler-rt runtimes,
it's not possible<br>
> with libc++, libc++abi, libunwind.<br>
><br>
> Our proposal is to move both compiler-rt and libc++,
libc++abi,<br>
> libunwind into a new location that would support
cross-compilation and<br>
> unify the layout:<br>
><br>
> headers:
$prefix/lib/clang/$version/include(/$triple)(/c++/v1)<br>
> libraries:
$prefix/lib/clang/$version/$triple/lib/$name.$ext<br>
<br>
I don't think it's a good idea to tie all the runtimes to
the compiler<br>
like that. It makes sense for the builtins to live there
since they are<br>
heavily coupled with the specific compiler version, but
I'm not<br>
convinced libc++/libc++abi/libunwind should.<br>
</blockquote>
<div><br>
</div>
<div>They may be less tied than sanitizers because they
don't rely on compiler instrumentation, but there's still
some dependency, e.g. in order to use coroutines in libc++
I need a version of Clang that has the appropriate
intrinsics.</div></div></div></div></blockquote></div><div text="#000000" bgcolor="#FFFFFF">
Or a version of GCC that supports them. Clang isn't the only client
of these runtimes.</div></blockquote><div> </div><div>Yes, but in that case you're likely going to build them separately from LLVM as a standalone project in which case they're going to use the standard layout same as today. What I'm proposing is intended for the llvm/runtimes (I'm not sure what better to call it) when cross-compiling runtimes as part of the LLVM toolchain build.</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div>I could build libc++ against an older version of Clang
which will disable features not supported by that Clang
version, but that version is again tied to that version of
Clang.</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
> This means that for compiler-rt, the main difference
would be moving the<br>
> runtime libraries to an target specific subdirectory
rather than<br>
> including the architecture in the library name; for
libc++, libc++abi,<br>
> libunwind, both headers and libraries will be moved
to a new, target<br>
> specific location.<br>
><br>
> In terms of implementation, we'll need to modify the
Clang driver to use<br>
> this location when looking for runtimes and C++
libraries and headers.<br>
> This should be a non-intrusive change: we'll modify
the driver to look<br>
> into the new location in addition to the existing
ones, so if the new<br>
> path doesn't exist, the driver will simply fallback
to the existing<br>
> behavior. When this is done, we need to modify the
CMake build to<br>
> install files into the new location.<br>
><br>
> This layout would be only used when runtimes are
built as part of the<br>
> llvm/runtimes tree or using LLVM_ENABLE_RUNTIMES in
the monorepo layout<br>
> (because this setup supports cross-compiling
runtimes). When built as<br>
> part of LLVM or standalone, libc++, libc++abi,
libunwind would still<br>
> install their files to $prefix/include and
$prefix/lib as today.<br>
><br>
> Once the overall scheme is agreed upon, we could also
consider<br>
> de-duplicating C++ headers across targets, by moving
shared headers into<br>
> a common directory, with only varying subset in
include/$triple.<br>
<br>
At the very least the __config_site headers cannot be
de-duplicated.<br>
</blockquote>
<div><br>
</div>
<div>I think it should be sufficient to have separate
__config headers for separate targets, that's really the
only thing that differs.<br>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Beyond that, I strongly disagree with de-duplicating them
in general<br>
because it will break --sysroot= unless you add a bunch of
symlinks...<br>
which don't exist on every host platform.</blockquote>
<div><br>
</div>
<div>I don't think you need symlinks, all you need is to put
both include/c++/v1 and include/$triple/c++/v1 to your
include paths, where the former contains all the common
headers while the latter contains the target specific
__config. This is exactly what GNU "multi-arch" layout
does.</div></div></div></div></blockquote></div><div text="#000000" bgcolor="#FFFFFF">
I guess, but then you're pulling pieces out of the sysroot, and
still breaking what --sysroot= is for.</div></blockquote><div><br></div><div>If you have a sysroot that contains everything you need and all you want from Clang is the tooling then yes, but different users have different use cases.<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
><br>
> To give an example, for x86_64 and aarch64 Linux,
this would look like:<br>
><br>
> $prefix/lib/clang/6.0.0/include/sanitizer<br>
> $prefix/lib/clang/6.0.0/include/c++/v1<br>
>
$prefix/lib/clang/6.0.0/include/x86_64-linux-gnu/c++/v1/__config<br>
> ...<br>
> $prefix/lib/clang/6.0.0/x86_64-linux-gnu/lib/<a href="http://libclang_rt.asan.so" rel="noreferrer" target="_blank">libclang_rt.asan.so</a><br>
> <<a href="http://libclang_rt.asan.so" rel="noreferrer" target="_blank">http://libclang_rt.asan.so</a>><br>
>
$prefix/lib/clang/6.0.0/x86_64-linux-gnu/lib/libc++.so<br>
> ...<br>
> $prefix/lib/clang/6.0.0/aarch64-linux-gnu/lib/<a href="http://libclang_rt.asan.so" rel="noreferrer" target="_blank">libclang_rt.asan.so</a><br>
> <<a href="http://libclang_rt.asan.so" rel="noreferrer" target="_blank">http://libclang_rt.asan.so</a>><br>
>
$prefix/lib/clang/6.0.0/aarch64-linux-gnu/lib/libc++.so<br>
> ...<br>
><br>
><br>
> _______________________________________________<br>
> LLVM Developers mailing list<br>
> <a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
> <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
><br>
<br>
--<br>
Jon Roelofs<br>
<a href="mailto:jonathan@codesourcery.com" target="_blank">jonathan@codesourcery.com</a><br>
CodeSourcery / Mentor Embedded / Siemens<br>
</blockquote>
</div>
</div>
</div>
</blockquote>
<br>
<pre class="m_687200001658868573m_-1440258751414844925m_3402099528619821175moz-signature" cols="72">--
Jon Roelofs
<a class="m_687200001658868573m_-1440258751414844925m_3402099528619821175moz-txt-link-abbreviated" href="mailto:jonathan@codesourcery.com" target="_blank">jonathan@codesourcery.com</a>
CodeSourcery / Mentor Embedded / Siemens</pre>
</div></blockquote></div></div></div>