[cfe-dev] Making libc++ on Linux user-friendly

[Sebastian Redl]

Hi,

I'm currently setting up a fresh Linux box (recent Mint, using GCC 4.8
as system compiler) as a dev environment, and since my stuff is using
cutting-edge C++, I want Clang and libc++ for this. Unfortunately,
getting this to work isn't fun at all.

Clang itself is easy.

libc++ is anything but. Reading the instructions provided[1], it looks
very much like building with libsupc++, pulled out of the libstdc++.so,
is the best way of doing things. Using libcxxrt (or libc++abi, but why
would I do that on Linux) is listed as having a little drawback:
"Unfortunately you can't simply run clang with "-stdlib=libc++" at this
point, as clang is set up to link for libc++ linked to libsupc++."

Therefore I go with this CMake command line:

|CC=clang CXX=clang++ cmake -G Ninja -DLIBCXX_CXX_ABI=libstdc++
-DLIBCXX_LIBSUPCXX_INCLUDE_PATHS="/usr/include/c++/4.8/;/usr/include/|||x86_64-linux-gnu/|c++/4.8/"
-DCMAKE_BUILD_TYPE=Release <libc++-source-dir>|

Note that I removed the install prefix so it defaults to /usr/local.
This means I need an ldconfig update after installing, but that part
works. Also, my Clang sits in /usr/local, so the header search path
setup in Clang does the right thing. I'm also using Ninja, but I tested
and it doesn't change anything compared to Makefiles.

After building and installing, I try to compile a small example program
that uses dynamic_cast and iostreams.

#include <iostream>
struct Base { virtual ~Base() {} };
struct Derived : Base {
  void x() { std::cout << "Hello, World!\n"; }
};
void foo(Base& b) {
  dynamic_cast<Derived&>(b).x();
}
int main() {
    Derived d;
    foo(d);
}

And I compile it using

$ clang++ -stdlib=libc++ -o hello-llvm hello.cpp

This fails:

/usr/bin/ld: /tmp/hello-7f7d71.o: undefined reference to symbol
'__cxa_free_exception@@CXXABI_1.3'
/usr/lib/x86_64-linux-gnu/libstdc++.so.6: error adding symbols: DSO
missing from command line
clang-3.5: error: linker command failed with exit code 1 (use -v to see
invocation)

Here's the problem: when building libc++, the linker finds the various
ABI functions in libstdc++, and is quite happy with them being there.
When Clang calls the linker for the actual program, though, it doesn't
pass along a link flag for libstdc++, only for libc++. Thus, the links
fails.

Ironically enough, I can't even manually add -lstdc++ to the command
line: Clang recognizes this as an attempt to link the standard library
and replaces with with -lc++ - so the final link command has two -lc++
flags and the link fails with the same error. I have to give the full
path to the library as an input file to make it work. This is not a good
situation.

If we strike the goal of having only one copy of the C++ ABI stuff in
the final program, even if it links to a C++ library using libstdc++,
then we have more options. Using libcxxrt is on the table, and so is
linking against the static libsupc++ library. The latter shouldn't use
any additional command line flags, so let's try this:

|CC=clang CXX=clang++ cmake -G Ninja -DLIBCXX_CXX_ABI=libsupc++
-DLIBCXX_LIBSUPCXX_INCLUDE_PATHS="/usr/include/c++/4.8/;/usr/include/|||x86_64-linux-gnu/|c++/4.8/"
-DCMAKE_BUILD_TYPE=Release <libc++-source-dir>|

This doesn't work either for some reason:

$ clang++ -stdlib=libc++ -o hello-llvm hello.cpp
/tmp/hello-3689e5.o: In function `foo(Base&)':
hello.cpp:(.text+0x39): undefined reference to `__dynamic_cast'
hello.cpp:(.text+0x4e): undefined reference to `__cxa_bad_cast'
clang-3.5: error: linker command failed with exit code 1 (use -v to see
invocation)

Using nm to inspect libc++.so, it appears that the linker just doesn't
pull in (or at least export) these two functions. This, in turn, is
probably because it doesn't use them (libc++ doesn't use dynamic_cast in
any compiled code; there's one use in the inline rethrow_exception, one
in the template dynamic_pointer_cast, and one in an unevaluated context
in type_traits). Neither is there any linker input instructing the
linker to pull in the symbol, or an appropriate use of --whole-archive.

This again can be worked around by explicitly specifying linking against
the source library, and here -lsupc++ works.


The bottom line is that none of the libc++ options on Linux work out of
the box without additional command line arguments beyond -stdlib=libc++,
and the best option (libstdc++, which prevents duplication of the ABI
stuff in a process) is actually the worst case, because -lstdc++ doesn't
work as a workaround.


This makes me unhappy. So I did some research into possible solutions.

The libsupc++ problem is independent of the others. It basically comes
down to using --whole-archive to pull in libsupc++.a completely into the
resulting .so, not just the parts that happen to be used. A patch
hacking the CMakeLists.txt to do exactly that is attached.


The other problem is this: how do we make it so that no additional
command line flags are required, no matter what ABI library is used?
Ideally, the libc++.so would just tell the linker to pull the ABI
library into the link, or claim to export the symbols itself but really
just forward to the underlying library.

The first solution is exactly what libtool does, but using the wrapper
isn't really an option, not to mention that it would require writing the
libtool specs manually or using libtool to build libc++. Neither of
these are attractive options. The linker itself does not have any way of
doing this, even though reading the documentation for --rpath-link makes
it sound like it does.

The second solution is what is done on MacOS. The MachO linker supports
the reexport_library option. Similarly, PE/COFF DLLs on Windows support
such forwarding symbols (kernel32.dll contains lots of forwarders to
ntdll.dll); an OldNewThing article[2] describes this and the comments
contain links to more detailed information.

It appears, however, from my reading of the ld man page and extensive
web searching, that there is no way to do the equivalent thing on Linux
without actually adding stub forwarders to the library in question. I
have asked on StackOverflow[3], but I don't really expect an answer.

So it seems there are these options:

1) Add stub forwarders to libc++. This is annoying, but it works with
libc++ modifications alone.
2) Have Clang find out by some means what flags, beyond -lc++, are
needed to link against libc++. This information would have to be
provided by libc++ somehow.
3) Have the user tell Clang. For example, -stdlib=libc++ could instead
be -stdlib=libc++-gnuabi or -stdlib=libc++-cxxrt, depending on the ABI
lib. Or the ABI could be a separate option, i.e. -stdlib=libc++
-abilib=cxxrt. Of course, this is little better than the state we have now.

Any thoughts on this? Preferred variant? Things I've overlooked?


[1] http://libcxx.llvm.org/
[2] http://blogs.msdn.com/b/oldnewthing/archive/2006/07/19/671238.aspx
[3]
http://stackoverflow.com/questions/22764734/linux-equivalent-of-windows-dll-forwarders-or-macos-reexport-library
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20140331/6a5a683f/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: libsupcxx.patch
Type: text/x-diff
Size: 522 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20140331/6a5a683f/attachment.patch>