[cfe-dev] ARM Linux libc++ / libunwind — Exceptions not being caught

[Andrew Brownsword via cfe-dev]

Hi Peter, thanks for the detailed reply...

> My steps were:
> 1.) Build trunk clang to use for building libc++ etc.
> 2.) Using cmake
> 
> cmake -GNinja\
>      /path/to/monorepo/llvm \
>      -DLLVM_ENABLE_PROJECTS="libcxxabi;libcxx;libunwind" \
>      -DCMAKE_BUILD_TYPE=Release \
>      -DLLVM_ENABLE_ASSERTIONS=true\
>      -DCMAKE_C_FLAGS='-mcpu=cortex-a57'\
>      -DCMAKE_CXX_FLAGS='-mcpu=cortex-a57'\
>      -DLLVM_TARGETS_TO_BUILD='ARM'\
>      -DLIBCXXABI_USE_LLVM_UNWINDER=On
> 3.) build
> ninja libunwind
> ninja cxxabi
> ninja cxx

I will attempt to emulate your steps.  I just switched to ninja from makefiles (for the toolchain build) a few days ago because almost everyone uses ninja for their examples and I’m trying to minimize differences… unfortunately that seemed to cause issues with how I was file sharing (my weak ARM SBC Linux host mounts a share from an OSX machine as that is better and faster than abusing the SDcards of the SBCs).  I think I have that sorted now, so hopefully the builds proceed more smoothly now.  Building the toolchain on such a weak machine isn’t the fastest thing in the world.  Can’t even use all the cores since the compiler’s memory footprint pushes the limits of the host’s 2GB.


> Some things you might want to try:
> - Use trunk clang rather than the 7.0 release. You may have found a
> release blocking bug if it is only a problem on the 7.0 branch.

Well I initially inadvertently was on tip of tree (i.e. 8.0.0) until I figured out the whole svn tags/7.0.0/rc2 scheme.  It had the same issue.


> - What happens to your example when you use libstd++ and the gcc
> unwinding implementation? I'm trying to rule out whether clang has not
> produced correct exception handling information for the program.

It works fine when I build my little repro case like this (some arguments may be superfluous, they are there for historical reasons):

	clang++ -funwind-tables -stdlib=libstdc++ -std=gnu++1z -I/usr/lib/arm-linux-gnueabihf/../include -pthread -c test.cpp -o test.o
	clang++ -funwind-tables -stdlib=libstdc++ -L/usr/local/lib -lpthread test.o -o test

It fails when I build it like this:

	clang++ -funwind-tables -stdlib=libc++ -std=gnu++1z -I/usr/lib/arm-linux-gnueabihf/../include -pthread -c test.cpp -o test.o
	clang++ -funwind-tables -stdlib=libc++ -L/usr/local/lib -lunwind -lc++ -lc++abi -lpthread test.o -o test

I’ve lost track of how many ways I’ve tried to build libunwind/libc++/libc++abi.  I’ve tried static vs shared libraries.  I’ve tried the supc++ route and some other variations — most failed to build.  There are quite a few options and configuration parameters, and its all rather hit-and-miss when you aren’t steeped in the mythos of the linux / llvm toolchain.

And no, using libstdc++ is not an attractive option.


> - If you are able to build libunwind with debug then I've had some
> success with the logging functions that are enabled when certain
> environment variables are set, such as LIBUNWIND_PRINT_UNWINDING. This
> might help determine if the abort is because the callsite of the throw
> isn't found in the .ARM.exidx exception table.

Okay, that sounds like a path of last resort!   :)    I did catch one variation of the problem (i.e. an earlier toolchain build) in gdb and it seemed to be stuck in an endless unwind loop with one of the unwinder’s C functions repeating.  That version bailed in a less pleasant manner than the current abort.


> - It may also be worth simplifying your example so that it is your
> executable that throws the exception and not a function in the libc++
> library itself, which in this case I believe would be in the libc++
> shared object itself.

Indeed, this was a useful thing to try — if I throw invalid_argument directly, it is caught properly!  That explains why my actual project hadn’t exhibited this problem before, as I was fairly certain it was throwing at least a handful of exceptions but not aborting (until I had a stoul call throw, which I have coded around… but having a broken toolchain is less than ideal).

So the problem is when libc++ throws…?  That implies I’m building the library wrong?


> I'm making the assumption you are building natively here. I have
> cross-compiled from x86_64 to AArch64 before but I haven't done it for
> Arm. The solution I found needed standalone builds of the libraries.
> If you can share a bit more about how you are trying to build we may
> be able to spot something wrong?

I am building natively on my ODROID HC-1 (8 32-bit cores) running Ubuntu 16.04.  I also have a Wandboard Quad (4 32-bit cores) running Ubuntu 16.04.  Both exhibit the same problem, but that’s not terribly surprising since I followed the same build path for the toolchain on each host.  The programs are cmake projects with makefiles, but given the simple repro case above, that seems unrelated.  The programs (my full app and the test case) work fine on my OSX host using the Xcode toolchain.