[all-commits] [llvm/llvm-project] f6ba98: [libc++abi] Use std::abort() instead of std::termi...

[Louis Dionne via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: f6ba9850c06197e7bb530b9949dddce1e8a169f0
      https://github.com/llvm/llvm-project/commit/f6ba9850c06197e7bb530b9949dddce1e8a169f0
  Author: Louis Dionne <ldionne.2 at gmail.com>
  Date:   2023-08-31 (Thu, 31 Aug 2023)

  Changed paths:
    M libcxxabi/src/demangle/ItaniumDemangle.h
    M libcxxabi/src/demangle/Utility.h
    M llvm/include/llvm/Demangle/ItaniumDemangle.h
    M llvm/include/llvm/Demangle/Utility.h

  Log Message:
  -----------
  [libc++abi] Use std::abort() instead of std::terminate() on failure to allocate

Inside the Itanium demangler, we would previously call std::terminate()
after failing to (re)allocate. However, programs are free to install a
custom terminate_handler which does non-trivial things. In fact, by
default libc++abi itself installs a demangling_terminate_handler() which
tries to demangle the currently active exception before aborting the
program.

In case of an out-of-memory exception caused by the system truly having
no more memory (as opposed to attempting to allocate INT_MAX just once),
we will end up trying to demangle the exception, failing to do so because
we can't grow the OutputBuffer inside ItaniumDemangle.h, and then calling
std::terminate() there. That will call the demangling_terminate_handler(),
which will then start this loop again. We eventually end up crashing due
to a stack overflow.

To fix this problem, this patch calls std::abort() directly from the
demangler instead of going through std::terminate(). After all, calling
std::abort() is the default behavior for std::terminate() according
to the Standard, so this behavior is definitely valid. The downside of
this approach is that in case of a "true" out-of-memory condition:
1. the program will throw an exception
2. std::terminate() will be called if uncaught
3. demangling_terminate_handler() will be called and will fail
4. abort() will be called

We'll end up aborting the program without mentioning the cause, which
normally looks like:

  terminating due to uncaught exception of type <TYPE>: <what()-MESSAGE>

Another option would be to properly handle failure-to-allocate inside
ItaniumDemangle.h and to propagate something like an error code or a
std::expected to the caller of all functions in the demangler that
can allocate. Then, we could make sure that __cxa_demangle returns
nullptr when it fails to demangle the input due to any error, as it is
supposed to (but today "true" out-of-memory conditions are not handled
properly). The demangling_terminate_handler() would then see that
__cxa_demangle failed to do its job and would still print the
appropriate message, simply using the non-demangled exception type.
However, this is akin to a partial rewrite of the demangler code since
a large number of functions would now have to return a std::expected
to account for out-of-memory conditions.

Using exceptions would be a lot simpler in terms of code changes and
would achieve the same result, however the demangler can't use exceptions
because it is used inside LLVM and libc++abi implements the exception
runtime anyway (so while it might be possible to use them in that
context, I'd argue we'd only be playing with fire).

rdar://110767664

Differential Revision: https://reviews.llvm.org/D155598