[libc++abi] Use std::abort() instead of std::terminate() on failure to allocate
Inside the Itanium demangler, we would previously call std::terminate()
after failing to (re)allocate. However, programs are free to install a
custom terminate_handler which does non-trivial things. In fact, by
default libc++abi itself installs a demangling_terminate_handler() which
tries to demangle the currently active exception before aborting the
program.
In case of an out-of-memory exception caused by the system truly having
no more memory (as opposed to attempting to allocate INT_MAX just once),
we will end up trying to demangle the exception, failing to do so because
we can't grow the OutputBuffer inside ItaniumDemangle.h, and then calling
std::terminate() there. That will call the demangling_terminate_handler(),
which will then start this loop again. We eventually end up crashing due
to a stack overflow.
To fix this problem, this patch calls std::abort() directly from the
demangler instead of going through std::terminate(). After all, calling
std::abort() is the default behavior for std::terminate() according
to the Standard, so this behavior is definitely valid. The downside of
this approach is that in case of a "true" out-of-memory condition:
1. the program will throw an exception
2. std::terminate() will be called if uncaught
3. demangling_terminate_handler() will be called and will fail
4. abort() will be called
We'll end up aborting the program without mentioning the cause, which
normally looks like:
terminating due to uncaught exception of type <TYPE>: <what()-MESSAGE>
Another option would be to properly handle failure-to-allocate inside
ItaniumDemangle.h and to propagate something like an error code or a
std::expected to the caller of all functions in the demangler that
can allocate. Then, we could make sure that __cxa_demangle returns
nullptr when it fails to demangle the input due to any error, as it is
supposed to (but today "true" out-of-memory conditions are not handled
properly). The demangling_terminate_handler() would then see that
__cxa_demangle failed to do its job and would still print the
appropriate message, simply using the non-demangled exception type.
However, this is akin to a partial rewrite of the demangler code since
a large number of functions would now have to return a std::expected
to account for out-of-memory conditions.
Using exceptions would be a lot simpler in terms of code changes and
would achieve the same result, however the demangler can't use exceptions
because it is used inside LLVM and libc++abi implements the exception
runtime anyway (so while it might be possible to use them in that
context, I'd argue we'd only be playing with fire).
rdar://110767664
Differential Revision: https://reviews.llvm.org/D155598
GitOrigin-RevId: f6ba9850c06197e7bb530b9949dddce1e8a169f0
2 files changed
