[libcxx-commits] [PATCH] D150610: [libc++] Make sure `operator new` never returns nullptr, even under -fno-exceptions

[Louis Dionne via Phabricator via libcxx-commits]

ldionne created this revision.
Herald added a project: All.
ldionne requested review of this revision.
Herald added projects: libc++, libc++abi.
Herald added a subscriber: libcxx-commits.
Herald added a reviewer: libc++.
Herald added a reviewer: libc++abi.

In D144319 <https://reviews.llvm.org/D144319>, Clang tried to land a change that would cause some functions
that are not supposed to return nullptr to optimize better. As reported
in https://reviews.llvm.org/D144319#4203982, libc++ started seeing failures
in its CI shortly after this change was landed.

As explained in D146379 <https://reviews.llvm.org/D146379>, the reason for these failures is that libc++'s
throwing `operator new` can in fact return nullptr when compiled with
exceptions disabled. However, this contradicts the Standard, which
clearly says that the throwing version of `operator new(size_t)`
should never return nullptr.

This is actually a long standing issue. I've previously seen a case where
LTO would optimize incorrectly based on the assumption that `operator new`
doesn't return nullptr, an assumption that was violated in that case
because libc++.dylib was compiled with -fno-exceptions.

Unfortunately, fixing this is kind of tricky. The Standard has a few
requirements for the allocation functions, some of which are impossible
to satisfy under -fno-exceptions:

1. `operator new(size_t)` must never return nullptr
2. `operator new(size_t, nothrow_t)` must call the throwing version and return nullptr on failure to allocate
3. We can't throw exceptions when compiled with `-fno-exceptions`

In the case where exceptions are enabled, things work nicely. `new(size_t)`
throws and `new(size_t, nothrow_t)` uses a try-catch to return nullptr.
However, when compiling the library with -fno-exceptions, we can't throw
an exception from `new(size_t)`, and we can't catch anything from
`new(size_t, nothrow_t)`. The only thing we can do from `new(size_t)`
is actually abort the program, which does not make it possible for
`new(size_t, nothrow_t)` to return nullptr. Whether that implementation
does satisfy the Standard is debatable, however that is the approach that
libstdc++ has taken and it seems that having slightly surprising behavior
for `new(size_t, nothrow_t)` is less vexing than having definitely
incorrect behavior for the more widely used `new(size_t)`.

This is extremely tricky because it constitutes a behavior change for
vendors who build the library with -fno-exceptions. Indeed, when built
with -fno-exceptions, `new(size_t, nothrow_t)` will now abort when it
fails to allocate instead of returning nullptr like it used to.

rdar://103958777


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D150610

Files:
  libcxx/src/new.cpp
  libcxx/test/libcxx/language.support/support.dynamic/new_dont_return_nullptr.pass.cpp
  libcxxabi/src/stdlib_new_delete.cpp

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D150610.522327.patch
Type: text/x-patch
Size: 8665 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/libcxx-commits/attachments/20230515/09189b77/attachment-0001.bin>