[libcxx-commits] [libcxx] [libc++] Optimize `std::exception_ptr` (PR #162773)
via libcxx-commits
libcxx-commits at lists.llvm.org
Fri Oct 10 11:27:48 PDT 2025
llvmbot wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-libcxx
Author: Adrian Vogelsgesang (vogelsgesang)
<details>
<summary>Changes</summary>
This commit optimizes the performance for `std::exception_ptr` for an empty exception objects.
To do so, we use 3 high-level approaches:
1. Moving the implementation from the libc++ library into the libc++ headers, thereby allowing the compiler to inline the function bodies.
2. Adding fast paths to the (now inlineable) bodies, checking for the empty case.
3. Adding move constuctor, assignment and `swap`
Those optimizations were implemented for the libc++abi, libsupc++ and libstdc++ ABIs.
Fixes #<!-- -->44892
Performance
-----------
With this change, the compiler can now completely constant-fold https://godbolt.org/z/NaNKe5. Also in cases where the compiler cannot statically prove that exception_ptr is empty, it can at least generate a fast-path check if the exception_ptr is empty, without calling into the library.
ABI compatibility
-----------------
We use a new visibility macro `_LIBCPP_EXPORTED_FROM_ABI_INLINEABLE` to ensure that the functions which are now declared in the header are still exported by the library. See the description in `VisibilityMacros.rst` for details. This approach was originally proposed by Nikolas Klauser in https://reviews.llvm.org/D122536
Moving implementations to the header
------------------------------------
To move the implementation to the header, we must know the selected LIBCXX_CXX_ABI also in the headers. For that purpose, the LIBCXX_CXX_ABI configuration is now exposed via `__config_site`.
While the Microsoft ABI and the "unimplemented" APIs do not benefit from this optimizations, I also moved their implementation to the headers for more uniformity. Mid-term, we probably have to expose all implementations in the header, anyway, because P3068R6 mandates all methods of `exception_ptr` to be constexpr.
Unifying libc++abi, libsupc++ and `none`
-----------------------------------------------
Both libc++abi and libsupc++ are reference-counted. The primary difference is the function called for ref-counting:
* libc++api uses `__cxa_{in,de}crement_exception_refcount`
* libsupc++ uses `__exception_ptr::_M_{addref,release}`
This commit factors out the common reference-counting logic into a shared header.
Our libsupc++ implementation so far did not take advantage of `_M_addref`/`_M_release`. For the performance benefits of this PR it was necessary to start using them.
The same reference-counted approach is also reused for the `none`/`unimplemented` implementation. This has the side effect, that users can now use empty `exception_ptr`s even in the `none` ABI. The abort is only triggered for non-empty `exception_ptr`s. Given this simplifies the code, I think change is acceptable.
Unifying nested_exception
-------------------------
The implementation for `nested_exception` was effectively identical across all ABIs. For most ABIs, the source code was literally identical. There were only two deviations:
* For libsupc++ or libstdc++, we did not use define the `~nested_exception` destructor.
* The abort in `nested_exception::rethrow_nested` was unnecessary as the same abort is also present in the `rethrow_exception` implementation. As such, we were able to simply remove that special casing.
The implementation is now unified directly in the `__nested_exception.h` header.
Standard conformance
--------------------
The available constructors, operators and methods of `exception_ptr` is not specified by the standard. As such, adding the move constructor and the assignment operator are standard conformant. libstdc++ made the same decision and also provides those headers.
---
Patch is 41.22 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/162773.diff
24 Files Affected:
- (modified) libcxx/CMakeLists.txt (+14)
- (modified) libcxx/cmake/Modules/HandleLibCXXABI.cmake (-4)
- (modified) libcxx/docs/DesignDocs/VisibilityMacros.rst (+19)
- (modified) libcxx/include/CMakeLists.txt (+4)
- (modified) libcxx/include/__config_site.in (+8)
- (modified) libcxx/include/__exception/exception_ptr.h (+111-24)
- (added) libcxx/include/__exception/exception_ptr_cxxabi.ipp (+50)
- (added) libcxx/include/__exception/exception_ptr_glibcxx.ipp (+47)
- (renamed) libcxx/include/__exception/exception_ptr_msvc.ipp (+14-25)
- (added) libcxx/include/__exception/exception_ptr_unimplemented.ipp (+36)
- (modified) libcxx/include/__exception/operations.h (-5)
- (modified) libcxx/src/CMakeLists.txt (-4)
- (modified) libcxx/src/exception.cpp (+26-18)
- (modified) libcxx/src/new_handler.cpp (+5-7)
- (modified) libcxx/src/support/runtime/exception_libcxxabi.ipp (+3-5)
- (modified) libcxx/src/support/runtime/exception_libcxxrt.ipp (-5)
- (removed) libcxx/src/support/runtime/exception_pointer_cxxabi.ipp (-64)
- (removed) libcxx/src/support/runtime/exception_pointer_glibcxx.ipp (-68)
- (removed) libcxx/src/support/runtime/exception_pointer_unimplemented.ipp (-62)
- (modified) libcxx/src/support/runtime/stdexcept_default.ipp (+1-1)
- (modified) libcxx/src/typeinfo.cpp (+2-4)
- (modified) libcxx/test/benchmarks/exception_ptr.bench.cpp (+16)
- (modified) libcxx/test/tools/clang_tidy_checks/hide_from_abi.cpp (+2-1)
- (modified) libcxx/utils/libcxx/header_information.py (+4)
``````````diff
diff --git a/libcxx/CMakeLists.txt b/libcxx/CMakeLists.txt
index a119850cd808e..4ff23f0038d0a 100644
--- a/libcxx/CMakeLists.txt
+++ b/libcxx/CMakeLists.txt
@@ -750,6 +750,20 @@ config_define(${LIBCXX_ENABLE_WIDE_CHARACTERS} _LIBCPP_HAS_WIDE_CHARACTERS)
config_define(${LIBCXX_ENABLE_TIME_ZONE_DATABASE} _LIBCPP_HAS_TIME_ZONE_DATABASE)
config_define(${LIBCXX_ENABLE_VENDOR_AVAILABILITY_ANNOTATIONS} _LIBCPP_HAS_VENDOR_AVAILABILITY_ANNOTATIONS)
+if (LIBCXX_CXX_ABI STREQUAL "none")
+ config_define(1 _LIBCPP_CXX_ABI_NONE)
+elseif (LIBCXX_CXX_ABI STREQUAL "libcxxabi" OR LIBCXX_CXX_ABI STREQUAL "system-libcxxabi")
+ config_define(1 _LIBCPP_CXX_ABI_LIBCXXABI)
+elseif (LIBCXX_CXX_ABI STREQUAL "libcxxrt")
+ config_define(1 _LIBCPP_CXX_ABI_LIBCXXRT)
+elseif (LIBCXX_CXX_ABI STREQUAL "libstdc++")
+ config_define(1 _LIBCPP_CXX_ABI_LIBSTDCXX)
+elseif (LIBCXX_CXX_ABI STREQUAL "libsupc++")
+ config_define(1 _LIBCPP_CXX_ABI_LIBSUPCXX)
+elseif (LIBCXX_CXX_ABI STREQUAL "vcruntime")
+ config_define(1 _LIBCPP_CXX_ABI_VCRUNTIME)
+endif()
+
# TODO: Remove in LLVM 21. We're leaving an error to make this fail explicitly.
if (LIBCXX_ENABLE_ASSERTIONS)
message(FATAL_ERROR "LIBCXX_ENABLE_ASSERTIONS has been removed. Please use LIBCXX_HARDENING_MODE instead.")
diff --git a/libcxx/cmake/Modules/HandleLibCXXABI.cmake b/libcxx/cmake/Modules/HandleLibCXXABI.cmake
index 52236f473f35d..35287cf380da6 100644
--- a/libcxx/cmake/Modules/HandleLibCXXABI.cmake
+++ b/libcxx/cmake/Modules/HandleLibCXXABI.cmake
@@ -119,7 +119,6 @@ elseif ("${LIBCXX_CXX_ABI}" STREQUAL "libsupc++")
elseif ("${LIBCXX_CXX_ABI}" STREQUAL "libcxxabi")
add_library(libcxx-abi-headers INTERFACE)
target_link_libraries(libcxx-abi-headers INTERFACE cxxabi-headers)
- target_compile_definitions(libcxx-abi-headers INTERFACE "-DLIBCXX_BUILDING_LIBCXXABI")
if (TARGET cxxabi_shared)
add_library(libcxx-abi-shared INTERFACE)
@@ -156,7 +155,6 @@ elseif ("${LIBCXX_CXX_ABI}" STREQUAL "system-libcxxabi")
add_library(libcxx-abi-headers INTERFACE)
import_private_headers(libcxx-abi-headers "${LIBCXX_CXX_ABI_INCLUDE_PATHS}" "cxxabi.h;__cxxabi_config.h")
- target_compile_definitions(libcxx-abi-headers INTERFACE "-DLIBCXX_BUILDING_LIBCXXABI")
import_shared_library(libcxx-abi-shared c++abi)
target_link_libraries(libcxx-abi-shared INTERFACE libcxx-abi-headers)
@@ -173,7 +171,6 @@ elseif ("${LIBCXX_CXX_ABI}" STREQUAL "libcxxrt")
add_library(libcxx-abi-headers INTERFACE)
import_private_headers(libcxx-abi-headers "${LIBCXX_CXX_ABI_INCLUDE_PATHS}"
"cxxabi.h;unwind.h;unwind-arm.h;unwind-itanium.h")
- target_compile_definitions(libcxx-abi-headers INTERFACE "-DLIBCXXRT")
import_shared_library(libcxx-abi-shared cxxrt)
target_link_libraries(libcxx-abi-shared INTERFACE libcxx-abi-headers)
@@ -191,7 +188,6 @@ elseif ("${LIBCXX_CXX_ABI}" STREQUAL "vcruntime")
# Don't link against any ABI library
elseif ("${LIBCXX_CXX_ABI}" STREQUAL "none")
add_library(libcxx-abi-headers INTERFACE)
- target_compile_definitions(libcxx-abi-headers INTERFACE "-D_LIBCPP_BUILDING_HAS_NO_ABI_LIBRARY")
add_library(libcxx-abi-shared INTERFACE)
target_link_libraries(libcxx-abi-shared INTERFACE libcxx-abi-headers)
diff --git a/libcxx/docs/DesignDocs/VisibilityMacros.rst b/libcxx/docs/DesignDocs/VisibilityMacros.rst
index db54b35386b19..949401012ae36 100644
--- a/libcxx/docs/DesignDocs/VisibilityMacros.rst
+++ b/libcxx/docs/DesignDocs/VisibilityMacros.rst
@@ -35,6 +35,25 @@ Visibility Macros
used on class templates. On classes it should only be used if the vtable
lives in the built library.
+**_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE**
+ Mark a symbol as exported from the libc++ library, while still providing an
+ inlineable definition that can be used by the compiler for optimization
+ purposes.
+
+ To use this macro on a class method, define the method body
+ *outside* of the class definition and annotate that definition with
+ `_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE`. Make sure to include the
+ header in at least one translation unit linked into the libc++ library.
+
+ This macro works by applying `[[gnu::gnu_inline]] inline` to the funciton
+ in the header, thereby suppressing code generation while still allowing the
+ compiler to use the function for optimization purposes.
+ During the build of libc++, we trigger code generation by expanding the
+ macro to `/*empty*/`. Since the function is no longer marked as `inline`,
+ it will be emitted even if not called. (For this reason its paramount to
+ not define methods in the class definition, since those definitions would
+ be implicitly `inline`.)
+
**_LIBCPP_OVERRIDABLE_FUNC_VIS**
Mark a symbol as being exported by the libc++ library, but allow it to be
overridden locally. On non-Windows, this is equivalent to `_LIBCPP_FUNC_VIS`.
diff --git a/libcxx/include/CMakeLists.txt b/libcxx/include/CMakeLists.txt
index ddace8bf8c728..2baf6fca70a5f 100644
--- a/libcxx/include/CMakeLists.txt
+++ b/libcxx/include/CMakeLists.txt
@@ -343,6 +343,10 @@ set(files
__debug_utils/strict_weak_ordering_check.h
__exception/exception.h
__exception/exception_ptr.h
+ __exception/exception_ptr_cxxabi.ipp
+ __exception/exception_ptr_glibcxx.ipp
+ __exception/exception_ptr_msvc.ipp
+ __exception/exception_ptr_unimplemented.ipp
__exception/nested_exception.h
__exception/operations.h
__exception/terminate.h
diff --git a/libcxx/include/__config_site.in b/libcxx/include/__config_site.in
index b68c0c8258366..58714ed231b88 100644
--- a/libcxx/include/__config_site.in
+++ b/libcxx/include/__config_site.in
@@ -33,6 +33,14 @@
#cmakedefine01 _LIBCPP_HAS_TIME_ZONE_DATABASE
#cmakedefine01 _LIBCPP_INSTRUMENTED_WITH_ASAN
+// LIBCXX_CXX_ABI backends
+#cmakedefine _LIBCPP_CXX_ABI_NONE
+#cmakedefine _LIBCPP_CXX_ABI_LIBCXXABI
+#cmakedefine _LIBCPP_CXX_ABI_LIBCXXRT
+#cmakedefine _LIBCPP_CXX_ABI_LIBSTDCXX
+#cmakedefine _LIBCPP_CXX_ABI_LIBSUPCXX
+#cmakedefine _LIBCPP_CXX_ABI_VCRUNTIME
+
// PSTL backends
#cmakedefine _LIBCPP_PSTL_BACKEND_SERIAL
#cmakedefine _LIBCPP_PSTL_BACKEND_STD_THREAD
diff --git a/libcxx/include/__exception/exception_ptr.h b/libcxx/include/__exception/exception_ptr.h
index 796fa924be121..5a30fc875bce0 100644
--- a/libcxx/include/__exception/exception_ptr.h
+++ b/libcxx/include/__exception/exception_ptr.h
@@ -23,9 +23,29 @@
# pragma GCC system_header
#endif
-#ifndef _LIBCPP_ABI_MICROSOFT
+// Previously, parts of exception_ptr were defined out-of-line, which prevented
+// useful compiler optimizations. Changing the out-of-line definitions to inline
+// definitions is an ABI break, however. To prevent this, we have to make sure
+// the symbols remain available in the libc++ library, in addition to being
+// defined inline here in this header.
+// To this end, we use _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE macro:
+// The macro is defined as empty for src/exception.cpp, forcing the definitions of
+// the functions to be emitted and included in the library. When users of libc++
+// compile their code, the __gnu_inline__ attribute will suppress generation of
+// these functions while making their definitions available for inlining.
+#ifdef _LIBCPP_EMIT_CODE_FOR_EXCEPTION_PTR
+# define _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE _LIBCPP_EXPORTED_FROM_ABI
+#else
+# if !__has_cpp_attribute(__gnu__::__gnu_inline__)
+# error "GNU inline attribute is not supported"
+# endif
+# define _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE [[__gnu__::__gnu_inline__]] inline
+#endif
-# if _LIBCPP_AVAILABILITY_HAS_INIT_PRIMARY_EXCEPTION
+_LIBCPP_DIAGNOSTIC_PUSH
+_LIBCPP_CLANG_DIAGNOSTIC_IGNORED("-Wgnu-inline-cpp-without-extern")
+
+#ifdef _LIBCPP_AVAILABILITY_HAS_INIT_PRIMARY_EXCEPTION
namespace __cxxabiv1 {
@@ -49,18 +69,25 @@ _LIBCPP_OVERRIDABLE_FUNC_VIS __cxa_exception* __cxa_init_primary_exception(
} // namespace __cxxabiv1
-# endif
-
#endif
_LIBCPP_BEGIN_UNVERSIONED_NAMESPACE_STD
-#ifndef _LIBCPP_ABI_MICROSOFT
+class _LIBCPP_EXPORTED_FROM_ABI exception_ptr;
+_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr current_exception() _NOEXCEPT;
+[[__noreturn__]] _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE void rethrow_exception(exception_ptr);
+
+#ifndef _LIBCPP_CXX_ABI_VCRUNTIME
class _LIBCPP_EXPORTED_FROM_ABI exception_ptr {
void* __ptr_;
- static exception_ptr __from_native_exception_pointer(void*) _NOEXCEPT;
+ // Customization points to adjust the reference counting for cxxabi or
+ // libsupc++/libstdc++
+ _LIBCPP_HIDE_FROM_ABI _LIBCPP_ALWAYS_INLINE static void __increment_refcount(void* __ptr) _NOEXCEPT;
+ _LIBCPP_HIDE_FROM_ABI _LIBCPP_ALWAYS_INLINE static void __decrement_refcount(void* __ptr) _NOEXCEPT;
+
+ _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE static exception_ptr __from_native_exception_pointer(void*) _NOEXCEPT;
template <class _Ep>
friend _LIBCPP_HIDE_FROM_ABI exception_ptr __make_exception_ptr_explicit(_Ep&) _NOEXCEPT;
@@ -74,9 +101,11 @@ class _LIBCPP_EXPORTED_FROM_ABI exception_ptr {
_LIBCPP_HIDE_FROM_ABI exception_ptr() _NOEXCEPT : __ptr_() {}
_LIBCPP_HIDE_FROM_ABI exception_ptr(nullptr_t) _NOEXCEPT : __ptr_() {}
- exception_ptr(const exception_ptr&) _NOEXCEPT;
- exception_ptr& operator=(const exception_ptr&) _NOEXCEPT;
- ~exception_ptr() _NOEXCEPT;
+ _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr(const exception_ptr&) _NOEXCEPT;
+ _LIBCPP_HIDE_FROM_ABI exception_ptr(exception_ptr&&) _NOEXCEPT;
+ _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr& operator=(const exception_ptr&) _NOEXCEPT;
+ _LIBCPP_HIDE_FROM_ABI exception_ptr& operator=(exception_ptr&&) _NOEXCEPT;
+ _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE ~exception_ptr() _NOEXCEPT;
_LIBCPP_HIDE_FROM_ABI explicit operator bool() const _NOEXCEPT { return __ptr_ != nullptr; }
@@ -88,10 +117,54 @@ class _LIBCPP_EXPORTED_FROM_ABI exception_ptr {
return !(__x == __y);
}
- friend _LIBCPP_EXPORTED_FROM_ABI exception_ptr current_exception() _NOEXCEPT;
- friend _LIBCPP_EXPORTED_FROM_ABI void rethrow_exception(exception_ptr);
+ friend _LIBCPP_HIDE_FROM_ABI void swap(exception_ptr& __x, exception_ptr& __y) {
+ void* __tmp = __x.__ptr_;
+ __x.__ptr_ = __y.__ptr_;
+ __y.__ptr_ = __tmp;
+ }
+
+ friend exception_ptr current_exception() _NOEXCEPT;
+ friend void rethrow_exception(exception_ptr);
};
+// Must be defined outside the class definition due to _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE
+_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr exception_ptr::__from_native_exception_pointer(void* __e) _NOEXCEPT {
+ __increment_refcount(__e);
+ exception_ptr __ptr;
+ __ptr.__ptr_ = __e;
+ return __ptr;
+}
+
+// Must be defined outside the class definition due to _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE
+_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr::exception_ptr(const exception_ptr& __other) _NOEXCEPT
+ : __ptr_(__other.__ptr_) {
+ __increment_refcount(__ptr_);
+}
+
+_LIBCPP_HIDE_FROM_ABI inline exception_ptr::exception_ptr(exception_ptr&& __other) _NOEXCEPT : __ptr_(__other.__ptr_) {
+ __other.__ptr_ = nullptr;
+}
+
+// Must be defined outside the class definition due to _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE
+_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr& exception_ptr::operator=(const exception_ptr& __other) _NOEXCEPT {
+ if (__ptr_ != __other.__ptr_) {
+ __increment_refcount(__other.__ptr_);
+ __decrement_refcount(__ptr_);
+ __ptr_ = __other.__ptr_;
+ }
+ return *this;
+}
+
+_LIBCPP_HIDE_FROM_ABI inline exception_ptr& exception_ptr::operator=(exception_ptr&& __other) _NOEXCEPT {
+ __decrement_refcount(__ptr_);
+ __ptr_ = __other.__ptr_;
+ __other.__ptr_ = nullptr;
+ return *this;
+}
+
+// Must be defined outside the class definition due to _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE
+_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr::~exception_ptr() _NOEXCEPT { __decrement_refcount(__ptr_); }
+
# if _LIBCPP_HAS_EXCEPTIONS
# if _LIBCPP_AVAILABILITY_HAS_INIT_PRIMARY_EXCEPTION
template <class _Ep>
@@ -159,7 +232,7 @@ _LIBCPP_HIDE_FROM_ABI exception_ptr make_exception_ptr(_Ep) _NOEXCEPT {
#else // _LIBCPP_ABI_MICROSOFT
-class _LIBCPP_EXPORTED_FROM_ABI exception_ptr {
+class exception_ptr {
_LIBCPP_DIAGNOSTIC_PUSH
_LIBCPP_CLANG_DIAGNOSTIC_IGNORED("-Wunused-private-field")
void* __ptr1_;
@@ -167,26 +240,26 @@ class _LIBCPP_EXPORTED_FROM_ABI exception_ptr {
_LIBCPP_DIAGNOSTIC_POP
public:
- exception_ptr() _NOEXCEPT;
- exception_ptr(nullptr_t) _NOEXCEPT;
- exception_ptr(const exception_ptr& __other) _NOEXCEPT;
- exception_ptr& operator=(const exception_ptr& __other) _NOEXCEPT;
- exception_ptr& operator=(nullptr_t) _NOEXCEPT;
- ~exception_ptr() _NOEXCEPT;
- explicit operator bool() const _NOEXCEPT;
+ _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr() _NOEXCEPT;
+ _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr(nullptr_t) _NOEXCEPT;
+ _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr(const exception_ptr& __other) _NOEXCEPT;
+ _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr& operator=(const exception_ptr& __other) _NOEXCEPT;
+ _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr& operator=(nullptr_t) _NOEXCEPT;
+ _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE ~exception_ptr() _NOEXCEPT;
+ _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE explicit operator bool() const _NOEXCEPT;
};
-_LIBCPP_EXPORTED_FROM_ABI bool operator==(const exception_ptr& __x, const exception_ptr& __y) _NOEXCEPT;
+_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE bool operator==(const exception_ptr& __x, const exception_ptr& __y) _NOEXCEPT;
inline _LIBCPP_HIDE_FROM_ABI bool operator!=(const exception_ptr& __x, const exception_ptr& __y) _NOEXCEPT {
return !(__x == __y);
}
-_LIBCPP_EXPORTED_FROM_ABI void swap(exception_ptr&, exception_ptr&) _NOEXCEPT;
+_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE void swap(exception_ptr&, exception_ptr&) _NOEXCEPT;
-_LIBCPP_EXPORTED_FROM_ABI exception_ptr __copy_exception_ptr(void* __except, const void* __ptr);
-_LIBCPP_EXPORTED_FROM_ABI exception_ptr current_exception() _NOEXCEPT;
-[[__noreturn__]] _LIBCPP_EXPORTED_FROM_ABI void rethrow_exception(exception_ptr);
+_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr __copy_exception_ptr(void* __except, const void* __ptr);
+_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr current_exception() _NOEXCEPT;
+[[__noreturn__]] _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE void rethrow_exception(exception_ptr);
// This is a built-in template function which automagically extracts the required
// information.
@@ -201,4 +274,18 @@ _LIBCPP_HIDE_FROM_ABI exception_ptr make_exception_ptr(_Ep __e) _NOEXCEPT {
#endif // _LIBCPP_ABI_MICROSOFT
_LIBCPP_END_UNVERSIONED_NAMESPACE_STD
+#if defined(_LIBCPP_CXX_ABI_NONE)
+# include <__exception/exception_ptr_unimplemented.ipp>
+#elif defined(_LIBCPP_CXX_ABI_LIBCXXABI) || defined(_LIBCPP_CXX_ABI_LIBCXXRT)
+# include <__exception/exception_ptr_cxxabi.ipp>
+#elif defined(_LIBCPP_CXX_ABI_LIBSTDCXX) || defined(_LIBCPP_CXX_ABI_LIBSUPCXX)
+# include <__exception/exception_ptr_glibcxx.ipp>
+#elif defined(_LIBCPP_CXX_ABI_VCRUNTIME)
+# include <__exception/exception_ptr_msvc.ipp>
+#else
+# error "Unsupported C++ ABI library"
+#endif
+
+_LIBCPP_DIAGNOSTIC_POP
+
#endif // _LIBCPP___EXCEPTION_EXCEPTION_PTR_H
diff --git a/libcxx/include/__exception/exception_ptr_cxxabi.ipp b/libcxx/include/__exception/exception_ptr_cxxabi.ipp
new file mode 100644
index 0000000000000..187a6e0ad94c9
--- /dev/null
+++ b/libcxx/include/__exception/exception_ptr_cxxabi.ipp
@@ -0,0 +1,50 @@
+// -*- C++ -*-
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include <__exception/terminate.h>
+
+namespace __cxxabiv1 {
+
+extern "C" {
+_LIBCPP_OVERRIDABLE_FUNC_VIS void __cxa_increment_exception_refcount(void*) _NOEXCEPT;
+_LIBCPP_OVERRIDABLE_FUNC_VIS void __cxa_decrement_exception_refcount(void*) _NOEXCEPT;
+_LIBCPP_OVERRIDABLE_FUNC_VIS void* __cxa_current_primary_exception() _NOEXCEPT;
+_LIBCPP_OVERRIDABLE_FUNC_VIS void __cxa_rethrow_primary_exception(void*);
+}
+
+} // namespace __cxxabiv1
+
+namespace std {
+
+_LIBCPP_HIDE_FROM_ABI _LIBCPP_ALWAYS_INLINE inline void exception_ptr::__increment_refcount(void* __ptr) _NOEXCEPT {
+ if (__ptr)
+ __cxxabiv1::__cxa_increment_exception_refcount(__ptr);
+}
+
+_LIBCPP_HIDE_FROM_ABI _LIBCPP_ALWAYS_INLINE inline void exception_ptr::__decrement_refcount(void* __ptr) _NOEXCEPT {
+ if (__ptr)
+ __cxxabiv1::__cxa_decrement_exception_refcount(__ptr);
+}
+
+_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr current_exception() _NOEXCEPT {
+ // It would be nicer if there was a constructor that took a ptr, then
+ // this whole function would be just:
+ // return exception_ptr(__cxa_current_primary_exception());
+ exception_ptr __ptr;
+ __ptr.__ptr_ = __cxxabiv1::__cxa_current_primary_exception();
+ return __ptr;
+}
+
+_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE void rethrow_exception(exception_ptr __ptr) {
+ __cxxabiv1::__cxa_rethrow_primary_exception(__ptr.__ptr_);
+ // if __ptr.__ptr_ is NULL, above returns so we terminate.
+ terminate();
+}
+
+} // namespace std
diff --git a/libcxx/include/__exception/exception_ptr_glibcxx.ipp b/libcxx/include/__exception/exception_ptr_glibcxx.ipp
new file mode 100644
index 0000000000000..3f054249bca83
--- /dev/null
+++ b/libcxx/include/__exception/exception_ptr_glibcxx.ipp
@@ -0,0 +1,47 @@
+// -*- C++ -*-
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+namespace std {
+
+// libsupc++ does not implement the dependent EH ABI and the functionality
+// it uses to implement std::exception_ptr (which it declares as an alias of
+// std::__exception_ptr::exception_ptr) is not directly exported to clients. So
+// we have little choice but to hijack std::__exception_ptr::exception_ptr's
+// _M_addref and _M_release (which are part of its ABI), and its
+// rethrow_exception(std::__exception_ptr::exception_ptr) function. Fortunately,
+// glibcxx's exception_ptr has the same layout as our exception_ptr and we can
+// reinterpret_cast between the two.
+namespace __exception_ptr {
+
+struct exception_ptr {
+ void* __ptr_;
+
+ void _M_addref() _GLIBCXX_USE_NOEXCEPT;
+ void _M_release() _GLIBCXX_USE_NOEXCEPT;
+};
+
+} // namespace __exception_ptr
+
+_LIBCPP_HIDE_FROM_ABI _LIBCPP_ALWAYS_INLINE inline void exception_ptr::__increment_refcount(void* __ptr) _NOEXCEPT {
+ if (__ptr)
+ reinterpret_cast<__exception_ptr::exception_ptr*>(this)->_M_addref();
+}
+
+_LIBCPP_HIDE_FROM_ABI _LIBCPP_ALWAYS_INLINE inline void exception_ptr::__decrement_refcount(void* __ptr) _NOEXCEPT {
+ if (__ptr)
+ reinterpret_cast<__exception_ptr::exception_ptr*>(this)->_M_release();
+}
+
+[[__noreturn__]] void rethrow_exception(__exception_ptr::exception_ptr);
+
+[[__noreturn__]] _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE void rethrow_exception(exception_ptr __ptr) {
+ rethrow_exception(reinterpret_cast<__exception_ptr::exception_ptr&>(__ptr));
+}
+
+} // namespace std
diff --git a/libcxx/src/support/runtime/exception_pointer_msvc.ipp b/libcxx/include/__exception/exception_ptr_msvc.ipp
similarity index 50%
rename from libcxx/src/support/runtime/exception_pointer_msvc.ipp
rename to libcxx/include/__exception/exception_ptr_msvc.ipp
index 2be5136176e32..d893045ba17b0 100644
--- a/libcxx/src/support/runtime/exception_pointer_msvc.ipp
+++ b/libcxx/include/__exception/exception_ptr_msvc.ipp
@@ -7,7 +7,6 @@
//
//===----------------------------------------------------------------------===//
-#include <stdio.h>
#include <stdlib.h>
_LIBCPP_CRT_FUNC void __cdecl __ExceptionPtrCreate(void*);
@@ -23,54 +22,44 @@ _LIBCPP_CRT_FUNC void __cdecl __ExceptionPtrCopyException(void*, const void*, co
namespace std {
-exception_ptr::exception_ptr() noexcept { __ExceptionPtrCreate(this); }
-exception_ptr::exception_ptr(nullptr_t) noexcept { __ExceptionPtrCreate(this); }
+_LIBCPP_EXPORTED_FROM_LIB_...
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/162773
More information about the libcxx-commits
mailing list