[libcxx-commits] [libcxx] [libcxx] Implementation of non lock-free atomic shared_ptr (PR #194215)
Vladislav Semykin via libcxx-commits
libcxx-commits at lists.llvm.org
Sun Apr 26 00:04:39 PDT 2026
https://github.com/ViNN280801 created https://github.com/llvm/llvm-project/pull/194215
## Motivation
Prior to this patch, `std::atomic<std::shared_ptr<T>>` and `std::atomic<std::weak_ptr<T>>` were unusable under libc++ - instantiation hit a hard `static_assert` (`std::atomic<T> requires that 'T' be a trivially
copyable type`) because `shared_ptr` is not trivially copyable. libstdc++ (GCC 12, 2022) and MSVC STL (2020) have shipped conforming implementations.
## Related Links
- Proposal - https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0718r2.html
- LWG 3661 - https://cplusplus.github.io/LWG/lwg-defects.html#3661
- LWG 3893 - https://cplusplus.github.io/LWG/lwg-defects.html#3893
- [util.smartptr.atomic] - https://eel.is/c++draft/util.smartptr.atomic
- [util.smartptr.atomic.shared] - https://eel.is/c++draft/util.smartptr.atomic.shared
- [util.smartptr.atomic.weak] - https://eel.is/c++draft/util.smartptr.atomic.weak
- [atomics.wait] - https://eel.is/c++draft/atomics.wait
- Last try - #78317
- Issue - #99980
- GCC libstdc++ implementation - https://github.com/gcc-mirror/gcc/blob/master/libstdc++-v3/include/bits/shared_ptr_atomic.h
- Microsoft STL implementation - https://github.com/microsoft/STL/blob/main/stl/inc/memory
## Testing
All the tests are passed:
```log
[14/15] Running libcxx tests
llvm-lit: /home/loveit0/Documents/Prog/llvm-project/libcxx/utils/libcxx/test/config.py:24: note: (llvm-libc++-shared.cfg.in) All available features: add-latomic-workaround, buildhost=linux, c++26, c++experimental, can-create-symlinks, character-conversion-warnings, clang, clang-23, clang-23.0, clang-23.0.0, diagnose-if-support, enable-benchmarks=dry-run, gcc-style-warnings, has-1024-bit-atomics, has-64-bit-atomics, has-fblocks, has-fconstexpr-steps, has-unix-headers, host-has-gdb-with-python, large_tests, libcpp-abi-version=1, libcpp-hardening-mode=none, libcpp-has-no-availability-markup, libcpp-has-thread-api-pthread, linux, locale.en_US.UTF-8, locale.ru_RU.UTF-8, long_tests, objective-c++, optimization=none, std-at-least-c++03, std-at-least-c++11, std-at-least-c++14, std-at-least-c++17, std-at-least-c++20, std-at-least-c++23, std-at-least-c++26, stdlib=libc++, stdlib=llvm-libc++, target=x86_64-unknown-linux-gnu, verify-support
Testing Time: 1245.71s
Total Discovered Tests: 11409
Unsupported : 1029 (9.02%)
Passed : 10353 (90.74%)
Expectedly Failed: 27 (0.24%)
./build/bin/llvm-lit -sv build/runtimes/runtimes-bins/libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic
llvm-lit: /home/loveit0/Documents/Prog/llvm-project/libcxx/utils/libcxx/test/config.py:24: note: (llvm-libc++-shared.cfg.in) All available features: add-latomic-workaround, buildhost=linux, c++26, c++experimental, can-create-symlinks, character-conversion-warnings, clang, clang-23, clang-23.0, clang-23.0.0, diagnose-if-support, enable-benchmarks=run, gcc-style-warnings, has-1024-bit-atomics, has-64-bit-atomics, has-fblocks, has-fconstexpr-steps, has-unix-headers, host-has-gdb-with-python, large_tests, libcpp-abi-version=1, libcpp-hardening-mode=none, libcpp-has-no-availability-markup, libcpp-has-thread-api-pthread, linux, locale.en_US.UTF-8, locale.ru_RU.UTF-8, long_tests, objective-c++, optimization=none, std-at-least-c++03, std-at-least-c++11, std-at-least-c++14, std-at-least-c++17, std-at-least-c++20, std-at-least-c++23, std-at-least-c++26, stdlib=libc++, stdlib=llvm-libc++, target=x86_64-unknown-linux-gnu, verify-support
Testing Time: 3.16s
Total Discovered Tests: 18
Passed: 18 (100.00%)
```
## Additional information
This is not a lock-free implementation, but I'm still on thoughts how to properly implement it, and I need more time to investigate existing imlpementations and think about the best way to do it.
>From 1a0999aa4c5bf41a96692de672d4ee41e89bbcad Mon Sep 17 00:00:00 2001
From: ViNN280801 <vladislav.semykin at gmail.com>
Date: Sun, 26 Apr 2026 10:03:26 +0300
Subject: [PATCH] [libcxx] Implementation of non lock-free atomic shared_ptr
Signed-off-by: ViNN280801 <vladislav.semykin at gmail.com>
---
libcxx/include/CMakeLists.txt | 2 +
libcxx/include/__atomic/atomic_sync_lite.h | 45 ++
libcxx/include/__memory/atomic_shared_ptr.h | 491 ++++++++++++++++++
libcxx/include/__memory/shared_ptr.h | 22 +-
libcxx/include/module.modulemap.in | 5 +
.../atomic_shared_ptr_aliasing.pass.cpp | 71 +++
.../atomic_shared_ptr_class.pass.cpp | 70 +++
.../atomic_shared_ptr_memory_order.verify.cpp | 58 +++
.../atomic_shared_ptr_nullptr.pass.cpp | 54 ++
.../atomic_shared_ptr_refcount.pass.cpp | 105 ++++
.../atomic_shared_ptr_stress.pass.cpp | 107 ++++
.../atomic_weak_ptr_class.pass.cpp | 91 ++++
12 files changed, 1119 insertions(+), 2 deletions(-)
create mode 100644 libcxx/include/__atomic/atomic_sync_lite.h
create mode 100644 libcxx/include/__memory/atomic_shared_ptr.h
create mode 100644 libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_shared_ptr_aliasing.pass.cpp
create mode 100644 libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_shared_ptr_class.pass.cpp
create mode 100644 libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_shared_ptr_memory_order.verify.cpp
create mode 100644 libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_shared_ptr_nullptr.pass.cpp
create mode 100644 libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_shared_ptr_refcount.pass.cpp
create mode 100644 libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_shared_ptr_stress.pass.cpp
create mode 100644 libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_weak_ptr_class.pass.cpp
diff --git a/libcxx/include/CMakeLists.txt b/libcxx/include/CMakeLists.txt
index 69a6590d18f85..d5402b6fc40b4 100644
--- a/libcxx/include/CMakeLists.txt
+++ b/libcxx/include/CMakeLists.txt
@@ -216,6 +216,7 @@ set(files
__atomic/atomic_lock_free.h
__atomic/atomic_ref.h
__atomic/atomic_sync.h
+ __atomic/atomic_sync_lite.h
__atomic/atomic_sync_timed.h
__atomic/atomic_waitable_traits.h
__atomic/check_memory_order.h
@@ -590,6 +591,7 @@ set(files
__memory/allocator_traits.h
__memory/array_cookie.h
__memory/assume_aligned.h
+ __memory/atomic_shared_ptr.h
__memory/auto_ptr.h
__memory/compressed_pair.h
__memory/concepts.h
diff --git a/libcxx/include/__atomic/atomic_sync_lite.h b/libcxx/include/__atomic/atomic_sync_lite.h
new file mode 100644
index 0000000000000..a0433e16b506a
--- /dev/null
+++ b/libcxx/include/__atomic/atomic_sync_lite.h
@@ -0,0 +1,45 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef _LIBCPP___ATOMIC_ATOMIC_SYNC_LITE_H
+#define _LIBCPP___ATOMIC_ATOMIC_SYNC_LITE_H
+
+#include <__atomic/contention_t.h>
+#include <__config>
+
+#if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
+# pragma GCC system_header
+#endif
+
+_LIBCPP_BEGIN_NAMESPACE_STD
+
+#if _LIBCPP_STD_VER >= 20 && _LIBCPP_HAS_THREADS
+
+# if !_LIBCPP_AVAILABILITY_HAS_NEW_SYNC
+// Old dylib interface kept for backwards compatibility.
+_LIBCPP_EXPORTED_FROM_ABI void __cxx_atomic_notify_one(void const volatile*) _NOEXCEPT;
+_LIBCPP_EXPORTED_FROM_ABI void __cxx_atomic_notify_all(void const volatile*) _NOEXCEPT;
+_LIBCPP_EXPORTED_FROM_ABI __cxx_contention_t __libcpp_atomic_monitor(void const volatile*) _NOEXCEPT;
+_LIBCPP_EXPORTED_FROM_ABI void __libcpp_atomic_wait(void const volatile*, __cxx_contention_t) _NOEXCEPT;
+# endif // !_LIBCPP_AVAILABILITY_HAS_NEW_SYNC
+
+// New dylib interface.
+_LIBCPP_AVAILABILITY_NEW_SYNC _LIBCPP_EXPORTED_FROM_ABI __cxx_contention_t
+__atomic_monitor_global(void const* __address) _NOEXCEPT;
+
+_LIBCPP_AVAILABILITY_NEW_SYNC _LIBCPP_EXPORTED_FROM_ABI void
+__atomic_wait_global_table(void const* __address, __cxx_contention_t __monitor_value) _NOEXCEPT;
+
+_LIBCPP_AVAILABILITY_NEW_SYNC _LIBCPP_EXPORTED_FROM_ABI void __atomic_notify_one_global_table(void const*) _NOEXCEPT;
+_LIBCPP_AVAILABILITY_NEW_SYNC _LIBCPP_EXPORTED_FROM_ABI void __atomic_notify_all_global_table(void const*) _NOEXCEPT;
+
+#endif // _LIBCPP_STD_VER >= 20 && _LIBCPP_HAS_THREADS
+
+_LIBCPP_END_NAMESPACE_STD
+
+#endif // _LIBCPP___ATOMIC_ATOMIC_SYNC_LITE_H
diff --git a/libcxx/include/__memory/atomic_shared_ptr.h b/libcxx/include/__memory/atomic_shared_ptr.h
new file mode 100644
index 0000000000000..000b56e08bae6
--- /dev/null
+++ b/libcxx/include/__memory/atomic_shared_ptr.h
@@ -0,0 +1,491 @@
+// -*- C++ -*-
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef _LIBCPP___MEMORY_ATOMIC_SHARED_PTR_H
+#define _LIBCPP___MEMORY_ATOMIC_SHARED_PTR_H
+
+#include <__atomic/atomic_sync_lite.h>
+#include <__atomic/check_memory_order.h>
+#include <__atomic/memory_order.h>
+#include <__atomic/support.h>
+#include <__config>
+#include <__cstddef/nullptr_t.h>
+#include <__memory/shared_count.h>
+#include <__utility/move.h>
+
+#include <cstdint>
+
+#if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
+# pragma GCC system_header
+#endif
+
+#if defined(__SANITIZE_THREAD__) || (defined(__has_feature) && __has_feature(thread_sanitizer))
+# if __has_include(<sanitizer/tsan_interface.h>)
+# include <sanitizer/tsan_interface.h>
+# define _LIBCPP_ATOMIC_SHARED_PTR_TSAN 1
+# endif
+#endif
+
+_LIBCPP_PUSH_MACROS
+#include <__undef_macros>
+
+// TSAN annotations model the lock-bit protocol on __ctrl_.
+#if defined(_LIBCPP_ATOMIC_SHARED_PTR_TSAN)
+# define _LIBCPP_ATOMIC_SP_TSAN_PRE_LOCK(addr) \
+ ::__tsan_mutex_pre_lock(reinterpret_cast<void*>(const_cast<__cxx_atomic_impl<uintptr_t>*>(addr)), 0)
+# define _LIBCPP_ATOMIC_SP_TSAN_POST_LOCK(addr) \
+ ::__tsan_mutex_post_lock(reinterpret_cast<void*>(const_cast<__cxx_atomic_impl<uintptr_t>*>(addr)), 0, 0)
+# define _LIBCPP_ATOMIC_SP_TSAN_PRE_UNLOCK(addr) \
+ ::__tsan_mutex_pre_unlock(reinterpret_cast<void*>(const_cast<__cxx_atomic_impl<uintptr_t>*>(addr)), 0)
+# define _LIBCPP_ATOMIC_SP_TSAN_POST_UNLOCK(addr) \
+ ::__tsan_mutex_post_unlock(reinterpret_cast<void*>(const_cast<__cxx_atomic_impl<uintptr_t>*>(addr)), 0)
+#else
+# define _LIBCPP_ATOMIC_SP_TSAN_PRE_LOCK(addr) ((void)(addr))
+# define _LIBCPP_ATOMIC_SP_TSAN_POST_LOCK(addr) ((void)(addr))
+# define _LIBCPP_ATOMIC_SP_TSAN_PRE_UNLOCK(addr) ((void)(addr))
+# define _LIBCPP_ATOMIC_SP_TSAN_POST_UNLOCK(addr) ((void)(addr))
+#endif
+
+_LIBCPP_BEGIN_NAMESPACE_STD
+
+#if _LIBCPP_STD_VER >= 20 && _LIBCPP_HAS_THREADS && _LIBCPP_HAS_ATOMIC_HEADER
+
+template <class _Tp>
+class shared_ptr;
+
+template <class _Tp>
+class weak_ptr;
+
+template <class _Tp>
+struct atomic;
+
+// Split state into pointer word and control word.
+// The control word stores control-block pointer plus lock/notify bits.
+struct __atomic_smart_ptr_storage {
+ static constexpr uintptr_t __lock_bit_ = uintptr_t{1};
+ static constexpr uintptr_t __notify_bit_ = uintptr_t{2};
+ static constexpr uintptr_t __ptr_mask_ = ~(__lock_bit_ | __notify_bit_);
+
+ _LIBCPP_HIDE_FROM_ABI static uintptr_t __encode(__shared_weak_count* __ctrl, uintptr_t __bits) _NOEXCEPT {
+ return (reinterpret_cast<uintptr_t>(__ctrl) & __ptr_mask_) | (__bits & ~__ptr_mask_);
+ }
+
+ _LIBCPP_HIDE_FROM_ABI static __shared_weak_count* __decode(uintptr_t __word) _NOEXCEPT {
+ return reinterpret_cast<__shared_weak_count*>(__word & __ptr_mask_);
+ }
+
+ _LIBCPP_HIDE_FROM_ABI static bool __has_lock(uintptr_t __word) _NOEXCEPT { return (__word & __lock_bit_) != 0; }
+ _LIBCPP_HIDE_FROM_ABI static bool __has_notify(uintptr_t __word) _NOEXCEPT { return (__word & __notify_bit_) != 0; }
+};
+
+_LIBCPP_HIDE_FROM_ABI inline void __atomic_smart_ptr_notify_one(const void* __address) _NOEXCEPT {
+# if _LIBCPP_AVAILABILITY_HAS_NEW_SYNC
+ std::__atomic_notify_one_global_table(__address);
+# else
+ std::__cxx_atomic_notify_one(reinterpret_cast<void const volatile*>(__address));
+# endif
+}
+
+_LIBCPP_HIDE_FROM_ABI inline void __atomic_smart_ptr_notify_all(const void* __address) _NOEXCEPT {
+# if _LIBCPP_AVAILABILITY_HAS_NEW_SYNC
+ std::__atomic_notify_all_global_table(__address);
+# else
+ std::__cxx_atomic_notify_all(reinterpret_cast<void const volatile*>(__address));
+# endif
+}
+
+template <class _Poll>
+_LIBCPP_HIDE_FROM_ABI inline void __atomic_smart_ptr_wait_on_address(const void* __address, _Poll&& __poll) _NOEXCEPT {
+ while (!__poll()) {
+# if _LIBCPP_AVAILABILITY_HAS_NEW_SYNC
+ auto __monitor_value = std::__atomic_monitor_global(__address);
+ if (__poll())
+ return;
+ std::__atomic_wait_global_table(__address, __monitor_value);
+# else
+ void const volatile* __volatile_address = reinterpret_cast<void const volatile*>(__address);
+ auto __monitor_value = std::__libcpp_atomic_monitor(__volatile_address);
+ if (__poll())
+ return;
+ std::__libcpp_atomic_wait(__volatile_address, __monitor_value);
+# endif
+ }
+}
+
+template <class _Element>
+struct __atomic_smart_ptr_fields {
+ mutable __cxx_atomic_impl<_Element*> __ptr_;
+ mutable __cxx_atomic_impl<uintptr_t> __ctrl_;
+
+ _LIBCPP_HIDE_FROM_ABI __atomic_smart_ptr_fields(_Element* __p, __shared_weak_count* __c) _NOEXCEPT
+ : __ptr_(__p),
+ __ctrl_(__atomic_smart_ptr_storage::__encode(__c, 0)) {}
+
+ _LIBCPP_HIDE_FROM_ABI const void* __ctrl_address() const _NOEXCEPT {
+ return static_cast<const void*>(__builtin_addressof(__ctrl_));
+ }
+
+ // Acquire lock bit on __ctrl_. Contended path sets notify bit and waits.
+ _LIBCPP_HIDE_FROM_ABI void __lock() const _NOEXCEPT {
+ _LIBCPP_ATOMIC_SP_TSAN_PRE_LOCK(&__ctrl_);
+ uintptr_t __expected = std::__cxx_atomic_load(__builtin_addressof(__ctrl_), memory_order_relaxed);
+ for (;;) {
+ if (!__atomic_smart_ptr_storage::__has_lock(__expected)) {
+ uintptr_t __desired = __expected | __atomic_smart_ptr_storage::__lock_bit_;
+ if (std::__cxx_atomic_compare_exchange_weak(
+ __builtin_addressof(__ctrl_),
+ __builtin_addressof(__expected),
+ __desired,
+ memory_order_acquire,
+ memory_order_relaxed)) {
+ _LIBCPP_ATOMIC_SP_TSAN_POST_LOCK(&__ctrl_);
+ return;
+ }
+ continue;
+ }
+
+ uintptr_t __with_notify = __expected | __atomic_smart_ptr_storage::__notify_bit_;
+ if (!__atomic_smart_ptr_storage::__has_notify(__expected)) {
+ if (!std::__cxx_atomic_compare_exchange_weak(
+ __builtin_addressof(__ctrl_),
+ __builtin_addressof(__expected),
+ __with_notify,
+ memory_order_relaxed,
+ memory_order_relaxed))
+ continue;
+ __expected = __with_notify;
+ }
+
+ std::__atomic_smart_ptr_wait_on_address(__ctrl_address(), [&] {
+ __expected = std::__cxx_atomic_load(__builtin_addressof(__ctrl_), memory_order_relaxed);
+ return !__atomic_smart_ptr_storage::__has_lock(__expected);
+ });
+ }
+ }
+
+ // Publish new control pointer, clear bits, and notify waiters if needed.
+ _LIBCPP_HIDE_FROM_ABI void __unlock(__shared_weak_count* __ctrl_to_publish) const _NOEXCEPT {
+ _LIBCPP_ATOMIC_SP_TSAN_PRE_UNLOCK(&__ctrl_);
+ uintptr_t __new_word = __atomic_smart_ptr_storage::__encode(__ctrl_to_publish, 0);
+ uintptr_t __previous = std::__cxx_atomic_exchange(__builtin_addressof(__ctrl_), __new_word, memory_order_release);
+ if (__atomic_smart_ptr_storage::__has_notify(__previous))
+ std::__atomic_smart_ptr_notify_all(__ctrl_address());
+ _LIBCPP_ATOMIC_SP_TSAN_POST_UNLOCK(&__ctrl_);
+ }
+};
+
+// [util.smartptr.atomic.shared]: same stored pointer and same ownership, or both empty.
+template <class _Element>
+_LIBCPP_HIDE_FROM_ABI inline bool __atomic_smart_ptr_equivalent(
+ _Element* __ptr,
+ __shared_weak_count* __ctrl,
+ _Element* __expected_ptr,
+ __shared_weak_count* __expected_ctrl) _NOEXCEPT {
+ if (__ctrl == nullptr && __expected_ctrl == nullptr)
+ return true;
+ return __ptr == __expected_ptr && __ctrl == __expected_ctrl;
+}
+
+template <class _Tp>
+struct atomic<shared_ptr<_Tp>> {
+ using value_type = shared_ptr<_Tp>;
+
+ static constexpr bool is_always_lock_free = false;
+
+ _LIBCPP_HIDE_FROM_ABI atomic() _NOEXCEPT : __fields_(nullptr, nullptr) {}
+ _LIBCPP_HIDE_FROM_ABI constexpr atomic(nullptr_t) _NOEXCEPT : __fields_(nullptr, nullptr) {}
+ _LIBCPP_HIDE_FROM_ABI atomic(shared_ptr<_Tp> __desired) _NOEXCEPT : __fields_(__desired.__ptr_, __desired.__cntrl_) {
+ __desired.__ptr_ = nullptr;
+ __desired.__cntrl_ = nullptr;
+ }
+
+ atomic(const atomic&) = delete;
+ atomic& operator=(const atomic&) = delete;
+
+ _LIBCPP_HIDE_FROM_ABI ~atomic() {
+ if (auto* __c = __atomic_smart_ptr_storage::__decode(
+ std::__cxx_atomic_load(__builtin_addressof(__fields_.__ctrl_), memory_order_relaxed)))
+ __c->__release_shared();
+ }
+
+ _LIBCPP_HIDE_FROM_ABI bool is_lock_free() const _NOEXCEPT { return false; }
+
+ _LIBCPP_HIDE_FROM_ABI void operator=(shared_ptr<_Tp> __desired) _NOEXCEPT { store(std::move(__desired)); }
+ _LIBCPP_HIDE_FROM_ABI void operator=(nullptr_t) _NOEXCEPT { store(nullptr); }
+ _LIBCPP_HIDE_FROM_ABI operator shared_ptr<_Tp>() const _NOEXCEPT { return load(); }
+
+ _LIBCPP_HIDE_FROM_ABI void store(shared_ptr<_Tp> __desired, memory_order __m = memory_order_seq_cst) _NOEXCEPT
+ _LIBCPP_CHECK_STORE_MEMORY_ORDER(__m) {
+ (void)__m;
+ _Tp* __desired_ptr = __desired.__ptr_;
+ __shared_weak_count* __desired_c = __desired.__cntrl_;
+ __desired.__ptr_ = nullptr;
+ __desired.__cntrl_ = nullptr;
+
+ __fields_.__lock();
+ __shared_weak_count* __old_c = __atomic_smart_ptr_storage::__decode(
+ std::__cxx_atomic_load(__builtin_addressof(__fields_.__ctrl_), memory_order_relaxed));
+ std::__cxx_atomic_store(__builtin_addressof(__fields_.__ptr_), __desired_ptr, memory_order_relaxed);
+ __fields_.__unlock(__desired_c);
+
+ if (__old_c)
+ __old_c->__release_shared();
+ }
+
+ _LIBCPP_HIDE_FROM_ABI shared_ptr<_Tp> load(memory_order __m = memory_order_seq_cst) const _NOEXCEPT
+ _LIBCPP_CHECK_LOAD_MEMORY_ORDER(__m) {
+ (void)__m;
+ __fields_.__lock();
+ _Tp* __ptr = std::__cxx_atomic_load(__builtin_addressof(__fields_.__ptr_), memory_order_relaxed);
+ __shared_weak_count* __c = __atomic_smart_ptr_storage::__decode(
+ std::__cxx_atomic_load(__builtin_addressof(__fields_.__ctrl_), memory_order_relaxed));
+ if (__c)
+ __c->__add_shared();
+ __fields_.__unlock(__c);
+ return shared_ptr<_Tp>::__create_with_control_block(__ptr, __c);
+ }
+
+ _LIBCPP_HIDE_FROM_ABI shared_ptr<_Tp>
+ exchange(shared_ptr<_Tp> __desired, memory_order __m = memory_order_seq_cst) _NOEXCEPT {
+ (void)__m;
+ _Tp* __desired_ptr = __desired.__ptr_;
+ __shared_weak_count* __desired_c = __desired.__cntrl_;
+ __desired.__ptr_ = nullptr;
+ __desired.__cntrl_ = nullptr;
+
+ __fields_.__lock();
+ _Tp* __old_ptr = std::__cxx_atomic_load(__builtin_addressof(__fields_.__ptr_), memory_order_relaxed);
+ __shared_weak_count* __old_c = __atomic_smart_ptr_storage::__decode(
+ std::__cxx_atomic_load(__builtin_addressof(__fields_.__ctrl_), memory_order_relaxed));
+ std::__cxx_atomic_store(__builtin_addressof(__fields_.__ptr_), __desired_ptr, memory_order_relaxed);
+ __fields_.__unlock(__desired_c);
+
+ return shared_ptr<_Tp>::__create_with_control_block(__old_ptr, __old_c);
+ }
+
+ _LIBCPP_HIDE_FROM_ABI bool compare_exchange_strong(
+ shared_ptr<_Tp>& __expected, shared_ptr<_Tp> __desired, memory_order __success, memory_order __failure) _NOEXCEPT
+ _LIBCPP_CHECK_EXCHANGE_MEMORY_ORDER(__success, __failure) {
+ (void)__success;
+ (void)__failure;
+ __fields_.__lock();
+ _Tp* __cur_ptr = std::__cxx_atomic_load(__builtin_addressof(__fields_.__ptr_), memory_order_relaxed);
+ __shared_weak_count* __cur_c = __atomic_smart_ptr_storage::__decode(
+ std::__cxx_atomic_load(__builtin_addressof(__fields_.__ctrl_), memory_order_relaxed));
+
+ if (__atomic_smart_ptr_equivalent(__cur_ptr, __cur_c, __expected.__ptr_, __expected.__cntrl_)) {
+ _Tp* __desired_ptr = __desired.__ptr_;
+ __shared_weak_count* __desired_c = __desired.__cntrl_;
+ __desired.__ptr_ = nullptr;
+ __desired.__cntrl_ = nullptr;
+
+ std::__cxx_atomic_store(__builtin_addressof(__fields_.__ptr_), __desired_ptr, memory_order_relaxed);
+ __fields_.__unlock(__desired_c);
+ if (__cur_c)
+ __cur_c->__release_shared();
+ return true;
+ }
+
+ if (__cur_c)
+ __cur_c->__add_shared();
+ __fields_.__unlock(__cur_c);
+ __expected = shared_ptr<_Tp>::__create_with_control_block(__cur_ptr, __cur_c);
+ return false;
+ }
+
+ _LIBCPP_HIDE_FROM_ABI bool compare_exchange_strong(
+ shared_ptr<_Tp>& __expected, shared_ptr<_Tp> __desired, memory_order __m = memory_order_seq_cst) _NOEXCEPT {
+ return compare_exchange_strong(__expected, std::move(__desired), __m, std::__to_failure_order(__m));
+ }
+
+ _LIBCPP_HIDE_FROM_ABI bool compare_exchange_weak(
+ shared_ptr<_Tp>& __expected, shared_ptr<_Tp> __desired, memory_order __success, memory_order __failure) _NOEXCEPT
+ _LIBCPP_CHECK_EXCHANGE_MEMORY_ORDER(__success, __failure) {
+ return compare_exchange_strong(__expected, std::move(__desired), __success, __failure);
+ }
+
+ _LIBCPP_HIDE_FROM_ABI bool compare_exchange_weak(
+ shared_ptr<_Tp>& __expected, shared_ptr<_Tp> __desired, memory_order __m = memory_order_seq_cst) _NOEXCEPT {
+ return compare_exchange_strong(__expected, std::move(__desired), __m, std::__to_failure_order(__m));
+ }
+
+ // Wait until the stored value is not equivalent to __old.
+ // __ctrl_ is the wait address; pointer changes are published with control updates.
+ _LIBCPP_HIDE_FROM_ABI void wait(shared_ptr<_Tp> __old, memory_order __m = memory_order_seq_cst) const _NOEXCEPT
+ _LIBCPP_CHECK_WAIT_MEMORY_ORDER(__m) {
+ _Tp* __old_ptr = __old.__ptr_;
+ __shared_weak_count* __old_c = __old.__cntrl_;
+
+ std::__atomic_smart_ptr_wait_on_address(__fields_.__ctrl_address(), [&] {
+ uintptr_t __word = std::__cxx_atomic_load(__builtin_addressof(__fields_.__ctrl_), __m);
+ __shared_weak_count* __cur_c = __atomic_smart_ptr_storage::__decode(__word);
+ if (__cur_c != __old_c)
+ return true;
+ _Tp* __cur_ptr = std::__cxx_atomic_load(__builtin_addressof(__fields_.__ptr_), __m);
+ return !__atomic_smart_ptr_equivalent(__cur_ptr, __cur_c, __old_ptr, __old_c);
+ });
+ }
+
+ _LIBCPP_HIDE_FROM_ABI void notify_one() _NOEXCEPT { std::__atomic_smart_ptr_notify_one(__fields_.__ctrl_address()); }
+ _LIBCPP_HIDE_FROM_ABI void notify_all() _NOEXCEPT { std::__atomic_smart_ptr_notify_all(__fields_.__ctrl_address()); }
+
+private:
+ __atomic_smart_ptr_fields<_Tp> __fields_;
+};
+
+template <class _Tp>
+struct atomic<weak_ptr<_Tp>> {
+ using value_type = weak_ptr<_Tp>;
+
+ static constexpr bool is_always_lock_free = false;
+
+ _LIBCPP_HIDE_FROM_ABI atomic() _NOEXCEPT : __fields_(nullptr, nullptr) {}
+ _LIBCPP_HIDE_FROM_ABI atomic(weak_ptr<_Tp> __desired) _NOEXCEPT : __fields_(__desired.__ptr_, __desired.__cntrl_) {
+ __desired.__ptr_ = nullptr;
+ __desired.__cntrl_ = nullptr;
+ }
+
+ atomic(const atomic&) = delete;
+ atomic& operator=(const atomic&) = delete;
+
+ _LIBCPP_HIDE_FROM_ABI ~atomic() {
+ if (auto* __c = __atomic_smart_ptr_storage::__decode(
+ std::__cxx_atomic_load(__builtin_addressof(__fields_.__ctrl_), memory_order_relaxed)))
+ __c->__release_weak();
+ }
+
+ _LIBCPP_HIDE_FROM_ABI bool is_lock_free() const _NOEXCEPT { return false; }
+
+ _LIBCPP_HIDE_FROM_ABI void operator=(weak_ptr<_Tp> __desired) _NOEXCEPT { store(std::move(__desired)); }
+ _LIBCPP_HIDE_FROM_ABI operator weak_ptr<_Tp>() const _NOEXCEPT { return load(); }
+
+ _LIBCPP_HIDE_FROM_ABI void store(weak_ptr<_Tp> __desired, memory_order __m = memory_order_seq_cst) _NOEXCEPT
+ _LIBCPP_CHECK_STORE_MEMORY_ORDER(__m) {
+ (void)__m;
+ _Tp* __desired_ptr = __desired.__ptr_;
+ __shared_weak_count* __desired_c = __desired.__cntrl_;
+ __desired.__ptr_ = nullptr;
+ __desired.__cntrl_ = nullptr;
+
+ __fields_.__lock();
+ __shared_weak_count* __old_c = __atomic_smart_ptr_storage::__decode(
+ std::__cxx_atomic_load(__builtin_addressof(__fields_.__ctrl_), memory_order_relaxed));
+ std::__cxx_atomic_store(__builtin_addressof(__fields_.__ptr_), __desired_ptr, memory_order_relaxed);
+ __fields_.__unlock(__desired_c);
+
+ if (__old_c)
+ __old_c->__release_weak();
+ }
+
+ _LIBCPP_HIDE_FROM_ABI weak_ptr<_Tp> load(memory_order __m = memory_order_seq_cst) const _NOEXCEPT
+ _LIBCPP_CHECK_LOAD_MEMORY_ORDER(__m) {
+ (void)__m;
+ __fields_.__lock();
+ _Tp* __ptr = std::__cxx_atomic_load(__builtin_addressof(__fields_.__ptr_), memory_order_relaxed);
+ __shared_weak_count* __c = __atomic_smart_ptr_storage::__decode(
+ std::__cxx_atomic_load(__builtin_addressof(__fields_.__ctrl_), memory_order_relaxed));
+ if (__c)
+ __c->__add_weak();
+ __fields_.__unlock(__c);
+ return weak_ptr<_Tp>::__create_with_control_block(__ptr, __c);
+ }
+
+ _LIBCPP_HIDE_FROM_ABI weak_ptr<_Tp>
+ exchange(weak_ptr<_Tp> __desired, memory_order __m = memory_order_seq_cst) _NOEXCEPT {
+ (void)__m;
+ _Tp* __desired_ptr = __desired.__ptr_;
+ __shared_weak_count* __desired_c = __desired.__cntrl_;
+ __desired.__ptr_ = nullptr;
+ __desired.__cntrl_ = nullptr;
+
+ __fields_.__lock();
+ _Tp* __old_ptr = std::__cxx_atomic_load(__builtin_addressof(__fields_.__ptr_), memory_order_relaxed);
+ __shared_weak_count* __old_c = __atomic_smart_ptr_storage::__decode(
+ std::__cxx_atomic_load(__builtin_addressof(__fields_.__ctrl_), memory_order_relaxed));
+ std::__cxx_atomic_store(__builtin_addressof(__fields_.__ptr_), __desired_ptr, memory_order_relaxed);
+ __fields_.__unlock(__desired_c);
+
+ return weak_ptr<_Tp>::__create_with_control_block(__old_ptr, __old_c);
+ }
+
+ _LIBCPP_HIDE_FROM_ABI bool compare_exchange_strong(
+ weak_ptr<_Tp>& __expected, weak_ptr<_Tp> __desired, memory_order __success, memory_order __failure) _NOEXCEPT
+ _LIBCPP_CHECK_EXCHANGE_MEMORY_ORDER(__success, __failure) {
+ (void)__success;
+ (void)__failure;
+ __fields_.__lock();
+ _Tp* __cur_ptr = std::__cxx_atomic_load(__builtin_addressof(__fields_.__ptr_), memory_order_relaxed);
+ __shared_weak_count* __cur_c = __atomic_smart_ptr_storage::__decode(
+ std::__cxx_atomic_load(__builtin_addressof(__fields_.__ctrl_), memory_order_relaxed));
+
+ if (__atomic_smart_ptr_equivalent(__cur_ptr, __cur_c, __expected.__ptr_, __expected.__cntrl_)) {
+ _Tp* __desired_ptr = __desired.__ptr_;
+ __shared_weak_count* __desired_c = __desired.__cntrl_;
+ __desired.__ptr_ = nullptr;
+ __desired.__cntrl_ = nullptr;
+
+ std::__cxx_atomic_store(__builtin_addressof(__fields_.__ptr_), __desired_ptr, memory_order_relaxed);
+ __fields_.__unlock(__desired_c);
+ if (__cur_c)
+ __cur_c->__release_weak();
+ return true;
+ }
+
+ if (__cur_c)
+ __cur_c->__add_weak();
+ __fields_.__unlock(__cur_c);
+ __expected = weak_ptr<_Tp>::__create_with_control_block(__cur_ptr, __cur_c);
+ return false;
+ }
+
+ _LIBCPP_HIDE_FROM_ABI bool compare_exchange_strong(
+ weak_ptr<_Tp>& __expected, weak_ptr<_Tp> __desired, memory_order __m = memory_order_seq_cst) _NOEXCEPT {
+ return compare_exchange_strong(__expected, std::move(__desired), __m, std::__to_failure_order(__m));
+ }
+
+ _LIBCPP_HIDE_FROM_ABI bool compare_exchange_weak(
+ weak_ptr<_Tp>& __expected, weak_ptr<_Tp> __desired, memory_order __success, memory_order __failure) _NOEXCEPT
+ _LIBCPP_CHECK_EXCHANGE_MEMORY_ORDER(__success, __failure) {
+ return compare_exchange_strong(__expected, std::move(__desired), __success, __failure);
+ }
+
+ _LIBCPP_HIDE_FROM_ABI bool compare_exchange_weak(
+ weak_ptr<_Tp>& __expected, weak_ptr<_Tp> __desired, memory_order __m = memory_order_seq_cst) _NOEXCEPT {
+ return compare_exchange_strong(__expected, std::move(__desired), __m, std::__to_failure_order(__m));
+ }
+
+ _LIBCPP_HIDE_FROM_ABI void wait(weak_ptr<_Tp> __old, memory_order __m = memory_order_seq_cst) const _NOEXCEPT
+ _LIBCPP_CHECK_WAIT_MEMORY_ORDER(__m) {
+ _Tp* __old_ptr = __old.__ptr_;
+ __shared_weak_count* __old_c = __old.__cntrl_;
+
+ std::__atomic_smart_ptr_wait_on_address(__fields_.__ctrl_address(), [&] {
+ uintptr_t __word = std::__cxx_atomic_load(__builtin_addressof(__fields_.__ctrl_), __m);
+ __shared_weak_count* __cur_c = __atomic_smart_ptr_storage::__decode(__word);
+ if (__cur_c != __old_c)
+ return true;
+ _Tp* __cur_ptr = std::__cxx_atomic_load(__builtin_addressof(__fields_.__ptr_), __m);
+ return !__atomic_smart_ptr_equivalent(__cur_ptr, __cur_c, __old_ptr, __old_c);
+ });
+ }
+
+ _LIBCPP_HIDE_FROM_ABI void notify_one() _NOEXCEPT { std::__atomic_smart_ptr_notify_one(__fields_.__ctrl_address()); }
+ _LIBCPP_HIDE_FROM_ABI void notify_all() _NOEXCEPT { std::__atomic_smart_ptr_notify_all(__fields_.__ctrl_address()); }
+
+private:
+ __atomic_smart_ptr_fields<_Tp> __fields_;
+};
+
+#endif // _LIBCPP_STD_VER >= 20 && _LIBCPP_HAS_THREADS && _LIBCPP_HAS_ATOMIC_HEADER
+
+_LIBCPP_END_NAMESPACE_STD
+
+_LIBCPP_POP_MACROS
+
+#endif // _LIBCPP___MEMORY_ATOMIC_SHARED_PTR_H
diff --git a/libcxx/include/__memory/shared_ptr.h b/libcxx/include/__memory/shared_ptr.h
index 4c86eb160ef1a..03b0d4c344349 100644
--- a/libcxx/include/__memory/shared_ptr.h
+++ b/libcxx/include/__memory/shared_ptr.h
@@ -626,6 +626,9 @@ class _LIBCPP_SHARED_PTR_TRIVIAL_ABI shared_ptr {
return __r;
}
+ template <class>
+ friend struct atomic;
+
private:
template <class _Yp, bool = is_function<_Yp>::value>
struct __shared_ptr_default_allocator {
@@ -1098,8 +1101,8 @@ template <class _Tp, class _Up>
#endif
template <class _Tp, class _Up>
-[[__nodiscard__]] inline
- _LIBCPP_HIDE_FROM_ABI shared_ptr<_Tp> dynamic_pointer_cast(const shared_ptr<_Up>& __r) _NOEXCEPT {
+[[__nodiscard__]] inline _LIBCPP_HIDE_FROM_ABI shared_ptr<_Tp>
+dynamic_pointer_cast(const shared_ptr<_Up>& __r) _NOEXCEPT {
typedef typename shared_ptr<_Tp>::element_type _ET;
_ET* __p = dynamic_cast<_ET*>(__r.get());
return __p ? shared_ptr<_Tp>(__r, __p) : shared_ptr<_Tp>();
@@ -1222,6 +1225,17 @@ class _LIBCPP_SHARED_PTR_TRIVIAL_ABI weak_ptr {
friend class weak_ptr;
template <class _Up>
friend class shared_ptr;
+
+ template <class>
+ friend struct atomic;
+
+ template <class _Yp, class _CntrlBlk>
+ _LIBCPP_HIDE_FROM_ABI static weak_ptr<_Tp> __create_with_control_block(_Yp* __p, _CntrlBlk* __cntrl) _NOEXCEPT {
+ weak_ptr<_Tp> __r;
+ __r.__ptr_ = __p;
+ __r.__cntrl_ = __cntrl;
+ return __r;
+ }
};
#if _LIBCPP_STD_VER >= 17
@@ -1538,6 +1552,10 @@ inline _LIBCPP_HIDE_FROM_ABI bool atomic_compare_exchange_weak_explicit(
_LIBCPP_END_NAMESPACE_STD
+#if _LIBCPP_STD_VER >= 20 && _LIBCPP_HAS_THREADS && _LIBCPP_HAS_ATOMIC_HEADER
+# include <__memory/atomic_shared_ptr.h>
+#endif
+
_LIBCPP_POP_MACROS
#endif // _LIBCPP___MEMORY_SHARED_PTR_H
diff --git a/libcxx/include/module.modulemap.in b/libcxx/include/module.modulemap.in
index b7c2ef91be551..c638bb1daf6c3 100644
--- a/libcxx/include/module.modulemap.in
+++ b/libcxx/include/module.modulemap.in
@@ -879,6 +879,7 @@ module std [system] {
module atomic_lock_free { header "__atomic/atomic_lock_free.h" }
module atomic_ref { header "__atomic/atomic_ref.h" }
module atomic_sync { header "__atomic/atomic_sync.h" }
+ module atomic_sync_lite { header "__atomic/atomic_sync_lite.h" }
module atomic_sync_timed { header "__atomic/atomic_sync_timed.h" }
module atomic_waitable_traits { header "__atomic/atomic_waitable_traits.h" }
module atomic {
@@ -1688,6 +1689,10 @@ module std [system] {
header "__memory/shared_ptr.h"
export std.functional.hash
}
+ module atomic_shared_ptr {
+ header "__memory/atomic_shared_ptr.h"
+ export std.memory.shared_ptr
+ }
module swap_allocator { header "__memory/swap_allocator.h" }
module temp_value { header "__memory/temp_value.h" }
module temporary_buffer {
diff --git a/libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_shared_ptr_aliasing.pass.cpp b/libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_shared_ptr_aliasing.pass.cpp
new file mode 100644
index 0000000000000..8c70c371475bd
--- /dev/null
+++ b/libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_shared_ptr_aliasing.pass.cpp
@@ -0,0 +1,71 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+// UNSUPPORTED: no-threads
+
+// [util.smartptr.atomic.shared] equivalence checks with aliasing shared_ptr.
+
+#include <atomic>
+#include <cassert>
+#include <memory>
+
+struct Pair {
+ int a;
+ int b;
+};
+
+int main(int, char**) {
+ auto owner = std::make_shared<Pair>(Pair{10, 20});
+
+ std::shared_ptr<int> view_a(owner, &owner->a);
+ std::shared_ptr<int> view_b(owner, &owner->b);
+ assert(view_a.use_count() == view_b.use_count());
+ assert(view_a.get() != view_b.get());
+
+ std::atomic<std::shared_ptr<int>> atom(view_a);
+
+ // Same control block, different stored pointer -> CAS must fail.
+ {
+ std::shared_ptr<int> expected = view_b;
+ auto fresh = std::make_shared<int>(99);
+ bool ok = atom.compare_exchange_strong(expected, fresh);
+ assert(!ok);
+ assert(expected.get() == view_a.get());
+ }
+
+ // Same stored pointer with the matching control block -> CAS must succeed.
+ {
+ std::shared_ptr<int> expected = view_a;
+ bool ok = atom.compare_exchange_strong(expected, view_b);
+ assert(ok);
+ auto got = atom.load();
+ assert(got.get() == view_b.get());
+ }
+
+ // Both empty are equivalent.
+ {
+ std::atomic<std::shared_ptr<int>> empty_atom;
+ std::shared_ptr<int> expected;
+ bool ok = empty_atom.compare_exchange_strong(expected, view_a);
+ assert(ok);
+ auto got = empty_atom.load();
+ assert(got.get() == view_a.get());
+ }
+
+ // Empty vs non-empty are not equivalent.
+ {
+ std::atomic<std::shared_ptr<int>> non_empty(view_a);
+ std::shared_ptr<int> expected;
+ bool ok = non_empty.compare_exchange_strong(expected, view_b);
+ assert(!ok);
+ assert(expected.get() == view_a.get());
+ }
+
+ return 0;
+}
diff --git a/libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_shared_ptr_class.pass.cpp b/libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_shared_ptr_class.pass.cpp
new file mode 100644
index 0000000000000..6eb3ab65fc2e1
--- /dev/null
+++ b/libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_shared_ptr_class.pass.cpp
@@ -0,0 +1,70 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+// UNSUPPORTED: no-threads
+
+#include <atomic>
+#include <cassert>
+#include <memory>
+#include <thread>
+
+int main(int, char**) {
+ std::atomic<std::shared_ptr<int>> a;
+
+ auto p1 = std::make_shared<int>(1);
+ auto p2 = std::make_shared<int>(2);
+
+ a.store(p1);
+ {
+ auto got = a.load();
+ assert(got && *got == 1);
+ }
+
+ {
+ auto old = a.exchange(p2);
+ assert(old && *old == 1);
+ auto got = a.load();
+ assert(got && *got == 2);
+ }
+
+ {
+ auto expected = p2;
+ bool ok = a.compare_exchange_strong(expected, p1);
+ assert(ok);
+ auto got = a.load();
+ assert(got && *got == 1);
+ }
+
+ {
+ auto expected = p2;
+ bool ok = a.compare_exchange_strong(expected, p2);
+ assert(!ok);
+ assert(expected && *expected == 1);
+ }
+
+#if __cpp_lib_atomic_wait >= 201907L
+ {
+ std::atomic<bool> started{false};
+ std::thread t([&] {
+ auto old = a.load();
+ started.store(true, std::memory_order_release);
+ a.wait(old);
+ });
+
+ while (!started.load(std::memory_order_acquire)) {
+ }
+
+ a.store(p2);
+ a.notify_all();
+ t.join();
+ }
+#endif
+
+ return 0;
+}
diff --git a/libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_shared_ptr_memory_order.verify.cpp b/libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_shared_ptr_memory_order.verify.cpp
new file mode 100644
index 0000000000000..047c2785cab6e
--- /dev/null
+++ b/libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_shared_ptr_memory_order.verify.cpp
@@ -0,0 +1,58 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// REQUIRES: diagnose-if-support
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+// UNSUPPORTED: no-threads
+
+#include <atomic>
+#include <memory>
+
+void test() {
+ std::atomic<std::shared_ptr<int>> atom;
+ std::shared_ptr<int> expected;
+ std::shared_ptr<int> desired;
+
+ atom.store(desired,
+ std::memory_order_consume); // expected-warning {{memory order argument to atomic operation is invalid}}
+ atom.store(desired,
+ std::memory_order_acquire); // expected-warning {{memory order argument to atomic operation is invalid}}
+ atom.store(desired,
+ std::memory_order_acq_rel); // expected-warning {{memory order argument to atomic operation is invalid}}
+
+ (void)atom.load(
+ std::memory_order_release); // expected-warning {{memory order argument to atomic operation is invalid}}
+ (void)atom.load(
+ std::memory_order_acq_rel); // expected-warning {{memory order argument to atomic operation is invalid}}
+
+ atom.wait(expected,
+ std::memory_order_release); // expected-warning {{memory order argument to atomic operation is invalid}}
+ atom.wait(expected,
+ std::memory_order_acq_rel); // expected-warning {{memory order argument to atomic operation is invalid}}
+
+ atom.compare_exchange_strong(
+ expected,
+ desired,
+ std::memory_order_seq_cst,
+ std::memory_order_release); // expected-warning {{memory order argument to atomic operation is invalid}}
+ atom.compare_exchange_strong(
+ expected,
+ desired,
+ std::memory_order_seq_cst,
+ std::memory_order_acq_rel); // expected-warning {{memory order argument to atomic operation is invalid}}
+ atom.compare_exchange_weak(
+ expected,
+ desired,
+ std::memory_order_seq_cst,
+ std::memory_order_release); // expected-warning {{memory order argument to atomic operation is invalid}}
+ atom.compare_exchange_weak(
+ expected,
+ desired,
+ std::memory_order_seq_cst,
+ std::memory_order_acq_rel); // expected-warning {{memory order argument to atomic operation is invalid}}
+}
diff --git a/libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_shared_ptr_nullptr.pass.cpp b/libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_shared_ptr_nullptr.pass.cpp
new file mode 100644
index 0000000000000..3662c491f72ee
--- /dev/null
+++ b/libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_shared_ptr_nullptr.pass.cpp
@@ -0,0 +1,54 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+// UNSUPPORTED: no-threads
+
+// LWG 3661 and LWG 3893 coverage.
+
+#include <atomic>
+#include <cassert>
+#include <memory>
+#include <type_traits>
+
+int main(int, char**) {
+ static_assert(std::is_constructible_v<std::atomic<std::shared_ptr<int>>, std::nullptr_t>);
+ static_assert(std::is_constructible_v<std::atomic<std::weak_ptr<int>>>);
+ static_assert(std::is_assignable_v<std::atomic<std::shared_ptr<int>>&, std::nullptr_t>);
+ static_assert(std::is_assignable_v<std::atomic<std::shared_ptr<int>>&, std::shared_ptr<int>>);
+
+ {
+ std::atomic<std::shared_ptr<int>> a(nullptr);
+ auto loaded = a.load();
+ assert(!loaded);
+ }
+
+ {
+ auto p = std::make_shared<int>(7);
+ std::atomic<std::shared_ptr<int>> a(p);
+ a = nullptr;
+ auto loaded = a.load();
+ assert(!loaded);
+ }
+
+ {
+ auto p = std::make_shared<int>(13);
+ std::atomic<std::shared_ptr<int>> a;
+ a = p;
+ auto loaded = a.load();
+ assert(loaded && *loaded == 13);
+ }
+
+ {
+ std::atomic<std::shared_ptr<int>> a;
+ std::shared_ptr<int> got = a;
+ assert(!got);
+ }
+
+ return 0;
+}
diff --git a/libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_shared_ptr_refcount.pass.cpp b/libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_shared_ptr_refcount.pass.cpp
new file mode 100644
index 0000000000000..197506cd5b0ac
--- /dev/null
+++ b/libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_shared_ptr_refcount.pass.cpp
@@ -0,0 +1,105 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+// UNSUPPORTED: no-threads
+
+#include <atomic>
+#include <cassert>
+#include <memory>
+
+namespace {
+struct Tracker {
+ static int alive;
+ Tracker() { ++alive; }
+ ~Tracker() { --alive; }
+};
+int Tracker::alive = 0;
+
+struct ReentrantDeleter {
+ std::atomic<std::shared_ptr<int>>* watch;
+ void operator()(int* p) const {
+ if (watch) {
+ auto current = watch->load();
+ (void)current;
+ }
+ delete p;
+ }
+};
+} // namespace
+
+int main(int, char**) {
+ // store releases the previous value.
+ {
+ std::atomic<std::shared_ptr<Tracker>> a;
+ a.store(std::make_shared<Tracker>());
+ assert(Tracker::alive == 1);
+ a.store(std::make_shared<Tracker>());
+ assert(Tracker::alive == 1);
+ a.store(nullptr);
+ assert(Tracker::alive == 0);
+ }
+
+ // exchange returns old ownership and preserves refcounts.
+ {
+ auto first = std::make_shared<Tracker>();
+ std::atomic<std::shared_ptr<Tracker>> a(first);
+ assert(first.use_count() == 2);
+ auto second = std::make_shared<Tracker>();
+ auto old = a.exchange(second);
+ assert(old.get() == first.get());
+ assert(first.use_count() == 2);
+ assert(second.use_count() == 2);
+ assert(Tracker::alive == 2);
+ }
+
+ // compare_exchange success/failure paths preserve ownership.
+ {
+ auto p1 = std::make_shared<Tracker>();
+ auto p2 = std::make_shared<Tracker>();
+ std::atomic<std::shared_ptr<Tracker>> a(p1);
+ {
+ std::shared_ptr<Tracker> expected = p2;
+ bool ok = a.compare_exchange_strong(expected, p1);
+ assert(!ok);
+ assert(expected.get() == p1.get());
+ }
+ {
+ std::shared_ptr<Tracker> expected = p1;
+ bool ok = a.compare_exchange_strong(expected, p2);
+ assert(ok);
+ }
+ assert(p1.use_count() == 1);
+ assert(p2.use_count() == 2);
+ }
+ assert(Tracker::alive == 0);
+
+ // atomic destruction releases held shared ownership.
+ {
+ auto p = std::make_shared<Tracker>();
+ {
+ std::atomic<std::shared_ptr<Tracker>> a(p);
+ assert(p.use_count() == 2);
+ }
+ assert(p.use_count() == 1);
+ }
+ assert(Tracker::alive == 0);
+
+ // Deleter must run after unlock; reentrant load must succeed.
+ {
+ std::atomic<std::shared_ptr<int>> a;
+ {
+ ReentrantDeleter d{&a};
+ std::shared_ptr<int> p(new int(7), d);
+ a.store(std::move(p));
+ }
+ a.store(std::shared_ptr<int>{});
+ }
+
+ return 0;
+}
diff --git a/libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_shared_ptr_stress.pass.cpp b/libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_shared_ptr_stress.pass.cpp
new file mode 100644
index 0000000000000..e02d74931fce6
--- /dev/null
+++ b/libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_shared_ptr_stress.pass.cpp
@@ -0,0 +1,107 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+// UNSUPPORTED: no-threads
+
+#include <array>
+#include <atomic>
+#include <cassert>
+#include <memory>
+#include <thread>
+#include <vector>
+
+namespace {
+constexpr int kCandidateCount = 4;
+constexpr int kWriters = 4;
+constexpr int kReaders = 4;
+constexpr int kIterations = 4000;
+
+template <class Atomic, class Make>
+void run(Atomic& a, Make make_value, const std::array<int, kCandidateCount>& expected) {
+ std::atomic<bool> stop{false};
+ std::vector<std::thread> threads;
+
+ for (int w = 0; w < kWriters; ++w) {
+ threads.emplace_back([&, w] {
+ for (int i = 0; i < kIterations; ++i) {
+ a.store(make_value(expected[(w + i) % kCandidateCount]));
+ }
+ });
+ }
+
+ for (int r = 0; r < kReaders; ++r) {
+ threads.emplace_back([&, r] {
+ while (!stop.load(std::memory_order_relaxed)) {
+ auto val = a.load();
+ std::shared_ptr<int> sp;
+ if constexpr (requires { val.lock(); }) {
+ sp = val.lock();
+ } else {
+ sp = val;
+ }
+ if (sp) {
+ int observed = *sp;
+ bool ok = false;
+ for (int candidate : expected) {
+ if (candidate == observed) {
+ ok = true;
+ break;
+ }
+ }
+ assert(ok);
+ }
+ (void)r;
+ }
+ });
+ }
+
+ for (int w = 0; w < kWriters; ++w) {
+ threads[w].join();
+ }
+ stop.store(true, std::memory_order_relaxed);
+ for (int r = 0; r < kReaders; ++r) {
+ threads[kWriters + r].join();
+ }
+}
+} // namespace
+
+int main(int, char**) {
+ const std::array<int, kCandidateCount> expected = {1, 2, 3, 4};
+ std::array<std::shared_ptr<int>, kCandidateCount> pool;
+ for (int i = 0; i < kCandidateCount; ++i) {
+ pool[i] = std::make_shared<int>(expected[i]);
+ }
+
+ {
+ std::atomic<std::shared_ptr<int>> a(pool[0]);
+ auto make_shared_value = [&](int v) {
+ for (int i = 0; i < kCandidateCount; ++i) {
+ if (expected[i] == v)
+ return pool[i];
+ }
+ return pool[0];
+ };
+ run(a, make_shared_value, expected);
+ }
+
+ {
+ std::atomic<std::weak_ptr<int>> a;
+ a.store(std::weak_ptr<int>(pool[0]));
+ auto make_weak_value = [&](int v) -> std::weak_ptr<int> {
+ for (int i = 0; i < kCandidateCount; ++i) {
+ if (expected[i] == v)
+ return std::weak_ptr<int>(pool[i]);
+ }
+ return std::weak_ptr<int>(pool[0]);
+ };
+ run(a, make_weak_value, expected);
+ }
+
+ return 0;
+}
diff --git a/libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_weak_ptr_class.pass.cpp b/libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_weak_ptr_class.pass.cpp
new file mode 100644
index 0000000000000..2b0c68234d3f0
--- /dev/null
+++ b/libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_weak_ptr_class.pass.cpp
@@ -0,0 +1,91 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+// UNSUPPORTED: no-threads
+
+#include <atomic>
+#include <cassert>
+#include <memory>
+#include <thread>
+
+int main(int, char**) {
+ std::atomic<std::weak_ptr<int>> a;
+
+ auto sp1 = std::make_shared<int>(1);
+ auto sp2 = std::make_shared<int>(2);
+ std::weak_ptr<int> wp1 = sp1;
+ std::weak_ptr<int> wp2 = sp2;
+
+ a.store(wp1);
+ {
+ std::weak_ptr<int> got = a.load();
+ auto locked = got.lock();
+ assert(locked && *locked == 1);
+ }
+
+ {
+ std::weak_ptr<int> old = a.exchange(wp2);
+ auto locked = old.lock();
+ assert(locked && *locked == 1);
+ std::weak_ptr<int> got = a.load();
+ auto locked2 = got.lock();
+ assert(locked2 && *locked2 == 2);
+ }
+
+ {
+ std::weak_ptr<int> expected = wp2;
+ bool ok = a.compare_exchange_strong(expected, wp1);
+ assert(ok);
+ std::weak_ptr<int> got = a.load();
+ auto locked = got.lock();
+ assert(locked && *locked == 1);
+ }
+
+ {
+ std::weak_ptr<int> expected = wp2;
+ bool ok = a.compare_exchange_strong(expected, wp2);
+ assert(!ok);
+ auto locked = expected.lock();
+ assert(locked && *locked == 1);
+ }
+
+ // Expired weak pointer remains representable and loadable.
+ {
+ auto sp3 = std::make_shared<int>(3);
+ std::weak_ptr<int> wp3 = sp3;
+ a.store(wp3);
+ sp3.reset();
+ std::weak_ptr<int> got = a.load();
+ assert(got.expired());
+ }
+
+#if __cpp_lib_atomic_wait >= 201907L
+ {
+ auto sp_for_wait = std::make_shared<int>(42);
+ std::weak_ptr<int> wp_for_wait = sp_for_wait;
+ a.store(wp_for_wait);
+
+ std::atomic<bool> started{false};
+ std::thread t([&] {
+ std::weak_ptr<int> old = a.load();
+ started.store(true, std::memory_order_release);
+ a.wait(old);
+ });
+
+ while (!started.load(std::memory_order_acquire)) {
+ }
+
+ a.store(wp1);
+ a.notify_all();
+ t.join();
+ }
+#endif
+
+ return 0;
+}
More information about the libcxx-commits
mailing list