[libcxx-commits] [libcxx] [libc++][hardening] Use bounded iterators in std::vector and std::string (PR #78929)
Konstantin Varlamov via libcxx-commits
libcxx-commits at lists.llvm.org
Mon Jul 22 14:15:46 PDT 2024
https://github.com/var-const updated https://github.com/llvm/llvm-project/pull/78929
>From 2990385a4d421f37c2ce9f5e382f0443156ef0d9 Mon Sep 17 00:00:00 2001
From: David Benjamin <davidben at google.com>
Date: Sat, 20 Jan 2024 17:06:05 -0500
Subject: [PATCH 1/7] [libc++][hardening] Use bounded iterators in std::vector
and std::string
This partially restores parity with the old, since removed debug build.
We now can re-enable a bunch of the disabled tests. Some things of note:
- bounded_iter's converting constructor has never worked. It needs a
friend declaration to access the other bound_iter instantiation's
private fields.
- The old debug iterators also checked that callers did not try to
compare iterators from different objects. bounded_iter does not
currently do this, so I've left those disabled. However, I think we
probably should add those. See
https://github.com/llvm/llvm-project/issues/78771#issuecomment-1902999181
- The std::vector iterators are bounded up to capacity, not size. This
makes for a weaker safety check. This is because the STL promises not
to invalidate iterators when appending up to the capacity. Since we
cannot retroactively update all the iterators on push_back(), I've
instead sized it to the capacaity. This is not as good, but at least
will stop the iterator from going off the end of the buffer.
There was also no test for this, so I've added one in the std
directory.
- std::string has two ambiguities to deal with. First, I opted not to
size it against the capacity. https://eel.is/c++draft/string.require#4
says iterators are invalided on an non-const operation. Second,
whether the iterator can reach the NUL terminator. The previous debug
tests and the special-case in https://eel.is/c++draft/string.access#2
suggest no. If either of these causes widespread problems, I figure we
can revisit.
- resize_and_overwrite.pass.cpp assumed std::string's iterator supported
s.begin().base(), but I see no promise of this in the standard. GCC
also doesn't support this. I fixed the test to use std::to_address.
- alignof.compile.pass.cpp's pointer isn't enough of a real pointer. (It
needs to satisfy NullablePointer, LegacyRandomAccessIterator, and
LegacyContiguousIterator.) __bounded_iter seems to instantiate enough
to notice. I've added a few more bits to satisfy it.
Fixes #78805
---
libcxx/include/__iterator/bounded_iter.h | 2 +
libcxx/include/string | 26 +++++++++++-
libcxx/include/vector | 30 ++++++++++++-
.../vector/debug.iterator.add.pass.cpp | 24 ++++++++---
.../vector/debug.iterator.decrement.pass.cpp | 8 ++--
.../debug.iterator.dereference.pass.cpp | 20 +++++++--
.../vector/debug.iterator.increment.pass.cpp | 24 ++++++++---
.../vector/debug.iterator.index.pass.cpp | 26 ++++++++++--
.../basic.string/alignof.compile.pass.cpp | 9 ++++
.../basic.string/sizeof.compile.pass.cpp | 9 ++++
.../debug.iterator.add.pass.cpp | 6 +--
.../debug.iterator.decrement.pass.cpp | 6 +--
.../debug.iterator.dereference.pass.cpp | 6 +--
.../debug.iterator.increment.pass.cpp | 6 +--
.../debug.iterator.index.pass.cpp | 9 ++--
.../push_back.invalidation.pass.cpp | 42 +++++++++++++++++++
.../resize_and_overwrite.pass.cpp | 5 ++-
17 files changed, 214 insertions(+), 44 deletions(-)
create mode 100644 libcxx/test/std/containers/sequences/vector/vector.modifiers/push_back.invalidation.pass.cpp
diff --git a/libcxx/include/__iterator/bounded_iter.h b/libcxx/include/__iterator/bounded_iter.h
index a1a941ffbaaf1..f4ab840c86f71 100644
--- a/libcxx/include/__iterator/bounded_iter.h
+++ b/libcxx/include/__iterator/bounded_iter.h
@@ -225,6 +225,8 @@ struct __bounded_iter {
private:
template <class>
friend struct pointer_traits;
+ template <class, class>
+ friend struct __bounded_iter;
_Iterator __current_; // current iterator
_Iterator __begin_, __end_; // valid range represented as [begin, end]
};
diff --git a/libcxx/include/string b/libcxx/include/string
index 1db803e822d72..61b781065115d 100644
--- a/libcxx/include/string
+++ b/libcxx/include/string
@@ -580,6 +580,7 @@ basic_string<char32_t> operator""s( const char32_t *str, size_t len );
#include <__functional/unary_function.h>
#include <__fwd/string.h>
#include <__ios/fpos.h>
+#include <__iterator/bounded_iter.h>
#include <__iterator/distance.h>
#include <__iterator/iterator_traits.h>
#include <__iterator/reverse_iterator.h>
@@ -786,9 +787,16 @@ public:
"[allocator.requirements] states that rebinding an allocator to the same type should result in the "
"original allocator");
- // TODO: Implement iterator bounds checking without requiring the global database.
+#ifdef _LIBCPP_ABI_BOUNDED_ITERATORS
+ // The pointer must be passed through __wrap_iter because
+ // __alloc_traits::pointer may not be detected as a continguous iterator on
+ // its own.
+ typedef __bounded_iter<__wrap_iter<pointer>> iterator;
+ typedef __bounded_iter<__wrap_iter<const_pointer>> const_iterator;
+#else
typedef __wrap_iter<pointer> iterator;
typedef __wrap_iter<const_pointer> const_iterator;
+#endif
typedef std::reverse_iterator<iterator> reverse_iterator;
typedef std::reverse_iterator<const_iterator> const_reverse_iterator;
@@ -918,11 +926,27 @@ private:
__init_with_sentinel(std::move(__first), std::move(__last));
}
+#ifdef _LIBCPP_ABI_BOUNDED_ITERATORS
+ _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 iterator __make_iterator(pointer __p) {
+ return __make_bounded_iter(
+ __wrap_iter<pointer>(__p),
+ __wrap_iter<pointer>(__get_pointer()),
+ __wrap_iter<pointer>(__get_pointer() + size()));
+ }
+
+ _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 const_iterator __make_const_iterator(const_pointer __p) const {
+ return __make_bounded_iter(
+ __wrap_iter<const_pointer>(__p),
+ __wrap_iter<const_pointer>(__get_pointer()),
+ __wrap_iter<const_pointer>(__get_pointer() + size()));
+ }
+#else
_LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 iterator __make_iterator(pointer __p) { return iterator(__p); }
_LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 const_iterator __make_const_iterator(const_pointer __p) const {
return const_iterator(__p);
}
+#endif
public:
_LIBCPP_TEMPLATE_DATA_VIS static const size_type npos = -1;
diff --git a/libcxx/include/vector b/libcxx/include/vector
index 976bde9b9048c..475b12be42fd1 100644
--- a/libcxx/include/vector
+++ b/libcxx/include/vector
@@ -328,6 +328,7 @@ template<class T, class charT> requires is-vector-bool-reference<T> // Since C++
#include <__functional/unary_function.h>
#include <__fwd/vector.h>
#include <__iterator/advance.h>
+#include <__iterator/bounded_iter.h>
#include <__iterator/distance.h>
#include <__iterator/iterator_traits.h>
#include <__iterator/reverse_iterator.h>
@@ -401,9 +402,16 @@ public:
typedef typename __alloc_traits::difference_type difference_type;
typedef typename __alloc_traits::pointer pointer;
typedef typename __alloc_traits::const_pointer const_pointer;
- // TODO: Implement iterator bounds checking without requiring the global database.
+#ifdef _LIBCPP_ABI_BOUNDED_ITERATORS
+ // The pointer must be passed through __wrap_iter because
+ // __alloc_traits::pointer may not be detected as a continguous iterator on
+ // its own.
+ typedef __bounded_iter<__wrap_iter<pointer>> iterator;
+ typedef __bounded_iter<__wrap_iter<const_pointer>> const_iterator;
+#else
typedef __wrap_iter<pointer> iterator;
typedef __wrap_iter<const_pointer> const_iterator;
+#endif
typedef std::reverse_iterator<iterator> reverse_iterator;
typedef std::reverse_iterator<const_iterator> const_reverse_iterator;
@@ -798,10 +806,30 @@ private:
_LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_HIDE_FROM_ABI void __append(size_type __n);
_LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_HIDE_FROM_ABI void __append(size_type __n, const_reference __x);
_LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_HIDE_FROM_ABI iterator __make_iter(pointer __p) _NOEXCEPT {
+#ifdef _LIBCPP_ABI_BOUNDED_ITERATORS
+ // Bound the iterator according to the capacity, rather than the size.
+ // Resizing a vector up to the capacity will not invalidate iterators, so,
+ // Without a way to update all live iterators on resize, we must
+ // conservatively bound the iterator by the capacity rather than the size.
+ return __make_bounded_iter(
+ __wrap_iter<pointer>(__p), __wrap_iter<pointer>(this->__begin_), __wrap_iter<pointer>(this->__end_cap()));
+#else
return iterator(__p);
+#endif
}
_LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_HIDE_FROM_ABI const_iterator __make_iter(const_pointer __p) const _NOEXCEPT {
+#ifdef _LIBCPP_ABI_BOUNDED_ITERATORS
+ // Bound the iterator according to the capacity, rather than the size.
+ // Resizing a vector up to the capacity will not invalidate iterators, so,
+ // Without a way to update all live iterators on resize, we must
+ // conservatively bound the iterator by the capacity rather than the size.
+ return __make_bounded_iter(
+ __wrap_iter<const_pointer>(__p),
+ __wrap_iter<const_pointer>(this->__begin_),
+ __wrap_iter<const_pointer>(this->__end_cap()));
+#else
return const_iterator(__p);
+#endif
}
_LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_HIDE_FROM_ABI void
__swap_out_circular_buffer(__split_buffer<value_type, allocator_type&>& __v);
diff --git a/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.add.pass.cpp b/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.add.pass.cpp
index 42021824ce6ae..1ccc71c6c1512 100644
--- a/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.add.pass.cpp
+++ b/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.add.pass.cpp
@@ -10,8 +10,8 @@
// Add to iterator out of bounds.
-// REQUIRES: has-unix-headers
-// UNSUPPORTED: !libcpp-has-legacy-debug-mode, c++03
+// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators
+// UNSUPPORTED: libcpp-hardening-mode=none, c++03
#include <vector>
#include <cassert>
@@ -19,27 +19,39 @@
#include "check_assertion.h"
#include "min_allocator.h"
+template <typename T, typename A>
+void fill_to_capacity(std::vector<T, A>& vec) {
+ // Fill vec up to its capacity. Our bounded iterators currently unable to
+ // catch accesses between size and capacity due to iterator stability
+ // guarantees. This function clears those effects.
+ while (vec.size() < vec.capacity()) {
+ vec.push_back(T());
+ }
+}
+
int main(int, char**) {
{
typedef int T;
typedef std::vector<T> C;
C c(1);
+ fill_to_capacity(c);
C::iterator i = c.begin();
- i += 1;
+ i += c.size();
assert(i == c.end());
i = c.begin();
- TEST_LIBCPP_ASSERT_FAILURE(i + 2, "Attempted to add/subtract an iterator outside its valid range");
+ TEST_LIBCPP_ASSERT_FAILURE(i + 2, "__bounded_iter::operator+=: Attempt to advance an iterator past the end");
}
{
typedef int T;
typedef std::vector<T, min_allocator<T> > C;
C c(1);
+ fill_to_capacity(c);
C::iterator i = c.begin();
- i += 1;
+ i += c.size();
assert(i == c.end());
i = c.begin();
- TEST_LIBCPP_ASSERT_FAILURE(i + 2, "Attempted to add/subtract an iterator outside its valid range");
+ TEST_LIBCPP_ASSERT_FAILURE(i + 2, "__bounded_iter::operator+=: Attempt to advance an iterator past the end");
}
return 0;
diff --git a/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.decrement.pass.cpp b/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.decrement.pass.cpp
index d134527a967e5..3f6092d9208f8 100644
--- a/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.decrement.pass.cpp
+++ b/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.decrement.pass.cpp
@@ -10,8 +10,8 @@
// Decrement iterator prior to begin.
-// REQUIRES: has-unix-headers
-// UNSUPPORTED: !libcpp-has-legacy-debug-mode, c++03
+// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators
+// UNSUPPORTED: libcpp-hardening-mode=none, c++03
#include <vector>
#include <cassert>
@@ -27,7 +27,7 @@ int main(int, char**) {
C::iterator i = c.end();
--i;
assert(i == c.begin());
- TEST_LIBCPP_ASSERT_FAILURE(--i, "Attempted to decrement a non-decrementable iterator");
+ TEST_LIBCPP_ASSERT_FAILURE(--i, "__bounded_iter::operator--: Attempt to rewind an iterator past the start");
}
{
@@ -37,7 +37,7 @@ int main(int, char**) {
C::iterator i = c.end();
--i;
assert(i == c.begin());
- TEST_LIBCPP_ASSERT_FAILURE(--i, "Attempted to decrement a non-decrementable iterator");
+ TEST_LIBCPP_ASSERT_FAILURE(--i, "__bounded_iter::operator--: Attempt to rewind an iterator past the start");
}
return 0;
diff --git a/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.dereference.pass.cpp b/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.dereference.pass.cpp
index 918cdd74b7916..a6e652de15489 100644
--- a/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.dereference.pass.cpp
+++ b/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.dereference.pass.cpp
@@ -10,29 +10,41 @@
// Dereference non-dereferenceable iterator.
-// REQUIRES: has-unix-headers
-// UNSUPPORTED: !libcpp-has-legacy-debug-mode, c++03
+// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators
+// UNSUPPORTED: libcpp-hardening-mode=none, c++03
#include <vector>
#include "check_assertion.h"
#include "min_allocator.h"
+template <typename T, typename A>
+void fill_to_capacity(std::vector<T, A>& vec) {
+ // Fill vec up to its capacity. Our bounded iterators currently unable to
+ // catch accesses between size and capacity due to iterator stability
+ // guarantees. This function clears those effects.
+ while (vec.size() < vec.capacity()) {
+ vec.push_back(T());
+ }
+}
+
int main(int, char**) {
{
typedef int T;
typedef std::vector<T> C;
C c(1);
+ fill_to_capacity(c);
C::iterator i = c.end();
- TEST_LIBCPP_ASSERT_FAILURE(*i, "Attempted to dereference a non-dereferenceable iterator");
+ TEST_LIBCPP_ASSERT_FAILURE(*i, "__bounded_iter::operator*: Attempt to dereference an iterator at the end");
}
{
typedef int T;
typedef std::vector<T, min_allocator<T> > C;
C c(1);
+ fill_to_capacity(c);
C::iterator i = c.end();
- TEST_LIBCPP_ASSERT_FAILURE(*i, "Attempted to dereference a non-dereferenceable iterator");
+ TEST_LIBCPP_ASSERT_FAILURE(*i, "__bounded_iter::operator*: Attempt to dereference an iterator at the end");
}
return 0;
diff --git a/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.increment.pass.cpp b/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.increment.pass.cpp
index d3e4b4ec3143f..b059e96073667 100644
--- a/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.increment.pass.cpp
+++ b/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.increment.pass.cpp
@@ -10,8 +10,8 @@
// Increment iterator past end.
-// REQUIRES: has-unix-headers
-// UNSUPPORTED: !libcpp-has-legacy-debug-mode, c++03
+// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators
+// UNSUPPORTED: libcpp-hardening-mode=none, c++03
#include <vector>
#include <cassert>
@@ -19,25 +19,37 @@
#include "check_assertion.h"
#include "min_allocator.h"
+template <typename T, typename A>
+void fill_to_capacity(std::vector<T, A>& vec) {
+ // Fill vec up to its capacity. Our bounded iterators currently unable to
+ // catch accesses between size and capacity due to iterator stability
+ // guarantees. This function clears those effects.
+ while (vec.size() < vec.capacity()) {
+ vec.push_back(T());
+ }
+}
+
int main(int, char**) {
{
typedef int T;
typedef std::vector<T> C;
C c(1);
+ fill_to_capacity(c);
C::iterator i = c.begin();
- ++i;
+ i += c.size();
assert(i == c.end());
- TEST_LIBCPP_ASSERT_FAILURE(++i, "Attempted to increment a non-incrementable iterator");
+ TEST_LIBCPP_ASSERT_FAILURE(++i, "__bounded_iter::operator++: Attempt to advance an iterator past the end");
}
{
typedef int T;
typedef std::vector<T, min_allocator<T> > C;
C c(1);
+ fill_to_capacity(c);
C::iterator i = c.begin();
- ++i;
+ i += c.size();
assert(i == c.end());
- TEST_LIBCPP_ASSERT_FAILURE(++i, "Attempted to increment a non-incrementable iterator");
+ TEST_LIBCPP_ASSERT_FAILURE(++i, "__bounded_iter::operator++: Attempt to advance an iterator past the end");
}
return 0;
diff --git a/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.index.pass.cpp b/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.index.pass.cpp
index 8e8f6a5dae69d..4f3b49b69b3e8 100644
--- a/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.index.pass.cpp
+++ b/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.index.pass.cpp
@@ -10,8 +10,8 @@
// Index iterator out of bounds.
-// REQUIRES: has-unix-headers
-// UNSUPPORTED: !libcpp-has-legacy-debug-mode, c++03
+// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators
+// UNSUPPORTED: libcpp-hardening-mode=none, c++03
#include <vector>
#include <cassert>
@@ -19,23 +19,41 @@
#include "check_assertion.h"
#include "min_allocator.h"
+template <typename T, typename A>
+void fill_to_capacity(std::vector<T, A>& vec) {
+ // Fill vec up to its capacity. Our bounded iterators currently unable to
+ // catch accesses between size and capacity due to iterator stability
+ // guarantees. This function clears those effects.
+ while (vec.size() < vec.capacity()) {
+ vec.push_back(T());
+ }
+}
+
int main(int, char**) {
{
typedef int T;
typedef std::vector<T> C;
C c(1);
+ fill_to_capacity(c);
C::iterator i = c.begin();
assert(i[0] == 0);
- TEST_LIBCPP_ASSERT_FAILURE(i[1], "Attempted to subscript an iterator outside its valid range");
+ TEST_LIBCPP_ASSERT_FAILURE(
+ i[c.size()], "__bounded_iter::operator[]: Attempt to index an iterator at or past the end");
+ TEST_LIBCPP_ASSERT_FAILURE(
+ i[c.size() + 1], "__bounded_iter::operator[]: Attempt to index an iterator at or past the end");
}
{
typedef int T;
typedef std::vector<T, min_allocator<T> > C;
C c(1);
+ fill_to_capacity(c);
C::iterator i = c.begin();
assert(i[0] == 0);
- TEST_LIBCPP_ASSERT_FAILURE(i[1], "Attempted to subscript an iterator outside its valid range");
+ TEST_LIBCPP_ASSERT_FAILURE(
+ i[c.size()], "__bounded_iter::operator[]: Attempt to index an iterator at or past the end");
+ TEST_LIBCPP_ASSERT_FAILURE(
+ i[c.size() + 1], "__bounded_iter::operator[]: Attempt to index an iterator at or past the end");
}
return 0;
diff --git a/libcxx/test/libcxx/strings/basic.string/alignof.compile.pass.cpp b/libcxx/test/libcxx/strings/basic.string/alignof.compile.pass.cpp
index 7b4d54ed410b0..00943ef8762f8 100644
--- a/libcxx/test/libcxx/strings/basic.string/alignof.compile.pass.cpp
+++ b/libcxx/test/libcxx/strings/basic.string/alignof.compile.pass.cpp
@@ -10,6 +10,7 @@
// UNSUPPORTED: c++03
+#include <iterator>
#include <string>
#include "test_macros.h"
@@ -18,6 +19,14 @@
template <class T>
class small_pointer {
+public:
+ using value_type = T;
+ using difference_type = std::int16_t;
+ using pointer = small_pointer;
+ using reference = T&;
+ using iterator_category = std::random_access_iterator_tag;
+
+private:
std::uint16_t offset;
};
diff --git a/libcxx/test/libcxx/strings/basic.string/sizeof.compile.pass.cpp b/libcxx/test/libcxx/strings/basic.string/sizeof.compile.pass.cpp
index 6e00e43618b2e..b85895ffcd837 100644
--- a/libcxx/test/libcxx/strings/basic.string/sizeof.compile.pass.cpp
+++ b/libcxx/test/libcxx/strings/basic.string/sizeof.compile.pass.cpp
@@ -8,6 +8,7 @@
// Ensure that we never change the size or alignment of `basic_string`
+#include <iterator>
#include <string>
#include "test_macros.h"
@@ -16,6 +17,14 @@
template <class T>
class small_pointer {
+public:
+ using value_type = T;
+ using difference_type = std::int16_t;
+ using pointer = small_pointer;
+ using reference = T&;
+ using iterator_category = std::random_access_iterator_tag;
+
+private:
std::uint16_t offset;
};
diff --git a/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.add.pass.cpp b/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.add.pass.cpp
index 8459284637dc5..5ab3d98fb19bf 100644
--- a/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.add.pass.cpp
+++ b/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.add.pass.cpp
@@ -10,8 +10,8 @@
// Add to iterator out of bounds.
-// REQUIRES: has-unix-headers
-// UNSUPPORTED: !libcpp-has-legacy-debug-mode, c++03
+// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators
+// UNSUPPORTED: libcpp-hardening-mode=none, c++03
#include <string>
#include <cassert>
@@ -26,7 +26,7 @@ void test() {
i += 1;
assert(i == c.end());
i = c.begin();
- TEST_LIBCPP_ASSERT_FAILURE(i += 2, "Attempted to add/subtract an iterator outside its valid range");
+ TEST_LIBCPP_ASSERT_FAILURE(i += 2, "__bounded_iter::operator+=: Attempt to advance an iterator past the end");
}
int main(int, char**) {
diff --git a/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.decrement.pass.cpp b/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.decrement.pass.cpp
index f1fa08d006a1e..6a1368c309bce 100644
--- a/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.decrement.pass.cpp
+++ b/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.decrement.pass.cpp
@@ -10,8 +10,8 @@
// Decrement iterator prior to begin.
-// REQUIRES: has-unix-headers
-// UNSUPPORTED: !libcpp-has-legacy-debug-mode, c++03
+// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators
+// UNSUPPORTED: libcpp-hardening-mode=none, c++03
#include <string>
#include <cassert>
@@ -25,7 +25,7 @@ void test() {
typename C::iterator i = c.end();
--i;
assert(i == c.begin());
- TEST_LIBCPP_ASSERT_FAILURE(--i, "Attempted to decrement a non-decrementable iterator");
+ TEST_LIBCPP_ASSERT_FAILURE(--i, "__bounded_iter::operator--: Attempt to rewind an iterator past the start");
}
int main(int, char**) {
diff --git a/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.dereference.pass.cpp b/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.dereference.pass.cpp
index 0bf295c6c4f4f..521d96a41c2fe 100644
--- a/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.dereference.pass.cpp
+++ b/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.dereference.pass.cpp
@@ -10,8 +10,8 @@
// Dereference non-dereferenceable iterator.
-// REQUIRES: has-unix-headers
-// UNSUPPORTED: !libcpp-has-legacy-debug-mode, c++03
+// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators
+// UNSUPPORTED: libcpp-hardening-mode=none, c++03
#include <string>
@@ -22,7 +22,7 @@ template <class C>
void test() {
C c(1, '\0');
typename C::iterator i = c.end();
- TEST_LIBCPP_ASSERT_FAILURE(*i, "Attempted to dereference a non-dereferenceable iterator");
+ TEST_LIBCPP_ASSERT_FAILURE(*i, "__bounded_iter::operator*: Attempt to dereference an iterator at the end");
}
int main(int, char**) {
diff --git a/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.increment.pass.cpp b/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.increment.pass.cpp
index 9cc9ab40bcdd6..28e2bc425d5e3 100644
--- a/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.increment.pass.cpp
+++ b/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.increment.pass.cpp
@@ -10,8 +10,8 @@
// Increment iterator past end.
-// REQUIRES: has-unix-headers
-// UNSUPPORTED: !libcpp-has-legacy-debug-mode, c++03
+// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators
+// UNSUPPORTED: libcpp-hardening-mode=none, c++03
#include <string>
#include <cassert>
@@ -25,7 +25,7 @@ void test() {
typename C::iterator i = c.begin();
++i;
assert(i == c.end());
- TEST_LIBCPP_ASSERT_FAILURE(++i, "Attempted to increment a non-incrementable iterator");
+ TEST_LIBCPP_ASSERT_FAILURE(++i, "__bounded_iter::operator++: Attempt to advance an iterator past the end");
}
int main(int, char**) {
diff --git a/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.index.pass.cpp b/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.index.pass.cpp
index 34060065d2046..8ac156c5fcf07 100644
--- a/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.index.pass.cpp
+++ b/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.index.pass.cpp
@@ -10,8 +10,8 @@
// Index iterator out of bounds.
-// REQUIRES: has-unix-headers
-// UNSUPPORTED: !libcpp-has-legacy-debug-mode, c++03
+// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators
+// UNSUPPORTED: libcpp-hardening-mode=none, c++03
#include <string>
#include <cassert>
@@ -23,9 +23,10 @@ template <class C>
void test() {
using T = decltype(std::uint8_t() - std::uint8_t());
C c(1, '\0');
- C::iterator i = c.begin();
+ typename C::iterator i = c.begin();
assert(i[0] == 0);
- TEST_LIBCPP_ASSERT_FAILURE(i[1], "Attempted to subscript an iterator outside its valid range");
+ TEST_LIBCPP_ASSERT_FAILURE(i[1], "__bounded_iter::operator[]: Attempt to index an iterator at or past the end");
+ TEST_LIBCPP_ASSERT_FAILURE(i[-1], "__bounded_iter::operator[]: Attempt to index an iterator past the start");
}
int main(int, char**) {
diff --git a/libcxx/test/std/containers/sequences/vector/vector.modifiers/push_back.invalidation.pass.cpp b/libcxx/test/std/containers/sequences/vector/vector.modifiers/push_back.invalidation.pass.cpp
new file mode 100644
index 0000000000000..a0d657064901a
--- /dev/null
+++ b/libcxx/test/std/containers/sequences/vector/vector.modifiers/push_back.invalidation.pass.cpp
@@ -0,0 +1,42 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// <vector>
+
+// void push_back(const value_type& x);
+//
+// If no reallocation happens, then references, pointers, and iterators before
+// the insertion point remain valid but those at or after the insertion point,
+// including the past-the-end iterator, are invalidated.
+
+#include <vector>
+#include <cassert>
+#include <cstddef>
+
+int main(int, char**) {
+ std::vector<int> vec;
+ vec.push_back(0);
+ vec.push_back(1);
+ vec.push_back(2);
+ vec.reserve(4);
+ std::size_t old_capacity = vec.capacity();
+ assert(old_capacity >= 4);
+
+ auto it = vec.begin();
+ vec.push_back(3);
+ assert(vec.capacity() == old_capacity);
+
+ // The capacity did not change, so the iterator remains valid and can reach
+ // the new element.
+ assert(*it == 0);
+ assert(*(it + 1) == 1);
+ assert(*(it + 2) == 2);
+ assert(*(it + 3) == 3);
+
+ return 0;
+}
diff --git a/libcxx/test/std/strings/basic.string/string.capacity/resize_and_overwrite.pass.cpp b/libcxx/test/std/strings/basic.string/string.capacity/resize_and_overwrite.pass.cpp
index edc8b67808b85..abd284852a189 100644
--- a/libcxx/test/std/strings/basic.string/string.capacity/resize_and_overwrite.pass.cpp
+++ b/libcxx/test/std/strings/basic.string/string.capacity/resize_and_overwrite.pass.cpp
@@ -15,6 +15,7 @@
#include <algorithm>
#include <cassert>
+#include <memory>
#include <string>
#include "make_string.h"
@@ -29,7 +30,7 @@ constexpr void test_appending(std::size_t k, size_t N, size_t new_capacity) {
s.resize_and_overwrite(new_capacity, [&](auto* p, auto n) {
assert(n == new_capacity);
LIBCPP_ASSERT(s.size() == new_capacity);
- LIBCPP_ASSERT(s.begin().base() == p);
+ LIBCPP_ASSERT(std::to_address(s.begin()) == p);
assert(std::all_of(p, p + k, [](const auto ch) { return ch == 'a'; }));
std::fill(p + k, p + n, 'b');
p[n] = 'c'; // will be overwritten
@@ -48,7 +49,7 @@ constexpr void test_truncating(std::size_t o, size_t N) {
s.resize_and_overwrite(N, [&](auto* p, auto n) {
assert(n == N);
LIBCPP_ASSERT(s.size() == n);
- LIBCPP_ASSERT(s.begin().base() == p);
+ LIBCPP_ASSERT(std::to_address(s.begin()) == p);
assert(std::all_of(p, p + n, [](auto ch) { return ch == 'a'; }));
p[n - 1] = 'b';
p[n] = 'c'; // will be overwritten
>From 015128b7eb2f5a2be628326dd8a235f5e85de4b1 Mon Sep 17 00:00:00 2001
From: Konstantin Varlamov <varconst at apple.com>
Date: Mon, 22 Jul 2024 01:23:21 -0700
Subject: [PATCH 2/7] Address feedback plus minor tweaks:
- create dedicated ABI macros
`_LIBCPP_ABI_BOUNDED_ITERATORS_IN_STRING` and
`_LIBCPP_ABI_BOUNDED_ITERATORS_IN_VECTOR` (and the associated Lit
features, etc.);
- add a release note and update the hardening documentation;
- add a comment in `<string>` to mention that its bounded iterators use
the size rather than the capacity;
- reword and slightly expand the comment in `<vector>` explaining the
use of the capacity rather than the size;
- tweak the comment that justifies the use of `__wrap_iter`;
- in `string`, move the conditional logic from outside to inside the
functions' scope;
- move the repeated `fill_to_capacity` function to a separate helper
header and slightly reword the comment;
- make `push_back.invalidation.pass.cpp` a death test and add a comment
on some limitations of what a bounded iterator can currently detect;
- fix another `small_pointer` helper class to fix compilation.
---
...-hardening-mode-fast-with-abi-breaks.cmake | 2 +
libcxx/docs/ReleaseNotes/19.rst | 14 +
libcxx/include/__config | 13 +
libcxx/include/string | 318 +++++++++---------
libcxx/include/vector | 47 +--
.../sequences/vector/abi.compile.pass.cpp | 8 +
.../vector/debug.iterator.add.pass.cpp | 13 +-
.../vector/debug.iterator.decrement.pass.cpp | 2 +-
.../debug.iterator.dereference.pass.cpp | 13 +-
.../vector/debug.iterator.increment.pass.cpp | 13 +-
.../vector/debug.iterator.index.pass.cpp | 13 +-
.../sequences/vector/fill_to_capacity.h | 24 ++
.../debug.iterator.add.pass.cpp | 2 +-
.../debug.iterator.decrement.pass.cpp | 2 +-
.../debug.iterator.dereference.pass.cpp | 2 +-
.../debug.iterator.increment.pass.cpp | 2 +-
.../debug.iterator.index.pass.cpp | 2 +-
.../push_back.invalidation.pass.cpp | 42 ---
libcxx/utils/libcxx/test/features.py | 2 +
19 files changed, 267 insertions(+), 267 deletions(-)
create mode 100644 libcxx/test/libcxx/containers/sequences/vector/fill_to_capacity.h
delete mode 100644 libcxx/test/std/containers/sequences/vector/vector.modifiers/push_back.invalidation.pass.cpp
diff --git a/libcxx/cmake/caches/Generic-hardening-mode-fast-with-abi-breaks.cmake b/libcxx/cmake/caches/Generic-hardening-mode-fast-with-abi-breaks.cmake
index 4860b590dcde9..4a9389fdcb41c 100644
--- a/libcxx/cmake/caches/Generic-hardening-mode-fast-with-abi-breaks.cmake
+++ b/libcxx/cmake/caches/Generic-hardening-mode-fast-with-abi-breaks.cmake
@@ -1,2 +1,4 @@
set(LIBCXX_HARDENING_MODE "fast" CACHE STRING "")
set(LIBCXX_ABI_DEFINES "_LIBCPP_ABI_BOUNDED_ITERATORS" CACHE STRING "")
+set(LIBCXX_ABI_DEFINES "_LIBCPP_ABI_BOUNDED_ITERATORS_IN_STRING" CACHE STRING "")
+set(LIBCXX_ABI_DEFINES "_LIBCPP_ABI_BOUNDED_ITERATORS_IN_VECTOR" CACHE STRING "")
diff --git a/libcxx/docs/ReleaseNotes/19.rst b/libcxx/docs/ReleaseNotes/19.rst
index 83fcd40bb80cc..9c4d603680f8b 100644
--- a/libcxx/docs/ReleaseNotes/19.rst
+++ b/libcxx/docs/ReleaseNotes/19.rst
@@ -76,6 +76,20 @@ Improvements and New Features
- The formatting library is updated to Unicode 15.1.0.
+- ``string`` and ``vector`` can now be configured to use bounded iterators when hardening is enabled. Note that checks
+ for iterator invalidation are currently not supported -- any accesses made through an invalidated bounded iterator
+ will still result in undefined behavior (bounded iterator follow the normal invalidation rules of the associated
+ container). ``string`` bounded iterators use the logical size of the container (``index < str.size()``) whereas
+ ``vector`` bounded iterators use the "physical" size of the container (``index < vec.capacity()``) which is a less
+ strict check; refer to the implementation for further details.
+
+ Bounded iterators can be enabled via the ``_LIBCPP_ABI_BOUNDED_ITERATORS_IN_STRING`` for ``string`` and via the
+ ``_LIBCPP_ABI_BOUNDED_ITERATORS_IN_VECTOR`` for ``vector``; note that checks will only be performed if the hardening
+ mode is set to ``fast`` or above (i.e., no checking is performed in the unchecked mode, even if bounded iterators are
+ enabled in the ABI configuration).
+
+ Note: bounded iterators currently are not supported for ``vector<bool>``.
+
Deprecations and Removals
-------------------------
diff --git a/libcxx/include/__config b/libcxx/include/__config
index 104a244cc82cc..f435687d4d2c8 100644
--- a/libcxx/include/__config
+++ b/libcxx/include/__config
@@ -206,6 +206,19 @@
// - `array`.
// #define _LIBCPP_ABI_BOUNDED_ITERATORS
+// Changes the iterator type of `basic_string` to a bounded iterator that keeps track of whether it's within the bounds
+// of the original container and asserts it on every dereference and when performing iterator arithmetics.
+//
+// ABI impact: changes the iterator type of `basic_string` and its specializations, such as `string` and `wstring`.
+// #define _LIBCPP_ABI_BOUNDED_ITERATORS_IN_STRING
+
+// Changes the iterator type of `vector` to a bounded iterator that keeps track of whether it's within the bounds of the
+// original container and asserts it on every dereference and when performing iterator arithmetics. Note: this doesn't
+// yet affect `vector<bool>`.
+//
+// ABI impact: changes the iterator type of `vector` (except `vector<bool>`).
+// #define _LIBCPP_ABI_BOUNDED_ITERATORS_IN_VECTOR
+
// } ABI
// HARDENING {
diff --git a/libcxx/include/string b/libcxx/include/string
index 61b781065115d..c684ad5c636cd 100644
--- a/libcxx/include/string
+++ b/libcxx/include/string
@@ -8,7 +8,7 @@
//===----------------------------------------------------------------------===//
#ifndef _LIBCPP_STRING
-#define _LIBCPP_STRING
+# define _LIBCPP_STRING
// clang-format off
@@ -568,101 +568,101 @@ basic_string<char32_t> operator""s( const char32_t *str, size_t len );
// clang-format on
-#include <__algorithm/max.h>
-#include <__algorithm/min.h>
-#include <__algorithm/remove.h>
-#include <__algorithm/remove_if.h>
-#include <__assert>
-#include <__config>
-#include <__debug_utils/sanitizers.h>
-#include <__format/enable_insertable.h>
-#include <__functional/hash.h>
-#include <__functional/unary_function.h>
-#include <__fwd/string.h>
-#include <__ios/fpos.h>
-#include <__iterator/bounded_iter.h>
-#include <__iterator/distance.h>
-#include <__iterator/iterator_traits.h>
-#include <__iterator/reverse_iterator.h>
-#include <__iterator/wrap_iter.h>
-#include <__memory/addressof.h>
-#include <__memory/allocate_at_least.h>
-#include <__memory/allocator.h>
-#include <__memory/allocator_traits.h>
-#include <__memory/compressed_pair.h>
-#include <__memory/construct_at.h>
-#include <__memory/pointer_traits.h>
-#include <__memory/swap_allocator.h>
-#include <__memory_resource/polymorphic_allocator.h>
-#include <__ranges/access.h>
-#include <__ranges/concepts.h>
-#include <__ranges/container_compatible_range.h>
-#include <__ranges/from_range.h>
-#include <__ranges/size.h>
-#include <__string/char_traits.h>
-#include <__string/extern_template_lists.h>
-#include <__type_traits/conditional.h>
-#include <__type_traits/is_allocator.h>
-#include <__type_traits/is_array.h>
-#include <__type_traits/is_convertible.h>
-#include <__type_traits/is_nothrow_assignable.h>
-#include <__type_traits/is_nothrow_constructible.h>
-#include <__type_traits/is_same.h>
-#include <__type_traits/is_standard_layout.h>
-#include <__type_traits/is_trivial.h>
-#include <__type_traits/is_trivially_relocatable.h>
-#include <__type_traits/noexcept_move_assign_container.h>
-#include <__type_traits/remove_cvref.h>
-#include <__type_traits/void_t.h>
-#include <__utility/auto_cast.h>
-#include <__utility/declval.h>
-#include <__utility/forward.h>
-#include <__utility/is_pointer_in_range.h>
-#include <__utility/move.h>
-#include <__utility/swap.h>
-#include <__utility/unreachable.h>
-#include <climits>
-#include <cstdio> // EOF
-#include <cstring>
-#include <limits>
-#include <stdexcept>
-#include <string_view>
-#include <version>
+# include <__algorithm/max.h>
+# include <__algorithm/min.h>
+# include <__algorithm/remove.h>
+# include <__algorithm/remove_if.h>
+# include <__assert>
+# include <__config>
+# include <__debug_utils/sanitizers.h>
+# include <__format/enable_insertable.h>
+# include <__functional/hash.h>
+# include <__functional/unary_function.h>
+# include <__fwd/string.h>
+# include <__ios/fpos.h>
+# include <__iterator/bounded_iter.h>
+# include <__iterator/distance.h>
+# include <__iterator/iterator_traits.h>
+# include <__iterator/reverse_iterator.h>
+# include <__iterator/wrap_iter.h>
+# include <__memory/addressof.h>
+# include <__memory/allocate_at_least.h>
+# include <__memory/allocator.h>
+# include <__memory/allocator_traits.h>
+# include <__memory/compressed_pair.h>
+# include <__memory/construct_at.h>
+# include <__memory/pointer_traits.h>
+# include <__memory/swap_allocator.h>
+# include <__memory_resource/polymorphic_allocator.h>
+# include <__ranges/access.h>
+# include <__ranges/concepts.h>
+# include <__ranges/container_compatible_range.h>
+# include <__ranges/from_range.h>
+# include <__ranges/size.h>
+# include <__string/char_traits.h>
+# include <__string/extern_template_lists.h>
+# include <__type_traits/conditional.h>
+# include <__type_traits/is_allocator.h>
+# include <__type_traits/is_array.h>
+# include <__type_traits/is_convertible.h>
+# include <__type_traits/is_nothrow_assignable.h>
+# include <__type_traits/is_nothrow_constructible.h>
+# include <__type_traits/is_same.h>
+# include <__type_traits/is_standard_layout.h>
+# include <__type_traits/is_trivial.h>
+# include <__type_traits/is_trivially_relocatable.h>
+# include <__type_traits/noexcept_move_assign_container.h>
+# include <__type_traits/remove_cvref.h>
+# include <__type_traits/void_t.h>
+# include <__utility/auto_cast.h>
+# include <__utility/declval.h>
+# include <__utility/forward.h>
+# include <__utility/is_pointer_in_range.h>
+# include <__utility/move.h>
+# include <__utility/swap.h>
+# include <__utility/unreachable.h>
+# include <climits>
+# include <cstdio> // EOF
+# include <cstring>
+# include <limits>
+# include <stdexcept>
+# include <string_view>
+# include <version>
-#ifndef _LIBCPP_HAS_NO_WIDE_CHARACTERS
-# include <cwchar>
-#endif
+# ifndef _LIBCPP_HAS_NO_WIDE_CHARACTERS
+# include <cwchar>
+# endif
// standard-mandated includes
// [iterator.range]
-#include <__iterator/access.h>
-#include <__iterator/data.h>
-#include <__iterator/empty.h>
-#include <__iterator/reverse_access.h>
-#include <__iterator/size.h>
+# include <__iterator/access.h>
+# include <__iterator/data.h>
+# include <__iterator/empty.h>
+# include <__iterator/reverse_access.h>
+# include <__iterator/size.h>
// [string.syn]
-#include <compare>
-#include <initializer_list>
+# include <compare>
+# include <initializer_list>
-#if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
-# pragma GCC system_header
-#endif
+# if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
+# pragma GCC system_header
+# endif
_LIBCPP_PUSH_MACROS
-#include <__undef_macros>
+# include <__undef_macros>
-#if !defined(_LIBCPP_HAS_NO_ASAN) && defined(_LIBCPP_INSTRUMENTED_WITH_ASAN)
-# define _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS __attribute__((__no_sanitize__("address")))
+# if !defined(_LIBCPP_HAS_NO_ASAN) && defined(_LIBCPP_INSTRUMENTED_WITH_ASAN)
+# define _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS __attribute__((__no_sanitize__("address")))
// This macro disables AddressSanitizer (ASan) instrumentation for a specific function,
// allowing memory accesses that would normally trigger ASan errors to proceed without crashing.
// This is useful for accessing parts of objects memory, which should not be accessed,
// such as unused bytes in short strings, that should never be accessed
// by other parts of the program.
-#else
-# define _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS
-#endif
+# else
+# define _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS
+# endif
_LIBCPP_BEGIN_NAMESPACE_STD
@@ -736,7 +736,7 @@ public:
//
// This string implementation doesn't contain any references into itself. It only contains a bit that says whether
// it is in small or large string mode, so the entire structure is trivially relocatable if its members are.
-#if !defined(_LIBCPP_HAS_NO_ASAN) && defined(_LIBCPP_INSTRUMENTED_WITH_ASAN)
+# if !defined(_LIBCPP_HAS_NO_ASAN) && defined(_LIBCPP_INSTRUMENTED_WITH_ASAN)
// When compiling with AddressSanitizer (ASan), basic_string cannot be trivially
// relocatable. Because the object's memory might be poisoned when its content
// is kept inside objects memory (short string optimization), instead of in allocated
@@ -744,36 +744,35 @@ public:
// the memory to avoid triggering false positives.
// Therefore it's crucial to ensure the destructor is called.
using __trivially_relocatable = void;
-#else
+# else
using __trivially_relocatable = __conditional_t<
__libcpp_is_trivially_relocatable<allocator_type>::value && __libcpp_is_trivially_relocatable<pointer>::value,
basic_string,
void>;
-#endif
-#if !defined(_LIBCPP_HAS_NO_ASAN) && defined(_LIBCPP_INSTRUMENTED_WITH_ASAN)
- _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20
- pointer __asan_volatile_wrapper(pointer const &__ptr) const {
+# endif
+# if !defined(_LIBCPP_HAS_NO_ASAN) && defined(_LIBCPP_INSTRUMENTED_WITH_ASAN)
+ _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 pointer __asan_volatile_wrapper(pointer const& __ptr) const {
if (__libcpp_is_constant_evaluated())
return __ptr;
pointer volatile __copy_ptr = __ptr;
- return const_cast<pointer &>(__copy_ptr);
+ return const_cast<pointer&>(__copy_ptr);
}
- _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20
- const_pointer __asan_volatile_wrapper(const_pointer const &__ptr) const {
+ _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 const_pointer
+ __asan_volatile_wrapper(const_pointer const& __ptr) const {
if (__libcpp_is_constant_evaluated())
return __ptr;
const_pointer volatile __copy_ptr = __ptr;
- return const_cast<const_pointer &>(__copy_ptr);
+ return const_cast<const_pointer&>(__copy_ptr);
}
-#define _LIBCPP_ASAN_VOLATILE_WRAPPER(PTR) __asan_volatile_wrapper(PTR)
-#else
-#define _LIBCPP_ASAN_VOLATILE_WRAPPER(PTR) PTR
-#endif
+# define _LIBCPP_ASAN_VOLATILE_WRAPPER(PTR) __asan_volatile_wrapper(PTR)
+# else
+# define _LIBCPP_ASAN_VOLATILE_WRAPPER(PTR) PTR
+# endif
static_assert((!is_array<value_type>::value), "Character type of basic_string must not be an array");
static_assert((is_standard_layout<value_type>::value), "Character type of basic_string must be standard-layout");
@@ -787,15 +786,15 @@ public:
"[allocator.requirements] states that rebinding an allocator to the same type should result in the "
"original allocator");
-#ifdef _LIBCPP_ABI_BOUNDED_ITERATORS
- // The pointer must be passed through __wrap_iter because
- // __alloc_traits::pointer may not be detected as a continguous iterator on
- // its own.
+#ifdef _LIBCPP_ABI_BOUNDED_ITERATORS_IN_STRING
+ // Users might provide custom allocators, and prior to C++20 we have no existing way to detect whether the allocator's
+ // pointer type is contiguous (though it has to be by the Standard). Using the wrapper type ensures the iterator is
+ // considered contiguous.
typedef __bounded_iter<__wrap_iter<pointer>> iterator;
typedef __bounded_iter<__wrap_iter<const_pointer>> const_iterator;
#else
- typedef __wrap_iter<pointer> iterator;
- typedef __wrap_iter<const_pointer> const_iterator;
+typedef __wrap_iter<pointer> iterator;
+typedef __wrap_iter<const_pointer> const_iterator;
#endif
typedef std::reverse_iterator<iterator> reverse_iterator;
typedef std::reverse_iterator<const_iterator> const_reverse_iterator;
@@ -843,34 +842,34 @@ private:
#else // _LIBCPP_ABI_ALTERNATE_STRING_LAYOUT
# ifdef _LIBCPP_BIG_ENDIAN
- static const size_type __endian_factor = 1;
+static const size_type __endian_factor = 1;
# else
- static const size_type __endian_factor = 2;
+static const size_type __endian_factor = 2;
# endif
- // Attribute 'packed' is used to keep the layout compatible with the
- // previous definition that did not use bit fields. This is because on
- // some platforms bit fields have a default size rather than the actual
- // size used, e.g., it is 4 bytes on AIX. See D128285 for details.
- struct __long {
- struct _LIBCPP_PACKED {
- size_type __is_long_ : 1;
- size_type __cap_ : sizeof(size_type) * CHAR_BIT - 1;
- };
- size_type __size_;
- pointer __data_;
+// Attribute 'packed' is used to keep the layout compatible with the
+// previous definition that did not use bit fields. This is because on
+// some platforms bit fields have a default size rather than the actual
+// size used, e.g., it is 4 bytes on AIX. See D128285 for details.
+struct __long {
+ struct _LIBCPP_PACKED {
+ size_type __is_long_ : 1;
+ size_type __cap_ : sizeof(size_type) * CHAR_BIT - 1;
};
+ size_type __size_;
+ pointer __data_;
+};
- enum { __min_cap = (sizeof(__long) - 1) / sizeof(value_type) > 2 ? (sizeof(__long) - 1) / sizeof(value_type) : 2 };
+enum { __min_cap = (sizeof(__long) - 1) / sizeof(value_type) > 2 ? (sizeof(__long) - 1) / sizeof(value_type) : 2 };
- struct __short {
- struct _LIBCPP_PACKED {
- unsigned char __is_long_ : 1;
- unsigned char __size_ : 7;
- };
- char __padding_[sizeof(value_type) - 1];
- value_type __data_[__min_cap];
+struct __short {
+ struct _LIBCPP_PACKED {
+ unsigned char __is_long_ : 1;
+ unsigned char __size_ : 7;
};
+ char __padding_[sizeof(value_type) - 1];
+ value_type __data_[__min_cap];
+};
#endif // _LIBCPP_ABI_ALTERNATE_STRING_LAYOUT
@@ -926,27 +925,34 @@ private:
__init_with_sentinel(std::move(__first), std::move(__last));
}
-#ifdef _LIBCPP_ABI_BOUNDED_ITERATORS
- _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 iterator __make_iterator(pointer __p) {
- return __make_bounded_iter(
- __wrap_iter<pointer>(__p),
- __wrap_iter<pointer>(__get_pointer()),
- __wrap_iter<pointer>(__get_pointer() + size()));
- }
-
- _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 const_iterator __make_const_iterator(const_pointer __p) const {
- return __make_bounded_iter(
- __wrap_iter<const_pointer>(__p),
- __wrap_iter<const_pointer>(__get_pointer()),
- __wrap_iter<const_pointer>(__get_pointer() + size()));
- }
+ _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 iterator __make_iterator(pointer __p) {
+#ifdef _LIBCPP_ABI_BOUNDED_ITERATORS_IN_STRING
+ // Bound the iterator according to the size (and not the capacity, unlike vector).
+ //
+ // By the Standard, string iterators are generally not guaranteed to stay valid when the container is modified,
+ // regardless of whether reallocation occurs. This allows us to check for out-of-bounds accesses using logical size,
+ // a stricter check, since correct code can never rely on being able to access newly-added elements via an existing
+ // iterator.
+ return std::__make_bounded_iter(
+ std::__wrap_iter<pointer>(__p),
+ std::__wrap_iter<pointer>(__get_pointer()),
+ std::__wrap_iter<pointer>(__get_pointer() + size()));
#else
- _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 iterator __make_iterator(pointer __p) { return iterator(__p); }
+ return iterator(__p);
+#endif // _LIBCPP_ABI_BOUNDED_ITERATORS_IN_STRING
+ }
_LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 const_iterator __make_const_iterator(const_pointer __p) const {
- return const_iterator(__p);
+#ifdef _LIBCPP_ABI_BOUNDED_ITERATORS_IN_STRING
+ // Bound the iterator according to the size (and not the capacity, unlike vector).
+ return std::__make_bounded_iter(
+ std::__wrap_iter<const_pointer>(__p),
+ std::__wrap_iter<const_pointer>(__get_pointer()),
+ std::__wrap_iter<const_pointer>(__get_pointer() + size()));
+#else
+ return const_iterator(__p);
+#endif // _LIBCPP_ABI_BOUNDED_ITERATORS_IN_STRING
}
-#endif
public:
_LIBCPP_TEMPLATE_DATA_VIS static const size_type npos = -1;
@@ -961,7 +967,7 @@ public:
#if _LIBCPP_STD_VER <= 14
_NOEXCEPT_(is_nothrow_copy_constructible<allocator_type>::value)
#else
- _NOEXCEPT
+ _NOEXCEPT
#endif
: __r_(__value_init_tag(), __a) {
__annotate_new(0);
@@ -1654,20 +1660,20 @@ public:
return basic_string(*this, __pos, __n);
}
#else
- _LIBCPP_HIDE_FROM_ABI constexpr basic_string substr(size_type __pos = 0, size_type __n = npos) const& {
- return basic_string(*this, __pos, __n);
- }
+_LIBCPP_HIDE_FROM_ABI constexpr basic_string substr(size_type __pos = 0, size_type __n = npos) const& {
+ return basic_string(*this, __pos, __n);
+}
- _LIBCPP_HIDE_FROM_ABI constexpr basic_string substr(size_type __pos = 0, size_type __n = npos) && {
- return basic_string(std::move(*this), __pos, __n);
- }
+_LIBCPP_HIDE_FROM_ABI constexpr basic_string substr(size_type __pos = 0, size_type __n = npos) && {
+ return basic_string(std::move(*this), __pos, __n);
+}
#endif
_LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 void swap(basic_string& __str)
#if _LIBCPP_STD_VER >= 14
_NOEXCEPT;
#else
- _NOEXCEPT_(!__alloc_traits::propagate_on_container_swap::value || __is_nothrow_swappable<allocator_type>::value);
+ _NOEXCEPT_(!__alloc_traits::propagate_on_container_swap::value || __is_nothrow_swappable<allocator_type>::value);
#endif
_LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 const value_type* c_str() const _NOEXCEPT { return data(); }
@@ -1858,8 +1864,8 @@ private:
std::construct_at(std::addressof(__begin[__i]));
}
#else
- (void)__begin;
- (void)__n;
+ (void)__begin;
+ (void)__n;
#endif // _LIBCPP_STD_VER >= 20
}
@@ -1948,10 +1954,12 @@ private:
_LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 const_pointer __get_long_pointer() const _NOEXCEPT {
return _LIBCPP_ASAN_VOLATILE_WRAPPER(__r_.first().__l.__data_);
}
- _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS pointer __get_short_pointer() _NOEXCEPT {
+ _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS pointer
+ __get_short_pointer() _NOEXCEPT {
return _LIBCPP_ASAN_VOLATILE_WRAPPER(pointer_traits<pointer>::pointer_to(__r_.first().__s.__data_[0]));
}
- _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS const_pointer __get_short_pointer() const _NOEXCEPT {
+ _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS const_pointer
+ __get_short_pointer() const _NOEXCEPT {
return _LIBCPP_ASAN_VOLATILE_WRAPPER(pointer_traits<const_pointer>::pointer_to(__r_.first().__s.__data_[0]));
}
_LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 pointer __get_pointer() _NOEXCEPT {
@@ -2185,11 +2193,11 @@ private:
std::__throw_out_of_range("basic_string");
}
- friend _LIBCPP_CONSTEXPR_SINCE_CXX20 basic_string operator+ <>(const basic_string&, const basic_string&);
- friend _LIBCPP_CONSTEXPR_SINCE_CXX20 basic_string operator+ <>(const value_type*, const basic_string&);
- friend _LIBCPP_CONSTEXPR_SINCE_CXX20 basic_string operator+ <>(value_type, const basic_string&);
- friend _LIBCPP_CONSTEXPR_SINCE_CXX20 basic_string operator+ <>(const basic_string&, const value_type*);
- friend _LIBCPP_CONSTEXPR_SINCE_CXX20 basic_string operator+ <>(const basic_string&, value_type);
+ friend _LIBCPP_CONSTEXPR_SINCE_CXX20 basic_string operator+<>(const basic_string&, const basic_string&);
+ friend _LIBCPP_CONSTEXPR_SINCE_CXX20 basic_string operator+<>(const value_type*, const basic_string&);
+ friend _LIBCPP_CONSTEXPR_SINCE_CXX20 basic_string operator+<>(value_type, const basic_string&);
+ friend _LIBCPP_CONSTEXPR_SINCE_CXX20 basic_string operator+<>(const basic_string&, const value_type*);
+ friend _LIBCPP_CONSTEXPR_SINCE_CXX20 basic_string operator+<>(const basic_string&, value_type);
};
// These declarations must appear before any functions are implicitly used
diff --git a/libcxx/include/vector b/libcxx/include/vector
index 475b12be42fd1..7783784d1e043 100644
--- a/libcxx/include/vector
+++ b/libcxx/include/vector
@@ -402,10 +402,10 @@ public:
typedef typename __alloc_traits::difference_type difference_type;
typedef typename __alloc_traits::pointer pointer;
typedef typename __alloc_traits::const_pointer const_pointer;
-#ifdef _LIBCPP_ABI_BOUNDED_ITERATORS
- // The pointer must be passed through __wrap_iter because
- // __alloc_traits::pointer may not be detected as a continguous iterator on
- // its own.
+#ifdef _LIBCPP_ABI_BOUNDED_ITERATORS_IN_VECTOR
+ // Users might provide custom allocators, and prior to C++20 we have no existing way to detect whether the allocator's
+ // pointer type is contiguous (though it has to be by the Standard). Using the wrapper type ensures the iterator is
+ // considered contiguous.
typedef __bounded_iter<__wrap_iter<pointer>> iterator;
typedef __bounded_iter<__wrap_iter<const_pointer>> const_iterator;
#else
@@ -805,32 +805,39 @@ private:
_LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_HIDE_FROM_ABI void __append(size_type __n);
_LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_HIDE_FROM_ABI void __append(size_type __n, const_reference __x);
+
_LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_HIDE_FROM_ABI iterator __make_iter(pointer __p) _NOEXCEPT {
-#ifdef _LIBCPP_ABI_BOUNDED_ITERATORS
+#ifdef _LIBCPP_ABI_BOUNDED_ITERATORS_IN_VECTOR
// Bound the iterator according to the capacity, rather than the size.
- // Resizing a vector up to the capacity will not invalidate iterators, so,
- // Without a way to update all live iterators on resize, we must
- // conservatively bound the iterator by the capacity rather than the size.
- return __make_bounded_iter(
- __wrap_iter<pointer>(__p), __wrap_iter<pointer>(this->__begin_), __wrap_iter<pointer>(this->__end_cap()));
+ //
+ // Vector guarantees that iterators stay valid as long as no reallocation occurs even if new elements are inserted
+ // into the container; for these cases, we need to make sure that the newly-inserted elements can be accessed
+ // through the bounded iterator without failing checks. The downside is that the bounded iterator won't catch
+ // access that is logically out-of-bounds, i.e., goes beyond the size, but is still within the capacity. With the
+ // current implementation, there is no connection between a bounded iterator and its associated container, so we
+ // don't have a way to update existing valid iterators when the container is resized, so we have to go with a more
+ // lax approach.
+ return std::__make_bounded_iter(
+ std::__wrap_iter<pointer>(__p),
+ std::__wrap_iter<pointer>(this->__begin_),
+ std::__wrap_iter<pointer>(this->__end_cap()));
#else
return iterator(__p);
-#endif
+#endif // _LIBCPP_ABI_BOUNDED_ITERATORS_IN_VECTOR
}
+
_LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_HIDE_FROM_ABI const_iterator __make_iter(const_pointer __p) const _NOEXCEPT {
-#ifdef _LIBCPP_ABI_BOUNDED_ITERATORS
+#ifdef _LIBCPP_ABI_BOUNDED_ITERATORS_IN_VECTOR
// Bound the iterator according to the capacity, rather than the size.
- // Resizing a vector up to the capacity will not invalidate iterators, so,
- // Without a way to update all live iterators on resize, we must
- // conservatively bound the iterator by the capacity rather than the size.
- return __make_bounded_iter(
- __wrap_iter<const_pointer>(__p),
- __wrap_iter<const_pointer>(this->__begin_),
- __wrap_iter<const_pointer>(this->__end_cap()));
+ return std::__make_bounded_iter(
+ std::__wrap_iter<const_pointer>(__p),
+ std::__wrap_iter<const_pointer>(this->__begin_),
+ std::__wrap_iter<const_pointer>(this->__end_cap()));
#else
return const_iterator(__p);
-#endif
+#endif // _LIBCPP_ABI_BOUNDED_ITERATORS_IN_VECTOR
}
+
_LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_HIDE_FROM_ABI void
__swap_out_circular_buffer(__split_buffer<value_type, allocator_type&>& __v);
_LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_HIDE_FROM_ABI pointer
diff --git a/libcxx/test/libcxx/containers/sequences/vector/abi.compile.pass.cpp b/libcxx/test/libcxx/containers/sequences/vector/abi.compile.pass.cpp
index b03f48434a0e0..5dcbdad41968a 100644
--- a/libcxx/test/libcxx/containers/sequences/vector/abi.compile.pass.cpp
+++ b/libcxx/test/libcxx/containers/sequences/vector/abi.compile.pass.cpp
@@ -14,6 +14,14 @@
template <class T>
class small_pointer {
+public:
+ using value_type = T;
+ using difference_type = std::int16_t;
+ using pointer = small_pointer;
+ using reference = T&;
+ using iterator_category = std::random_access_iterator_tag;
+
+private:
std::uint16_t offset;
};
diff --git a/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.add.pass.cpp b/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.add.pass.cpp
index 1ccc71c6c1512..a066ad30ebd71 100644
--- a/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.add.pass.cpp
+++ b/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.add.pass.cpp
@@ -10,25 +10,16 @@
// Add to iterator out of bounds.
-// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators
+// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators-in-vector
// UNSUPPORTED: libcpp-hardening-mode=none, c++03
#include <vector>
#include <cassert>
#include "check_assertion.h"
+#include "fill_to_capacity.h"
#include "min_allocator.h"
-template <typename T, typename A>
-void fill_to_capacity(std::vector<T, A>& vec) {
- // Fill vec up to its capacity. Our bounded iterators currently unable to
- // catch accesses between size and capacity due to iterator stability
- // guarantees. This function clears those effects.
- while (vec.size() < vec.capacity()) {
- vec.push_back(T());
- }
-}
-
int main(int, char**) {
{
typedef int T;
diff --git a/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.decrement.pass.cpp b/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.decrement.pass.cpp
index 3f6092d9208f8..59b9c16a6aa0e 100644
--- a/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.decrement.pass.cpp
+++ b/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.decrement.pass.cpp
@@ -10,7 +10,7 @@
// Decrement iterator prior to begin.
-// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators
+// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators-in-vector
// UNSUPPORTED: libcpp-hardening-mode=none, c++03
#include <vector>
diff --git a/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.dereference.pass.cpp b/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.dereference.pass.cpp
index a6e652de15489..877d3655fbe2e 100644
--- a/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.dereference.pass.cpp
+++ b/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.dereference.pass.cpp
@@ -10,24 +10,15 @@
// Dereference non-dereferenceable iterator.
-// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators
+// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators-in-vector
// UNSUPPORTED: libcpp-hardening-mode=none, c++03
#include <vector>
#include "check_assertion.h"
+#include "fill_to_capacity.h"
#include "min_allocator.h"
-template <typename T, typename A>
-void fill_to_capacity(std::vector<T, A>& vec) {
- // Fill vec up to its capacity. Our bounded iterators currently unable to
- // catch accesses between size and capacity due to iterator stability
- // guarantees. This function clears those effects.
- while (vec.size() < vec.capacity()) {
- vec.push_back(T());
- }
-}
-
int main(int, char**) {
{
typedef int T;
diff --git a/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.increment.pass.cpp b/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.increment.pass.cpp
index b059e96073667..e540f40f8c476 100644
--- a/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.increment.pass.cpp
+++ b/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.increment.pass.cpp
@@ -10,25 +10,16 @@
// Increment iterator past end.
-// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators
+// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators-in-vector
// UNSUPPORTED: libcpp-hardening-mode=none, c++03
#include <vector>
#include <cassert>
#include "check_assertion.h"
+#include "fill_to_capacity.h"
#include "min_allocator.h"
-template <typename T, typename A>
-void fill_to_capacity(std::vector<T, A>& vec) {
- // Fill vec up to its capacity. Our bounded iterators currently unable to
- // catch accesses between size and capacity due to iterator stability
- // guarantees. This function clears those effects.
- while (vec.size() < vec.capacity()) {
- vec.push_back(T());
- }
-}
-
int main(int, char**) {
{
typedef int T;
diff --git a/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.index.pass.cpp b/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.index.pass.cpp
index 4f3b49b69b3e8..63354af5af022 100644
--- a/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.index.pass.cpp
+++ b/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.index.pass.cpp
@@ -10,25 +10,16 @@
// Index iterator out of bounds.
-// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators
+// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators-in-vector
// UNSUPPORTED: libcpp-hardening-mode=none, c++03
#include <vector>
#include <cassert>
#include "check_assertion.h"
+#include "fill_to_capacity.h"
#include "min_allocator.h"
-template <typename T, typename A>
-void fill_to_capacity(std::vector<T, A>& vec) {
- // Fill vec up to its capacity. Our bounded iterators currently unable to
- // catch accesses between size and capacity due to iterator stability
- // guarantees. This function clears those effects.
- while (vec.size() < vec.capacity()) {
- vec.push_back(T());
- }
-}
-
int main(int, char**) {
{
typedef int T;
diff --git a/libcxx/test/libcxx/containers/sequences/vector/fill_to_capacity.h b/libcxx/test/libcxx/containers/sequences/vector/fill_to_capacity.h
new file mode 100644
index 0000000000000..abf88c477fece
--- /dev/null
+++ b/libcxx/test/libcxx/containers/sequences/vector/fill_to_capacity.h
@@ -0,0 +1,24 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LIBCXX_TEST_LIBCXX_CONTAINERS_SEQUENCES_VECTOR_FILL_TO_CAPACITY_H
+#define LIBCXX_TEST_LIBCXX_CONTAINERS_SEQUENCES_VECTOR_FILL_TO_CAPACITY_H
+
+#include <vector>
+
+template <typename T, typename A>
+void fill_to_capacity(std::vector<T, A>& vec) {
+ // Fill the given vector up to its capacity. Our bounded iterators are currently unable to catch an out-of-bounds
+ // access that goes beyond the container's logical storage (above the size) but is still within its physical storage
+ // (below the capacity) due to iterator stability guarantees. Filling a vector makes this distinction go away.
+ while (vec.size() < vec.capacity()) {
+ vec.push_back(T());
+ }
+}
+
+#endif // LIBCXX_TEST_LIBCXX_CONTAINERS_SEQUENCES_VECTOR_FILL_TO_CAPACITY_H
diff --git a/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.add.pass.cpp b/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.add.pass.cpp
index 5ab3d98fb19bf..56c9d63d0dbaf 100644
--- a/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.add.pass.cpp
+++ b/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.add.pass.cpp
@@ -10,7 +10,7 @@
// Add to iterator out of bounds.
-// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators
+// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators-in-string
// UNSUPPORTED: libcpp-hardening-mode=none, c++03
#include <string>
diff --git a/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.decrement.pass.cpp b/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.decrement.pass.cpp
index 6a1368c309bce..43a9739bf936f 100644
--- a/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.decrement.pass.cpp
+++ b/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.decrement.pass.cpp
@@ -10,7 +10,7 @@
// Decrement iterator prior to begin.
-// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators
+// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators-in-string
// UNSUPPORTED: libcpp-hardening-mode=none, c++03
#include <string>
diff --git a/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.dereference.pass.cpp b/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.dereference.pass.cpp
index 521d96a41c2fe..e2326be021033 100644
--- a/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.dereference.pass.cpp
+++ b/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.dereference.pass.cpp
@@ -10,7 +10,7 @@
// Dereference non-dereferenceable iterator.
-// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators
+// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators-in-string
// UNSUPPORTED: libcpp-hardening-mode=none, c++03
#include <string>
diff --git a/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.increment.pass.cpp b/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.increment.pass.cpp
index 28e2bc425d5e3..a7453f3115197 100644
--- a/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.increment.pass.cpp
+++ b/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.increment.pass.cpp
@@ -10,7 +10,7 @@
// Increment iterator past end.
-// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators
+// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators-in-string
// UNSUPPORTED: libcpp-hardening-mode=none, c++03
#include <string>
diff --git a/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.index.pass.cpp b/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.index.pass.cpp
index 8ac156c5fcf07..e7d384413b589 100644
--- a/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.index.pass.cpp
+++ b/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.index.pass.cpp
@@ -10,7 +10,7 @@
// Index iterator out of bounds.
-// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators
+// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators-in-string
// UNSUPPORTED: libcpp-hardening-mode=none, c++03
#include <string>
diff --git a/libcxx/test/std/containers/sequences/vector/vector.modifiers/push_back.invalidation.pass.cpp b/libcxx/test/std/containers/sequences/vector/vector.modifiers/push_back.invalidation.pass.cpp
deleted file mode 100644
index a0d657064901a..0000000000000
--- a/libcxx/test/std/containers/sequences/vector/vector.modifiers/push_back.invalidation.pass.cpp
+++ /dev/null
@@ -1,42 +0,0 @@
-//===----------------------------------------------------------------------===//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-
-// <vector>
-
-// void push_back(const value_type& x);
-//
-// If no reallocation happens, then references, pointers, and iterators before
-// the insertion point remain valid but those at or after the insertion point,
-// including the past-the-end iterator, are invalidated.
-
-#include <vector>
-#include <cassert>
-#include <cstddef>
-
-int main(int, char**) {
- std::vector<int> vec;
- vec.push_back(0);
- vec.push_back(1);
- vec.push_back(2);
- vec.reserve(4);
- std::size_t old_capacity = vec.capacity();
- assert(old_capacity >= 4);
-
- auto it = vec.begin();
- vec.push_back(3);
- assert(vec.capacity() == old_capacity);
-
- // The capacity did not change, so the iterator remains valid and can reach
- // the new element.
- assert(*it == 0);
- assert(*(it + 1) == 1);
- assert(*(it + 2) == 2);
- assert(*(it + 3) == 3);
-
- return 0;
-}
diff --git a/libcxx/utils/libcxx/test/features.py b/libcxx/utils/libcxx/test/features.py
index c81b56b1af547..e8e9293ebfc80 100644
--- a/libcxx/utils/libcxx/test/features.py
+++ b/libcxx/utils/libcxx/test/features.py
@@ -312,6 +312,8 @@ def _getAndroidDeviceApi(cfg):
"_LIBCPP_NO_VCRUNTIME": "libcpp-no-vcruntime",
"_LIBCPP_ABI_VERSION": "libcpp-abi-version",
"_LIBCPP_ABI_BOUNDED_ITERATORS": "libcpp-has-abi-bounded-iterators",
+ "_LIBCPP_ABI_BOUNDED_ITERATORS_IN_STRING": "libcpp-has-abi-bounded-iterators-in-string",
+ "_LIBCPP_ABI_BOUNDED_ITERATORS_IN_VECTOR": "libcpp-has-abi-bounded-iterators-in-vector",
"_LIBCPP_HAS_NO_FILESYSTEM": "no-filesystem",
"_LIBCPP_HAS_NO_RANDOM_DEVICE": "no-random-device",
"_LIBCPP_HAS_NO_LOCALIZATION": "no-localization",
>From fb68b42703aee0b7c61a6cfb668d453362364a1e Mon Sep 17 00:00:00 2001
From: Konstantin Varlamov <varconst at apple.com>
Date: Mon, 22 Jul 2024 01:26:30 -0700
Subject: [PATCH 3/7] Fix string formatting
---
libcxx/include/string | 248 +++++++++++++++++++++---------------------
1 file changed, 124 insertions(+), 124 deletions(-)
diff --git a/libcxx/include/string b/libcxx/include/string
index c684ad5c636cd..07b7b342b2b77 100644
--- a/libcxx/include/string
+++ b/libcxx/include/string
@@ -8,7 +8,7 @@
//===----------------------------------------------------------------------===//
#ifndef _LIBCPP_STRING
-# define _LIBCPP_STRING
+#define _LIBCPP_STRING
// clang-format off
@@ -568,101 +568,101 @@ basic_string<char32_t> operator""s( const char32_t *str, size_t len );
// clang-format on
-# include <__algorithm/max.h>
-# include <__algorithm/min.h>
-# include <__algorithm/remove.h>
-# include <__algorithm/remove_if.h>
-# include <__assert>
-# include <__config>
-# include <__debug_utils/sanitizers.h>
-# include <__format/enable_insertable.h>
-# include <__functional/hash.h>
-# include <__functional/unary_function.h>
-# include <__fwd/string.h>
-# include <__ios/fpos.h>
-# include <__iterator/bounded_iter.h>
-# include <__iterator/distance.h>
-# include <__iterator/iterator_traits.h>
-# include <__iterator/reverse_iterator.h>
-# include <__iterator/wrap_iter.h>
-# include <__memory/addressof.h>
-# include <__memory/allocate_at_least.h>
-# include <__memory/allocator.h>
-# include <__memory/allocator_traits.h>
-# include <__memory/compressed_pair.h>
-# include <__memory/construct_at.h>
-# include <__memory/pointer_traits.h>
-# include <__memory/swap_allocator.h>
-# include <__memory_resource/polymorphic_allocator.h>
-# include <__ranges/access.h>
-# include <__ranges/concepts.h>
-# include <__ranges/container_compatible_range.h>
-# include <__ranges/from_range.h>
-# include <__ranges/size.h>
-# include <__string/char_traits.h>
-# include <__string/extern_template_lists.h>
-# include <__type_traits/conditional.h>
-# include <__type_traits/is_allocator.h>
-# include <__type_traits/is_array.h>
-# include <__type_traits/is_convertible.h>
-# include <__type_traits/is_nothrow_assignable.h>
-# include <__type_traits/is_nothrow_constructible.h>
-# include <__type_traits/is_same.h>
-# include <__type_traits/is_standard_layout.h>
-# include <__type_traits/is_trivial.h>
-# include <__type_traits/is_trivially_relocatable.h>
-# include <__type_traits/noexcept_move_assign_container.h>
-# include <__type_traits/remove_cvref.h>
-# include <__type_traits/void_t.h>
-# include <__utility/auto_cast.h>
-# include <__utility/declval.h>
-# include <__utility/forward.h>
-# include <__utility/is_pointer_in_range.h>
-# include <__utility/move.h>
-# include <__utility/swap.h>
-# include <__utility/unreachable.h>
-# include <climits>
-# include <cstdio> // EOF
-# include <cstring>
-# include <limits>
-# include <stdexcept>
-# include <string_view>
-# include <version>
+#include <__algorithm/max.h>
+#include <__algorithm/min.h>
+#include <__algorithm/remove.h>
+#include <__algorithm/remove_if.h>
+#include <__assert>
+#include <__config>
+#include <__debug_utils/sanitizers.h>
+#include <__format/enable_insertable.h>
+#include <__functional/hash.h>
+#include <__functional/unary_function.h>
+#include <__fwd/string.h>
+#include <__ios/fpos.h>
+#include <__iterator/bounded_iter.h>
+#include <__iterator/distance.h>
+#include <__iterator/iterator_traits.h>
+#include <__iterator/reverse_iterator.h>
+#include <__iterator/wrap_iter.h>
+#include <__memory/addressof.h>
+#include <__memory/allocate_at_least.h>
+#include <__memory/allocator.h>
+#include <__memory/allocator_traits.h>
+#include <__memory/compressed_pair.h>
+#include <__memory/construct_at.h>
+#include <__memory/pointer_traits.h>
+#include <__memory/swap_allocator.h>
+#include <__memory_resource/polymorphic_allocator.h>
+#include <__ranges/access.h>
+#include <__ranges/concepts.h>
+#include <__ranges/container_compatible_range.h>
+#include <__ranges/from_range.h>
+#include <__ranges/size.h>
+#include <__string/char_traits.h>
+#include <__string/extern_template_lists.h>
+#include <__type_traits/conditional.h>
+#include <__type_traits/is_allocator.h>
+#include <__type_traits/is_array.h>
+#include <__type_traits/is_convertible.h>
+#include <__type_traits/is_nothrow_assignable.h>
+#include <__type_traits/is_nothrow_constructible.h>
+#include <__type_traits/is_same.h>
+#include <__type_traits/is_standard_layout.h>
+#include <__type_traits/is_trivial.h>
+#include <__type_traits/is_trivially_relocatable.h>
+#include <__type_traits/noexcept_move_assign_container.h>
+#include <__type_traits/remove_cvref.h>
+#include <__type_traits/void_t.h>
+#include <__utility/auto_cast.h>
+#include <__utility/declval.h>
+#include <__utility/forward.h>
+#include <__utility/is_pointer_in_range.h>
+#include <__utility/move.h>
+#include <__utility/swap.h>
+#include <__utility/unreachable.h>
+#include <climits>
+#include <cstdio> // EOF
+#include <cstring>
+#include <limits>
+#include <stdexcept>
+#include <string_view>
+#include <version>
-# ifndef _LIBCPP_HAS_NO_WIDE_CHARACTERS
-# include <cwchar>
-# endif
+#ifndef _LIBCPP_HAS_NO_WIDE_CHARACTERS
+# include <cwchar>
+#endif
// standard-mandated includes
// [iterator.range]
-# include <__iterator/access.h>
-# include <__iterator/data.h>
-# include <__iterator/empty.h>
-# include <__iterator/reverse_access.h>
-# include <__iterator/size.h>
+#include <__iterator/access.h>
+#include <__iterator/data.h>
+#include <__iterator/empty.h>
+#include <__iterator/reverse_access.h>
+#include <__iterator/size.h>
// [string.syn]
-# include <compare>
-# include <initializer_list>
+#include <compare>
+#include <initializer_list>
-# if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
-# pragma GCC system_header
-# endif
+#if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
+# pragma GCC system_header
+#endif
_LIBCPP_PUSH_MACROS
-# include <__undef_macros>
+#include <__undef_macros>
-# if !defined(_LIBCPP_HAS_NO_ASAN) && defined(_LIBCPP_INSTRUMENTED_WITH_ASAN)
-# define _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS __attribute__((__no_sanitize__("address")))
+#if !defined(_LIBCPP_HAS_NO_ASAN) && defined(_LIBCPP_INSTRUMENTED_WITH_ASAN)
+# define _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS __attribute__((__no_sanitize__("address")))
// This macro disables AddressSanitizer (ASan) instrumentation for a specific function,
// allowing memory accesses that would normally trigger ASan errors to proceed without crashing.
// This is useful for accessing parts of objects memory, which should not be accessed,
// such as unused bytes in short strings, that should never be accessed
// by other parts of the program.
-# else
-# define _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS
-# endif
+#else
+# define _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS
+#endif
_LIBCPP_BEGIN_NAMESPACE_STD
@@ -736,7 +736,7 @@ public:
//
// This string implementation doesn't contain any references into itself. It only contains a bit that says whether
// it is in small or large string mode, so the entire structure is trivially relocatable if its members are.
-# if !defined(_LIBCPP_HAS_NO_ASAN) && defined(_LIBCPP_INSTRUMENTED_WITH_ASAN)
+#if !defined(_LIBCPP_HAS_NO_ASAN) && defined(_LIBCPP_INSTRUMENTED_WITH_ASAN)
// When compiling with AddressSanitizer (ASan), basic_string cannot be trivially
// relocatable. Because the object's memory might be poisoned when its content
// is kept inside objects memory (short string optimization), instead of in allocated
@@ -744,13 +744,13 @@ public:
// the memory to avoid triggering false positives.
// Therefore it's crucial to ensure the destructor is called.
using __trivially_relocatable = void;
-# else
+#else
using __trivially_relocatable = __conditional_t<
__libcpp_is_trivially_relocatable<allocator_type>::value && __libcpp_is_trivially_relocatable<pointer>::value,
basic_string,
void>;
-# endif
-# if !defined(_LIBCPP_HAS_NO_ASAN) && defined(_LIBCPP_INSTRUMENTED_WITH_ASAN)
+#endif
+#if !defined(_LIBCPP_HAS_NO_ASAN) && defined(_LIBCPP_INSTRUMENTED_WITH_ASAN)
_LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 pointer __asan_volatile_wrapper(pointer const& __ptr) const {
if (__libcpp_is_constant_evaluated())
return __ptr;
@@ -769,10 +769,10 @@ public:
return const_cast<const_pointer&>(__copy_ptr);
}
-# define _LIBCPP_ASAN_VOLATILE_WRAPPER(PTR) __asan_volatile_wrapper(PTR)
-# else
-# define _LIBCPP_ASAN_VOLATILE_WRAPPER(PTR) PTR
-# endif
+# define _LIBCPP_ASAN_VOLATILE_WRAPPER(PTR) __asan_volatile_wrapper(PTR)
+#else
+# define _LIBCPP_ASAN_VOLATILE_WRAPPER(PTR) PTR
+#endif
static_assert((!is_array<value_type>::value), "Character type of basic_string must not be an array");
static_assert((is_standard_layout<value_type>::value), "Character type of basic_string must be standard-layout");
@@ -793,8 +793,8 @@ public:
typedef __bounded_iter<__wrap_iter<pointer>> iterator;
typedef __bounded_iter<__wrap_iter<const_pointer>> const_iterator;
#else
-typedef __wrap_iter<pointer> iterator;
-typedef __wrap_iter<const_pointer> const_iterator;
+ typedef __wrap_iter<pointer> iterator;
+ typedef __wrap_iter<const_pointer> const_iterator;
#endif
typedef std::reverse_iterator<iterator> reverse_iterator;
typedef std::reverse_iterator<const_iterator> const_reverse_iterator;
@@ -842,34 +842,34 @@ private:
#else // _LIBCPP_ABI_ALTERNATE_STRING_LAYOUT
# ifdef _LIBCPP_BIG_ENDIAN
-static const size_type __endian_factor = 1;
+ static const size_type __endian_factor = 1;
# else
-static const size_type __endian_factor = 2;
+ static const size_type __endian_factor = 2;
# endif
-// Attribute 'packed' is used to keep the layout compatible with the
-// previous definition that did not use bit fields. This is because on
-// some platforms bit fields have a default size rather than the actual
-// size used, e.g., it is 4 bytes on AIX. See D128285 for details.
-struct __long {
- struct _LIBCPP_PACKED {
- size_type __is_long_ : 1;
- size_type __cap_ : sizeof(size_type) * CHAR_BIT - 1;
+ // Attribute 'packed' is used to keep the layout compatible with the
+ // previous definition that did not use bit fields. This is because on
+ // some platforms bit fields have a default size rather than the actual
+ // size used, e.g., it is 4 bytes on AIX. See D128285 for details.
+ struct __long {
+ struct _LIBCPP_PACKED {
+ size_type __is_long_ : 1;
+ size_type __cap_ : sizeof(size_type) * CHAR_BIT - 1;
+ };
+ size_type __size_;
+ pointer __data_;
};
- size_type __size_;
- pointer __data_;
-};
-enum { __min_cap = (sizeof(__long) - 1) / sizeof(value_type) > 2 ? (sizeof(__long) - 1) / sizeof(value_type) : 2 };
+ enum { __min_cap = (sizeof(__long) - 1) / sizeof(value_type) > 2 ? (sizeof(__long) - 1) / sizeof(value_type) : 2 };
-struct __short {
- struct _LIBCPP_PACKED {
- unsigned char __is_long_ : 1;
- unsigned char __size_ : 7;
+ struct __short {
+ struct _LIBCPP_PACKED {
+ unsigned char __is_long_ : 1;
+ unsigned char __size_ : 7;
+ };
+ char __padding_[sizeof(value_type) - 1];
+ value_type __data_[__min_cap];
};
- char __padding_[sizeof(value_type) - 1];
- value_type __data_[__min_cap];
-};
#endif // _LIBCPP_ABI_ALTERNATE_STRING_LAYOUT
@@ -925,7 +925,7 @@ struct __short {
__init_with_sentinel(std::move(__first), std::move(__last));
}
- _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 iterator __make_iterator(pointer __p) {
+ _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 iterator __make_iterator(pointer __p) {
#ifdef _LIBCPP_ABI_BOUNDED_ITERATORS_IN_STRING
// Bound the iterator according to the size (and not the capacity, unlike vector).
//
@@ -940,7 +940,7 @@ struct __short {
#else
return iterator(__p);
#endif // _LIBCPP_ABI_BOUNDED_ITERATORS_IN_STRING
- }
+ }
_LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 const_iterator __make_const_iterator(const_pointer __p) const {
#ifdef _LIBCPP_ABI_BOUNDED_ITERATORS_IN_STRING
@@ -950,7 +950,7 @@ struct __short {
std::__wrap_iter<const_pointer>(__get_pointer()),
std::__wrap_iter<const_pointer>(__get_pointer() + size()));
#else
- return const_iterator(__p);
+ return const_iterator(__p);
#endif // _LIBCPP_ABI_BOUNDED_ITERATORS_IN_STRING
}
@@ -967,7 +967,7 @@ public:
#if _LIBCPP_STD_VER <= 14
_NOEXCEPT_(is_nothrow_copy_constructible<allocator_type>::value)
#else
- _NOEXCEPT
+ _NOEXCEPT
#endif
: __r_(__value_init_tag(), __a) {
__annotate_new(0);
@@ -1660,20 +1660,20 @@ public:
return basic_string(*this, __pos, __n);
}
#else
-_LIBCPP_HIDE_FROM_ABI constexpr basic_string substr(size_type __pos = 0, size_type __n = npos) const& {
- return basic_string(*this, __pos, __n);
-}
+ _LIBCPP_HIDE_FROM_ABI constexpr basic_string substr(size_type __pos = 0, size_type __n = npos) const& {
+ return basic_string(*this, __pos, __n);
+ }
-_LIBCPP_HIDE_FROM_ABI constexpr basic_string substr(size_type __pos = 0, size_type __n = npos) && {
- return basic_string(std::move(*this), __pos, __n);
-}
+ _LIBCPP_HIDE_FROM_ABI constexpr basic_string substr(size_type __pos = 0, size_type __n = npos) && {
+ return basic_string(std::move(*this), __pos, __n);
+ }
#endif
_LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 void swap(basic_string& __str)
#if _LIBCPP_STD_VER >= 14
_NOEXCEPT;
#else
- _NOEXCEPT_(!__alloc_traits::propagate_on_container_swap::value || __is_nothrow_swappable<allocator_type>::value);
+ _NOEXCEPT_(!__alloc_traits::propagate_on_container_swap::value || __is_nothrow_swappable<allocator_type>::value);
#endif
_LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 const value_type* c_str() const _NOEXCEPT { return data(); }
@@ -1864,8 +1864,8 @@ private:
std::construct_at(std::addressof(__begin[__i]));
}
#else
- (void)__begin;
- (void)__n;
+ (void)__begin;
+ (void)__n;
#endif // _LIBCPP_STD_VER >= 20
}
>From 691bf4b53cda84991a4bce344a31dbfea0cbbf5b Mon Sep 17 00:00:00 2001
From: Konstantin Varlamov <varconst at apple.com>
Date: Mon, 22 Jul 2024 14:02:54 -0700
Subject: [PATCH 4/7] Add a forgotten test file
---
.../assert.push_back.invalidation.pass.cpp | 56 +++++++++++++++++++
1 file changed, 56 insertions(+)
create mode 100644 libcxx/test/std/containers/sequences/vector/vector.modifiers/assert.push_back.invalidation.pass.cpp
diff --git a/libcxx/test/std/containers/sequences/vector/vector.modifiers/assert.push_back.invalidation.pass.cpp b/libcxx/test/std/containers/sequences/vector/vector.modifiers/assert.push_back.invalidation.pass.cpp
new file mode 100644
index 0000000000000..4b5beba413d59
--- /dev/null
+++ b/libcxx/test/std/containers/sequences/vector/vector.modifiers/assert.push_back.invalidation.pass.cpp
@@ -0,0 +1,56 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// <vector>
+
+// void push_back(const value_type& x);
+//
+// If no reallocation happens, then references, pointers, and iterators before
+// the insertion point remain valid but those at or after the insertion point,
+// including the past-the-end iterator, are invalidated.
+
+// REQUIRES: has-unix-headers, libcpp-has-abi-bounded-iterators-in-vector
+// UNSUPPORTED: c++03
+// UNSUPPORTED: libcpp-hardening-mode=none
+// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing
+
+#include <vector>
+#include <cassert>
+#include <cstddef>
+
+#include "check_assertion.h"
+
+int main(int, char**) {
+ std::vector<int> vec;
+ vec.reserve(4);
+ std::size_t old_capacity = vec.capacity();
+ assert(old_capacity >= 4);
+
+ vec.push_back(0);
+ vec.push_back(1);
+ vec.push_back(2);
+ auto it = vec.begin();
+ vec.push_back(3);
+ assert(vec.capacity() == old_capacity);
+
+ // The capacity did not change, so the iterator remains valid and can reach the new element.
+ assert(*it == 0);
+ assert(*(it + 1) == 1);
+ assert(*(it + 2) == 2);
+ assert(*(it + 3) == 3);
+
+ while (vec.capacity() == old_capacity) {
+ vec.push_back(42);
+ }
+ TEST_LIBCPP_ASSERT_FAILURE(
+ *(it + old_capacity), "__bounded_iter::operator*: Attempt to dereference an iterator at the end");
+ // Unfortunately, the bounded iterator does not detect that it's been invalidated and will still allow attempts to
+ // dereference elements 0 to 4 (even though they refer to memory that's been reallocated).
+
+ return 0;
+}
>From f3e942c6197e53ec5f908991baff318223a29cb4 Mon Sep 17 00:00:00 2001
From: Konstantin Varlamov <varconst at apple.com>
Date: Mon, 22 Jul 2024 14:07:55 -0700
Subject: [PATCH 5/7] More comments and docs
---
libcxx/docs/Hardening.rst | 26 ++++++++++++++++++++++++--
libcxx/include/__configuration/abi.h | 13 +++++++++++++
2 files changed, 37 insertions(+), 2 deletions(-)
diff --git a/libcxx/docs/Hardening.rst b/libcxx/docs/Hardening.rst
index 9aac059d27ecf..67791a5e55ac7 100644
--- a/libcxx/docs/Hardening.rst
+++ b/libcxx/docs/Hardening.rst
@@ -325,6 +325,22 @@ Vendors can use the following ABI options to enable additional hardening checks:
- ``span``;
- ``string_view``.
+- ``_LIBCPP_ABI_BOUNDED_ITERATORS_IN_STRING`` -- changes the iterator type of
+ ``basic_string`` to a bounded iterator that keeps track of whether it's within
+ the bounds of the original container and asserts it on every dereference and
+ when performing iterator arithmetics.
+
+ ABI impact: changes the iterator type of ``basic_string`` and its
+ specializations, such as ``string`` and ``wstring``.
+
+- ``_LIBCPP_ABI_BOUNDED_ITERATORS_IN_VECTOR`` -- changes the iterator type of
+ ``vector`` to a bounded iterator that keeps track of whether it's within the
+ bounds of the original container and asserts it on every dereference and when
+ performing iterator arithmetics. Note: this doesn't yet affect
+ ``vector<bool>``.
+
+ ABI impact: changes the iterator type of ``vector`` (except ``vector<bool>``).
+
ABI tags
--------
@@ -367,10 +383,10 @@ Hardened containers status
- ❌
* - ``vector``
- ✅
- - ❌
+ - ✅ (see note)
* - ``string``
- ✅
- - ❌
+ - ✅ (see note)
* - ``list``
- ✅
- ❌
@@ -429,6 +445,12 @@ Hardened containers status
- ❌
- N/A
+Note: for ``vector`` and ``string``, the iterator does not check for
+invalidation (accesses made via an invalidated iterator still lead to undefined
+behavior)
+
+Note: ``vector<bool>`` iterator is not currently hardened.
+
Testing
=======
diff --git a/libcxx/include/__configuration/abi.h b/libcxx/include/__configuration/abi.h
index 710548d90a649..0422b645727d8 100644
--- a/libcxx/include/__configuration/abi.h
+++ b/libcxx/include/__configuration/abi.h
@@ -141,6 +141,19 @@
// - `string_view`.
// #define _LIBCPP_ABI_BOUNDED_ITERATORS
+// Changes the iterator type of `basic_string` to a bounded iterator that keeps track of whether it's within the bounds
+// of the original container and asserts it on every dereference and when performing iterator arithmetics.
+//
+// ABI impact: changes the iterator type of `basic_string` and its specializations, such as `string` and `wstring`.
+// #define _LIBCPP_ABI_BOUNDED_ITERATORS_IN_STRING
+
+// Changes the iterator type of `vector` to a bounded iterator that keeps track of whether it's within the bounds of the
+// original container and asserts it on every dereference and when performing iterator arithmetics. Note: this doesn't
+// yet affect `vector<bool>`.
+//
+// ABI impact: changes the iterator type of `vector` (except `vector<bool>`).
+// #define _LIBCPP_ABI_BOUNDED_ITERATORS_IN_VECTOR
+
#if defined(_LIBCPP_COMPILER_CLANG_BASED)
# if defined(__APPLE__)
# if defined(__i386__) || defined(__x86_64__)
>From f1926a882055939d2c06be812cd7dd1f6bf033af Mon Sep 17 00:00:00 2001
From: Konstantin Varlamov <varconst at apple.com>
Date: Mon, 22 Jul 2024 14:11:35 -0700
Subject: [PATCH 6/7] Rename assertion test files
---
.../{debug.iterator.add.pass.cpp => assert.iterator.add.pass.cpp} | 0
...ator.decrement.pass.cpp => assert.iterator.decrement.pass.cpp} | 0
....dereference.pass.cpp => assert.iterator.dereference.pass.cpp} | 0
...ator.increment.pass.cpp => assert.iterator.increment.pass.cpp} | 0
...bug.iterator.index.pass.cpp => assert.iterator.index.pass.cpp} | 0
.../{debug.iterator.add.pass.cpp => assert.iterator.add.pass.cpp} | 0
...ator.decrement.pass.cpp => assert.iterator.decrement.pass.cpp} | 0
....dereference.pass.cpp => assert.iterator.dereference.pass.cpp} | 0
...ator.increment.pass.cpp => assert.iterator.increment.pass.cpp} | 0
...bug.iterator.index.pass.cpp => assert.iterator.index.pass.cpp} | 0
10 files changed, 0 insertions(+), 0 deletions(-)
rename libcxx/test/libcxx/containers/sequences/vector/{debug.iterator.add.pass.cpp => assert.iterator.add.pass.cpp} (100%)
rename libcxx/test/libcxx/containers/sequences/vector/{debug.iterator.decrement.pass.cpp => assert.iterator.decrement.pass.cpp} (100%)
rename libcxx/test/libcxx/containers/sequences/vector/{debug.iterator.dereference.pass.cpp => assert.iterator.dereference.pass.cpp} (100%)
rename libcxx/test/libcxx/containers/sequences/vector/{debug.iterator.increment.pass.cpp => assert.iterator.increment.pass.cpp} (100%)
rename libcxx/test/libcxx/containers/sequences/vector/{debug.iterator.index.pass.cpp => assert.iterator.index.pass.cpp} (100%)
rename libcxx/test/libcxx/strings/basic.string/string.iterators/{debug.iterator.add.pass.cpp => assert.iterator.add.pass.cpp} (100%)
rename libcxx/test/libcxx/strings/basic.string/string.iterators/{debug.iterator.decrement.pass.cpp => assert.iterator.decrement.pass.cpp} (100%)
rename libcxx/test/libcxx/strings/basic.string/string.iterators/{debug.iterator.dereference.pass.cpp => assert.iterator.dereference.pass.cpp} (100%)
rename libcxx/test/libcxx/strings/basic.string/string.iterators/{debug.iterator.increment.pass.cpp => assert.iterator.increment.pass.cpp} (100%)
rename libcxx/test/libcxx/strings/basic.string/string.iterators/{debug.iterator.index.pass.cpp => assert.iterator.index.pass.cpp} (100%)
diff --git a/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.add.pass.cpp b/libcxx/test/libcxx/containers/sequences/vector/assert.iterator.add.pass.cpp
similarity index 100%
rename from libcxx/test/libcxx/containers/sequences/vector/debug.iterator.add.pass.cpp
rename to libcxx/test/libcxx/containers/sequences/vector/assert.iterator.add.pass.cpp
diff --git a/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.decrement.pass.cpp b/libcxx/test/libcxx/containers/sequences/vector/assert.iterator.decrement.pass.cpp
similarity index 100%
rename from libcxx/test/libcxx/containers/sequences/vector/debug.iterator.decrement.pass.cpp
rename to libcxx/test/libcxx/containers/sequences/vector/assert.iterator.decrement.pass.cpp
diff --git a/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.dereference.pass.cpp b/libcxx/test/libcxx/containers/sequences/vector/assert.iterator.dereference.pass.cpp
similarity index 100%
rename from libcxx/test/libcxx/containers/sequences/vector/debug.iterator.dereference.pass.cpp
rename to libcxx/test/libcxx/containers/sequences/vector/assert.iterator.dereference.pass.cpp
diff --git a/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.increment.pass.cpp b/libcxx/test/libcxx/containers/sequences/vector/assert.iterator.increment.pass.cpp
similarity index 100%
rename from libcxx/test/libcxx/containers/sequences/vector/debug.iterator.increment.pass.cpp
rename to libcxx/test/libcxx/containers/sequences/vector/assert.iterator.increment.pass.cpp
diff --git a/libcxx/test/libcxx/containers/sequences/vector/debug.iterator.index.pass.cpp b/libcxx/test/libcxx/containers/sequences/vector/assert.iterator.index.pass.cpp
similarity index 100%
rename from libcxx/test/libcxx/containers/sequences/vector/debug.iterator.index.pass.cpp
rename to libcxx/test/libcxx/containers/sequences/vector/assert.iterator.index.pass.cpp
diff --git a/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.add.pass.cpp b/libcxx/test/libcxx/strings/basic.string/string.iterators/assert.iterator.add.pass.cpp
similarity index 100%
rename from libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.add.pass.cpp
rename to libcxx/test/libcxx/strings/basic.string/string.iterators/assert.iterator.add.pass.cpp
diff --git a/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.decrement.pass.cpp b/libcxx/test/libcxx/strings/basic.string/string.iterators/assert.iterator.decrement.pass.cpp
similarity index 100%
rename from libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.decrement.pass.cpp
rename to libcxx/test/libcxx/strings/basic.string/string.iterators/assert.iterator.decrement.pass.cpp
diff --git a/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.dereference.pass.cpp b/libcxx/test/libcxx/strings/basic.string/string.iterators/assert.iterator.dereference.pass.cpp
similarity index 100%
rename from libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.dereference.pass.cpp
rename to libcxx/test/libcxx/strings/basic.string/string.iterators/assert.iterator.dereference.pass.cpp
diff --git a/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.increment.pass.cpp b/libcxx/test/libcxx/strings/basic.string/string.iterators/assert.iterator.increment.pass.cpp
similarity index 100%
rename from libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.increment.pass.cpp
rename to libcxx/test/libcxx/strings/basic.string/string.iterators/assert.iterator.increment.pass.cpp
diff --git a/libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.index.pass.cpp b/libcxx/test/libcxx/strings/basic.string/string.iterators/assert.iterator.index.pass.cpp
similarity index 100%
rename from libcxx/test/libcxx/strings/basic.string/string.iterators/debug.iterator.index.pass.cpp
rename to libcxx/test/libcxx/strings/basic.string/string.iterators/assert.iterator.index.pass.cpp
>From 7e7776e6a86da0a712f64c06c26f0d8199433293 Mon Sep 17 00:00:00 2001
From: Konstantin Varlamov <varconst at apple.com>
Date: Mon, 22 Jul 2024 14:15:10 -0700
Subject: [PATCH 7/7] Tweak the release note to make it clear the ABI macros
are intended only for vendors.
---
libcxx/docs/ReleaseNotes/19.rst | 22 +++++++++++-----------
1 file changed, 11 insertions(+), 11 deletions(-)
diff --git a/libcxx/docs/ReleaseNotes/19.rst b/libcxx/docs/ReleaseNotes/19.rst
index 2cba372b046ab..439f552db59a8 100644
--- a/libcxx/docs/ReleaseNotes/19.rst
+++ b/libcxx/docs/ReleaseNotes/19.rst
@@ -87,17 +87,17 @@ Improvements and New Features
- ``std::ignore``\s ``const __ignore_t& operator=(_Tp&&) const`` was changed to
``const __ignore_type& operator=(const _Tp&) const noexcept`` for all language versions.
-- ``string`` and ``vector`` can now be configured to use bounded iterators when hardening is enabled. Note that checks
- for iterator invalidation are currently not supported -- any accesses made through an invalidated bounded iterator
- will still result in undefined behavior (bounded iterator follow the normal invalidation rules of the associated
- container). ``string`` bounded iterators use the logical size of the container (``index < str.size()``) whereas
- ``vector`` bounded iterators use the "physical" size of the container (``index < vec.capacity()``) which is a less
- strict check; refer to the implementation for further details.
-
- Bounded iterators can be enabled via the ``_LIBCPP_ABI_BOUNDED_ITERATORS_IN_STRING`` for ``string`` and via the
- ``_LIBCPP_ABI_BOUNDED_ITERATORS_IN_VECTOR`` for ``vector``; note that checks will only be performed if the hardening
- mode is set to ``fast`` or above (i.e., no checking is performed in the unchecked mode, even if bounded iterators are
- enabled in the ABI configuration).
+- Vendors can now configure the ABI so that ``string`` and ``vector`` will use bounded iterators when hardening is
+ enabled. Note that checks for iterator invalidation are currently not supported -- any accesses made through an
+ invalidated bounded iterator will still result in undefined behavior (bounded iterators follow the normal invalidation
+ rules of the associated container). ``string`` bounded iterators use the logical size of the container (``index
+ < str.size()``) whereas ``vector`` bounded iterators use the "physical" size of the container (``index
+ < vec.capacity()``) which is a less strict check; refer to the implementation for further details.
+
+ Bounded iterators can be enabled via the ``_LIBCPP_ABI_BOUNDED_ITERATORS_IN_STRING`` ABI macro for ``string`` and via
+ the ``_LIBCPP_ABI_BOUNDED_ITERATORS_IN_VECTOR`` ABI macro for ``vector``; note that checks will only be performed if
+ the hardening mode is set to ``fast`` or above (i.e., no checking is performed in the unchecked mode, even if bounded
+ iterators are enabled in the ABI configuration).
Note: bounded iterators currently are not supported for ``vector<bool>``.
More information about the libcxx-commits
mailing list