[libcxx-commits] [libcxx] [ASan][libc++] std::basic_string annotations (PR #72677)

via libcxx-commits libcxx-commits at lists.llvm.org
Fri Nov 17 08:59:08 PST 2023


llvmbot wrote:


<!--LLVM PR SUMMARY COMMENT-->

@llvm/pr-subscribers-libcxx

Author: Tacet (AdvenamTacet)

<details>
<summary>Changes</summary>

This commit turns on basic annotations for std::basic_string, similar to those in std::vector and std::deque.

**Notice:** those annotations work if and only if libc++ and libc++abi are instrumented (with ASan) as well!

Originally proposed here: https://reviews.llvm.org/D132769

Related patches:
Turning on annotations for short strings: https://reviews.llvm.org/D147680

Turning on annotations for all allocators: https://reviews.llvm.org/D146214

This revision is a part of a series of patches extending AddressSanitizer C++ container overflow detection capabilities by adding annotations, similar to those existing in std::vector, to std::string and std::deque collections. These changes allow ASan to detect cases when the instrumented program accesses memory which is internally allocated by the collection but is still not in-use (accesses before or after the stored elements for std::deque, or between the size and capacity bounds for std::string).

The motivation for the research and those changes was a bug, found by Trail of Bits, in a real code where an out-of-bounds read could happen as two strings were compared via a std::equals function that took iter1_begin, iter1_end, iter2_begin iterators (with a custom comparison function). When object iter1 was longer than iter2, read out-of-bounds on iter2 could happen. Container sanitization would detect it.

This Pull Request adds annotations for `std::basic_string`. Long string is very similar to std::vector, and therefore works well with `__sanitizer_annotate_contiguous_container` from LLVM 15 and therefore annotations works if that implementation is compiled with previous LLVM. However, only the standard allocator is supported.

As D132522 extended possibilities of `__sanitizer_annotate_contiguous_container`, now annotations may be supported with all allocators, but that support will be added in a next PR. Only strings with default allocator are annotated with that commit.

If you have any questions, please email:
- advenam.tacet@<!-- -->trailofbits.com
- disconnect3d@<!-- -->trailofbits.com

---

Patch is 129.85 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/72677.diff


83 Files Affected:

- (modified) libcxx/include/string (+225-57) 
- (modified) libcxx/test/std/strings/basic.string/string.capacity/capacity.pass.cpp (+1) 
- (modified) libcxx/test/std/strings/basic.string/string.capacity/clear.pass.cpp (+7) 
- (modified) libcxx/test/std/strings/basic.string/string.capacity/reserve.pass.cpp (+3) 
- (added) libcxx/test/std/strings/basic.string/string.capacity/reserve_size.asan.pass.cpp (+81) 
- (modified) libcxx/test/std/strings/basic.string/string.capacity/reserve_size.pass.cpp (+1) 
- (modified) libcxx/test/std/strings/basic.string/string.capacity/resize_and_overwrite.pass.cpp (+4) 
- (modified) libcxx/test/std/strings/basic.string/string.capacity/resize_size.pass.cpp (+10-5) 
- (modified) libcxx/test/std/strings/basic.string/string.capacity/resize_size_char.pass.cpp (+19-5) 
- (modified) libcxx/test/std/strings/basic.string/string.capacity/shrink_to_fit.pass.cpp (+8) 
- (modified) libcxx/test/std/strings/basic.string/string.cons/T_size_size.pass.cpp (+17-13) 
- (modified) libcxx/test/std/strings/basic.string/string.cons/alloc.pass.cpp (+17-13) 
- (modified) libcxx/test/std/strings/basic.string/string.cons/brace_assignment.pass.cpp (+3) 
- (modified) libcxx/test/std/strings/basic.string/string.cons/char_assignment.pass.cpp (+2) 
- (modified) libcxx/test/std/strings/basic.string/string.cons/copy.pass.cpp (+4) 
- (modified) libcxx/test/std/strings/basic.string/string.cons/copy_alloc.pass.cpp (+3) 
- (modified) libcxx/test/std/strings/basic.string/string.cons/copy_assignment.pass.cpp (+3) 
- (modified) libcxx/test/std/strings/basic.string/string.cons/default.pass.cpp (+2) 
- (modified) libcxx/test/std/strings/basic.string/string.cons/initializer_list.pass.cpp (+9) 
- (modified) libcxx/test/std/strings/basic.string/string.cons/initializer_list_assignment.pass.cpp (+10-1) 
- (modified) libcxx/test/std/strings/basic.string/string.cons/iter_alloc.pass.cpp (+3) 
- (modified) libcxx/test/std/strings/basic.string/string.cons/iter_alloc_deduction.pass.cpp (+1) 
- (modified) libcxx/test/std/strings/basic.string/string.cons/move.pass.cpp (+4) 
- (modified) libcxx/test/std/strings/basic.string/string.cons/move_alloc.pass.cpp (+4) 
- (modified) libcxx/test/std/strings/basic.string/string.cons/move_assignment.pass.cpp (+5) 
- (modified) libcxx/test/std/strings/basic.string/string.cons/pointer_alloc.pass.cpp (+3) 
- (modified) libcxx/test/std/strings/basic.string/string.cons/pointer_assignment.pass.cpp (+3) 
- (modified) libcxx/test/std/strings/basic.string/string.cons/pointer_size_alloc.pass.cpp (+3) 
- (modified) libcxx/test/std/strings/basic.string/string.cons/size_char_alloc.pass.cpp (+3) 
- (modified) libcxx/test/std/strings/basic.string/string.cons/string_view.pass.cpp (+5) 
- (modified) libcxx/test/std/strings/basic.string/string.cons/string_view_assignment.pass.cpp (+3) 
- (modified) libcxx/test/std/strings/basic.string/string.cons/substr.pass.cpp (+35-27) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_append/initializer_list.pass.cpp (+3-1) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_append/iterator.pass.cpp (+2) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_append/pointer.pass.cpp (+2) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_append/pointer_size.pass.cpp (+2) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_append/push_back.pass.cpp (+2) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_append/size_char.pass.cpp (+2) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_append/string.pass.cpp (+2) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_append/string_size_size.pass.cpp (+9-5) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_assign/T_size_size.pass.cpp (+9-5) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_assign/initializer_list.pass.cpp (+3) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_assign/iterator.pass.cpp (+2) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_assign/pointer.pass.cpp (+2) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_assign/pointer_size.pass.cpp (+2) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_assign/size_char.pass.cpp (+2) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_assign/string.pass.cpp (+3) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_assign/string_size_size.pass.cpp (+9-5) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_copy/copy.pass.cpp (+7-3) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_erase/iter.pass.cpp (+2) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_erase/iter_iter.pass.cpp (+2) 
- (added) libcxx/test/std/strings/basic.string/string.modifiers/string_erase/pop_back.asan.pass.cpp (+65) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_erase/pop_back.pass.cpp (+3) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_erase/size_size.pass.cpp (+12-8) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_insert/iter_char.pass.cpp (+13-9) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_insert/iter_initializer_list.pass.cpp (+2) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_insert/iter_iter_iter.pass.cpp (+2) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_insert/iter_size_char.pass.cpp (+2) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_insert/size_pointer.pass.cpp (+11-7) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_insert/size_pointer_size.pass.cpp (+11-7) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_insert/size_size_char.pass.cpp (+11-7) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_insert/size_string.pass.cpp (+11-7) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_insert/size_string_size_size.pass.cpp (+11-7) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_op_plus_equal/char.pass.cpp (+2) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_op_plus_equal/initializer_list.pass.cpp (+2) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_op_plus_equal/pointer.pass.cpp (+2) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_op_plus_equal/string.pass.cpp (+2) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_replace/iter_iter_string.pass.cpp (+3) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_replace/size_size_T_size_size.pass.cpp (+10-7) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_replace/size_size_pointer.pass.cpp (+15-10) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_replace/size_size_pointer_size.pass.cpp (+15-10) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_replace/size_size_size_char.pass.cpp (+14-10) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_replace/size_size_string.pass.cpp (+14-10) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_replace/size_size_string_size_size.pass.cpp (+14-10) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_replace/size_size_string_view.pass.cpp (+14-10) 
- (added) libcxx/test/std/strings/basic.string/string.modifiers/string_swap/swap.asan.pass.cpp (+102) 
- (modified) libcxx/test/std/strings/basic.string/string.modifiers/string_swap/swap.pass.cpp (+3) 
- (modified) libcxx/test/std/strings/basic.string/string.nonmembers/string.special/swap.pass.cpp (+3) 
- (modified) libcxx/test/std/strings/basic.string/string.nonmembers/string_op+/char_string.pass.cpp (+2) 
- (modified) libcxx/test/std/strings/basic.string/string.nonmembers/string_op+/string_char.pass.cpp (+2) 
- (modified) libcxx/test/std/strings/basic.string/string.nonmembers/string_op+/string_pointer.pass.cpp (+2) 
- (modified) libcxx/test/std/strings/basic.string/string.nonmembers/string_op+/string_string.pass.cpp (+2) 
- (modified) libcxx/test/support/asan_testing.h (+50-12) 


``````````diff
diff --git a/libcxx/include/string b/libcxx/include/string
index 9c2efac8006bd72..9362b1bcd16fa19 100644
--- a/libcxx/include/string
+++ b/libcxx/include/string
@@ -645,6 +645,16 @@ basic_string<char32_t> operator""s( const char32_t *str, size_t len );
 _LIBCPP_PUSH_MACROS
 #include <__undef_macros>
 
+#ifndef _LIBCPP_HAS_NO_ASAN
+#  define _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS __attribute__((__no_sanitize__("address")))
+// Sometimes string access own memory, which should not be accessed by different parts of program
+// (eg. not-in-use bytes of buffer in short string) and are poisoned by ASan.
+// This macro turns off instrumentation in a function, so memory accesses which normally would crash
+// work as expected.
+#else
+#  define _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS
+#endif
+#define _LIBCPP_SHORT_STRING_ANNOTATIONS_ALLOWED false
 
 _LIBCPP_BEGIN_NAMESPACE_STD
 
@@ -702,6 +712,9 @@ struct __init_with_sentinel_tag {};
 template<class _CharT, class _Traits, class _Allocator>
 class basic_string
 {
+private:
+  using __default_allocator_type = allocator<_CharT>;
+
 public:
     typedef basic_string                                 __self;
     typedef basic_string_view<_CharT, _Traits>           __self_view;
@@ -888,34 +901,46 @@ public:
 #endif
       : __r_(__value_init_tag(), __a) {}
 
-  _LIBCPP_CONSTEXPR_SINCE_CXX20 basic_string(const basic_string& __str)
+  _LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS basic_string(const basic_string& __str)
       : __r_(__default_init_tag(), __alloc_traits::select_on_container_copy_construction(__str.__alloc())) {
     if (!__str.__is_long())
+    {
       __r_.first() = __str.__r_.first();
+      __annotate_new(__get_short_size());
+    }
     else
       __init_copy_ctor_external(std::__to_address(__str.__get_long_pointer()), __str.__get_long_size());
   }
 
-  _LIBCPP_CONSTEXPR_SINCE_CXX20 basic_string(const basic_string& __str, const allocator_type& __a)
+  _LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS basic_string(const basic_string& __str, const allocator_type& __a)
       : __r_(__default_init_tag(), __a) {
     if (!__str.__is_long())
+    {
       __r_.first() = __str.__r_.first();
+      __annotate_new(__get_short_size());
+    }
     else
       __init_copy_ctor_external(std::__to_address(__str.__get_long_pointer()), __str.__get_long_size());
   }
 
 #ifndef _LIBCPP_CXX03_LANG
-  _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 basic_string(basic_string&& __str)
+  _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20
+  basic_string(basic_string&& __str)
 #  if _LIBCPP_STD_VER <= 14
       _NOEXCEPT_(is_nothrow_move_constructible<allocator_type>::value)
 #  else
       _NOEXCEPT
 #  endif
-      : __r_(std::move(__str.__r_)) {
+      // Turning off ASan instrumentation for variable initialization with _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS
+      // is inconsistent and that initialization may be annotated.
+      // Therefore, to copy __str memory, we have to unpoison it first (if object is poisoned and not external buffer,
+      // so short string case).
+      : __r_( ( (__str.__is_long() ? 0 : (__str.__annotate_delete(), 0)), std::move(__str.__r_)) ) {
     __str.__r_.first() = __rep();
   }
 
-  _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 basic_string(basic_string&& __str, const allocator_type& __a)
+  _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS
+  basic_string(basic_string&& __str, const allocator_type& __a)
       : __r_(__default_init_tag(), __a) {
     if (__str.__is_long() && __a != __str.__alloc()) // copy, not move
       __init(std::__to_address(__str.__get_long_pointer()), __str.__get_long_size());
@@ -923,6 +948,9 @@ public:
       if (__libcpp_is_constant_evaluated())
         __r_.first() = __rep();
       __r_.first() = __str.__r_.first();
+      if (!__str.__is_long()) {
+        __str.__annotate_delete();
+      }
       __str.__r_.first() = __rep();
     }
   }
@@ -965,11 +993,11 @@ public:
   }
 
 #if _LIBCPP_STD_VER >= 23
-  _LIBCPP_HIDE_FROM_ABI constexpr
+  _LIBCPP_HIDE_FROM_ABI _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS constexpr
   basic_string(basic_string&& __str, size_type __pos, const _Allocator& __alloc = _Allocator())
       : basic_string(std::move(__str), __pos, npos, __alloc) {}
 
-  _LIBCPP_HIDE_FROM_ABI constexpr
+  _LIBCPP_HIDE_FROM_ABI _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS constexpr
   basic_string(basic_string&& __str, size_type __pos, size_type __n, const _Allocator& __alloc = _Allocator())
       : __r_(__default_init_tag(), __alloc) {
     if (__pos > __str.size())
@@ -1081,6 +1109,7 @@ public:
 #endif // _LIBCPP_CXX03_LANG
 
   inline _LIBCPP_CONSTEXPR_SINCE_CXX20 ~basic_string() {
+    __annotate_delete();
     if (__is_long())
       __alloc_traits::deallocate(__alloc(), __get_long_pointer(), __get_long_cap());
   }
@@ -1098,8 +1127,8 @@ public:
     }
 
 #ifndef _LIBCPP_CXX03_LANG
-  _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 basic_string& operator=(basic_string&& __str)
-      _NOEXCEPT_((__noexcept_move_assign_container<_Allocator, __alloc_traits>::value)) {
+    _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS basic_string&
+    operator=(basic_string&& __str) _NOEXCEPT_((__noexcept_move_assign_container<_Allocator, __alloc_traits>::value)) {
     __move_assign(__str, integral_constant<bool, __alloc_traits::propagate_on_container_move_assignment::value>());
     return *this;
   }
@@ -1112,7 +1141,7 @@ public:
 #if _LIBCPP_STD_VER >= 23
     basic_string& operator=(nullptr_t) = delete;
 #endif
-    _LIBCPP_CONSTEXPR_SINCE_CXX20 basic_string& operator=(value_type __c);
+    _LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS basic_string& operator=(value_type __c);
 
     _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20
     iterator begin() _NOEXCEPT
@@ -1329,7 +1358,7 @@ public:
   }
 
 #if _LIBCPP_STD_VER >= 20
-  _LIBCPP_HIDE_FROM_ABI constexpr
+  _LIBCPP_HIDE_FROM_ABI constexpr _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS
   void __move_assign(basic_string&& __str, size_type __pos, size_type __len) {
     // Pilfer the allocation from __str.
     _LIBCPP_ASSERT_INTERNAL(__alloc() == __str.__alloc(), "__move_assign called with wrong allocator");
@@ -1345,7 +1374,7 @@ public:
     _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20
     basic_string& assign(const basic_string& __str) { return *this = __str; }
 #ifndef _LIBCPP_CXX03_LANG
-    _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20
+    _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS
     basic_string& assign(basic_string&& __str)
         _NOEXCEPT_((__noexcept_move_assign_container<_Allocator, __alloc_traits>::value))
         {*this = std::move(__str); return *this;}
@@ -1736,7 +1765,7 @@ private:
 
     _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 void __shrink_or_extend(size_type __target_capacity);
 
-    _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20
+    _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS
     bool __is_long() const _NOEXCEPT {
         if (__libcpp_is_constant_evaluated() && __builtin_constant_p(__r_.first().__l.__is_long_)) {
             return __r_.first().__l.__is_long_;
@@ -1776,6 +1805,7 @@ private:
         value_type* __p;
         if (__cap - __sz >= __n)
         {
+          __annotate_increase(__n);
             __p = std::__to_address(__get_pointer());
             size_type __n_move = __sz - __ip;
             if (__n_move != 0)
@@ -1802,7 +1832,7 @@ private:
     _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX14 allocator_type& __alloc() _NOEXCEPT { return __r_.second(); }
     _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR const allocator_type& __alloc() const _NOEXCEPT { return __r_.second(); }
 
-    _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20
+    _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS
     void __set_short_size(size_type __s) _NOEXCEPT {
         _LIBCPP_ASSERT_INTERNAL(
             __s < __min_cap, "__s should never be greater than or equal to the short string capacity");
@@ -1810,7 +1840,7 @@ private:
         __r_.first().__s.__is_long_ = false;
     }
 
-    _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20
+    _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS
     size_type __get_short_size() const _NOEXCEPT {
         _LIBCPP_ASSERT_INTERNAL(
             !__r_.first().__s.__is_long_, "String has to be short when trying to get the short size");
@@ -1860,6 +1890,45 @@ private:
     const_pointer __get_pointer() const _NOEXCEPT
         {return __is_long() ? __get_long_pointer() : __get_short_pointer();}
 
+    // The following functions are no-ops outside of AddressSanitizer mode.
+#ifndef _LIBCPP_HAS_NO_ASAN
+    _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 void __annotate_contiguous_container(
+        const void* __old_mid, const void* __new_mid) const {
+        const void* __begin = data();
+        const void* __end = data() + capacity() + 1;
+        if (!__libcpp_is_constant_evaluated() && __begin != nullptr && is_same<allocator_type, __default_allocator_type>::value)
+          __sanitizer_annotate_contiguous_container(__begin, __end, __old_mid, __new_mid);
+    }
+#else
+    _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 void
+    __annotate_contiguous_container(const void*, const void*) const {}
+#endif
+
+    // ASan: short string is poisoned if and only if this function returns true.
+    _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 bool __asan_short_string_is_annotated() const _NOEXCEPT {
+      return _LIBCPP_SHORT_STRING_ANNOTATIONS_ALLOWED && !__libcpp_is_constant_evaluated();
+    }
+
+    _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 void __annotate_new(size_type __current_size) const _NOEXCEPT {
+      if (!__libcpp_is_constant_evaluated() && (__asan_short_string_is_annotated() || __is_long()))
+        __annotate_contiguous_container(data() + capacity() + 1, data() + __current_size + 1);
+    }
+
+    _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 void __annotate_delete() const _NOEXCEPT {
+      if (!__libcpp_is_constant_evaluated() && (__asan_short_string_is_annotated() || __is_long()))
+        __annotate_contiguous_container(data() + size() + 1, data() + capacity() + 1);
+    }
+
+    _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 void __annotate_increase(size_type __n) const _NOEXCEPT {
+      if (!__libcpp_is_constant_evaluated() && (__asan_short_string_is_annotated() || __is_long()))
+        __annotate_contiguous_container(data() + size() + 1, data() + size() + 1 + __n);
+    }
+
+    _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 void __annotate_shrink(size_type __old_size) const _NOEXCEPT {
+      if (!__libcpp_is_constant_evaluated() && (__asan_short_string_is_annotated() || __is_long()))
+        __annotate_contiguous_container(data() + __old_size + 1, data() + size() + 1);
+    }
+
     template <size_type __a> static
         _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20
         size_type __align_it(size_type __s) _NOEXCEPT
@@ -1962,6 +2031,7 @@ private:
                 }
                 else
                 {
+                    __annotate_delete();
                     allocator_type __a = __str.__alloc();
                     auto __allocation = std::__allocate_at_least(__a, __str.__get_long_cap());
                     __begin_lifetime(__allocation.ptr, __allocation.count);
@@ -1971,6 +2041,7 @@ private:
                     __set_long_pointer(__allocation.ptr);
                     __set_long_cap(__allocation.count);
                     __set_long_size(__str.size());
+                    __annotate_new(__get_long_size());
                 }
             }
         }
@@ -2018,18 +2089,28 @@ private:
 
     // Assigns the value in __s, guaranteed to be __n < __min_cap in length.
     inline _LIBCPP_CONSTEXPR_SINCE_CXX20 basic_string& __assign_short(const value_type* __s, size_type __n) {
+      size_type __old_size = size();
+      if (__n > __old_size)
+        __annotate_increase(__n - __old_size);
       pointer __p = __is_long()
                         ? (__set_long_size(__n), __get_long_pointer())
                         : (__set_short_size(__n), __get_short_pointer());
       traits_type::move(std::__to_address(__p), __s, __n);
       traits_type::assign(__p[__n], value_type());
+      if (__old_size > __n)
+        __annotate_shrink(__old_size);
       return *this;
     }
 
     _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20
     basic_string& __null_terminate_at(value_type* __p, size_type __newsz) {
+      size_type __old_size = size();
+      if (__newsz > __old_size)
+        __annotate_increase(__newsz - __old_size);
       __set_size(__newsz);
       traits_type::assign(__p[__newsz], value_type());
+      if (__old_size > __newsz)
+        __annotate_shrink(__old_size);
       return *this;
     }
 
@@ -2136,6 +2217,7 @@ void basic_string<_CharT, _Traits, _Allocator>::__init(const value_type* __s,
     }
     traits_type::copy(std::__to_address(__p), __s, __sz);
     traits_type::assign(__p[__sz], value_type());
+    __annotate_new(__sz);
 }
 
 template <class _CharT, class _Traits, class _Allocator>
@@ -2164,6 +2246,7 @@ basic_string<_CharT, _Traits, _Allocator>::__init(const value_type* __s, size_ty
     }
     traits_type::copy(std::__to_address(__p), __s, __sz);
     traits_type::assign(__p[__sz], value_type());
+    __annotate_new(__sz);
 }
 
 template <class _CharT, class _Traits, class _Allocator>
@@ -2188,6 +2271,7 @@ void basic_string<_CharT, _Traits, _Allocator>::__init_copy_ctor_external(
     __set_long_size(__sz);
   }
   traits_type::copy(std::__to_address(__p), __s, __sz + 1);
+  __annotate_new(__sz);
 }
 
 template <class _CharT, class _Traits, class _Allocator>
@@ -2217,6 +2301,7 @@ basic_string<_CharT, _Traits, _Allocator>::__init(size_type __n, value_type __c)
     }
     traits_type::assign(std::__to_address(__p), __n, __c);
     traits_type::assign(__p[__n], value_type());
+    __annotate_new(__n);
 }
 
 template <class _CharT, class _Traits, class _Allocator>
@@ -2243,6 +2328,7 @@ void basic_string<_CharT, _Traits, _Allocator>::__init_with_sentinel(_InputItera
     }
     catch (...)
     {
+        __annotate_delete();
         if (__is_long())
             __alloc_traits::deallocate(__alloc(), __get_long_pointer(), __get_long_cap());
         throw;
@@ -2298,11 +2384,13 @@ void basic_string<_CharT, _Traits, _Allocator>::__init_with_size(
     }
     catch (...)
     {
+        __annotate_delete();
         if (__is_long())
             __alloc_traits::deallocate(__alloc(), __get_long_pointer(), __get_long_cap());
         throw;
     }
 #endif  // _LIBCPP_HAS_NO_EXCEPTIONS
+    __annotate_new(__sz);
 }
 
 template <class _CharT, class _Traits, class _Allocator>
@@ -2319,6 +2407,7 @@ basic_string<_CharT, _Traits, _Allocator>::__grow_by_and_replace
     size_type __cap = __old_cap < __ms / 2 - __alignment ?
                           __recommend(std::max(__old_cap + __delta_cap, 2 * __old_cap)) :
                           __ms - 1;
+    __annotate_delete();
     auto __allocation = std::__allocate_at_least(__alloc(), __cap + 1);
     pointer __p = __allocation.ptr;
     __begin_lifetime(__p, __allocation.count);
@@ -2338,6 +2427,7 @@ basic_string<_CharT, _Traits, _Allocator>::__grow_by_and_replace
     __old_sz = __n_copy + __n_add + __sec_cp_sz;
     __set_long_size(__old_sz);
     traits_type::assign(__p[__old_sz], value_type());
+    __annotate_new(__old_cap + __delta_cap);
 }
 
 // __grow_by is deprecated because it does not set the size. It may not update the size when the size is changed, and it
@@ -2360,6 +2450,7 @@ basic_string<_CharT, _Traits, _Allocator>::__grow_by(size_type __old_cap, size_t
     size_type __cap = __old_cap < __ms / 2 - __alignment ?
                           __recommend(std::max(__old_cap + __delta_cap, 2 * __old_cap)) :
                           __ms - 1;
+    __annotate_delete();
     auto __allocation = std::__allocate_at_least(__alloc(), __cap + 1);
     pointer __p = __allocation.ptr;
     __begin_lifetime(__p, __allocation.count);
@@ -2402,10 +2493,15 @@ basic_string<_CharT, _Traits, _Allocator>::__assign_no_alias(
     const value_type* __s, size_type __n) {
   size_type __cap = __is_short ? static_cast<size_type>(__min_cap) : __get_long_cap();
   if (__n < __cap) {
+    size_type __old_size = __is_short ? __get_short_size() : __get_long_size();
+    if (__n > __old_size)
+      __annotate_increase(__n - __old_size);
     pointer __p = __is_short ? __get_short_pointer() : __get_long_pointer();
     __is_short ? __set_short_size(__n) : __set_long_size(__n);
     traits_type::copy(std::__to_address(__p), __s, __n);
     traits_type::assign(__p[__n], value_type());
+    if (__old_size > __n)
+      __annotate_shrink(__old_size);
   } else {
     size_type __sz = __is_short ? __get_short_size() : __get_long_size();
     __grow_by_and_replace(__cap - 1, __n - __cap + 1, __sz, 0, __sz, __n, __s);
@@ -2420,6 +2516,9 @@ basic_string<_CharT, _Traits, _Allocator>::__assign_external(
     const value_type* __s, size_type __n) {
   size_type __cap = capacity();
   if (__cap >= __n) {
+    size_type __old_size = size();
+    if (__n > __old_size)
+      __annotate_increase(__n - __old_size);
     value_type* __p = std::__to_address(__get_pointer());
     traits_type::move(__p, __s, __n);
     return __null_terminate_at(__p, __n);
@@ -2447,11 +2546,15 @@ basic_string<_CharT, _Traits, _Allocator>&
 basic_string<_CharT, _Traits, _Allocator>::assign(size_type __n, value_type __c)
 {
     size_type __cap = capacity();
+    size_type __old_size = size();
     if (__cap < __n)
     {
         size_type __sz = size();
         __grow_by_without_replace(__cap, __n - __cap, __sz, 0, __sz);
+        __annotate_increase(__n);
     }
+    else if(__n > __old_size)
+        __annotate_increase(__n - __old_size);
     value_type* __p = std::__to_address(__get_pointer());
     traits_type::assign(__p, __n, __c);
     return __null_terminate_at(__p, __n);
@@ -2462,24 +2565,26 @@ _LIBCPP_CONSTEXPR_SINCE_CXX20
 basic_string<_CharT, _Traits, _Allocator>&
 basic_string<_CharT, _Traits, _Allocator>::operator=(value_type __c)
 {
-    pointer __p;
-    if (__is_long())
-    {
-        __p = __get_long_pointer();
-        __set_long_size(1);
-    }
-    else
-    {
-        __p = __get_short_pointer();
-        __set_short_size(1);
-    }
-    traits_type::assign(*__p, __c);
-    traits_type::assign(*++__p, value_type());
-    return *this;
+  pointer __p;
+  size_type __old_size = size();
+  if (__old_size == 0)
+    __annotate_increase(1);
+  if (__is_long()) {
+    __p = __get_long_pointer();
+    __set_long_size(1);
+  } else {
+    __p = __get_short_pointer();
+    __set_short_size(1);
+  }
+  traits_type::assign(*__p, __c);
+  traits_type::assign(*++__p, value_type());
+  if (__old_size > 1)
+    __annotate_shrink(__old_size);
+  return *this;
 }
 
 template <class _CharT, class _Traits, class _Allocator>
-_LIBCPP_CONSTEXPR_SINCE_CXX20
+_LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS
 basic_string<_CharT, _Traits, _Allocator>&
 basic_string<_CharT, _Traits, _Allocator>::operator=(const basic_string& __str)
 {
@@ -2487,7 +2592,12 @@ basic_string<_CharT, _Traits, _Allocator>::operator=(const basic_string& __str)
     __copy_assign_alloc(__str);
     if (!__is_long()) {
       if (!__str.__is_long()) {
+        size_type __old_size = __get_short_size();
+        if (__get_short_size() < __str.__get_short_size())
+          __annotate_increase(__str.__get_short_size() - __get_short_size());
         __r_.first() = __str.__r_.first();
+        if (__old_size > __get_short_size())
+          __annotate_shrink(__old_size);
       } else {
         return __assign_no_alias<true>(__str.data(), __str.size());
       }
@@ -2513,7 +2623,7 @@ basic_string<_CharT, _Traits, _Allocator>::__move_assign(basic_string& __str, fa
 }
 
 template <class _CharT, class _Traits, class _Allocator>
-inline _LIBCPP_CONSTEXPR_SINCE_CXX20
+inline _LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS
 void
 basic_string<_CharT, _Traits, _Allocator>::__move_assign(basic_string& __str, true_t...
[truncated]

``````````

</details>


https://github.com/llvm/llvm-project/pull/72677


More information about the libcxx-commits mailing list