[all-commits] [llvm/llvm-project] 9ed205: [ASan][libc++] std::basic_string annotations (#72677)

[Tacet via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 9ed20568e7de53dce85f1631d7d8c1415e7930ae
      https://github.com/llvm/llvm-project/commit/9ed20568e7de53dce85f1631d7d8c1415e7930ae
  Author: Tacet <advenam.tacet at trailofbits.com>
  Date:   2023-12-13 (Wed, 13 Dec 2023)

  Changed paths:
    M libcxx/CMakeLists.txt
    M libcxx/include/__config_site.in
    M libcxx/include/string
    M libcxx/test/std/strings/basic.string/string.capacity/capacity.pass.cpp
    M libcxx/test/std/strings/basic.string/string.capacity/clear.pass.cpp
    M libcxx/test/std/strings/basic.string/string.capacity/reserve.pass.cpp
    A libcxx/test/std/strings/basic.string/string.capacity/reserve_size.asan.pass.cpp
    M libcxx/test/std/strings/basic.string/string.capacity/reserve_size.pass.cpp
    M libcxx/test/std/strings/basic.string/string.capacity/resize_and_overwrite.pass.cpp
    M libcxx/test/std/strings/basic.string/string.capacity/resize_size.pass.cpp
    M libcxx/test/std/strings/basic.string/string.capacity/resize_size_char.pass.cpp
    M libcxx/test/std/strings/basic.string/string.capacity/shrink_to_fit.pass.cpp
    M libcxx/test/std/strings/basic.string/string.cons/T_size_size.pass.cpp
    M libcxx/test/std/strings/basic.string/string.cons/alloc.pass.cpp
    M libcxx/test/std/strings/basic.string/string.cons/brace_assignment.pass.cpp
    M libcxx/test/std/strings/basic.string/string.cons/char_assignment.pass.cpp
    M libcxx/test/std/strings/basic.string/string.cons/copy.pass.cpp
    M libcxx/test/std/strings/basic.string/string.cons/copy_alloc.pass.cpp
    M libcxx/test/std/strings/basic.string/string.cons/copy_assignment.pass.cpp
    M libcxx/test/std/strings/basic.string/string.cons/default.pass.cpp
    M libcxx/test/std/strings/basic.string/string.cons/from_range.pass.cpp
    M libcxx/test/std/strings/basic.string/string.cons/from_range_deduction.pass.cpp
    M libcxx/test/std/strings/basic.string/string.cons/initializer_list.pass.cpp
    M libcxx/test/std/strings/basic.string/string.cons/initializer_list_assignment.pass.cpp
    M libcxx/test/std/strings/basic.string/string.cons/iter_alloc.pass.cpp
    M libcxx/test/std/strings/basic.string/string.cons/iter_alloc_deduction.pass.cpp
    M libcxx/test/std/strings/basic.string/string.cons/move.pass.cpp
    M libcxx/test/std/strings/basic.string/string.cons/move_alloc.pass.cpp
    M libcxx/test/std/strings/basic.string/string.cons/move_assignment.pass.cpp
    M libcxx/test/std/strings/basic.string/string.cons/pointer_alloc.pass.cpp
    M libcxx/test/std/strings/basic.string/string.cons/pointer_assignment.pass.cpp
    M libcxx/test/std/strings/basic.string/string.cons/pointer_size_alloc.pass.cpp
    M libcxx/test/std/strings/basic.string/string.cons/size_char_alloc.pass.cpp
    M libcxx/test/std/strings/basic.string/string.cons/string_view.pass.cpp
    M libcxx/test/std/strings/basic.string/string.cons/string_view_assignment.pass.cpp
    M libcxx/test/std/strings/basic.string/string.cons/substr.pass.cpp
    M libcxx/test/std/strings/basic.string/string.cons/substr_rvalue.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_append/append_range.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_append/initializer_list.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_append/iterator.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_append/pointer.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_append/pointer_size.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_append/push_back.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_append/size_char.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_append/string.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_append/string_size_size.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_assign/T_size_size.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_assign/assign_range.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_assign/initializer_list.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_assign/iterator.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_assign/pointer.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_assign/pointer_size.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_assign/size_char.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_assign/string.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_assign/string_size_size.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_copy/copy.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_erase/iter.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_erase/iter_iter.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_erase/pop_back.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_erase/size_size.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_insert/iter_char.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_insert/iter_initializer_list.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_insert/iter_iter_iter.infinite_recursion.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_insert/iter_iter_iter.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_insert/iter_size_char.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_insert/size_pointer.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_insert/size_pointer_size.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_insert/size_size_char.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_insert/size_string.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_insert/size_string_size_size.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_op_plus_equal/char.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_op_plus_equal/initializer_list.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_op_plus_equal/pointer.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_op_plus_equal/string.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_replace/iter_iter_string.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_replace/size_size_T_size_size.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_replace/size_size_pointer.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_replace/size_size_pointer_size.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_replace/size_size_size_char.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_replace/size_size_string.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_replace/size_size_string_size_size.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_replace/size_size_string_view.pass.cpp
    A libcxx/test/std/strings/basic.string/string.modifiers/string_swap/swap.asan.pass.cpp
    M libcxx/test/std/strings/basic.string/string.modifiers/string_swap/swap.pass.cpp
    M libcxx/test/std/strings/basic.string/string.nonmembers/string.special/swap.pass.cpp
    M libcxx/test/std/strings/basic.string/string.nonmembers/string_op+/char_string.pass.cpp
    M libcxx/test/std/strings/basic.string/string.nonmembers/string_op+/string_char.pass.cpp
    M libcxx/test/std/strings/basic.string/string.nonmembers/string_op+/string_pointer.pass.cpp
    M libcxx/test/std/strings/basic.string/string.nonmembers/string_op+/string_string.pass.cpp
    M libcxx/test/std/strings/basic.string/string.ops/string_substr/substr.pass.cpp
    M libcxx/test/std/strings/basic.string/string.ops/string_substr/substr_rvalue.pass.cpp
    M libcxx/test/support/asan_testing.h

  Log Message:
  -----------
  [ASan][libc++] std::basic_string annotations (#72677)

This commit introduces basic annotations for `std::basic_string`,
mirroring the approach used in `std::vector` and `std::deque`.
Initially, only long strings with the default allocator will be
annotated. Short strings (_SSO - short string optimization_) and strings
with non-default allocators will be annotated in the near future, with
separate commits dedicated to enabling them. The process will be similar
to the workflow employed for enabling annotations in `std::deque`.

**Please note**: these annotations function effectively only when libc++
and libc++abi dylibs are instrumented (with ASan). This aligns with the
prevailing behavior of Memory Sanitizer.

To avoid breaking everything, this commit also appends
`_LIBCPP_INSTRUMENTED_WITH_ASAN` to `__config_site` whenever libc++ is
compiled with ASan. If this macro is not defined, string annotations are
not enabled. However, linking a binary that does **not** annotate
strings with a dynamic library that annotates strings, is not permitted.

Originally proposed here: https://reviews.llvm.org/D132769

Related patches on Phabricator:
- Turning on annotations for short strings:
https://reviews.llvm.org/D147680
- Turning on annotations for all allocators:
https://reviews.llvm.org/D146214

This PR is a part of a series of patches extending AddressSanitizer C++
container overflow detection capabilities by adding annotations, similar
to those existing in `std::vector` and `std::deque` collections. These
enhancements empower ASan to effectively detect instances where the
instrumented program attempts to access memory within a collection's
internal allocation that remains unused. This includes cases where
access occurs before or after the stored elements in `std::deque`, or
between the `std::basic_string`'s size (including the null terminator)
and capacity bounds.

The introduction of these annotations was spurred by a real-world
software bug discovered by Trail of Bits, involving an out-of-bounds
memory access during the comparison of two strings using the
`std::equals` function. This function was taking iterators
(`iter1_begin`, `iter1_end`, `iter2_begin`) to perform the comparison,
using a custom comparison function. When the `iter1` object exceeded the
length of `iter2`, an out-of-bounds read could occur on the `iter2`
object. Container sanitization, upon enabling these annotations, would
effectively identify and flag this potential vulnerability.

This Pull Request introduces basic annotations for `std::basic_string`.
Long strings exhibit structural similarities to `std::vector` and will
be annotated accordingly. Short strings are already implemented, but
will be turned on separately in a forthcoming commit. Look at [a
comment](https://github.com/llvm/llvm-project/pull/72677#issuecomment-1850554465)
below to read about SSO issues at current moment.

Due to the functionality introduced in
[D132522](https://github.com/llvm/llvm-project/commit/dd1b7b797a116eed588fd752fbe61d34deeb24e4),
the `__sanitizer_annotate_contiguous_container` function now offers
compatibility with all allocators. However, enabling this support will
be done in a subsequent commit. For the time being, only strings with
the default allocator will be annotated.

If you have any questions, please email:
- advenam.tacet at trailofbits.com
- disconnect3d at trailofbits.com