[all-commits] [llvm/llvm-project] 615b74: [ASan][libc++] Turn on ASan annotations for short ...

[Tacet via All-commits]

  Branch: refs/heads/users/AdvenamTacet/short-string-annotations-v2-testing
  Home:   https://github.com/llvm/llvm-project
  Commit: 615b74729e0c1cb56f6f1bc8dc3977dd4736ea04
      https://github.com/llvm/llvm-project/commit/615b74729e0c1cb56f6f1bc8dc3977dd4736ea04
  Author: Tacet <advenam.tacet at trailofbits.com>
  Date:   2024-01-22 (Mon, 22 Jan 2024)

  Changed paths:
    M libcxx/include/string
    A libcxx/test/libcxx/containers/strings/basic.string/asan_deque_integration.pass.cpp
    A libcxx/test/libcxx/containers/strings/basic.string/asan_short.pass.cpp
    A libcxx/test/libcxx/containers/strings/basic.string/asan_vector_integration.pass.cpp
    M libcxx/test/support/asan_testing.h

  Log Message:
  -----------
  [ASan][libc++] Turn on ASan annotations for short strings

This commit turns on ASan annotations in `std::basic_string` for short
stings (SSO case).

Originally suggested here: https://reviews.llvm.org/D147680

String annotations added here:
https://github.com/llvm/llvm-project/pull/72677

Requires to pass CI without fails:
- https://github.com/llvm/llvm-project/pull/75845
- https://github.com/llvm/llvm-project/pull/75858

Annotating `std::basic_string` with default allocator is implemented in
https://github.com/llvm/llvm-project/pull/72677 but annotations for
short strings (SSO - Short String Optimization) are turned off there.
This commit turns them on. This also removes
`_LIBCPP_SHORT_STRING_ANNOTATIONS_ALLOWED`, because we do not plan to
support turning on and off short string annotations.

Support in ASan API exists since
https://github.com/llvm/llvm-project/commit/dd1b7b797a116eed588fd752fbe61d34deeb24e4.
You can turn off annotations for a specific allocator based on changes
from
https://github.com/llvm/llvm-project/commit/2fa1bec7a20bb23f2e6620085adb257dafaa3be0.

This PR is a part of a series of patches extending AddressSanitizer C++
container overflow detection capabilities by adding annotations, similar
to those existing in `std::vector` and `std::deque` collections. These
enhancements empower ASan to effectively detect instances where the
instrumented program attempts to access memory within a collection's
internal allocation that remains unused. This includes cases where
access occurs before or after the stored elements in `std::deque`, or
between the `std::basic_string`'s size (including the null terminator)
and capacity bounds.

The introduction of these annotations was spurred by a real-world
software bug discovered by Trail of Bits, involving an out-of-bounds
memory access during the comparison of two strings using the
`std::equals` function. This function was taking iterators
(`iter1_begin`, `iter1_end`, `iter2_begin`) to perform the comparison,
using a custom comparison function. When the `iter1` object exceeded the
length of `iter2`, an out-of-bounds read could occur on the `iter2`
object. Container sanitization, upon enabling these annotations, would
effectively identify and flag this potential vulnerability.

If you have any questions, please email:

    advenam.tacet at trailofbits.com
    disconnect3d at trailofbits.com


  Commit: 4f46e9f34ab2852c9181ab2681aeed6460fba834
      https://github.com/llvm/llvm-project/commit/4f46e9f34ab2852c9181ab2681aeed6460fba834
  Author: Advenam Tacet <advenam.tacet at trailofbits.com>
  Date:   2024-01-22 (Mon, 22 Jan 2024)

  Changed paths:
    M llvm/include/llvm/ADT/FunctionExtras.h
    M llvm/include/llvm/Support/JSON.h

  Log Message:
  -----------
  Unpoison memory before reinterpret

This commit unpoisons memory before using memory as different types/before memset.
Buildobts were failing and therefore short string annotations were turned off.
Here: https://github.com/llvm/llvm-project/pull/78627

This commit unpoisons memory before debug "scribble across the rest of the storage",
as well as before reinterpret cast of an union.


  Commit: 93cc7515864af26df0b20373ddcb5e19e3537cca
      https://github.com/llvm/llvm-project/commit/93cc7515864af26df0b20373ddcb5e19e3537cca
  Author: Advenam Tacet <advenam.tacet at trailofbits.com>
  Date:   2024-01-22 (Mon, 22 Jan 2024)

  Changed paths:
    M libcxx/test/libcxx/containers/strings/basic.string/asan_deque_integration.pass.cpp
    M libcxx/test/libcxx/containers/strings/basic.string/asan_vector_integration.pass.cpp

  Log Message:
  -----------
  Speed up ASan tests

This commit lowers values in `std::vector` tests to as good as previous ones, but faster.
One tests caused a problem with buildbots: https://lab.llvm.org/buildbot/#/builders/168/builds/18126/steps/11/logs/stdio
Test added here: https://github.com/llvm/llvm-project/pull/75882


Compare: https://github.com/llvm/llvm-project/compare/615b74729e0c%5E...93cc7515864a