[llvm-dev] [RFC] ASan: patches to support 32-byte shadow granularity

[Walter Lee via llvm-dev]

I've finished my initial set of patches to make 32-byte shadow
granularity work on x86.  Here is a summary of the changes from last
week:

- As discussed, I added a full redzone after every stack variable.

- We discussed adding a -fsanitize-address-granularity=N flag, but I
  found the following existing flag has been sufficient for my
  purposes: -asan-mapping-scale N.  If anyone thinks I should add the
  flag anyways, possibly replacing the latter, please let me know.

- I've modified the build so that we always run the ASan
  instrumentation test suite for shadow scale values of 3 and 5.

- I've gone through the asan test suites to make them run cleanly for
  both shadow scale=3 and shadow scale=5.  Here are the tests I have
  disabled, grouped by categories:

  - Instrumentation/AddressSanitizer tests.  Most tests work out of
    the box, and I ported some basic tests, leaving the following:

    llvm/test/Instrumentation/AddressSanitizer/lifetime-throw.ll
    llvm/test/Instrumentation/AddressSanitizer/lifetime-uar-uas.ll
    llvm/test/Instrumentation/AddressSanitizer/lifetime.ll
    llvm/test/Instrumentation/AddressSanitizer/stack-poisoning-and-lifetime-be.ll
    llvm/test/Instrumentation/AddressSanitizer/stack-poisoning-and-lifetime.ll
    llvm/test/Instrumentation/AddressSanitizer/stack-poisoning.ll
    llvm/test/Instrumentation/AddressSanitizer/stack_layout.ll

  - Asm instrumentation not supported:
    Instrumentation/AddressSanitizer/X86/*
    compiler-rt/lib/asan/tests/asan_asm_test.cc
    compiler-rt/test/asan/TestCases/Linux/asan-asm-stacktrace-test.cc

  - Prelinking not supported:
    compiler-rt/test/asan/TestCases/Linux/asan_prelink_test.cc

  - Intra-object padding not supported:
    compiler-rt/test/asan/TestCases/intra-object-overflow.cc

  - Calls __asan_poison_memory_region in middle of shadow byte:
    compiler-rt/lib/asan/tests/asan_interface_test.cc
      SimplePoisonMemoryRegionTest
      OverlappingPoisonMemoryRegionTest
      PoisoningStressTest
    compiler-rt/test/asan/TestCases/small_memcpy_test.cc
    compiler-rt/test/asan/TestCases/strtol_strict.c
    compiler-rt/test/asan/TestCases/strtoll_strict.c

  - Has hardwired memory map:
    compiler-rt/test/asan/TestCases/Linux/cuda_test.cc
    compiler-rt/test/asan/TestCases/Linux/kernel-area.cc
    compiler-rt/test/asan/TestCases/Linux/nohugepage_test.cc

  - Miscellaneous:
    compiler-rt/test/asan/TestCases/Linux/allocator_oom_test.cc
      This test assumes amount of available memory.
    compiler-rt/test/asan/TestCases/stack-buffer-overflow-with-position.cc
      This fails because we don't have enough redzones to disambiguate
      overflow of one stack object vs underflow of the next stack
      object.

- Here is the full list of revisions.  I'll add review requests
  shortly.

[asan] Add CMake hook to override default shadow scale
       https://reviews.llvm.org/D39469
[asan] Fix size/alignment issues with non-default shadow scale
       https://reviews.llvm.org/D39470
[asan] Fix small X86_64 ShadowOffset for non-default shadow scale
       https://reviews.llvm.org/D39471
[asan] Ensure that the minimum redzone is at least SHADOW_GRANULARITY
       https://reviews.llvm.org/D39472
[sanitizers] Increase alignment of low level allocator
       https://reviews.llvm.org/D39473
[asan] Avoid assert failure for non-default shadow scale
       https://reviews.llvm.org/D39474
[asan] Add full redzone after every stack variable
       https://reviews.llvm.org/D39475
[gtest] Increase stack size for child process in EXPECT_DEATH implementation
       https://reviews.llvm.org/D39771
[asan] Add lit feature for custom shadow scale
       https://reviews.llvm.org/D39772
[asan] Port tests to shadow scale of 5
       https://reviews.llvm.org/D39773
[asan] Disable unsupported tests for custom shadow scale
       https://reviews.llvm.org/D39774
[asan] Test ASan instrumentation for shadow scale value of 5
       https://reviews.llvm.org/D39775

Thanks,

Walter