[all-commits] [llvm/llvm-project] 0784b1: Re-exec TSan with no ASLR if memory layout is inco...

[Thurston Dang via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 0784b1eefa36d4acbb0dacd2d18796e26313b6c5
      https://github.com/llvm/llvm-project/commit/0784b1eefa36d4acbb0dacd2d18796e26313b6c5
  Author: Thurston Dang <thurston at google.com>
  Date:   2024-01-19 (Fri, 19 Jan 2024)

  Changed paths:
    M compiler-rt/lib/tsan/rtl/tsan_platform.h
    M compiler-rt/lib/tsan/rtl/tsan_platform_linux.cpp
    M compiler-rt/lib/tsan/rtl/tsan_platform_mac.cpp
    M compiler-rt/lib/tsan/rtl/tsan_platform_posix.cpp

  Log Message:
  -----------
  Re-exec TSan with no ASLR if memory layout is incompatible on Linux (#78351)

TSan's shadow mappings only support 30-bits of ASLR entropy on x86
Linux, and it is not practical to support the maximum of 32-bits (due to pointer compression and the overhead of shadow mappings). Instead, this patch changes TSan to re-exec without ASLR if it encounters an 
incompatible memory layout, as suggested by Dmitry in
https://github.com/google/sanitizers/issues/1716.
If ASLR is already disabled but the memory layout is still incompatible,
it will abort.

This patch involves a bit of refactoring, because the old code is:
1. InitializePlatformEarly()
2. InitializeAllocator()
3. InitializePlatform(): CheckAndProtect()

but it may already segfault during InitializeAllocator() if the memory
layout is incompatible, before we get a chance to check in
CheckAndProtect().

This patch adds CheckAndProtect() during InitializePlatformEarly(), before the allocator is initialized. Naturally, it is necessary to ensure that CheckAndProtect() does *not* allow the heap regions to be occupied  here, hence we generalize CheckAndProtect() to optionally check the heap
regions. We keep the original behavior of CheckAndProtect() in InitializePlatform() as a last line of defense.

We need to be careful not to prematurely abort if ASLR is disabled but TSan was going to re-exec for other reasons (e.g., unlimited stack size); we implement this by moving all the re-exec logic into ReExecIfNeeded().