[compiler-rt] Remove maximum OSX version for sanitizers (PR #77543)
Chris Apple via llvm-commits
llvm-commits at lists.llvm.org
Tue Jan 9 17:34:09 PST 2024
https://github.com/cjappl created https://github.com/llvm/llvm-project/pull/77543
Remove a block preventing newer versions of the MacOS SDK from being selected for compiling the sanitizers.
As a backup proposal (if this is unacceptable), I think we should consider increasing the maximum version to (minimally) 10.15, or 14.0 (latest as of writing) to allow the sanitizers to intercept more modern calls like `aligned_alloc` and `shared_mutex`, introduced in 10.15 and 10.12 respectively.
# History
The limitation on the max version of OSX was committed here, originally set to 10.9:
https://github.com/llvm/llvm-project/commit/41c52889b90ded563e1aa56cfa7764e083323b00
> It's not a good idea to build the sanitizers with e.g. -DCMAKE_OSX_DEPLOYMENT_TARGET=10.12, because some deprecated functions that we intercept will cause build errors. Let's limit the allowed deployment targets to 10.9 (which is the default anyway), and warn when it's set above."
Later, it was updated to be 10.10 in this commit:
https://github.com/llvm/llvm-project/commit/b87fc09dceeff9cb838884961d817c0ab0257176
> Bump default value for `SANITIZER_MIN_OSX_VERSION` to 10.10 (from 10.9). TSan does not work on macOS 10.9 and a nice error message is preferable to an "unreferenced symbol" error when loading the TSan runtime.
>
> We could try to only bump the deployment target for TSan, but we would have to invest into adding support for this to our CMake build and it does not seem worth it. macOS 10.10 was released in 2014.
This means the original error was that the sanitizers could not build with the errors produced in a newer version than 10.9, and later 10.10.
# Why we may want to go higher than 10.10 (or not limit at all)
There are some sanitizers that do not intercept functions that they could because these APIs do not exist in the version we are targeting.
Take for example `aligned_alloc`, introduced in 10.15:
```
> cat /Library/Developer/CommandLineTools/SDKs/MacOSX14.0.sdk/usr/include/malloc/_malloc.h
...
void *aligned_alloc(size_t __alignment, size_t __size) ... __OSX_AVAILABLE(10.15) ...
...
```
We have a few interceptors that cannot intercept aligned_alloc:
```
> rg INTERCEPT_ALIGNED_ALLOC
compiler-rt/lib/sanitizer_common/sanitizer_platform_interceptors.h
495:#define SANITIZER_INTERCEPT_ALIGNED_ALLOC (!SI_MAC)
compiler-rt/lib/lsan/lsan_interceptors.cpp
145:#if SANITIZER_INTERCEPT_ALIGNED_ALLOC
151:#define LSAN_MAYBE_INTERCEPT_ALIGNED_ALLOC INTERCEPT_FUNCTION(aligned_alloc)
153:#define LSAN_MAYBE_INTERCEPT_ALIGNED_ALLOC
555: LSAN_MAYBE_INTERCEPT_ALIGNED_ALLOC;
```
TSan seemingly would use aligned_alloc if it were available to it (but does not use the SANITIZER_INTERCEPT_ALIGNED_ALLOC call)
```cpp
compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp
#if !SANITIZER_APPLE
TSAN_INTERCEPTOR(void*, aligned_alloc, uptr align, uptr sz) {
if (in_symbolizer())
return InternalAlloc(sz, nullptr, align);
SCOPED_INTERCEPTOR_RAW(aligned_alloc, align, sz);
return user_aligned_alloc(thr, pc, align, sz);
}
...
#endif
```
ASan shows aligned_alloc in their linux interceptors, but no such call exists in their mac interceptors
```
> cat compiler-rt/lib/asan/asan_malloc_linux.cpp | grep -C 2 "aligned_alloc"
#if SANITIZER_INTERCEPT_ALIGNED_ALLOC
INTERCEPTOR(void*, aligned_alloc, uptr boundary, uptr size) {
GET_STACK_TRACE_MALLOC;
return asan_aligned_alloc(boundary, size, &stack);
}
#endif // SANITIZER_INTERCEPT_ALIGNED_ALLOC
> cat compiler-rt/lib/sanitizer_common/sanitizer_malloc_mac.inc | grep -C 2 "aligned_alloc"
... (nothing)
```
Additionally, any future interceptor that wants to intercept calls introduced later than 10.10 will be blocked. For instance, in my own development I have been blocked intercepting `shared_mutex` (introduced 10.12) and `aligned_alloc` (introduced in 10.15).
# Testing
I have tested 3 versions, the existing 10.10, 10.15 (the latest I personally care about), and 14.0 (the latest MacOS available as of writing). I was attempting to confirm that no regressions happened in the newer versions, and the limitation could be removed.
Each of these tests I:
1. Ran `check-ubsan`
2. Ran `check-asan`
3. Ran `check-ubsan`
5. Compiled `clang`, `compiler-rt` and `llvm-symbolizer`
## Summary
No versions tested that were newer than 10.10 had any regressions or errors compiling. `check-asan` failed 137 tests against all versions, remaining the same, the others had no failures.
The newer versions (10.15 and 14.0) **did report a number of deprecation warnings, but no compilation errors.**
# Full testing results
Testing done on this machine:
```
> system_profiler SPSoftwareDataType SPHardwareDataType
Software:
System Software Overview:
System Version: macOS 14.1.1 (23B81)
Kernel Version: Darwin 23.1.0
Boot Volume: Macintosh HD
Boot Mode: Normal
Computer Name: ...
User Name: ...
Secure Virtual Memory: Enabled
System Integrity Protection: Enabled
Time since boot: 22 days, 4 hours, 43 minutes
Hardware:
Hardware Overview:
Model Name: MacBook Pro
Model Identifier: MacBookPro17,1
Model Number: Z11C000E4LL/A
Chip: Apple M1
Total Number of Cores: 8 (4 performance and 4 efficiency)
Memory: 16 GB
System Firmware Version: 10151.41.12
OS Loader Version: 10151.41.12
```
## With MacOS version 10.10 (Existing default today)
```
cmake -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_SHARED_LIBS=ON \
-DCOMPILER_RT_BUILD_SANITIZERS=ON \
-DLLVM_ENABLE_PROJECTS="clang;compiler-rt" \
-DLLVM_TARGETS_TO_BUILD=Native \
-DSANITIZER_MIN_OSX_VERSION="10.10" \
../llvm
```
```bash
> cat CMakeCache.txt | grep SANITIZER_MIN
SANITIZER_MIN_OSX_VERSION:STRING=10.10
```
### UBSan
```
> ninja check-ubsan
Total Discovered Tests: 261
Unsupported : 29 (11.11%)
Passed : 231 (88.51%)
Expectedly Failed: 1 (0.38%)
3 warning(s) in tests
```
No failures
### ASan
```
> ninja check-asan
Testing Time: 198.77s
Total Discovered Tests: 748
Unsupported : 237 (31.68%)
Passed : 372 (49.73%)
Expectedly Failed: 2 (0.27%)
Failed : 137 (18.32%)
```
Failed: 137 tests
### TSan
```
> ninja check-tsan
Total Discovered Tests: 441
Unsupported : 57 (12.93%)
Passed : 383 (86.85%)
Expectedly Failed: 1 (0.23%)
1 warning(s) in tests
```
No failures
### Building main targets
```
> ninja clang compiler-rt llvm-symbolizer
...
> echo $?
0
```
Succeeds
## With MacOS version 10.15
```
cmake -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_SHARED_LIBS=ON \
-DCOMPILER_RT_BUILD_SANITIZERS=ON \
-DLLVM_ENABLE_PROJECTS="clang;compiler-rt" \
-DLLVM_TARGETS_TO_BUILD=Native \
-DSANITIZER_MIN_OSX_VERSION="10.15" \
../llvm
```
```bash
> cat CMakeCache.txt | grep SANITIZER_MIN
SANITIZER_MIN_OSX_VERSION:STRING=10.15
```
### UBSan
```
> ninja check-ubsan
Total Discovered Tests: 261
Unsupported : 29 (11.11%)
Passed : 231 (88.51%)
Expectedly Failed: 1 (0.38%)
3 warning(s) in tests
```
No failures
### ASan
```
> ninja check-asan
Total Discovered Tests: 748
Unsupported : 237 (31.68%)
Passed : 372 (49.73%)
Expectedly Failed: 2 (0.27%)
Failed : 137 (18.32%)
```
Failed: 137 tests - SAME AS 10.10
### TSan
```
> ninja check-tsan
Total Discovered Tests: 441
Unsupported : 57 (12.93%)
Passed : 383 (86.85%)
Expectedly Failed: 1 (0.23%)
1 warning(s) in tests
```
No failures
### Building main targets
```
> ninja clang compiler-rt llvm-symbolizer
...
> echo $?
0
```
Succeeds
## With MacOS version 14.0 (latest released)
```
cmake -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_SHARED_LIBS=ON \
-DCOMPILER_RT_BUILD_SANITIZERS=ON \
-DLLVM_ENABLE_PROJECTS="clang;compiler-rt" \
-DLLVM_TARGETS_TO_BUILD=Native \
-DSANITIZER_MIN_OSX_VERSION="14.0" \
../llvm
```
```bash
> cat CMakeCache.txt | grep SANITIZER_MIN
SANITIZER_MIN_OSX_VERSION:STRING=14.0
```
### UBSan
```
> ninja check-ubsan
Total Discovered Tests: 261
Unsupported : 29 (11.11%)
Passed : 231 (88.51%)
Expectedly Failed: 1 (0.38%)
3 warning(s) in tests
```
No failures
### ASan
```
> ninja check-asan
Total Discovered Tests: 748
Unsupported : 237 (31.68%)
Passed : 372 (49.73%)
Expectedly Failed: 2 (0.27%)
Failed : 137 (18.32%)
```
Failed: 137 tests - SAME AS 10.10
### TSan
```
> ninja check-tsan
Total Discovered Tests: 261
Unsupported : 29 (11.11%)
Passed : 231 (88.51%)
Expectedly Failed: 1 (0.38%)
3 warning(s) in tests
```
No failures
### Building main targets
```
> ninja clang compiler-rt llvm-symbolizer
...
> echo $?
0
```
Succeeds
## Future PRs
To not overload this PR, I omitted these, but they should be done in quick succession, which I'm happy to do.
A few sanitizers unnecessarily do not include interceptors for `aligned_alloc` which could be easily fixed.
1. Improve the `SANITIZER_INTERCEPT_ALIGNED_ALLOC (!SI_MAC)` to be version dependent, if the version is greater than 10.15, this should evaluate to true
2. Change `TSAN_INTERCEPTOR` to use the `SANITIZER_INTERCEPT_ALIGNED_ALLOC` macro, currently it is blocked out behind just `!SANITIZER_APPLE`
3. Add `aligned_alloc` to `compiler-rt/lib/sanitizer_common/sanitizer_malloc_mac.inc`, as it is missing, but all other allocations exist there.
Lastly, we may want to consider increasing the default version from 10.10:
```cmake
set(DEFAULT_SANITIZER_MIN_OSX_VERSION 10.10)
```
See "Why we may want to go higher" for details on these changes.
>From ae68a982d25ae1c17a599a136cabd16b741854e8 Mon Sep 17 00:00:00 2001
From: Chris Apple <14171107+cjappl at users.noreply.github.com>
Date: Tue, 9 Jan 2024 17:28:19 -0800
Subject: [PATCH] Removed maximum OSX version for sanitizers
---
compiler-rt/cmake/config-ix.cmake | 5 -----
1 file changed, 5 deletions(-)
diff --git a/compiler-rt/cmake/config-ix.cmake b/compiler-rt/cmake/config-ix.cmake
index 142ab60f7152db..2ca18ebb4ad489 100644
--- a/compiler-rt/cmake/config-ix.cmake
+++ b/compiler-rt/cmake/config-ix.cmake
@@ -479,11 +479,6 @@ if(APPLE)
if(MIN_OSX_VERSION VERSION_LESS "10.7")
message(FATAL_ERROR "macOS deployment target '${SANITIZER_MIN_OSX_VERSION}' is too old.")
endif()
- if(MIN_OSX_VERSION VERSION_GREATER ${DEFAULT_SANITIZER_MIN_OSX_VERSION})
- message(WARNING "macOS deployment target '${SANITIZER_MIN_OSX_VERSION}' is too new, setting to '${DEFAULT_SANITIZER_MIN_OSX_VERSION}' instead.")
- set(MIN_OSX_VERSION ${DEFAULT_SANITIZER_MIN_OSX_VERSION})
- endif()
-
endif()
set(SANITIZER_MIN_OSX_VERSION "${MIN_OSX_VERSION}" CACHE STRING
More information about the llvm-commits
mailing list