[PATCH] D83247: [compiler-rt][asan][hwasan] Refactor shadow setup into sanitizer_common (NFCI)

Fri Jul 17 14:50:52 PDT 2020

tejohnson added a comment.

In D83247#2159463 <https://reviews.llvm.org/D83247#2159463>, @tejohnson wrote:

> In D83247#2159421 <https://reviews.llvm.org/D83247#2159421>, @phosek wrote:
>
> > This broke Fuchsia's toolchain build which is now failing with the following error:
> >
> >    FAILED: /b/s/w/ir/k/recipe_cleanup/clangro8g9J/llvm_build/lib/clang/12.0.0/lib/x86_64-unknown-fuchsia/libclang_rt.ubsan_standalone.so 
> >    : && /b/s/w/ir/k/recipe_cleanup/clangro8g9J/llvm_build/./bin/clang++ --target=x86_64-unknown-fuchsia --sysroot=/b/s/w/ir/k/cipd/sdk/arch/x64/sysroot -fPIC --target=x86_64-unknown-fuchsia -I/b/s/w/ir/k/cipd/sdk/pkg/fdio/include -fPIC -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -ffunction-sections -fdata-sections -ffile-prefix-map=/b/s/w/ir/k/recipe_cleanup/clangro8g9J/llvm_build/runtimes/runtimes-x86_64-unknown-fuchsia-bins=../recipe_cleanup/clangro8g9J/llvm_build/runtimes/runtimes-x86_64-unknown-fuchsia-bins -ffile-prefix-map=/b/s/w/ir/k/llvm-project/= -no-canonical-prefixes -Wall -std=c++14 -Wno-unused-parameter -O2 -g  -L/b/s/w/ir/k/cipd/sdk/arch/x64/lib    -nodefaultlibs -Wl,-z,text -Wl,-z,defs,-z,now,-z,relro -Wl,--version-script,/b/s/w/ir/k/recipe_cleanup/clangro8g9J/llvm_build/runtimes/runtimes-x86_64-unknown-fuchsia-bins/compiler-rt/lib/ubsan/clang_rt.ubsan_standalone-dynamic-x86_64.vers -shared -Wl,-soname,libclang_rt.ubsan_standalone.so -o /b/s/w/ir/k/recipe_cleanup/clangro8g9J/llvm_build/lib/clang/12.0.0/lib/x86_64-unknown-fuchsia/libclang_rt.ubsan_standalone.so compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_allocator.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_common.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_deadlock_detector1.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_deadlock_detector2.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_errno.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_file.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_flags.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_flag_parser.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_fuchsia.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_libc.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_libignore.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_linux.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_linux_s390.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_mac.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_netbsd.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_openbsd.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_persistent_allocator.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_platform_limits_freebsd.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_platform_limits_linux.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_platform_limits_netbsd.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_platform_limits_openbsd.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_platform_limits_posix.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_platform_limits_solaris.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_posix.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_printf.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_procmaps_common.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_procmaps_bsd.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_procmaps_fuchsia.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_procmaps_linux.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_procmaps_mac.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_procmaps_solaris.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_rtems.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_solaris.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_stoptheworld_fuchsia.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_stoptheworld_mac.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_suppressions.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_tls_get_addr.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_thread_registry.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_type_traits.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_win.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_termination.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.x86_64.dir/sanitizer_common_libcdep.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.x86_64.dir/sanitizer_allocator_checks.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.x86_64.dir/sanitizer_linux_libcdep.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.x86_64.dir/sanitizer_mac_libcdep.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.x86_64.dir/sanitizer_posix_libcdep.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.x86_64.dir/sanitizer_stoptheworld_linux_libcdep.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.x86_64.dir/sanitizer_stoptheworld_netbsd_libcdep.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonCoverage.x86_64.dir/sancov_flags.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonCoverage.x86_64.dir/sanitizer_coverage_fuchsia.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonCoverage.x86_64.dir/sanitizer_coverage_libcdep_new.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonCoverage.x86_64.dir/sanitizer_coverage_win_sections.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_allocator_report.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_stackdepot.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_stacktrace.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_stacktrace_libcdep.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_stacktrace_printer.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_stacktrace_sparc.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_symbolizer.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_symbolizer_libbacktrace.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_symbolizer_libcdep.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_symbolizer_mac.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_symbolizer_markup.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_symbolizer_posix_libcdep.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_symbolizer_report.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_symbolizer_win.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_unwind_linux_libcdep.cpp.obj compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_unwind_win.cpp.obj compiler-rt/lib/ubsan/CMakeFiles/RTUbsan.x86_64.dir/ubsan_diag.cpp.obj compiler-rt/lib/ubsan/CMakeFiles/RTUbsan.x86_64.dir/ubsan_init.cpp.obj compiler-rt/lib/ubsan/CMakeFiles/RTUbsan.x86_64.dir/ubsan_flags.cpp.obj compiler-rt/lib/ubsan/CMakeFiles/RTUbsan.x86_64.dir/ubsan_handlers.cpp.obj compiler-rt/lib/ubsan/CMakeFiles/RTUbsan.x86_64.dir/ubsan_monitor.cpp.obj compiler-rt/lib/ubsan/CMakeFiles/RTUbsan.x86_64.dir/ubsan_value.cpp.obj compiler-rt/lib/ubsan/CMakeFiles/RTUbsan_cxx.x86_64.dir/ubsan_handlers_cxx.cpp.obj compiler-rt/lib/ubsan/CMakeFiles/RTUbsan_cxx.x86_64.dir/ubsan_type_hash.cpp.obj compiler-rt/lib/ubsan/CMakeFiles/RTUbsan_cxx.x86_64.dir/ubsan_type_hash_itanium.cpp.obj compiler-rt/lib/ubsan/CMakeFiles/RTUbsan_cxx.x86_64.dir/ubsan_type_hash_win.cpp.obj compiler-rt/lib/ubsan/CMakeFiles/RTUbsan_standalone.x86_64.dir/ubsan_diag_standalone.cpp.obj compiler-rt/lib/ubsan/CMakeFiles/RTUbsan_standalone.x86_64.dir/ubsan_init_standalone.cpp.obj compiler-rt/lib/ubsan/CMakeFiles/RTUbsan_standalone.x86_64.dir/ubsan_signals_standalone.cpp.obj compiler-rt/lib/interception/CMakeFiles/RTInterception.x86_64.dir/interception_linux.cpp.obj compiler-rt/lib/interception/CMakeFiles/RTInterception.x86_64.dir/interception_mac.cpp.obj compiler-rt/lib/interception/CMakeFiles/RTInterception.x86_64.dir/interception_win.cpp.obj compiler-rt/lib/interception/CMakeFiles/RTInterception.x86_64.dir/interception_type_test.cpp.obj compiler-rt/lib/ubsan/CMakeFiles/RTUbsan_dynamic_version_script_dummy.x86_64.dir/dummy.cpp.obj -L/b/s/w/ir/k/cipd/sdk/arch/x64/lib  -L/b/s/w/ir/k/cipd/sdk/arch/x64/sysroot/lib /b/s/w/ir/k/recipe_cleanup/clangro8g9J/llvm_build/lib/x86_64-unknown-fuchsia/c++/libunwind.a /b/s/w/ir/k/recipe_cleanup/clangro8g9J/llvm_build/lib/x86_64-unknown-fuchsia/c++/libc++abi.so.1.0 -lc -lzircon -ldl -lrt -lpthread /b/s/w/ir/k/recipe_cleanup/clangro8g9J/llvm_build/lib/clang/12.0.0/lib/x86_64-unknown-fuchsia/libclang_rt.builtins.a -lc && :
> >    ld.lld: error: undefined symbol: __sanitizer::MmapFixedSuperNoReserve(unsigned long, unsigned long, char const*)
> >   >>> referenced by sanitizer_common_libcdep.cpp:150 (compiler-rt/lib/sanitizer_common/sanitizer_common_libcdep.cpp:150)
> >   >>>               compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.x86_64.dir/sanitizer_common_libcdep.cpp.obj:(__sanitizer::ReserveShadowMemoryRange(unsigned long, unsigned long, char const*, bool))
> >   
> >    ld.lld: error: undefined symbol: __sanitizer::DontDumpShadowMemory(unsigned long, unsigned long)
> >   >>> referenced by sanitizer_common_libcdep.cpp:159 (compiler-rt/lib/sanitizer_common/sanitizer_common_libcdep.cpp:159)
> >   >>>               compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.x86_64.dir/sanitizer_common_libcdep.cpp.obj:(__sanitizer::ReserveShadowMemoryRange(unsigned long, unsigned long, char const*, bool))
> >   
> >    ld.lld: error: undefined symbol: __sanitizer::MmapFixedNoReserve(unsigned long, unsigned long, char const*)
> >   >>> referenced by sanitizer_common_libcdep.cpp:151 (compiler-rt/lib/sanitizer_common/sanitizer_common_libcdep.cpp:151)
> >   >>>               compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.x86_64.dir/sanitizer_common_libcdep.cpp.obj:(__sanitizer::ReserveShadowMemoryRange(unsigned long, unsigned long, char const*, bool))
> >    clang++: error: ld.lld command failed with exit code 1 (use -v to see invocation)
> >   
> >
> > That function isn't used on Fuchsia, but it seems like `--gc-sections` isn't being used here causing the failure. It might be sufficient to just add `#if !SANITIZER_FUCHSIA` around this function as a workaround.
>
>
> Sorry about that! I see that this code when it was in asan before my patch was guarded by:
>  #if !SANITIZER_FUCHSIA && !SANITIZER_RTEMS
>  so probably I should guard it the same way here.
>
> Also, while the error above doesn't show any undefs coming from ProtectGap, I'm assuming that will give the same issue as it was moved to the same place and references MmapFixedNoAccess. Both were guarded before in asan and I'll add the guard here and commit shortly. I guess hwasan wasn't built on Fuchscia since it didn't have the guards around its version?

Fixed in db15b8ab90793e3707f72e7668c3844922fdf356 <https://reviews.llvm.org/rGdb15b8ab90793e3707f72e7668c3844922fdf356>

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D83247/new/

https://reviews.llvm.org/D83247