[llvm-dev] Attempt to modify memory sanitizer for support of X86
Evgenii Stepanov via llvm-dev
llvm-dev at lists.llvm.org
Tue Jun 21 13:07:11 PDT 2016
It's hard to say based on these code snippets, but it looks like you
have changed the memory mapping (i.e. app-to-shadow address
calculation) without updating the compiler side (MemorySanitizer.cpp).
On Tue, Jun 21, 2016 at 9:48 AM, Pusl, Stefan via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> Hello,
>
>
>
> thanks for your comments.
>
>
>
> I know, that there is not so much address space in case of 32-bit
> addressing, but for a lot of applications it is enough. I used address
> sanitizer with 32-bit addressing and due to the good performance I wanted to
> use memory sanitizer with 32-bit addressing.
>
>
>
> Instead of looking at /proc/<PID>/maps I changed msan.cc to print out the
> process mapping every time:
>
>
>
> diff --git a/lib/msan/msan.cc b/lib/msan/msan.cc
>
> index d2981f0..97f97c4 100644
>
> --- a/lib/msan/msan.cc
>
> +++ b/lib/msan/msan.cc
>
> @@ -408,6 +408,8 @@ void __msan_init() {
>
> Die();
>
> }
>
> + DumpProcessMap();
>
> +
>
> Symbolizer::GetOrInit()->AddHooks(EnterSymbolizer, ExitSymbolizer);
>
> InitializeCoverage(common_flags()->coverage,
> common_flags()->coverage_dir);
>
>
>
> When I now run my example, then I get the output:
>
>
>
> ==17434==Process memory map follows:
>
> 0x10000000-0x20000000
>
> 0x20000000-0x50000000
>
> 0x5659b000-0x5666a000 /home/pusl/code/example32.out
>
> 0x5666a000-0x5666b000 /home/pusl/code/example32.out
>
> 0x5666b000-0x5666e000 /home/pusl/code/example32.out
>
> 0x5666e000-0x576c6000
>
> 0x60000000-0x70000000
>
> 0x70000000-0x80000000
>
> 0x80000000-0x90000000
>
> 0x90000000-0xf0000000
>
> 0xf735a000-0xf7506000
>
> 0xf7506000-0xf76b3000 /lib32/libc-2.23.so
>
> 0xf76b3000-0xf76b5000 /lib32/libc-2.23.so
>
> 0xf76b5000-0xf76b6000 /lib32/libc-2.23.so
>
> 0xf76b6000-0xf76b9000
>
> 0xf76b9000-0xf76d5000 /usr/lib32/libgcc_s.so.1
>
> 0xf76d5000-0xf76d6000 /usr/lib32/libgcc_s.so.1
>
> 0xf76d6000-0xf76d9000 /lib32/libdl-2.23.so
>
> 0xf76d9000-0xf76da000 /lib32/libdl-2.23.so
>
> 0xf76da000-0xf76db000 /lib32/libdl-2.23.so
>
> 0xf76db000-0xf772e000 /lib32/libm-2.23.so
>
> 0xf772e000-0xf772f000 /lib32/libm-2.23.so
>
> 0xf772f000-0xf7730000 /lib32/libm-2.23.so
>
> 0xf7730000-0xf7737000 /lib32/librt-2.23.so
>
> 0xf7737000-0xf7738000 /lib32/librt-2.23.so
>
> 0xf7738000-0xf7739000 /lib32/librt-2.23.so
>
> 0xf7739000-0xf7752000 /lib32/libpthread-2.23.so
>
> 0xf7752000-0xf7753000 /lib32/libpthread-2.23.so
>
> 0xf7753000-0xf7754000 /lib32/libpthread-2.23.so
>
> 0xf7754000-0xf7757000
>
> 0xf7766000-0xf776e000
>
> 0xf776e000-0xf7770000 [vvar]
>
> 0xf7770000-0xf7771000 [vdso]
>
> 0xf7771000-0xf7793000 /lib32/ld-2.23.so
>
> 0xf7793000-0xf7794000
>
> 0xf7794000-0xf7795000 /lib32/ld-2.23.so
>
> 0xf7795000-0xf7796000 /lib32/ld-2.23.so
>
> 0xffa83000-0xffaa4000 [stack]
>
> ==17434==End of process memory map.
>
> Speicherzugriffsfehler (Speicherabzug geschrieben)
>
>
>
> When I look at the mapping, then the given code is always located in an
> app-region, given by kMemoryLayout. So I have misunderstand something
> completely or I oversee something.
>
>
>
> Any comments are welcome.
>
>
>
> Stefan
>
>
>
>
>
>
>
> From: Kostya Serebryany [mailto:kcc at google.com]
> Sent: Friday, June 17, 2016 7:13 PM
> To: Pusl, Stefan <stefan.pusl at intel.com>; Evgenii Stepanov
> <eugenis at google.com>
> Cc: llvm-dev at lists.llvm.org
> Subject: Re: [llvm-dev] Attempt to modify memory sanitizer for support of
> X86
>
>
>
> +eugenis (the msan co-author and current maintainer)
>
>
>
> I don't remember any hard blockers for 32-bit msan.
>
> Soft blockers are:
>
> * extra work and extra maintenance
>
> * too little address space, hard to fit any non-trivial application.
>
>
>
> You may start looking at /proc/<PID>/maps to see what is the memory region
> you
>
> are failing on and how it is related to kMemoryLayout.
>
>
>
> Just curious, why are you trying to do this?
>
>
>
> --kcc
>
>
>
> On Fri, Jun 17, 2016 at 8:02 AM, Pusl, Stefan via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>
> Hello,
>
>
>
> I’m quite new to LLVM, but I’m interested in memory sanitizer. The petty
> thing is, that 32bit addressing on Linux is not supported. Thus I tried to
> take the latest version of the software and to modify it, using Ubuntu 16.04
> as operating system. Given the example
>
>
>
> example.c:
>
> #include <stdio.h>
>
> #include <stdlib.h>
>
>
>
> int main(int argc, char **argv) {
>
> int *a = (int *)malloc(10*sizeof(10));
>
> a[5] = 0;
>
> if (a[argc])
>
> printf("xx\n");
>
> return 0;
>
> }
>
>
>
> which I compiled with the command
>
>
>
> clang -fsanitize=memory -m32 -fPIE -pie -fno-omit-frame-pointer -g
> -Wl,-Map,example32.map example.c -oexample32.out
>
>
>
> I get now a segmentation fault during the execution:
>
>
>
> (gdb) r
>
> Starting program: /home/pusl/code/example32.out
>
> [Thread debugging using libthread_db enabled]
>
> Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
>
>
>
> Program received signal SIGSEGV, Segmentation fault.
>
> 0x565e14f9 in main (argc=1448445640, argv=0xf7f1d3dc) at example.c:4
>
>
>
>
>
> I tried to figure out the issue, but as a newcomer to LLVM I don’t have any
> further idea now to get nearer to the root cause. Any new idea is welcome.
>
>
>
> I changed 5 files. The diffs are:
>
>
>
> - cmake/config-ix.cmake:
>
> diff --git a/cmake/config-ix.cmake b/cmake/config-ix.cmake
>
> index 04e33a0..d02e319 100644
>
> --- a/cmake/config-ix.cmake
>
> +++ b/cmake/config-ix.cmake
>
> @@ -149,7 +149,7 @@ set(ALL_ASAN_SUPPORTED_ARCH ${X86} ${X86_64} ${ARM32}
> ${ARM64}
>
> ${MIPS32} ${MIPS64} ${PPC64})
>
> set(ALL_DFSAN_SUPPORTED_ARCH ${X86_64} ${MIPS64} ${ARM64})
>
> set(ALL_LSAN_SUPPORTED_ARCH ${X86_64} ${MIPS64} ${ARM64})
>
> -set(ALL_MSAN_SUPPORTED_ARCH ${X86_64} ${MIPS64} ${ARM64})
>
> +set(ALL_MSAN_SUPPORTED_ARCH ${X86} ${X86_64} ${MIPS64} ${ARM64})
>
> set(ALL_PROFILE_SUPPORTED_ARCH ${X86} ${X86_64} ${ARM32} ${ARM64} ${PPC64}
>
> ${MIPS32} ${MIPS64})
>
> set(ALL_TSAN_SUPPORTED_ARCH ${X86_64} ${MIPS64} ${ARM64} ${PPC64})
>
>
>
> - lib/msan/msan.h:
>
> diff --git a/lib/msan/msan.h b/lib/msan/msan.h
>
> index be74f6e..10b5b05 100644
>
> --- a/lib/msan/msan.h
>
> +++ b/lib/msan/msan.h
>
> @@ -147,6 +147,26 @@ const MappingDesc kMemoryLayout[] = {
>
> #define MEM_TO_SHADOW(mem) (LINEARIZE_MEM((mem)) + 0x100000000000ULL)
>
> #define SHADOW_TO_ORIGIN(shadow) (((uptr)(shadow)) + 0x280000000000)
>
> +#elif SANITIZER_LINUX && SANITIZER_WORDSIZE == 32
>
> +
>
> +const MappingDesc kMemoryLayout[] = {
>
> + {0x00000000ULL, 0x10000000ULL, MappingDesc::APP, "app-1"},
>
> + {0x10000000ULL, 0x20000000ULL, MappingDesc::SHADOW, "shadow-1"},
>
> + {0x20000000ULL, 0x30000000ULL, MappingDesc::ORIGIN, "origin-1"},
>
> + {0x30000000ULL, 0x50000000ULL, MappingDesc::INVALID, "invalid"},
>
> + {0x50000000ULL, 0x60000000ULL, MappingDesc::APP, "app-2"},
>
> + {0x60000000ULL, 0x70000000ULL, MappingDesc::SHADOW, "shadow-2"},
>
> + {0x70000000ULL, 0x80000000ULL, MappingDesc::ORIGIN, "origin-2"},
>
> + {0x80000000ULL, 0x90000000ULL, MappingDesc::SHADOW, "shadow-3"},
>
> + {0x90000000ULL, 0xa0000000ULL, MappingDesc::ORIGIN, "origin-3"},
>
> + {0xa0000000ULL, 0xf0000000ULL, MappingDesc::INVALID, "invalid"},
>
> + {0xf0000000ULL, 0xffffffffULL, MappingDesc::APP, "app-3"},
>
> +};
>
> +#define MOVE_MEM(mem) \
>
> + ((uptr)(mem) & ~0x80000000ULL)
>
> +#define MEM_TO_SHADOW(mem) (MOVE_MEM((mem)) + 0x10000000ULL)
>
> +#define SHADOW_TO_ORIGIN(mem) (((uptr)(mem)) + 0x10000000ULL)
>
> +
>
> #elif SANITIZER_LINUX && SANITIZER_WORDSIZE == 64
>
> #ifdef MSAN_LINUX_X86_64_OLD_MAPPING
>
>
>
> - lib/msan/msan_allocator.cc:
>
> diff --git a/lib/msan/msan_allocator.cc b/lib/msan/msan_allocator.cc
>
> index b7d3947..dae6d72 100644
>
> --- a/lib/msan/msan_allocator.cc
>
> +++ b/lib/msan/msan_allocator.cc
>
> @@ -49,7 +49,15 @@ struct MsanMapUnmapCallback {
>
> typedef SizeClassAllocator32<0, SANITIZER_MMAP_RANGE_SIZE,
> sizeof(Metadata),
>
> SizeClassMap, kRegionSizeLog, ByteMap,
>
> MsanMapUnmapCallback> PrimaryAllocator;
>
> -
>
> +#elif defined (__i386__)
>
> +static const uptr kMaxAllowedMallocSize = 3UL << 30;
>
> + static const uptr kRegionSizeLog = 20;
>
> + static const uptr kNumRegions = SANITIZER_MMAP_RANGE_SIZE >>
> kRegionSizeLog;
>
> + typedef FlatByteMap<kNumRegions> ByteMap;
>
> + typedef CompactSizeClassMap SizeClassMap;
>
> + typedef SizeClassAllocator32<0, SANITIZER_MMAP_RANGE_SIZE,
> sizeof(Metadata),
>
> + SizeClassMap, kRegionSizeLog, ByteMap,
>
> + MsanMapUnmapCallback> PrimaryAllocator;
>
> #elif defined(__x86_64__)
>
> #if SANITIZER_LINUX && !defined(MSAN_LINUX_X86_64_OLD_MAPPING)
>
> static const uptr kAllocatorSpace = 0x700000000000ULL;
>
>
>
> - lib/msan/msan_linux.cc:
>
> diff --git a/lib/msan/msan_linux.cc b/lib/msan/msan_linux.cc
>
> index d6a9588..ca59564 100644
>
> --- a/lib/msan/msan_linux.cc
>
> +++ b/lib/msan/msan_linux.cc
>
> @@ -82,7 +82,7 @@ static void CheckMemoryLayoutSanity() {
>
> CHECK_LT(start, end);
>
> CHECK_EQ(prev_end, start);
>
> CHECK(addr_is_type(start, type));
>
> - CHECK(addr_is_type((start + end) / 2, type));
>
> + CHECK(addr_is_type(start/2 + end/2, type));
>
> CHECK(addr_is_type(end - 1, type));
>
> if (type == MappingDesc::APP) {
>
> uptr addr = start;
>
> @@ -90,7 +90,7 @@ static void CheckMemoryLayoutSanity() {
>
> CHECK(MEM_IS_ORIGIN(MEM_TO_ORIGIN(addr)));
>
> CHECK_EQ(MEM_TO_ORIGIN(addr), SHADOW_TO_ORIGIN(MEM_TO_SHADOW(addr)));
>
> - addr = (start + end) / 2;
>
> + addr = start/2 + end/2;
>
> CHECK(MEM_IS_SHADOW(MEM_TO_SHADOW(addr)));
>
> CHECK(MEM_IS_ORIGIN(MEM_TO_ORIGIN(addr)));
>
> CHECK_EQ(MEM_TO_ORIGIN(addr), SHADOW_TO_ORIGIN(MEM_TO_SHADOW(addr)));
>
>
>
> - lib/sanitizer_common/sanitizer_platform_interceptors.h:
>
> --- a/lib/sanitizer_common/sanitizer_platform_interceptors.h
>
> +++ b/lib/sanitizer_common/sanitizer_platform_interceptors.h
>
> @@ -211,7 +211,7 @@
>
> SI_FREEBSD || SI_MAC || SI_LINUX_NOT_ANDROID
>
> #define SANITIZER_INTERCEPT_ETHER_R SI_FREEBSD || SI_LINUX_NOT_ANDROID
>
> #define SANITIZER_INTERCEPT_SHMCTL \
>
> - ((SI_FREEBSD || SI_LINUX_NOT_ANDROID) && SANITIZER_WORDSIZE == 64)
>
> + ((SI_FREEBSD || SI_LINUX_NOT_ANDROID) && (SANITIZER_WORDSIZE == 64 ||
> defined(__i386__)))
>
> #define SANITIZER_INTERCEPT_RANDOM_R SI_LINUX_NOT_ANDROID
>
> #define SANITIZER_INTERCEPT_PTHREAD_ATTR_GET SI_NOT_WINDOWS
>
> #define SANITIZER_INTERCEPT_PTHREAD_ATTR_GETINHERITSCHED \
>
>
>
> Stefan Pusl
>
>
>
>
>
> Intel Deutschland GmbH
> Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany
> Tel: +49 89 99 8853-0, www.intel.de
> Managing Directors: Christin Eisenschmid, Christian Lamprechter
> Chairperson of the Supervisory Board: Nicole Lau
> Registered Office: Munich
> Commercial Register: Amtsgericht Muenchen HRB 186928
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>
> Intel Deutschland GmbH
> Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany
> Tel: +49 89 99 8853-0, www.intel.de
> Managing Directors: Christin Eisenschmid, Christian Lamprechter
> Chairperson of the Supervisory Board: Nicole Lau
> Registered Office: Munich
> Commercial Register: Amtsgericht Muenchen HRB 186928
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
More information about the llvm-dev
mailing list