[llvm-dev] Attempt to modify memory sanitizer for support of X86

Pusl, Stefan via llvm-dev llvm-dev at lists.llvm.org
Tue Jun 21 09:48:16 PDT 2016


Hello,

thanks for your comments.

I know, that there is not so much address space in case of 32-bit addressing, but for a lot of applications it is enough. I used address sanitizer with 32-bit addressing and due to the good performance I wanted to use memory sanitizer with 32-bit addressing.

Instead of looking at /proc/<PID>/maps I changed msan.cc to print out the process mapping every time:

diff --git a/lib/msan/msan.cc b/lib/msan/msan.cc
index d2981f0..97f97c4 100644
--- a/lib/msan/msan.cc
+++ b/lib/msan/msan.cc
@@ -408,6 +408,8 @@ void __msan_init() {
     Die();
   }
+  DumpProcessMap();
+
   Symbolizer::GetOrInit()->AddHooks(EnterSymbolizer, ExitSymbolizer);
   InitializeCoverage(common_flags()->coverage, common_flags()->coverage_dir);

When I now run my example, then I get the output:

==17434==Process memory map follows:
        0x10000000-0x20000000
        0x20000000-0x50000000
        0x5659b000-0x5666a000   /home/pusl/code/example32.out
        0x5666a000-0x5666b000   /home/pusl/code/example32.out
        0x5666b000-0x5666e000   /home/pusl/code/example32.out
        0x5666e000-0x576c6000
        0x60000000-0x70000000
        0x70000000-0x80000000
        0x80000000-0x90000000
        0x90000000-0xf0000000
        0xf735a000-0xf7506000
        0xf7506000-0xf76b3000   /lib32/libc-2.23.so
        0xf76b3000-0xf76b5000   /lib32/libc-2.23.so
        0xf76b5000-0xf76b6000   /lib32/libc-2.23.so
        0xf76b6000-0xf76b9000
        0xf76b9000-0xf76d5000   /usr/lib32/libgcc_s.so.1
        0xf76d5000-0xf76d6000   /usr/lib32/libgcc_s.so.1
        0xf76d6000-0xf76d9000   /lib32/libdl-2.23.so
        0xf76d9000-0xf76da000   /lib32/libdl-2.23.so
        0xf76da000-0xf76db000   /lib32/libdl-2.23.so
        0xf76db000-0xf772e000   /lib32/libm-2.23.so
        0xf772e000-0xf772f000   /lib32/libm-2.23.so
        0xf772f000-0xf7730000   /lib32/libm-2.23.so
        0xf7730000-0xf7737000   /lib32/librt-2.23.so
        0xf7737000-0xf7738000   /lib32/librt-2.23.so
        0xf7738000-0xf7739000   /lib32/librt-2.23.so
        0xf7739000-0xf7752000   /lib32/libpthread-2.23.so
        0xf7752000-0xf7753000   /lib32/libpthread-2.23.so
        0xf7753000-0xf7754000   /lib32/libpthread-2.23.so
        0xf7754000-0xf7757000
        0xf7766000-0xf776e000
        0xf776e000-0xf7770000   [vvar]
        0xf7770000-0xf7771000   [vdso]
        0xf7771000-0xf7793000   /lib32/ld-2.23.so
        0xf7793000-0xf7794000
        0xf7794000-0xf7795000   /lib32/ld-2.23.so
        0xf7795000-0xf7796000   /lib32/ld-2.23.so
        0xffa83000-0xffaa4000   [stack]
==17434==End of process memory map.
Speicherzugriffsfehler (Speicherabzug geschrieben)

When I look at the mapping, then the given code is always located in an app-region, given by kMemoryLayout. So I have misunderstand something completely or I oversee something.

Any comments are welcome.

Stefan



From: Kostya Serebryany [mailto:kcc at google.com]
Sent: Friday, June 17, 2016 7:13 PM
To: Pusl, Stefan <stefan.pusl at intel.com>; Evgenii Stepanov <eugenis at google.com>
Cc: llvm-dev at lists.llvm.org
Subject: Re: [llvm-dev] Attempt to modify memory sanitizer for support of X86

+eugenis (the msan co-author and current maintainer)

I don't remember any hard blockers for 32-bit msan.
Soft blockers are:
  * extra work and extra maintenance
  * too little address space, hard to fit any non-trivial application.

You may start looking at /proc/<PID>/maps to see what is the memory region you
are failing on and how it is related to kMemoryLayout.

Just curious, why are you trying to do this?

--kcc

On Fri, Jun 17, 2016 at 8:02 AM, Pusl, Stefan via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
Hello,

I’m quite new to LLVM, but I’m interested in memory sanitizer. The petty thing is, that 32bit addressing on Linux is not supported. Thus I tried to take the latest version of the software and to modify it, using Ubuntu 16.04 as operating system. Given the example

example.c:
#include <stdio.h>
#include <stdlib.h>

int main(int argc, char **argv) {
   int *a = (int *)malloc(10*sizeof(10));
   a[5] = 0;
   if (a[argc])
      printf("xx\n");
   return 0;
}

which I compiled with the command

clang -fsanitize=memory -m32 -fPIE -pie -fno-omit-frame-pointer -g -Wl,-Map,example32.map example.c -oexample32.out

I get now a segmentation fault during the execution:

(gdb) r
Starting program: /home/pusl/code/example32.out
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x565e14f9 in main (argc=1448445640, argv=0xf7f1d3dc) at example.c:4


I tried to figure out the issue, but as a newcomer to LLVM I don’t have any further idea now to get nearer to the root cause. Any new idea is welcome.

I changed 5 files. The diffs are:


-          cmake/config-ix.cmake:
diff --git a/cmake/config-ix.cmake b/cmake/config-ix.cmake
index 04e33a0..d02e319 100644
--- a/cmake/config-ix.cmake
+++ b/cmake/config-ix.cmake
@@ -149,7 +149,7 @@ set(ALL_ASAN_SUPPORTED_ARCH ${X86} ${X86_64} ${ARM32} ${ARM64}
     ${MIPS32} ${MIPS64} ${PPC64})
set(ALL_DFSAN_SUPPORTED_ARCH ${X86_64} ${MIPS64} ${ARM64})
set(ALL_LSAN_SUPPORTED_ARCH ${X86_64} ${MIPS64} ${ARM64})
-set(ALL_MSAN_SUPPORTED_ARCH ${X86_64} ${MIPS64} ${ARM64})
+set(ALL_MSAN_SUPPORTED_ARCH ${X86} ${X86_64} ${MIPS64} ${ARM64})
set(ALL_PROFILE_SUPPORTED_ARCH ${X86} ${X86_64} ${ARM32} ${ARM64} ${PPC64}
     ${MIPS32} ${MIPS64})
set(ALL_TSAN_SUPPORTED_ARCH ${X86_64} ${MIPS64} ${ARM64} ${PPC64})


-          lib/msan/msan.h:
diff --git a/lib/msan/msan.h b/lib/msan/msan.h
index be74f6e..10b5b05 100644
--- a/lib/msan/msan.h
+++ b/lib/msan/msan.h
@@ -147,6 +147,26 @@ const MappingDesc kMemoryLayout[] = {
#define MEM_TO_SHADOW(mem) (LINEARIZE_MEM((mem)) + 0x100000000000ULL)
#define SHADOW_TO_ORIGIN(shadow) (((uptr)(shadow)) + 0x280000000000)
+#elif SANITIZER_LINUX && SANITIZER_WORDSIZE == 32
+
+const MappingDesc kMemoryLayout[] = {
+    {0x00000000ULL, 0x10000000ULL, MappingDesc::APP, "app-1"},
+    {0x10000000ULL, 0x20000000ULL, MappingDesc::SHADOW, "shadow-1"},
+    {0x20000000ULL, 0x30000000ULL, MappingDesc::ORIGIN, "origin-1"},
+    {0x30000000ULL, 0x50000000ULL, MappingDesc::INVALID, "invalid"},
+    {0x50000000ULL, 0x60000000ULL, MappingDesc::APP, "app-2"},
+    {0x60000000ULL, 0x70000000ULL, MappingDesc::SHADOW, "shadow-2"},
+    {0x70000000ULL, 0x80000000ULL, MappingDesc::ORIGIN, "origin-2"},
+    {0x80000000ULL, 0x90000000ULL, MappingDesc::SHADOW, "shadow-3"},
+    {0x90000000ULL, 0xa0000000ULL, MappingDesc::ORIGIN, "origin-3"},
+    {0xa0000000ULL, 0xf0000000ULL, MappingDesc::INVALID, "invalid"},
+    {0xf0000000ULL, 0xffffffffULL, MappingDesc::APP, "app-3"},
+};
+#define MOVE_MEM(mem) \
+  ((uptr)(mem) & ~0x80000000ULL)
+#define MEM_TO_SHADOW(mem) (MOVE_MEM((mem)) + 0x10000000ULL)
+#define SHADOW_TO_ORIGIN(mem) (((uptr)(mem)) + 0x10000000ULL)
+
#elif SANITIZER_LINUX && SANITIZER_WORDSIZE == 64
 #ifdef MSAN_LINUX_X86_64_OLD_MAPPING


-          lib/msan/msan_allocator.cc:
diff --git a/lib/msan/msan_allocator.cc b/lib/msan/msan_allocator.cc
index b7d3947..dae6d72 100644
--- a/lib/msan/msan_allocator.cc
+++ b/lib/msan/msan_allocator.cc
@@ -49,7 +49,15 @@ struct MsanMapUnmapCallback {
   typedef SizeClassAllocator32<0, SANITIZER_MMAP_RANGE_SIZE, sizeof(Metadata),
                                SizeClassMap, kRegionSizeLog, ByteMap,
                                MsanMapUnmapCallback> PrimaryAllocator;
-
+#elif defined (__i386__)
+static const uptr kMaxAllowedMallocSize = 3UL << 30;
+  static const uptr kRegionSizeLog = 20;
+  static const uptr kNumRegions = SANITIZER_MMAP_RANGE_SIZE >> kRegionSizeLog;
+  typedef FlatByteMap<kNumRegions> ByteMap;
+  typedef CompactSizeClassMap SizeClassMap;
+  typedef SizeClassAllocator32<0, SANITIZER_MMAP_RANGE_SIZE, sizeof(Metadata),
+                               SizeClassMap, kRegionSizeLog, ByteMap,
+                               MsanMapUnmapCallback> PrimaryAllocator;
#elif defined(__x86_64__)
#if SANITIZER_LINUX && !defined(MSAN_LINUX_X86_64_OLD_MAPPING)
   static const uptr kAllocatorSpace = 0x700000000000ULL;


-          lib/msan/msan_linux.cc:
diff --git a/lib/msan/msan_linux.cc b/lib/msan/msan_linux.cc
index d6a9588..ca59564 100644
--- a/lib/msan/msan_linux.cc
+++ b/lib/msan/msan_linux.cc
@@ -82,7 +82,7 @@ static void CheckMemoryLayoutSanity() {
     CHECK_LT(start, end);
     CHECK_EQ(prev_end, start);
     CHECK(addr_is_type(start, type));
-    CHECK(addr_is_type((start + end) / 2, type));
+    CHECK(addr_is_type(start/2 + end/2, type));
     CHECK(addr_is_type(end - 1, type));
     if (type == MappingDesc::APP) {
       uptr addr = start;
@@ -90,7 +90,7 @@ static void CheckMemoryLayoutSanity() {
       CHECK(MEM_IS_ORIGIN(MEM_TO_ORIGIN(addr)));
       CHECK_EQ(MEM_TO_ORIGIN(addr), SHADOW_TO_ORIGIN(MEM_TO_SHADOW(addr)));
-      addr = (start + end) / 2;
+      addr = start/2 + end/2;
       CHECK(MEM_IS_SHADOW(MEM_TO_SHADOW(addr)));
       CHECK(MEM_IS_ORIGIN(MEM_TO_ORIGIN(addr)));
       CHECK_EQ(MEM_TO_ORIGIN(addr), SHADOW_TO_ORIGIN(MEM_TO_SHADOW(addr)));


-          lib/sanitizer_common/sanitizer_platform_interceptors.h:
--- a/lib/sanitizer_common/sanitizer_platform_interceptors.h
+++ b/lib/sanitizer_common/sanitizer_platform_interceptors.h
@@ -211,7 +211,7 @@
   SI_FREEBSD || SI_MAC || SI_LINUX_NOT_ANDROID
#define SANITIZER_INTERCEPT_ETHER_R SI_FREEBSD || SI_LINUX_NOT_ANDROID
#define SANITIZER_INTERCEPT_SHMCTL \
-  ((SI_FREEBSD || SI_LINUX_NOT_ANDROID) && SANITIZER_WORDSIZE == 64)
+  ((SI_FREEBSD || SI_LINUX_NOT_ANDROID) && (SANITIZER_WORDSIZE == 64 || defined(__i386__)))
#define SANITIZER_INTERCEPT_RANDOM_R SI_LINUX_NOT_ANDROID
#define SANITIZER_INTERCEPT_PTHREAD_ATTR_GET SI_NOT_WINDOWS
#define SANITIZER_INTERCEPT_PTHREAD_ATTR_GETINHERITSCHED \

Stefan Pusl



Intel Deutschland GmbH
Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de<http://www.intel.de>
Managing Directors: Christin Eisenschmid, Christian Lamprechter
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928

_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Intel Deutschland GmbH
Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de
Managing Directors: Christin Eisenschmid, Christian Lamprechter
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160621/26d56a93/attachment-0001.html>


More information about the llvm-dev mailing list