[PATCH] D56757: [hwasan] Madvise away unused shadow.

[Evgenii Stepanov via Phabricator via llvm-commits]

eugenis created this revision.
eugenis added reviewers: kcc, pcc.
Herald added subscribers: kubamracek, srhines.

Whenever a large shadow region is tagged to zero, madvise(DONT_NEED)
as much of it as possible.

This reduces shadow RSS on Android by 45% or so, and total memory use
by 2-4%, probably even more on long running multithreaded programs.
CPU time seems to be in the noise.


https://reviews.llvm.org/D56757

Files:
  compiler-rt/lib/hwasan/hwasan.cc
  compiler-rt/lib/hwasan/hwasan_poisoning.cc


Index: compiler-rt/lib/hwasan/hwasan_poisoning.cc
===================================================================
--- compiler-rt/lib/hwasan/hwasan_poisoning.cc
+++ compiler-rt/lib/hwasan/hwasan_poisoning.cc
@@ -24,7 +24,24 @@
   CHECK(IsAligned(size, kShadowAlignment));
   uptr shadow_start = MemToShadow(p);
   uptr shadow_size = MemToShadowSize(size);
-  internal_memset((void *)shadow_start, tag, shadow_size);
+
+  uptr page_size = GetPageSizeCached();
+  uptr page_start = RoundUpTo(shadow_start, page_size);
+  uptr page_end = RoundDownTo(shadow_start + shadow_size, page_size);
+  uptr threshold = common_flags()->clear_shadow_mmap_threshold;
+  if (SANITIZER_LINUX &&
+      UNLIKELY(page_end >= page_start + threshold && tag == 0)) {
+    if (page_start > shadow_start)
+      internal_memset((void *)shadow_start, tag, page_start - shadow_start);
+    if (shadow_start + shadow_size > page_end)
+      internal_memset((void *)page_end, tag,
+                      shadow_start + shadow_size - page_end);
+    // For an anonymous private mapping MADV_DONTNEED will return a zero page on
+    // Linux.
+    ReleaseMemoryPagesToOS(page_start, page_end);
+  } else {
+    internal_memset((void *)shadow_start, tag, shadow_size);
+  }
   return AddTagToPointer(p, tag);
 }
 
Index: compiler-rt/lib/hwasan/hwasan.cc
===================================================================
--- compiler-rt/lib/hwasan/hwasan.cc
+++ compiler-rt/lib/hwasan/hwasan.cc
@@ -88,6 +88,8 @@
     cf.check_printf = false;
     cf.intercept_tls_get_addr = true;
     cf.exitcode = 99;
+    // 8 shadow pages ~~ 512kB, small enough to cover common stack sizes.
+    cf.clear_shadow_mmap_threshold = 4096 * (SANITIZER_ANDROID ? 2 : 8);
     // Sigtrap is used in error reporting.
     cf.handle_sigtrap = kHandleSignalExclusive;
 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D56757.181919.patch
Type: text/x-patch
Size: 1823 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20190116/eef89292/attachment.bin>