[compiler-rt] 5a1525c - [scudo][standalone] Precommit pages

[Chia-hung Duan via llvm-commits]

Author: Yaneury Fermin
Date: 2022-12-20T17:37:49Z
New Revision: 5a1525c57635f3633e069c23f20ca12fe92ecced

URL: https://github.com/llvm/llvm-project/commit/5a1525c57635f3633e069c23f20ca12fe92ecced
DIFF: https://github.com/llvm/llvm-project/commit/5a1525c57635f3633e069c23f20ca12fe92ecced.diff

LOG: [scudo][standalone] Precommit pages

On Fuchsia, this CL changes garbage collection
to precommit all pages if the |Buffer| doesn't
fit into the static buffer size.

A test program (scudotest) was used that deliberately
grows a size class high water mark to the point where
the pre-allocated static buffer is no longer used for
garbage collection.

Traces showed that precommiting the Vmar removes ~30 page faults
and ~.22ms of wall time.*

Before: https://ui.perfetto.dev/#!/?s=7da19fc3f59448eef51fd6fd03283bb87b702cf1a565bcbe6c9c28371671
After: https://ui.perfetto.dev/#!/?s=97707cd99b2c9efd1e6569b2deb97e3d16f8be532c59a0cc12463c37fbb1d8

*: Use the added `zx_vmar_op_range` as a reference point to observe
the differences.

For more context, see https://fxbug.dev/115594.

Reviewed By: Chia-hungDuan

Differential Revision: https://reviews.llvm.org/D140320

Added: 
    

Modified: 
    compiler-rt/lib/scudo/standalone/common.h
    compiler-rt/lib/scudo/standalone/fuchsia.cpp
    compiler-rt/lib/scudo/standalone/release.h

Removed: 
    


################################################################################
diff  --git a/compiler-rt/lib/scudo/standalone/common.h b/compiler-rt/lib/scudo/standalone/common.h
index d03ffd8d4fede..601481671d71d 100644
--- a/compiler-rt/lib/scudo/standalone/common.h
+++ b/compiler-rt/lib/scudo/standalone/common.h
@@ -149,6 +149,7 @@ bool getRandom(void *Buffer, uptr Length, bool Blocking = false);
 #define MAP_NOACCESS (1U << 1)
 #define MAP_RESIZABLE (1U << 2)
 #define MAP_MEMTAG (1U << 3)
+#define MAP_PRECOMMIT (1U << 4)
 
 // Our platform memory mapping use is restricted to 3 scenarios:
 // - reserve memory at a random address (MAP_NOACCESS);

diff  --git a/compiler-rt/lib/scudo/standalone/fuchsia.cpp b/compiler-rt/lib/scudo/standalone/fuchsia.cpp
index 4ce3b4ce6d181..70e4e714f2cb9 100644
--- a/compiler-rt/lib/scudo/standalone/fuchsia.cpp
+++ b/compiler-rt/lib/scudo/standalone/fuchsia.cpp
@@ -97,6 +97,17 @@ void *map(void *Addr, uptr Size, const char *Name, uptr Flags,
   if (Offset)
     MapFlags |= ZX_VM_SPECIFIC;
   Status = _zx_vmar_map(Vmar, MapFlags, Offset, Vmo, VmoSize, Size, &P);
+  if (UNLIKELY(Status != ZX_OK)) {
+    if (Status != ZX_ERR_NO_MEMORY || !AllowNoMem)
+      dieOnMapUnmapError(Status == ZX_ERR_NO_MEMORY ? Size : 0);
+    return nullptr;
+  }
+
+  if (Flags & MAP_PRECOMMIT) {
+    Status = _zx_vmar_op_range(Vmar, ZX_VMAR_OP_COMMIT, P, Size,
+                               /*buffer=*/nullptr, /*buffer_size=*/0);
+  }
+
   // No need to track the Vmo if we don't intend on resizing it. Close it.
   if (Flags & MAP_RESIZABLE) {
     DCHECK(Data);
@@ -112,6 +123,7 @@ void *map(void *Addr, uptr Size, const char *Name, uptr Flags,
       dieOnMapUnmapError(Status == ZX_ERR_NO_MEMORY ? Size : 0);
     return nullptr;
   }
+
   if (Data)
     Data->VmoSize += Size;
 

diff  --git a/compiler-rt/lib/scudo/standalone/release.h b/compiler-rt/lib/scudo/standalone/release.h
index ef5f23208dff4..6de3b15534d0c 100644
--- a/compiler-rt/lib/scudo/standalone/release.h
+++ b/compiler-rt/lib/scudo/standalone/release.h
@@ -107,9 +107,15 @@ class RegionPageMap {
       Buffer = &StaticBuffer[0];
       memset(Buffer, 0, BufferSize);
     } else {
+      // When using a heap-based buffer, precommit the pages backing the
+      // Vmar by passing |MAP_PRECOMMIT| flag. This allows an optimization
+      // where page fault exceptions are skipped as the allocated memory
+      // is accessed.
+      const uptr MmapFlags =
+          MAP_ALLOWNOMEM | (SCUDO_FUCHSIA ? MAP_PRECOMMIT : 0);
       Buffer = reinterpret_cast<uptr *>(
           map(nullptr, roundUpTo(BufferSize, getPageSizeCached()),
-              "scudo:counters", MAP_ALLOWNOMEM, &MapData));
+              "scudo:counters", MmapFlags, &MapData));
     }
   }