[RFC] Using large pages for large hash tables

[Rafael Espíndola via llvm-commits]

I did a quick and dirty experiment to use large pages when a hash
table gets big. The results for lld are pretty impressive (see
attached file, but basically 1.04X faster link of files with debug
info).

I tested disabling madvise and the performance goes back to what it
was, so it is really the large pages that improves the performance.

The main question is then what the interface should look like. On
linux the abstraction could be

std::pair<void *, size_t> mallocLarge(size_t Size);

which return the allocated memory and how much was actually allocated.
The pointer can be passed to free once it is no longer needed.

The fallback implementation just calls malloc and returns Size unmodified.

On linux x86_64 if size is larger than 2MiB we use posix_memalign and madvise.

Would the same interface work on windows?

Cheers,
Rafael
-------------- next part --------------
diff --git a/include/llvm/ADT/DenseMap.h b/include/llvm/ADT/DenseMap.h
index 1eb9abc..6931995 100644
--- a/include/llvm/ADT/DenseMap.h
+++ b/include/llvm/ADT/DenseMap.h
@@ -27,6 +27,7 @@
 #include <iterator>
 #include <limits>
 #include <new>
+#include <sys/mman.h>
 #include <utility>
 
 namespace llvm {
@@ -726,7 +727,21 @@ private:
       return false;
     }
 
-    Buckets = static_cast<BucketT*>(operator new(sizeof(BucketT) * NumBuckets));
+    const size_t TwoM = 2 * 1024 * 1024;
+    size_t Alloc = sizeof(BucketT) * NumBuckets;
+    if (Alloc < TwoM) {
+      Buckets = static_cast<BucketT *>(operator new(Alloc));
+      return true;
+    }
+
+    Alloc = alignTo(Alloc, TwoM);
+    NumBuckets = PowerOf2Floor(Alloc / sizeof(BucketT));
+    int R = posix_memalign((void **)&Buckets, TwoM, Alloc);
+    assert(R == 0);
+    (void)R;
+    R = madvise(Buckets, Alloc, MADV_HUGEPAGE);
+    assert(R == 0);
+    (void)R;
     return true;
   }
 };
-------------- next part --------------
A non-text attachment was scrubbed...
Name: speed
Type: application/octet-stream
Size: 749 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20161017/bfbc1dae/attachment.obj>