[llvm] Tweak BumpPtrAllocator to benefit the hot path (PR #90571)
via llvm-commits
llvm-commits at lists.llvm.org
Tue Apr 30 01:03:30 PDT 2024
llvmbot wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-llvm-support
Author: Owen Anderson (resistor)
<details>
<summary>Changes</summary>
This takes the form of three consecutive but related changes:
- Mark the fast path of BumpPtrAllocator as likely-taken.
- Move the slow path of BumpPtrAllocator to a separate function.
- Mark the slow path of BumpPtrAllocator as noinline.
Overall, this saves geomean 0.4% userspace instructions on CTMark -O3,
and 0.98% on CTMark -O0 -g.
http://llvm-compile-time-tracker.com/compare.php?from=e1622e189e8c0ef457bfac528f90a7a930d9aad2&to=9eb53a4ed3af4a55e769ae1dd22d034b63d046e3&stat=instructions%3Au
---
Full diff: https://github.com/llvm/llvm-project/pull/90571.diff
1 Files Affected:
- (modified) llvm/include/llvm/Support/Allocator.h (+8-3)
``````````diff
diff --git a/llvm/include/llvm/Support/Allocator.h b/llvm/include/llvm/Support/Allocator.h
index c1e5c6d2853bd5..873d453650187e 100644
--- a/llvm/include/llvm/Support/Allocator.h
+++ b/llvm/include/llvm/Support/Allocator.h
@@ -159,9 +159,9 @@ class BumpPtrAllocatorImpl
#endif
// Check if we have enough space.
- if (Adjustment + SizeToAllocate <= size_t(End - CurPtr)
- // We can't return nullptr even for a zero-sized allocation!
- && CurPtr != nullptr) {
+ if (LLVM_LIKELY(Adjustment + SizeToAllocate <= size_t(End - CurPtr)
+ // We can't return nullptr even for a zero-sized allocation!
+ && CurPtr != nullptr)) {
char *AlignedPtr = CurPtr + Adjustment;
CurPtr = AlignedPtr + SizeToAllocate;
// Update the allocation point of this memory block in MemorySanitizer.
@@ -173,6 +173,11 @@ class BumpPtrAllocatorImpl
return AlignedPtr;
}
+ return AllocateSlow(SizeToAllocate, Alignment);
+ }
+
+ LLVM_ATTRIBUTE_RETURNS_NONNULL LLVM_ATTRIBUTE_NOINLINE void *
+ AllocateSlow(size_t SizeToAllocate, Align Alignment) {
// If Size is really big, allocate a separate slab for it.
size_t PaddedSize = SizeToAllocate + Alignment.value() - 1;
if (PaddedSize > SizeThreshold) {
``````````
</details>
https://github.com/llvm/llvm-project/pull/90571
More information about the llvm-commits
mailing list