[LLVMdev] MmapAllocator

[Török Edwin]

On Mon, 9 Aug 2010 10:17:27 -0700
Jakob Stoklund Olesen <stoklund at 2pi.dk> wrote:

> 
> On Aug 9, 2010, at 9:54 AM, Török Edwin wrote:
> 
> > With mmap() it is always possible to fully release the memory once
> > you are done using it.
> 
> Sure. Is that the goal, though? 

If goal is to reduce fragmentation, possibly. You
don't know if you have fragmentation or not, the JITed app may fragment
memory for example.

> Why isn't malloc doing it already?

Because it can't. sbrk() can only increase/decrease memory usage at the
end (like a stack), you can't release something in the middle.
Thats one of the reasons why we wrote a pool-based memory allocator for
ClamAV.

> 
> > With malloc() no, it takes just 1 allocation at the end of the heap
> > to keep all the rest allocated. That wouldn't be a problem if libc
> > would use mmap() as the low-level allocator for malloc but it
> > doesn't. It uses sbrk() mostly for small (<128k) allocations, and
> > even with mmaps it caches them for a while.
> 
> Recommended reading:
> http://people.freebsd.org/~jasone/jemalloc/bsdcan2006/jemalloc.pdf

If jemalloc provides same or better memory usage than
MMapAllocator, I think it'd be better to have a JEMallocAllocator
instead.
I think jemalloc is fairly portable (firefox uses it), isn't it?

> 
> > I think that is because mmap() is slow in multithreaded apps, since
> > it needs to take a process level lock, which also contends with the
> > lock taken by pagefaults from other existing mmaps (in fact that
> > lock is held during disk I/O!).
> 
> Sounds awesome, let's do that ;-)

Multithreaded performance should probably be benchmarked on a real app
with MMapAllocator, and with the MallocAllocator.

> 
> You are also leaving a bunch of 4K holes in your address space. On
> 32-bit systems, address space is a scarce resource.

Doesn't BumpPtrAllocator use a larger chunk size?

Best regards,
--Edwin