Benchmarking file output strategies

Rafael EspĂ­ndola rafael.espindola at gmail.com
Mon Dec 15 12:53:58 PST 2014


On 13 December 2014 at 17:03, Reid Kleckner <rnk at google.com> wrote:
> My best theory is that mmap has to map zero pages first. Typically
> everything is mapped to one zero page, and then every new page write causes
> a COW page fault. That might explain Sean's observations of lots of
> functions that execute #bytes/4096 times.
>
> I wonder if there are some tweaks to the mmap path that would help, like
> only mapping 10MB of output buffer per file and then remapping it to a +10MB
> offset. There are probably also things like MAP_POPULATE or explicitly
> asking for a few huge pages that might help.

One of the advantages of the mmap solution is the convenience of
having it all mapped at once and not having to worry about seeking. If
we only mmap a chunk at a time part of that advantage goes away.

We could have an interface that has seek/write and is implemented on
windows with mmap and on posix with actual seeks and writes. I might
give that a try, but first:

Sean point on the write code path got me thinking that maybe it is the
crt that is slow. I will see If can change the test to use WriteFile
directly and if that helps.

Cheers,
Rafael



More information about the llvm-commits mailing list