FYI patch: using write to output the file

Sean Silva via llvm-commits llvm-commits at lists.llvm.org
Tue Mar 8 15:36:04 PST 2016


On Tue, Mar 8, 2016 at 3:24 PM, Rui Ueyama via llvm-commits <
llvm-commits at lists.llvm.org> wrote:

> I don't know if it could theoretically make it faster. I think that if
> your write to a mmap'ed buffer actually have to hit a disk, that access
> blocks the thread, but as we only write to a mmap'ed area, so it virtually
> never blocks for our case, no?
>

It will block in at least two situations:

- when faulting in a page for the first time, the kernel will at least have
to zero it and create page table entries. Thankfully this doesn't hit disk
(assuming plenty of memory), but does require the kernel to dig up a zeroed
page or zero a free page and twiddle page tables; this happens under the
address space lock IIRC so it may present a bottleneck if we need to write
in parallel. This could be substantially mitigated with large pages, but we
cannot rely on those across platforms.

- when closing the file, on Unix the close() will block waiting for the
file to be written to disk. And in fact the kernel likely will only start
writing to disk when the close() is called (I have observed this in test
cases). With write, the kernel has already started the I/O operation by the
time the write returns. Hence write() will likely result in better
pipelining of I/O with the CPU effort.

-- Sean Silva




> On Tue, Mar 8, 2016 at 3:15 PM, Rafael EspĂ­ndola <
> rafael.espindola at gmail.com> wrote:
>
>> The attached patch changes lld to use posix_memalign + write instead of
>> mmap.
>>
>> When linking "small" binaries like clang there is almost no
>> difference. When linking scylladb I get a 1.019X slowdown when writing
>> to tmpfs and a 1.040 speedup when writing to btrfs.
>>
>> At this stage I would say this just shows there is potential, but we
>> have to at least benchmark this on a few filesystems.
>>
>> It would also be interesting to try doing one write for each section.
>>
>> Cheers,
>> Rafael
>>
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160308/322f9a82/attachment.html>


More information about the llvm-commits mailing list