[llvm-dev] LLD: Using sendfile(2) to copy file contents

[Rui Ueyama via llvm-dev]

This is a short summary of an experiment that I did for the linker.

One of the major tasks of the linker is to copy file contents from input
object files to an output file. I was wondering what's the fastest way to
copy data from one file to another, so I conducted an experiment.

Currently, LLD copies file contents using memcpy (input files and an output
file are mapped to memory.) mmap+memcpy is not known as the fastest way to
copy file contents.

Linux has sendfile system call. The system call takes two file descriptors
and copies contents from one to another (it used to take only a socket as a
destination, but these days it can take any file.) It is usually much
faster than memcpy to copy files. For example, it is about 3x faster than
cp command to copy large files on my machine (on SSD/ext4).

I made a change to LLVM and LLD to use sendfile instead of memcpy to copy
section contents. Here's the time to link clang with debug info.

    memcpy: 12.96 seconds
    sendfile: 12.82 seconds

sendfile(2) was slightly faster but not that much. But if you disable
string merging (by passing -O0 parameter to the linker), the difference
becomes noticeable.

    memcpy: 7.85 seconds
    sendfile: 6.94 seconds

I think it is because, with -O0, the linker has to copy more contents than
without -O0. It creates 2x larger executable than without -O0. As the
amount of data the linker needs to copy gets larger, sendfile gets more
effective.

By the way, gold takes 27.05 seconds to link it.

With the results, I'm *not* going to submit that change. There are two
reasons. First, the optimization seems too system-specific, and I'm not yet
sure if it's always effective even on Linux. Second, the current
implementations of MemoryBuffer and FileOutputBuffer are not
sendfile(2)-friendly because they close file descriptors immediately after
mapping them to memory. My patch is too hacky to submit.

Being said that, the results clearly show that there's room for future
optimization. I think we want to revisit it when we want to do a low-level
optimization on link speed.

Rui
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160605/811a604f/attachment.html>