[llvm-dev] LLD: Using sendfile(2) to copy file contents

[Davide Italiano via llvm-dev]

On Sun, Jun 5, 2016 at 1:19 PM, Rui Ueyama via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> This is a short summary of an experiment that I did for the linker.
>
> One of the major tasks of the linker is to copy file contents from input
> object files to an output file. I was wondering what's the fastest way to
> copy data from one file to another, so I conducted an experiment.
>
> Currently, LLD copies file contents using memcpy (input files and an output
> file are mapped to memory.) mmap+memcpy is not known as the fastest way to
> copy file contents.
>
> Linux has sendfile system call. The system call takes two file descriptors
> and copies contents from one to another (it used to take only a socket as a
> destination, but these days it can take any file.) It is usually much faster
> than memcpy to copy files. For example, it is about 3x faster than cp
> command to copy large files on my machine (on SSD/ext4).
>
> I made a change to LLVM and LLD to use sendfile instead of memcpy to copy
> section contents. Here's the time to link clang with debug info.
>
>     memcpy: 12.96 seconds
>     sendfile: 12.82 seconds
>
> sendfile(2) was slightly faster but not that much. But if you disable string
> merging (by passing -O0 parameter to the linker), the difference becomes
> noticeable.
>
>     memcpy: 7.85 seconds
>     sendfile: 6.94 seconds
>
> I think it is because, with -O0, the linker has to copy more contents than
> without -O0. It creates 2x larger executable than without -O0. As the amount
> of data the linker needs to copy gets larger, sendfile gets more effective.
>
> By the way, gold takes 27.05 seconds to link it.
>

With or without string merging?

Thanks,

-- 
Davide

"There are no solved problems; there are only problems that are more
or less solved" -- Henri Poincare