[PATCH] D69294: [LLD][ELF] Support -[no-]mmap-output-file

Tue Oct 22 13:26:26 PDT 2019

terrelln added a comment.

I'll update the description to answer questions, and make the requested changes. I've also answered questions inline below.

> I'm a little sad that BtrFS behaves poorly under a certain condition, and I feel like the filesystem is at fault than the linker

I'd say it is a combination of the linker and BtrFS causing the problem. With compression enabled, BtrFS has to write 128 KB at a time. When there is memory pressure, because the linker uses ~50 GB of memory, the kernel is forced to flush the page cache, so BtrFS is forced to do a write. I suspect you'd see the same problem on any compressed filesystem when under memory pressure. Reducing the memory footprint of the linker would also alleviate the problem.

It is possible the kernel page eviction algorithm could be improved if it knew the "block size" of the filesystem, and I've asked our BtrFS folks if they see a way this case can be improved.

> Do the flushes of random accessed dirty pages make it slow under memory pressure?

Yeah, BtrFS is forced to flush the dirty pages, but it has to write 128 KB at a time. This drastically slows down non-sequential write patterns since the same 128 KB block is written multiple times.

> FileOutputBuffer::F_in_memory

Thats the parent diff D69293 <https://reviews.llvm.org/D69293>

> What is compression is turned off? Have you measured the performance differences with and without --no-mmap-output-file? I think I have asked too much, but if you have access to some ext4/XFS/ZFS machines, the numbers will be useful... CentOS has defaulted to XFS and ext4 is the most widely used fs on Linux.

When compression is disabled I don't see the same problem, and I don't expect to see it on other filesystems (without compression enabled). The reason is we don't see as much write amplification. When compression is disabled flushing a 4 KB page only writes 4 KB. And, the kernel should only flush colder pages, so the write amplification should be minimized. But, maybe the kernel isn't as smart about flushing a cold 4 KB page in a warm 128 KB range.

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D69294/new/

https://reviews.llvm.org/D69294