[PATCH] D94267: [PDB] Defer relocating .debug$S until commit time and parallelize it

Reid Kleckner via Phabricator via llvm-commits llvm-commits at lists.llvm.org
Mon Jan 11 17:21:27 PST 2021


rnk added a comment.

In D94267#2491643 <https://reviews.llvm.org/D94267#2491643>, @aganea wrote:

> This is mostly caused by contention when page faulting on either mmap'ed files or zero-pages. KeYieldProcessorEx is spinning while waiting for a lock for bringing pages into the 'working set'.

Interesting, I've heard similar things about LLD ELF. I wonder how much of this is IO and how much of this is locks around modifying the process page directory.

I haven't profiled this out the way you have it here, but I've noticed that LLD's performance is really sensitive to the filesystem cache. If you can fit all input objs into RAM, then LLD runs slow on the first run as all these page faults load all the bits of the objects, and then on the second link, it runs really fast. Zach used to joke that LLD was an incremental linker, it just leverages the FS cache.

Similar to inserting prefetches, I was wondering if there were some APIs we can use to load the obj in phases:

- reserve memory for the entire file
- commit only the portions of the object used for symbol table, section table, and relocations
- resolve symbols, run linker GC
- commit section content memory for sections marked live, do not load memory for non-live sections

This would be much more explicit, similar to an explicit seeks and reads, explicitly getting the data from the FS when you need it.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D94267/new/

https://reviews.llvm.org/D94267



More information about the llvm-commits mailing list