[PATCH] D94267: [PDB] Defer relocating .debug$S until commit time and parallelize it

[Reid Kleckner via Phabricator via llvm-commits]

rnk created this revision.
rnk added reviewers: aganea, thakis.
Herald added subscribers: mgrang, hiraditya.
rnk requested review of this revision.
Herald added a project: LLVM.

This is a pretty classic optimization. Instead of processing symbol
records and copying them to temporary storage, do a first pass to
measure how large the module symbol stream will be, and then copy the
data into place in the PDB file. This requires defering relocation until
much later, which accounts for most of the complexity in this patch.

This patch avoids copying the contents of all live .debug$S sections
into heap memory, which is worth about 20% of private memory usage when
making PDBs. However, this is not an unmitigated performance win,
because it can be faster to read dense, temporary, heap data than it is
to iterate symbol records in object file backed memory a second time.

Results on release chrome.dll:
peak mem: 5164.89MB -> 4072.19MB (-1,092.7MB, -21.2%)
wall-j1:  0m30.844s -> 0m32.094s (slightly slower)
wall-j3:  0m20.968s -> 0m20.312s (slightly faster)
wall-j8:  0m19.062s -> 0m17.672s (meaningfully faster)

I gathered similar numbers for a debug, component build of content.dll
in Chrome, and the performance impact of this change was in the noise.
The memory usage reduction was visible and similar.

Because of the new parallelism in the PDB commit phase, more cores makes
the new approach faster. I'm assuming that most C++ developer machines
these days are at least quad core, so I think this is a win.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D94267

Files:
  lld/COFF/Chunks.cpp
  lld/COFF/Chunks.h
  lld/COFF/PDB.cpp
  llvm/include/llvm/DebugInfo/PDB/Native/DbiModuleDescriptorBuilder.h
  llvm/lib/DebugInfo/PDB/Native/DbiModuleDescriptorBuilder.cpp
  llvm/lib/DebugInfo/PDB/Native/DbiStreamBuilder.cpp

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D94267.315243.patch
Type: text/x-patch
Size: 52766 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20210107/5d922589/attachment-0001.bin>