[all-commits] [llvm/llvm-project] 6529d7: [PDB] Defer relocating .debug$S until commit time ...

[Reid Kleckner via All-commits]

  Branch: refs/heads/master
  Home:   https://github.com/llvm/llvm-project
  Commit: 6529d7c5a45b1b9588e512013b02f891d71bc134
      https://github.com/llvm/llvm-project/commit/6529d7c5a45b1b9588e512013b02f891d71bc134
  Author: Reid Kleckner <rnk at google.com>
  Date:   2021-01-12 (Tue, 12 Jan 2021)

  Changed paths:
    M lld/COFF/Chunks.cpp
    M lld/COFF/Chunks.h
    M lld/COFF/PDB.cpp
    M llvm/include/llvm/DebugInfo/PDB/Native/DbiModuleDescriptorBuilder.h
    M llvm/lib/DebugInfo/PDB/Native/DbiModuleDescriptorBuilder.cpp
    M llvm/lib/DebugInfo/PDB/Native/DbiStreamBuilder.cpp

  Log Message:
  -----------
  [PDB] Defer relocating .debug$S until commit time and parallelize it

This is a pretty classic optimization. Instead of processing symbol
records and copying them to temporary storage, do a first pass to
measure how large the module symbol stream will be, and then copy the
data into place in the PDB file. This requires defering relocation until
much later, which accounts for most of the complexity in this patch.

This patch avoids copying the contents of all live .debug$S sections
into heap memory, which is worth about 20% of private memory usage when
making PDBs. However, this is not an unmitigated performance win,
because it can be faster to read dense, temporary, heap data than it is
to iterate symbol records in object file backed memory a second time.

Results on release chrome.dll:
peak mem: 5164.89MB -> 4072.19MB (-1,092.7MB, -21.2%)
wall-j1:  0m30.844s -> 0m32.094s (slightly slower)
wall-j3:  0m20.968s -> 0m20.312s (slightly faster)
wall-j8:  0m19.062s -> 0m17.672s (meaningfully faster)

I gathered similar numbers for a debug, component build of content.dll
in Chrome, and the performance impact of this change was in the noise.
The memory usage reduction was visible and similar.

Because of the new parallelism in the PDB commit phase, more cores makes
the new approach faster. I'm assuming that most C++ developer machines
these days are at least quad core, so I think this is a win.

Differential Revision: https://reviews.llvm.org/D94267