[PATCH] D117853: [ELF] Parallelize --compress-debug-sections=zlib

[Fangrui Song via Phabricator via llvm-commits]

MaskRay created this revision.
MaskRay added reviewers: alexander-shaposhnikov, bd1976llvm, ikudrin, peter.smith.
Herald added subscribers: arichardson, mgorny, emaste.
MaskRay requested review of this revision.
Herald added a project: LLVM.
Herald added a subscriber: llvm-commits.

Currently debug sections are compressed serially.
This patch splits a section into 2MiB shards and calls zlib `deflake`
parallelly. We use `Z_SYNC_FLUSH` for every non-shard shard and `Z_FINISH` for
the last shard. When linking Debug build clang (265MiB SHF_ALLOC sections,
920MiB debug info), my --threads=8 link is 2.54x as fast.
Because the zlib dictionary is not reused among shards, the compression ratio is
smaller but only slightly.

  % bloaty clang.new -- clang.old
      FILE SIZE        VM SIZE
   --------------  --------------
    +0.2% +74.2Ki  [ = ]       0    .debug_line
    +0.1% +72.3Ki  [ = ]       0    .debug_str
    +0.0% +69.9Ki  [ = ]       0    .debug_info
    +0.1%    +976  [ = ]       0    .debug_abbrev
    +0.0%    +882  [ = ]       0    .debug_ranges
    +0.0%  +218Ki  [ = ]       0    TOTAL

The 2MiB shard size, 0.5 initial buffer size, and 1.5 grow rate are pretty
arbitrary and work for some program I have tested. I'd likely to hear what
parameters you think appropriate.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D117853

Files:
  lld/ELF/CMakeLists.txt
  lld/ELF/OutputSections.cpp
  lld/ELF/OutputSections.h
  lld/test/ELF/compressed-debug-level.test

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D117853.401859.patch
Type: text/x-patch
Size: 7516 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20220121/a7d0520b/attachment.bin>