[PATCH] D52917: Avoid unnecessary buffer allocation and memcpy for compressed sections.

[Rui Ueyama via Phabricator via llvm-commits]

ruiu created this revision.
ruiu added reviewers: grimar, MaskRay, dblaikie.
Herald added subscribers: JDevlieghere, arichardson, emaste.
Herald added a reviewer: javed.absar.
Herald added a reviewer: espindola.

Previously, we uncompress all compressed sections before doing anything.
That works, and that is conceptually simple, but that could results in
a waste of CPU time and memory if uncompressed sections are then
discarded or just copied to the output buffer.

In particular, if .debug_gnu_pub{names,types} are compressed and if no
-gdb-index option is given, we wasted CPU and memory because we
uncompress them into newly allocated bufers and then memcpy the buffers
to the output buffer. That temporary buffer was redundant.

This patch changes how to uncompress sections. Now, compressed sections
are uncompressed lazily. To do that, `Data` member of `InputSectionBase`
is now hidden from outside, and `data()` accessor automatically expands
an compressed buffer if necessary.

If no one calls `data()`, then `writeTo()` directly uncompresses
compressed data into the output buffer. That eliminates the redundant
memory allocation and redundant memcpy.

This patch significantly reduces memory consumption (20 GiB max RSS to
15 Gib) for an executable whose .debug_gnu_pub{names,types} are in total
5 GiB in an uncompressed form.


https://reviews.llvm.org/D52917

Files:
  lld/ELF/AArch64ErrataFix.cpp
  lld/ELF/DWARF.cpp
  lld/ELF/Driver.cpp
  lld/ELF/EhFrame.cpp
  lld/ELF/ICF.cpp
  lld/ELF/InputFiles.cpp
  lld/ELF/InputSection.cpp
  lld/ELF/InputSection.h
  lld/ELF/MarkLive.cpp
  lld/ELF/Relocations.cpp
  lld/ELF/SyntheticSections.cpp
  lld/ELF/SyntheticSections.h
  lld/test/ELF/compressed-debug-input-err.s
  lld/test/ELF/relocatable-compressed-input.s
  lld/test/ELF/strip-debug.s

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D52917.168411.patch
Type: text/x-patch
Size: 22784 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20181004/ab92ef26/attachment.bin>