[all-commits] [llvm/llvm-project] a96fe1: [ELF][LTO] Call madvise(MADV_DONTNEED) on MemoryBu...

[Fangrui Song via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: a96fe1bf3b320d62f1564ded5f7259eef1869cf9
      https://github.com/llvm/llvm-project/commit/a96fe1bf3b320d62f1564ded5f7259eef1869cf9
  Author: Fangrui Song <i at maskray.me>
  Date:   2021-12-30 (Thu, 30 Dec 2021)

  Changed paths:
    M lld/ELF/Driver.cpp
    M lld/ELF/Driver.h
    M lld/ELF/InputFiles.cpp
    M lld/ELF/InputFiles.h

  Log Message:
  -----------
  [ELF][LTO] Call madvise(MADV_DONTNEED) on MemoryBuffer instances

@tejohnson noticed that freeing MemoryBuffer instances right before
`lto->compile` can save RSS, likely because the memory can be reused by
LTO indexing (e.g. ThinLTO import/export lists).).

For ELFFileBase instances, symbol and section names are backed by MemoryBuffer,
so destroying MemoryBuffer would make some infrequent passes (parseSymbolVersion,
reportBackrefs) crash and make debugging difficult.
For a BitcodeFile, its content is completely unused, but destroying its
MemoryBuffer makes the buffer identifier inaccessible and may introduce
constraints for future changes.
This patch leverages madvise(MADV_DONTNEED) which achieves the major gain
without the latent issues.

`Maximum resident set size (kbytes): ` for a large --thinlto-index-only link:

* current behavior: 10146104KiB
* destroy MemoryBuffer instances: 8555240KiB
* madvise(MADV_DONTNEED) just bitcodeFiles  and lazyBitcodeFiles: 8737372KiB
* madvise(MADV_DONTNEED) all MemoryBuffers: 8739796KiB  (16% decrease)

Depends on D116366

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D116367