[all-commits] [llvm/llvm-project] 3a1152: [lld-macho] Move ICF earlier to avoid emitting red...

Thu Jul 1 18:23:07 PDT 2021

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 3a11528d97a788781de82f939f502abe7fbd729d
      https://github.com/llvm/llvm-project/commit/3a11528d97a788781de82f939f502abe7fbd729d
  Author: Jez Ng <jezng at fb.com>
  Date:   2021-07-01 (Thu, 01 Jul 2021)

  Changed paths:
    M lld/MachO/ConcatOutputSection.cpp
    M lld/MachO/ConcatOutputSection.h
    M lld/MachO/Driver.cpp
    M lld/MachO/ICF.cpp
    M lld/MachO/ICF.h
    M lld/MachO/InputSection.cpp
    M lld/MachO/InputSection.h
    M lld/MachO/MarkLive.cpp
    M lld/MachO/SyntheticSections.cpp
    M lld/MachO/SyntheticSections.h
    M lld/MachO/UnwindInfoSection.cpp
    M lld/MachO/UnwindInfoSection.h
    M lld/MachO/Writer.cpp

  Log Message:
  -----------
  [lld-macho] Move ICF earlier to avoid emitting redundant binds

This is a pretty big refactoring diff, so here are the motivations:

Previously, ICF ran after scanRelocations(), where we emitting
bind/rebase opcodes etc. So we had a bunch of redundant leftovers after
ICF. Having ICF run before Writer seems like a better design, and is
what LLD-ELF does, so this diff refactors it accordingly.

However, ICF had two dependencies on things occurring in Writer: 1) it
needs literals to be deduplicated beforehand and 2) it needs to know
which functions have unwind info, which was being handled by
`UnwindInfoSection::prepareRelocations()`.

In order to do literal deduplication earlier, we need to add literal
input sections to their corresponding output sections. So instead of
putting all input sections into the big `inputSections` vector, and then
filtering them by type later on, I've changed things so that literal
sections get added directly to their output sections during the 'gather'
phase. Likewise for compact unwind sections -- they get added directly
to the UnwindInfoSection now. This latter change is not strictly
necessary, but makes it easier for ICF to determine which functions have
unwind info.

Adding literal sections directly to their output sections means that we
can no longer determine `inputOrder` from iterating over
`inputSections`. Instead, we store that order explicitly on
InputSection. Bloating the size of InputSection for this purpose would
be unfortunate -- but LLD-ELF has already solved this problem: it reuses
`outSecOff` to store this order value.

One downside of this refactor is that we now make an additional pass
over the unwind info relocations to figure out which functions have
unwind info, since want to know that before `processRelocations()`. I've
made sure to run that extra loop only if ICF is enabled, so there should
be no overhead in non-optimizing runs of the linker.

The upside of all this is that the `inputSections` vector now contains
only ConcatInputSections that are destined for ConcatOutputSections, so
we can clean up a bunch of code that just existed to filter out other
elements from that vector.

I will test for the lack of redundant binds/rebases in the upcoming
cfstring deduplication diff. While binds/rebases can also happen in the
regular `.text` section, they're more common in `.data` sections, so it
seems more natural to test it that way.

This change is perf-neutral when linking chromium_framework.

Reviewed By: oontvoo

Differential Revision: https://reviews.llvm.org/D105044

  Commit: ac2dd06b91ae7264fa7d396c15c7647510ede231
      https://github.com/llvm/llvm-project/commit/ac2dd06b91ae7264fa7d396c15c7647510ede231
  Author: Jez Ng <jezng at fb.com>
  Date:   2021-07-01 (Thu, 01 Jul 2021)

  Changed paths:
    M lld/MachO/ICF.cpp
    M lld/MachO/InputFiles.cpp
    M lld/MachO/InputSection.cpp
    M lld/MachO/InputSection.h
    M lld/test/MachO/Inputs/MacOSX.sdk/System/Library/Frameworks/CoreFoundation.framework/CoreFoundation.tbd
    A lld/test/MachO/cfstring-dedup.s
    A lld/test/MachO/invalid/cfstring.s

  Log Message:
  -----------
  [lld-macho] Deduplicate CFStrings

`__cfstring` is a special literal section, so instead of breaking it up
at symbol boundaries, we break it up at fixed-width boundaries (since
each literal is the same size). Symbols can only occur at one of those
boundaries, so this is strictly more powerful than
`.subsections_via_symbols`.

With that in place, we then run the section through ICF.

This change is about perf-neutral when linking chromium_framework.

Reviewed By: #lld-macho, gkm

Differential Revision: https://reviews.llvm.org/D105045

  Commit: 08715e6c47f68b7ea985fbd76d4945dfdff0a9aa
      https://github.com/llvm/llvm-project/commit/08715e6c47f68b7ea985fbd76d4945dfdff0a9aa
  Author: Jez Ng <jezng at fb.com>
  Date:   2021-07-01 (Thu, 01 Jul 2021)

  Changed paths:
    M lld/MachO/SyntheticSections.h

  Log Message:
  -----------
  [lld-macho][nfc] Remove unnecessary vertical spacing

This makes NonLazyPointerSectionBase's style more in line with the rest
of the classes in its file.

  Commit: f6b6e7214366fc12469aba2fe16495e5f7a375a6
      https://github.com/llvm/llvm-project/commit/f6b6e7214366fc12469aba2fe16495e5f7a375a6
  Author: Jez Ng <jezng at fb.com>
  Date:   2021-07-01 (Thu, 01 Jul 2021)

  Changed paths:
    M lld/MachO/ConcatOutputSection.cpp
    M lld/MachO/Driver.cpp
    M lld/MachO/Dwarf.cpp
    M lld/MachO/ICF.cpp
    M lld/MachO/InputFiles.cpp
    M lld/MachO/InputSection.cpp
    M lld/MachO/InputSection.h
    M lld/MachO/MarkLive.cpp
    M lld/MachO/SymbolTable.cpp
    M lld/MachO/Symbols.h
    M lld/MachO/SyntheticSections.cpp
    M lld/MachO/UnwindInfoSection.cpp
    M lld/MachO/Writer.cpp

  Log Message:
  -----------
  [lld-macho] Factor out common InputSection members

We have been creating many ConcatInputSections with identical values due
to .subsections_via_symbols. This diff factors out the identical values
into a Shared struct, to reduce memory consumption and make copying
cheaper.

I also changed `callSiteCount` from a uint32_t to a 31-bit field to save an
extra word.

All in all, this takes InputSection from 120 to 72 bytes (and
ConcatInputSection from 160 to 112 bytes), i.e. 30% size reduction in
ConcatInputSection.

Numbers for linking chromium_framework on my 3.2 GHz 16-Core Intel Xeon W:

      N           Min           Max        Median           Avg        Stddev
  x  20          4.14          4.24          4.18         4.183   0.027548999
  +  20          4.04          4.11         4.075        4.0775   0.018027756
  Difference at 95.0% confidence
          -0.1055 +/- 0.0149005
          -2.52211% +/- 0.356215%
          (Student's t, pooled s = 0.0232803)

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D105305

Compare: https://github.com/llvm/llvm-project/compare/e895a670f8bc...f6b6e7214366