[all-commits] [llvm/llvm-project] 5d88f2: [lld-macho] Deduplicate fixed-width literals

Fri Jun 11 16:50:46 PDT 2021

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 5d88f2dd947825750d487f91b3b3735d61f7aa8e
      https://github.com/llvm/llvm-project/commit/5d88f2dd947825750d487f91b3b3735d61f7aa8e
  Author: Jez Ng <jezng at fb.com>
  Date:   2021-06-11 (Fri, 11 Jun 2021)

  Changed paths:
    M lld/MachO/InputFiles.cpp
    M lld/MachO/InputSection.cpp
    M lld/MachO/InputSection.h
    M lld/MachO/SyntheticSections.cpp
    M lld/MachO/SyntheticSections.h
    M lld/MachO/Writer.cpp
    A lld/test/MachO/literal-dedup.s
    M lld/test/MachO/mattrs.ll

  Log Message:
  -----------
  [lld-macho] Deduplicate fixed-width literals

Conceptually, the implementation is pretty straightforward: we put each
literal value into a hashtable, and then write out the keys of that
hashtable at the end.

In contrast with ELF, the Mach-O format does not support variable-length
literals that aren't strings. Its literals are either 4, 8, or 16 bytes
in length. LLD-ELF dedups its literals via sorting + uniq'ing, but since
we don't need to worry about overly-long values, we should be able to do
a faster job by just hashing.

That said, the implementation right now is far from optimal, because we
add to those hashtables serially. To parallelize this, we'll need a
basic concurrent hashtable (only needs to support concurrent writes w/o
interleave reads), which shouldn't be to hard to implement, but I'd like
to punt on it for now.

Numbers for linking chromium_framework on my 3.2 GHz 16-Core Intel Xeon W:

      N           Min           Max        Median           Avg        Stddev
  x  20          4.27          4.39         4.315        4.3225   0.033225703
  +  20          4.36          4.82          4.44        4.4845    0.13152846
  Difference at 95.0% confidence
          0.162 +/- 0.0613971
          3.74783% +/- 1.42041%
          (Student's t, pooled s = 0.0959262)

This corresponds to binary size savings of 2MB out of 335MB, or 0.6%.
It's not a great tradeoff as-is, but as mentioned our implementation can
be signficantly optimized, and literal dedup will unlock more
opportunities for ICF to identify identical structures that reference
the same literals.

Reviewed By: #lld-macho, gkm

Differential Revision: https://reviews.llvm.org/D103113

  Commit: 7f2ba39b1688230c663986485f605cf8df90f895
      https://github.com/llvm/llvm-project/commit/7f2ba39b1688230c663986485f605cf8df90f895
  Author: Jez Ng <jezng at fb.com>
  Date:   2021-06-11 (Fri, 11 Jun 2021)

  Changed paths:
    M lld/MachO/Driver.cpp
    M lld/MachO/InputSection.h
    M lld/MachO/MarkLive.cpp
    M lld/MachO/SymbolTable.cpp
    M lld/MachO/Symbols.cpp
    M lld/MachO/Symbols.h
    M lld/MachO/UnwindInfoSection.cpp
    M lld/MachO/UnwindInfoSection.h
    M lld/MachO/Writer.cpp

  Log Message:
  -----------
  [lld-macho][nfc] Move liveness-tracking fields into ConcatInputSection

These fields currently live in the parent InputSection class,
but they should be specific to ConcatInputSection, since the other
InputSection classes (that contain literals) aren't atomically live or
dead -- rather their component string/int literals should have
individual liveness states. (An upcoming diff will add liveness bits for
StringPieces and fixed-sized literals.)

I also factored out some asserts for isCoalescedWeak() in MarkLive.cpp.
We now avoid putting coalesced sections in the `inputSections` vector,
so we don't have to check/assert against it everywhere.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D103977

  Commit: 681cfeb59119f2ddc5418e4e3a48c632dcd1aa3e
      https://github.com/llvm/llvm-project/commit/681cfeb59119f2ddc5418e4e3a48c632dcd1aa3e
  Author: Jez Ng <jezng at fb.com>
  Date:   2021-06-11 (Fri, 11 Jun 2021)

  Changed paths:
    M lld/MachO/ConcatOutputSection.cpp
    M lld/MachO/Driver.cpp
    M lld/MachO/InputFiles.cpp
    M lld/MachO/InputSection.cpp
    M lld/MachO/InputSection.h
    M lld/MachO/SyntheticSections.cpp

  Log Message:
  -----------
  [lld-macho][nfc] Have InputSection ctors take some parameters

This is motivated by an upcoming diff in which the
WordLiteralInputSection ctor sets itself up based on the value of its
section flags. As such, it needs to be passed the `flags` value as part
of its ctor parameters, instead of having them assigned after the fact
in `parseSection()`. While refactoring code to make that possible, I
figured it would make sense for the other InputSections to also take
their initial values as ctor parameters.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D103978

  Commit: 464d3dc3d11eec105023011fd41abb1871f46d5d
      https://github.com/llvm/llvm-project/commit/464d3dc3d11eec105023011fd41abb1871f46d5d
  Author: Jez Ng <jezng at fb.com>
  Date:   2021-06-11 (Fri, 11 Jun 2021)

  Changed paths:
    M lld/MachO/InputSection.cpp
    M lld/MachO/InputSection.h
    M lld/MachO/MarkLive.cpp
    M lld/MachO/SyntheticSections.cpp
    M lld/test/MachO/dead-strip.s

  Log Message:
  -----------
  [lld-macho] Have dead-stripping work with literal sections

Literal sections are not atomically live or dead. Rather,
liveness is tracked for each individual literal they contain. CStrings
have their liveness tracked via a `live` bit in StringPiece, and
fixed-width literals have theirs tracked via a BitVector.

The live-marking code now needs to track the offset within each section
that is to be marked live, in order to identify the literal at that
particular offset.

Numbers for linking chromium_framework on my 3.2 GHz 16-Core Intel Xeon W
with both `-dead_strip` and `--deduplicate-literals`, with and without this diff
applied:

```
    N           Min           Max        Median           Avg        Stddev
x  20          4.32          4.44         4.375         4.372    0.03105174
+  20           4.3          4.39          4.36        4.3595   0.023277502
No difference proven at 95.0% confidence
```
This gives us size savings of about 0.4%.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D103979

Compare: https://github.com/llvm/llvm-project/compare/b90f9bea9673...464d3dc3d11e