[PATCH] D102964: [lld-macho] Implement cstring merging / deduplication

[Jez Ng via Phabricator via llvm-commits]

int3 created this revision.
int3 added a reviewer: lld-macho.
Herald added subscribers: dang, mgrang, mgorny.
Herald added a reviewer: gkm.
Herald added a project: lld-macho.
int3 requested review of this revision.
Herald added a project: LLVM.
Herald added a subscriber: llvm-commits.

Our implementation draws heavily from LLD-ELF's, which in turn delegates
its string merging to llvm-mc's StringTableBuilder. The messiness of
this diff is largely due to the fact that we've previously assumed that
all InputSections get concatenated together to form the output. This is
no longer true with cstrings, which can now be tail-merged. As such,
calculating output offsets for cstrings is a bit more complicated than
just adding the offset within the InputSection to its offset within the
containing OutputSection.

Another complication stems from the fact that we previously maintained
section ordering implicitly: we created InputSections as we parsed each
file in command-line order, and passed on this ordering when we created
OutputSections and OutputSegments by iterating over these InputSections.
The implicitness of the ordering made it difficult to refactor the code
to handle a new type of InputSection (i.e. the CStringInputSection). As
such, I've codified the ordering explicitly via `inputOrder` fields.
This also allows us to use `sort` instead of `stable_sort` for sorting
the output.

Some ELF-contrast notes: What ELF calls merge sections (marked by
`SHF_MERGE`), Mach-O calls literal sections (marked by `S_CSTRING_LITERALS`
or `S_{4,8,16}BYTE_LITERALS`). Semantically they seem pretty similar,
but while ELF allow for variable-width byte literals, Mach-O only allows
fixed-width ones. Since the maximum width is 16, we can likely have a
more efficient implementation by hashing these literal values. I will
attempt to handle fixed-width literals in an upcoming diff.

While the Mach-O format doesn't use the word 'merge', to avoid
confusion, I've renamed our `MergedOutputSection` to
`ConcatOutputSection`. I believe it's a more descriptive name too.

***Performance Numbers***

CString deduplication reduces chromium_framework from 345MB to 335MB, or
about a 2.8% reduction.

Performance numbers for linking chromium_framework on my 3.2 GHz
16-Core Intel Xeon W:

      N           Min           Max        Median           Avg        Stddev
  x  20          4.23          4.35          4.27         4.274   0.030157481
  +  20          4.36          4.45         4.395        4.3995   0.026051568
  Difference at 95.0% confidence
          0.1255 +/- 0.0180361
          2.93636% +/- 0.421996%
          (Student's t, pooled s = 0.0281794)

As expected, cstring merging incurs some non-trivial overhead.

When passing `--no-literal-merge`, I see a slight improvement relative
to an LLD built before this diff. This is possibly due to the use of
non-stable sort:

      N           Min           Max        Median           Avg        Stddev
  x  20          4.23          4.35          4.27         4.274   0.030157481
  +  20          4.19          4.31          4.23        4.2355   0.027042851
  Difference at 95.0% confidence
          -0.0385 +/- 0.0183325
          -0.900796% +/- 0.428931%
          (Student's t, pooled s = 0.0286425)


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D102964

Files:
  lld/MachO/CMakeLists.txt
  lld/MachO/ConcatOutputSection.cpp
  lld/MachO/ConcatOutputSection.h
  lld/MachO/Config.h
  lld/MachO/Driver.cpp
  lld/MachO/InputFiles.cpp
  lld/MachO/InputFiles.h
  lld/MachO/InputSection.cpp
  lld/MachO/InputSection.h
  lld/MachO/MergedOutputSection.cpp
  lld/MachO/MergedOutputSection.h
  lld/MachO/Options.td
  lld/MachO/OutputSection.h
  lld/MachO/OutputSegment.cpp
  lld/MachO/OutputSegment.h
  lld/MachO/Symbols.cpp
  lld/MachO/SyntheticSections.cpp
  lld/MachO/SyntheticSections.h
  lld/MachO/UnwindInfoSection.cpp
  lld/MachO/UnwindInfoSection.h
  lld/MachO/Writer.cpp
  lld/test/MachO/cstring-merging.s
  lld/test/MachO/invalid/cstring-merging.s
  lld/test/MachO/invalid/reserved-section-name.s
  lld/test/MachO/load-command-sequence.s
  lld/test/MachO/section-order.s
  lld/test/MachO/subsections-section-relocs.s
  lld/test/MachO/weak-binding.s
  lld/test/MachO/x86-64-relocs.s

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D102964.347169.patch
Type: text/x-patch
Size: 54086 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20210522/f25847b2/attachment.bin>