[PATCH] D46299: [wip] ELF: Add a feature for sharing rodata with an embedded binary.

[Peter Collingbourne via Phabricator via llvm-commits]

pcc created this revision.
pcc added reviewers: ruiu, espindola.
Herald added subscribers: arichardson, mgorny, emaste.

There is sometimes a need to distribute a binary that is compatible
with two or more architectures -- this is commonly known as a "fat
binary".

This patch allows you to create a fat binary with rodata sections
that are shared between architectures, in order to reduce overall
binary size. The way that it works is that you link one binary in the
usual way with a flag requesting that a hint file be written with the
locations of the rodata sections and strings, and then link another
binary with flags requesting that the first binary is embedded into
the second one and giving the path to the hint file.

Using the hint file, the linker removes rodata sections and strings
from the second binary whose content matches a section or string in the
first binary, and redirects any references to that section or string
to the matching one in the first binary, which is then embedded in
the rodata section in the second binary.

How can the embedded binary be loaded? Some dynamic loaders have a
feature that allows libraries to be loaded from an offset within a
file (such as Android's ANDROID_DLEXT_USE_LIBRARY_FD_OFFSET [1]),
so a program could arrange to use that, provided that it knows the
location of the embedded file. The location can be found using a
new program header, PT_LLVM_EMBED, which could also potentially be
used by a dynamic loader to load the embedded binary for the correct
architecture automatically.

The program header could also be used by another program to create
an appropriate container in another fat binary format, such as FatELF
[2], or the APK format used on Android [3], which is based on ZIP. A
flag allows you to specify a minimum amount of padding before the
embedded file, in case the container format requires a header to
appear immediately before the file (as ZIP does).

I'm mostly sharing this for feedback on the overall idea and design.
TODO:

- Write documentation. We probably need documentation for the ELF extension (i.e. the program header) as well as user documentation for the feature.
- Write tests.
- Clean up the code.

[1] https://android.googlesource.com/platform/bionic/+/da1994ced6fd870a9ad198c0435d3e5ebbf65636/libc/include/android/dlext.h#88
[2] https://icculus.org/fatelf/
[3] https://en.wikipedia.org/wiki/Android_application_package


https://reviews.llvm.org/D46299

Files:
  lld/ELF/CMakeLists.txt
  lld/ELF/Config.h
  lld/ELF/Driver.cpp
  lld/ELF/EmbedFile.cpp
  lld/ELF/EmbedFile.h
  lld/ELF/InputSection.cpp
  lld/ELF/InputSection.h
  lld/ELF/LinkerScript.cpp
  lld/ELF/MapFile.cpp
  lld/ELF/Options.td
  lld/ELF/SyntheticSections.cpp
  lld/ELF/SyntheticSections.h
  lld/ELF/Writer.cpp
  lld/ELF/Writer.h
  llvm/include/llvm/BinaryFormat/ELF.h

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D46299.144670.patch
Type: text/x-patch
Size: 22028 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20180501/dabd31da/attachment.bin>