[cfe-dev] [RFC] clang-ifso: A new clang-based tool for generating interface libraries.

[Puyan Lotfi via cfe-dev]

Hi All,

The following is our proposal for clang-ifso, we hope to get some valuable
feedback from the community on this and we hope this work helps SDK authors
in the future:

On platforms such as Darwin and Windows, there exist library interface
files (TAPI tbd files, Windows Import Library files) that appear to the
compile-time linker as just another library file (i.e. dylib, dll, etc) but
are in-fact only empty listings of symbols to interface functions that were
intended for exposure by the library writer (i.e. their `.text` sections
are stripped, but they have the API symbols needed for linking). These
library interfaces can be used to both limit access to internals of a
library at static compile time, and can be used to speedup link time in the
case of linking with extremely large dynamic libraries. Aside from
providing more controlled API exposure and reduced memory usage in linking,
there is also the benefit of having a much smaller distribution size for
development SDKs (in the case where you’ve got an SDK with applications
that are built and linked on a PC but then deployed to run on a totally
different device). Finally, these interface libraries can in many cases be
used to break up build dependencies as well if they can be cached or
generated quickly in some way prior to building all the different libraries
in a build.

clang-ifso is a tool that intends to bring the concept of interface
libraries to ELF shared objects. We call it ifso as a shorthand for
InterFace-Shared-Object: as in, we intend to support ELF by producing a .so
that looks just like a regular .so file to the linker but has most of the
.text and other contents dropped and has only the intended API interface
symbols populated. That means a .so file generated by clang-ifso contains
only a stripped .text section plus the `.dynsym` and `.dynstr` sections
along with the global symbols populated. Currently it is in a prototype
state and is capable of generating object-yaml or ELF, and is implemented
using the ClangTool interface.

As mentioned at the beginning, there is prior art in this area with things
like TBD files on Darwin as well as Microsoft’s import libraries. As an
aside, there is evidence that ELF versions of ifsos are also deployed in
production in some settings, so this is not completely new with ELF either
but the effort to make all of the work upstream is. One other way that
clang-ifso does differ from a lot of the prior work is that it uses the
clang parser to glean what visible NamedDecls are present in the library
headers.

Because clang-ifso operates from the clang level we can and do enable the
programmer to use visibility attributes to expose or hide whatever
interface libraries they want in their actual code rather than relying on a
separate file to strip symbols as a post-build step. And since all
compile-time linking will be done with the ifso rather than the .so, all
clients of the library will need to completely rely on whatever was exposed
through visibility attributes.

We are eager to to hear feedback and ideas in this space. Code for
clang-ifso is available at:

https://github.com/plotfi/llvm-project/tree/master/clang/tools/clang-ifso.

PL
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20190404/707677a0/attachment.html>