[cfe-dev] [RFC] Open sourcing and contributing TAPI back to the LLVM community

[Juergen Ributzka via cfe-dev]

Hi @ll,

Over the past years I have been looking into how to reduce the size of the SDK that ships with Xcode and how to improve build times for the overall OS inside Apple. The result is a tool called TAPI, which is used at Apple for all things related to text-based dynamic library files (.tbd).

What are text-based dynamic library files?
Text-based dynamic library files (TBDs) are a textual representation of the information in a dynamic library / shared library that is required by the static linker - basically a symbol list of the exported symbols.

Apple’s SDKs originally used Mach-O Dynamic Library Stubs. Mach-O Dynamic Library Stubs are dynamic library files, but with all the text and data stripped out. TBD files were introduced to replaced Mach-O Dynamic Library Stub files in the SDK to further reduce its overall size.

Over time the TAPI tool has grown and is used now in a variety of ways.

Dynamic Library Stubbing:
As mentioned above, TAPI is used to read the content of dynamic library / shared library and generates a textual representation that can be used by the static linker. The current implementation reads MachO files, but it could be extended to also provide the same functionality for other object file formats.

Framework / Dynamic Library Verification:
The symbols that are exported from a dynamic library should ideally match, or at least contain, all the API that is specified in the associated header files. TAPI performs this verification by parsing the header files with CLANG and compare the findings to the exported symbols from the library.

InstallAPI:
InstallAPI is a new build phase that generates the TBD file from header files only. This allows a dependency of the library to build concurrently even before the library has been built itself. This can be used to increase parallelism in the build or larger projects or operating systems.

Misc:
- display and operate on TBD files
- automatically generate API tests from header files
- libtapi, which is used by the linker (ld64) to parse the TBD files


The functionality of the tool is currently limited to Mach-O object files, but that is not a technical limitation. In making the tool open source I hope others will be able to take advantage of it too and extend its functionality to other object file formats.


I initially developed the project as a CLANG project, but that was mostly for practical reasons (out-of-tree development, separate repo, etc). For the curious ones I pushed the repo to github (https://github.com/ributzka/tapi <https://github.com/ributzka/tapi>).

I imagine, for example, that the reading/writing of TBD files is something that would fit better into the LLVM sources, which makes it available to other libraries and tools (e.g. LLVMObject, llvm-nm, lld, ...).

I created a small patch that integrates it with llvm-nm and LLVMObject. This patch is not complete and I will split it up into smaller patches for review. I am providing it as a reference to get the discussion started.

Please let me know what you think and bikeshed away :)

Thanks

Cheers,
Juergen




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20170907/4555e2da/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tapi-llvm-nm.patch.tar.bz2
Type: application/x-bzip2
Size: 16670 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20170907/4555e2da/attachment.bin>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20170907/4555e2da/attachment-0001.html>