[cfe-dev] [RFC] Open sourcing and contributing TAPI back to the LLVM community

[James Y Knight via cfe-dev]

I'm not really clear on the actual benefits of the TBD file, and why Apple
migrated to them in the first place. Shouldn't a dynamic library containing
only the relevant parts (e.g. the dynamic symbol table) be roughly
comparable in size? And, much simpler to support? I assume that's
effectively what "Mach-O Dynamic Library Stubs" actually _were_, before the
introduction of TBD files, so presumably there were good reasons for
switching?

If anyone wants to do something similar for another platform (that is to
say, ELF; COFF already has import libraries), I'd suggest that the sensible
way to do so would be to generate actual shared object files which contain
only the appropriate interface definitions.

Regardless of any of that, given that TBD files _are_ an integral part of
the apple platform, supporting them is certainly a necessity in order to
have a working apple linker. So, if making LLD work for Apple/MachO is the
justification for adding TBD support to LLVM, that seems self-evidently a
reasonable thing to do. On the other hand, it looks like the LLD mach-o
code is unmaintained and nobody seems to be much interested in it. And
having code for reading TBD files in LLVM seems not terribly interesting,
unless it is as part of a project to make the LLD MachO linker actually
functional and supported.


On Thu, Sep 7, 2017 at 8:01 PM, Juergen Ributzka via cfe-dev <
cfe-dev at lists.llvm.org> wrote:

> Hi @ll,
>
> Over the past years I have been looking into how to reduce the size of the
> SDK that ships with Xcode and how to improve build times for the overall OS
> inside Apple. The result is a tool called TAPI, which is used at Apple for
> all things related to text-based dynamic library files (.tbd).
>
> *What are text-based dynamic library files?*
> Text-based dynamic library files (TBDs) are a textual representation of
> the information in a dynamic library / shared library that is required by
> the static linker - basically a symbol list of the exported symbols.
>
> Apple’s SDKs originally used Mach-O Dynamic Library Stubs. Mach-O Dynamic
> Library Stubs are dynamic library files, but with all the text and data
> stripped out. TBD files were introduced to replaced Mach-O Dynamic Library
> Stub files in the SDK to further reduce its overall size.
>
> Over time the TAPI tool has grown and is used now in a variety of ways.
>
> *Dynamic Library Stubbing:*
> As mentioned above, TAPI is used to read the content of dynamic library /
> shared library and generates a textual representation that can be used by
> the static linker. The current implementation reads MachO files, but it
> could be extended to also provide the same functionality for other object
> file formats.
>
> *Framework / Dynamic Library Verification:*
> The symbols that are exported from a dynamic library should ideally match,
> or at least contain, all the API that is specified in the associated header
> files. TAPI performs this verification by parsing the header files with
> CLANG and compare the findings to the exported symbols from the library.
>
> *InstallAPI:*
> InstallAPI is a new build phase that generates the TBD file from header
> files only. This allows a dependency of the library to build concurrently
> even before the library has been built itself. This can be used to increase
> parallelism in the build or larger projects or operating systems.
>
> *Misc:*
> - display and operate on TBD files
> - automatically generate API tests from header files
> - libtapi, which is used by the linker (ld64) to parse the TBD files
>
>
> The functionality of the tool is currently limited to Mach-O object files,
> but that is not a technical limitation. In making the tool open source I
> hope others will be able to take advantage of it too and extend its
> functionality to other object file formats.
>
>
> I initially developed the project as a CLANG project, but that was mostly
> for practical reasons (out-of-tree development, separate repo, etc). For
> the curious ones I pushed the repo to github (
> https://github.com/ributzka/tapi).
>
> I imagine, for example, that the reading/writing of TBD files is something
> that would fit better into the LLVM sources, which makes it available to
> other libraries and tools (e.g. LLVMObject, llvm-nm, lld, ...).
>
> I created a small patch that integrates it with llvm-nm and LLVMObject.
> This patch is not complete and I will split it up into smaller patches for
> review. I am providing it as a reference to get the discussion started.
>
> Please let me know what you think and bikeshed away :)
>
> Thanks
>
> Cheers,
> Juergen
>
>
>
>
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20180409/c553d2a2/attachment.html>