[llvm-dev] [RFC] Open sourcing and contributing TAPI back to the LLVM community

[Davide Italiano via llvm-dev]

On Thu, Sep 7, 2017 at 5:01 PM, Juergen Ributzka via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> Hi @ll,
>
> Over the past years I have been looking into how to reduce the size of the
> SDK that ships with Xcode and how to improve build times for the overall OS
> inside Apple. The result is a tool called TAPI, which is used at Apple for
> all things related to text-based dynamic library files (.tbd).
>
> What are text-based dynamic library files?
> Text-based dynamic library files (TBDs) are a textual representation of the
> information in a dynamic library / shared library that is required by the
> static linker - basically a symbol list of the exported symbols.
>
> Apple’s SDKs originally used Mach-O Dynamic Library Stubs. Mach-O Dynamic
> Library Stubs are dynamic library files, but with all the text and data
> stripped out. TBD files were introduced to replaced Mach-O Dynamic Library
> Stub files in the SDK to further reduce its overall size.
>
> Over time the TAPI tool has grown and is used now in a variety of ways.
>
> Dynamic Library Stubbing:
> As mentioned above, TAPI is used to read the content of dynamic library /
> shared library and generates a textual representation that can be used by
> the static linker. The current implementation reads MachO files, but it
> could be extended to also provide the same functionality for other object
> file formats.
>
> Framework / Dynamic Library Verification:
> The symbols that are exported from a dynamic library should ideally match,
> or at least contain, all the API that is specified in the associated header
> files. TAPI performs this verification by parsing the header files with
> CLANG and compare the findings to the exported symbols from the library.
>
> InstallAPI:
> InstallAPI is a new build phase that generates the TBD file from header
> files only. This allows a dependency of the library to build concurrently
> even before the library has been built itself. This can be used to increase
> parallelism in the build or larger projects or operating systems.
>
> Misc:
> - display and operate on TBD files
> - automatically generate API tests from header files
> - libtapi, which is used by the linker (ld64) to parse the TBD files
>

I'm interested in whether you plan to have this integrated in lld as well.
As far as I understand, this is going to be the de-facto way of
shipping for Mach-O binaries (at least, the ones released by Apple).
Please correct me if I'm wrong.
I tried to self-host lld on El Capitan and it fails because lld
doesn't really know about TBD files.
This, unfortunately, makes the linker not really usable for modern Mac
OS releases.

>
> The functionality of the tool is currently limited to Mach-O object files,
> but that is not a technical limitation. In making the tool open source I
> hope others will be able to take advantage of it too and extend its
> functionality to other object file formats.
>
>
> I initially developed the project as a CLANG project, but that was mostly
> for practical reasons (out-of-tree development, separate repo, etc). For the
> curious ones I pushed the repo to github (https://github.com/ributzka/tapi).
>
> I imagine, for example, that the reading/writing of TBD files is something
> that would fit better into the LLVM sources, which makes it available to
> other libraries and tools (e.g. LLVMObject, llvm-nm, lld, ...).
>
> I created a small patch that integrates it with llvm-nm and LLVMObject. This
> patch is not complete and I will split it up into smaller patches for
> review. I am providing it as a reference to get the discussion started.
>
> Please let me know what you think and bikeshed away :)
>
> Thanks
>
> Cheers,
> Juergen
>
>
>
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>

-- 
Davide

"There are no solved problems; there are only problems that are more
or less solved" -- Henri Poincare