[cfe-dev] [RFC] clang-ifso: A new clang-based tool for generating interface libraries.

[Petr Hosek via cfe-dev]

llvm-elfabi is going to generate ELF binaries directly as well (we
originally considered extending lld to support tbe files directly but we
later decided not to do that to avoid introducing the additional
complexity). It'd be great if clang-ifso could emit tbe files and use the
same implementation for generating ELF binaries to avoid duplication
between the two tools.

On Thu, Apr 4, 2019 at 5:42 PM Puyan Lotfi <puyan.lotfi.llvm at gmail.com>
wrote:

> Shoaib, Petr:
>
> I don’t yet know the full details of elf-tapi, but the two ways I believe
> these differs are:
>
> 1) clang-ifso is a clang based tool that scans from the source level and
> uses visibility attributes to put together a set of symbols to expose in
> the ifso file.
>
> 2) clang-ifso primarily aims to generate the ELF binary directly, but also
> can generate yaml that yaml2obj can generate the ifso from. Currently the
> aim is to avoid introducing linker changes.
>
> It’s possible clang-ifso could also support generating said text stubs
> (tbe’s) too.
>
> PL
>
>
> On Thu, Apr 4, 2019 at 5:23 PM Petr Hosek <phosek at chromium.org> wrote:
>
>> To expand on Shoaib's answer, llvm-elfabi
>> <https://github.com/llvm/llvm-project/tree/master/llvm/tools/llvm-elfabi> is
>> a very similar tool that we've been working on for the past few months in
>> upstream. The current implementation is still incomplete, but once D56569
>> <https://reviews.llvm.org/D56569>, D55839
>> <https://reviews.llvm.org/D55839> and D55864
>> <https://reviews.llvm.org/D55864> land (hopefully within the next few
>> weeks), it should be possible to use llvm-elfabi to produce text stubs
>> (called .tbe files) and generate ELF interface libraries out of those.
>>
>> On Thu, Apr 4, 2019 at 5:18 PM Shoaib Meenai via cfe-dev <
>> cfe-dev at lists.llvm.org> wrote:
>>
>>> How does this compare to the ELF TAPI implementation in LLVM (
>>> http://lists.llvm.org/pipermail/llvm-dev/2018-September/126472.html and
>>> https://reviews.llvm.org/D53051)?
>>>
>>>
>>>
>>> *From: *cfe-dev <cfe-dev-bounces at lists.llvm.org> on behalf of cfe-dev <
>>> cfe-dev at lists.llvm.org>
>>> *Reply-To: *Puyan Lotfi <puyan.lotfi.llvm at gmail.com>
>>> *Date: *Thursday, April 4, 2019 at 4:52 PM
>>> *To: *cfe-dev <cfe-dev at lists.llvm.org>
>>> *Cc: *Puyan Lotfi <puyan at puyan.org>
>>> *Subject: *[cfe-dev] [RFC] clang-ifso: A new clang-based tool for
>>> generating interface libraries.
>>>
>>>
>>>
>>> Hi All,
>>>
>>>
>>>
>>> The following is our proposal for clang-ifso, we hope to get some
>>> valuable feedback from the community on this and we hope this work helps
>>> SDK authors in the future:
>>>
>>>
>>>
>>> On platforms such as Darwin and Windows, there exist library interface
>>> files (TAPI tbd files, Windows Import Library files) that appear to the
>>> compile-time linker as just another library file (i.e. dylib, dll, etc) but
>>> are in-fact only empty listings of symbols to interface functions that were
>>> intended for exposure by the library writer (i.e. their `.text` sections
>>> are stripped, but they have the API symbols needed for linking). These
>>> library interfaces can be used to both limit access to internals of a
>>> library at static compile time, and can be used to speedup link time in the
>>> case of linking with extremely large dynamic libraries. Aside from
>>> providing more controlled API exposure and reduced memory usage in linking,
>>> there is also the benefit of having a much smaller distribution size for
>>> development SDKs (in the case where you’ve got an SDK with applications
>>> that are built and linked on a PC but then deployed to run on a totally
>>> different device). Finally, these interface libraries can in many cases be
>>> used to break up build dependencies as well if they can be cached or
>>> generated quickly in some way prior to building all the different libraries
>>> in a build.
>>>
>>>
>>>
>>> clang-ifso is a tool that intends to bring the concept of interface
>>> libraries to ELF shared objects. We call it ifso as a shorthand for
>>> InterFace-Shared-Object: as in, we intend to support ELF by producing a .so
>>> that looks just like a regular .so file to the linker but has most of the
>>> .text and other contents dropped and has only the intended API interface
>>> symbols populated. That means a .so file generated by clang-ifso contains
>>> only a stripped .text section plus the `.dynsym` and `.dynstr` sections
>>> along with the global symbols populated. Currently it is in a prototype
>>> state and is capable of generating object-yaml or ELF, and is implemented
>>> using the ClangTool interface.
>>>
>>>
>>>
>>> As mentioned at the beginning, there is prior art in this area with
>>> things like TBD files on Darwin as well as Microsoft’s import libraries. As
>>> an aside, there is evidence that ELF versions of ifsos are also deployed in
>>> production in some settings, so this is not completely new with ELF either
>>> but the effort to make all of the work upstream is. One other way that
>>> clang-ifso does differ from a lot of the prior work is that it uses the
>>> clang parser to glean what visible NamedDecls are present in the library
>>> headers.
>>>
>>>
>>>
>>> Because clang-ifso operates from the clang level we can and do enable
>>> the programmer to use visibility attributes to expose or hide whatever
>>> interface libraries they want in their actual code rather than relying on a
>>> separate file to strip symbols as a post-build step. And since all
>>> compile-time linking will be done with the ifso rather than the .so, all
>>> clients of the library will need to completely rely on whatever was exposed
>>> through visibility attributes.
>>>
>>>
>>>
>>> We are eager to to hear feedback and ideas in this space. Code for
>>> clang-ifso is available at:
>>>
>>>
>>>
>>> https://github.com/plotfi/llvm-project/tree/master/clang/tools/clang-ifso
>>> .
>>>
>>>
>>>
>>> PL
>>>
>>>
>>>
>>>
>>>
>> _______________________________________________
>>> cfe-dev mailing list
>>> cfe-dev at lists.llvm.org
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20190404/80db3f3f/attachment.html>