[llvm-dev] RFC: Contributing llvm-libtool (LLVM version of Apple's libtool)

[Shoaib Meenai via llvm-dev]

(replies inline, as best as Outlook can manage ... the parts I'm replying to should be indented)

On 6/10/20, 11:13 AM, "Chris Lattner" <clattner at nondot.org> wrote:

    On Jun 9, 2020, at 4:05 PM, Shoaib Meenai via llvm-dev <llvm-dev at lists.llvm.org> wrote:
    > We're planning to contribute an LLVM version of Apple's libtool utility. To quote its man page, "Libtool with -static is intended to replace ar(5) and ranlib". In other words, libtool is the preferred tool to create archives for Apple platforms, and it also has good defaults for that platform (e.g. automatically creating a table of contents with the right format). It provides a convenient interface for developers, and we have a lot of usage of it (instead of ar) for that reason in various builds. In particular, it handles universal and archive input files well; for universal input files, you can either specify a single architecture or have libtool create a universal output file, and for archive input files, the individual objects in the input archive are added to the output archive. LLVM's build system uses libtool on Apple platforms to handle universal binaries correctly, for example.

    Nice!

Thank you!

    > To be clear, we're *not* planning to contribute an equivalent to GNU libtool, which has the same name but (as far as I can tell) a very different scope. Any naming ideas to avoid confusion on this front would be appreciated.

    I think that this will be a common point of confusion.  How is this handled in the lld space?

For the linker executables, the MSVC linker is called link.exe, so the Windows port of LLD was just named lld-link. The ELF and Darwin system linkers are both called "ld", but Apple's linker is named ld64, so the ELF port became ld.lld and the Mach-O port became ld64.lld.

For the source code layout, the lld directory has subdirectories for the different ports (COFF, ELF, Mach-O, WASM).

    If you need a specific name, I’d recommend either naming this something like llvm-libtool-macho or llvm-libtool-darwin.

Yeah, those make sense. We already have an option in the build system for optionally creating symlinks for tool names (e.g. if you want an objdump symlink that points to llvm-objdump), so we can just add the option to create a "libtool" symlink for people who want a drop-in replacement. I like the llvm-libtool-darwin name better, since libObject already refers to the specific archive format as "Darwin", but that's not a strong preference.

    An alternative is to plan for the tool to eventually grow to support the features of GNU libtool, allowing a contributor who may become interested in that to work on it within the same code base.

Unfortunately, GNU libtool appears to have a significantly different scope and interface, to the point where I'm not sure how much code you'd actually be able to share in the libtool code base itself (versus just making use of the same libObject primitives). GNU libtool supports compilation, installation and uninstallation, and program execution with library paths set (e.g. for running gdb), in addition to just creating static and dynamic libraries. On the other hand, universal libraries are a Darwin-specific concept which GNU libtool has no notion of.

We could take the approach used by LLD and llvm-objcopy, where we create a directory for the tool (llvm/tools/llvm-libtool in this case), and have subdirectories for the ports of the tool (Darwin in our case, and someone interested could create a GNU subdirectory later). That way there's a natural place to put any code that's common between the ports.

    > One design point we'd appreciate input on is how to structure this new tool. Apple's libtool supports creating either static or dynamic libraries. It produces static libraries directly, but for dynamic libraries, it calls the linker. Our initial focus and contribution will be the functionality for static libraries, but we want our design to take the possibility of the dynamic library generation being added later into account. For this reason, we're planning to create llvm-libtool as a standalone tool instead of as a frontend for llvm-ar. (llvm-ranlib is an example of a frontend for llvm-ar, where it's the same underlying binary which just does different option parsing depending on how it's invoked.)

    Yes, I agree.  LLVM’s library based design makes this pretty natural.

    > Much of the archive writing logic that llvm-ar uses is already factored out to libObject (the writeArchive function), and we can factor more of it out as needed, so we shouldn't end up with too much duplicated code between llvm-ar and llvm-libtool. At the same time, making llvm-libtool its own tool gives us the freedom to e.g. factor out parts of llvm-lipo into a library to handle universal inputs and outputs directly in llvm-libtool rather than having to invoke lipo separately, handle dynamic library generation in a natural way (whereas it would be odd for llvm-ar to contain code for generating dynamic libraries if we were to make llvm-libtool a frontend for llvm-ar, since llvm-ar is a tool specifically for creating archives), and potentially integrate with LLD for Mach-O as a library in the future instead of spawning the linker separately. Are there any concerns with this plan?

    +1 from me.

Great, thanks!

    -Chris