[LLVMdev] LLVM as a shared library

[Filip Pizlo]

This is exciting!

I would be happy to help.


> On Aug 5, 2014, at 12:38 PM, Chris Bieneman <beanz at apple.com> wrote:
> 
> Hello LLVM community,
> 
> Over the last few years the LLVM team here at Apple and development teams elsewhere have been busily working on finding new and interesting uses for LLVM. Some of these uses are traditional compilers, but a growing number of them aren’t. Some of LLVM’s new clients, like WebKit, are embedding LLVM into existing applications. These embedded uses of LLVM have their own unique challenges.
> 
> Over the next few months, a few of us at Apple are going to be working on tackling a few new problems that we would like solved in open source so other projects can benefit from them. Some of these efforts will be non-trivial, so we’d like to start a few discussions over the next few weeks.
> 
> Our primary goals are to (1) make it easier to embed LLVM into external projects as a shared library, and (2) generally improve the performance of LLVM as a shared library.
> 
> The list of the problems we’re currently planning to tackle is:
> 
> (1) Reduce or eliminate static initializers, global constructors, and global destructors
> (2) Clean up cross compiling in the CMake build system
> (3) Update LLVM debugging mechanisms for being part of a dynamic library
> (4) Move overridden sys calls (like abort) into the tools, rather than the libraries
> (5) Update TableGen to support stripping unused content (i.e. Intrinsics for backends you’re not building)

Also:

(6) Determine if command line options are the best way of passing configuration settings into LLVM.

It’s an awkward abstraction when LLVM is embedded. I suspect (6) will be closely related to (1) since command line option parsing was the hardest impediment to getting rid of static initializers.

My understanding of the shared library proposal is that the library only exposes the C API since the C++ API is not intended to allow for binary compatibility.  So, I think we need to either add the following as either an explicit goal of the shared library work, or as a closely related project:

(7) Make the C API truly great.

I think it’s harmful to LLVM in the long run if external embedders use the C++ API.  I think that one way of ensuring that they don’t have an excuse to do it is to flesh out some things:

- Add more tests of the C API to ensure that people don’t break it accidentally and to give more gravitas to the C API backwards compatibility claims.
- Increase C API coverage.
	- For example, WebKit currently sidesteps the C API to pass some commandline options to LLVM.  We don’t want that.
	- Add more support for reasoning about targets and triples.  WebKit still has to hardcode triples in some places even though it only ever does in-process JITing where host==target.  That’s weird.
	- Expose debugging and runtime stuff and make sure that there’s a coherent integration story with the MCJIT C API.
		- Currently it’s difficult to round-trip debug info: creating it in C is awkward and parsing DWARF sections that MCJIT generates involves lots of weirdness.  WebKit has its own DWARF parser for this, which shouldn’t be necessary.
		- WebKit is about to have its own copies of both a compactunwind and EH frame parser.  The contributor who “wrote” the EH frame parser actually just took it from LLVM.  The licenses are compatible, but nonetheless, copy-paste from LLVM into WebKit should be discouraged.
- Engage with non-WebKit embedders that currently use the C++ API to figure out what it would take to get them to switch to the C API.

I think that a lot of time when C API discussions arise, lots of embedders give excuses for using the C++ API.  WebKit used the C API for generating IR and even doing some IR manipulation, and for driving the MCJIT.  It’s been a positive experience and we enjoy the binary compatibility that it gives us.  I think it would be great to see if other embedders can do the same.

-Filip

> 
> We will be sending more specific proposals and patches for each of the changes listed above starting this week. If you’re interested in these problems and their solutions, please speak up and help us develop a solution that will work for your needs and ours.
> 
> Thanks,
> -Chris
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev