[PATCH] Expose custom MC-JIT memory allocation through the C API

[Filip Pizlo]

I considered using an opaque struct with getters and setters. But instead I went with the old-school C idiom of having a struct that the user memset's to zero up to the size they saw:

memset(&functions, 0, sizeof(functions));

And then the LLVM bindings also memset according to what LLVM sees and does a copy:

memset(&myFunctions, 0, sizeof(myFunctions));
memcpy(&myFunctions, PassedFunctions, SizeOfPassedFunctions);

This ensures both forward source compatibility and forward binary compatibility, except if we wanted to remove a function:

Source compatibility for added functions: the user's compiler would see a larger sizeof(functions), and the memset() would zero-initialize those pointers, causing the bindings to provide default implementations. 

Binary compatibility for added functions: the user would end up passing a value of SizeOfPassedFunctions that is smaller than LLVM expected, and LLVM would zero-initialize the added functions. 

AFAIK, this is no less robust than an opaque struct. Both handle added functions gracefully, and neither can handle removed functions gracefully unless we do something crazy. The un-opaque struct just makes writing the code a bit easier, both for LLVM and for the client. But that's just my opinion. :-)

I am curious what y'all think about the weirder functions like registerEHFrames. It feels weird that this is part of the MM to begin with. 

-Filip

On May 16, 2013, at 1:50 PM, Sean Silva <silvas at purdue.edu> wrote:

> 
> 
> 
> On Thu, May 16, 2013 at 1:28 PM, Filip Pizlo <fpizlo at apple.com> wrote:
>> 
>> On May 16, 2013, at 10:42 AM, Sean Silva <silvas at purdue.edu> wrote:
>> 
>>> Is basing the JSC fourth tier on LLVM something that you guys have committed to, or mainly exploratory? You seem to describe it as a "study" on <https://bugs.webkit.org/show_bug.cgi?id=112840>.
>> 
>> If we can get LLVM to provide a speed-up over our own optimizing JIT, then it will be turned on in WebKit trunk.  As you can see from that bug, we've put a lot of work into this so far, and still have a lot of work ahead of us.  The results so far are promising and I like where it's going,
> 
> Great!
>  
>> but given the amount of work remaining I cannot commit to anything.
> 
> Given that this is generally useful functionality that will probably be needed by any serious use case, and that your work is already pretty far along, it's probably fine to expose this in the C API.
> 
> (Now to review the patch).
> 
> Factoring out RTDyldMemoryManager into its own header should be its own patch. This code move is probably a good idea to do anyway independently of adding functionality to the C API.
> 
> As for the API change, my concern is that it potentially exposes too much. As far as I can tell, `struct LLVMMCJITMemoryManagerFunctions` is basically a thin wrapper around the vtable of RTDyldMemoryManager, which raises the question of what will happen if RTDyldMemoryManager changes.
> 
> Rafael, Andrew: could you take a look at this patch? In particular, is this API stable enough that it will be OK to proxy the RTDyldMemoryManager API like this?
> 
> +    if (options.SizeOfMCJMMFunctions > sizeof(functions)) {
> +      *OutError = strdup(
> +        "Refusing to use MCJIT memory manager functions struct that is larger "
> +        "than my own; assuming LLVM library mismatch.");
> +      return 1;
> +    }
> 
> In order to avoid this, it would be better to expose a an opaque struct, and have all manipulation of that struct happen through getter/setter functions, which will push library mismatch errors to link time rather than runtime and overall be easier to maintain/extend. That opaque struct could also hold the `void *` callback data. Sadly, the surrounding code already falls into the brittle "sizeof" pattern.
> 
> -- Sean Silva
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20130516/1d720c9f/attachment.html>