[LLVMdev] Lazy compilation in MCJIT (was RE: Old JIT Status (i.e., can we delete it?))

[David Chisnall]

On 27 Nov 2012, at 00:23, Albert Graef wrote:

> Yes. Let's not forget what JIT actually means; this *is* what makes a
> compiler a JIT compiler. In Pure you can add new functions and change
> existing ones at any time, by just typing an equation at the command
> prompt of Pure's interactive frontend. Other dynamic languages work in a
> similar fashion. The old JIT makes this very easy to do; you just
> (re)compile the function to IR on the fly, and the JIT will then take
> care of lowering the function to native code when it's first needed. For
> me this has always been one of the coolest features of LLVM, a feature
> which really sets it apart from traditional compiler technology.

Have you done any profiling of this?  I admit that I haven't since 2.9, but when I did I found that the time spent in the JIT itself was negligible compared to the time spent in optimisation, even with a fairly reduced set of passes.  And even that was pretty small if you are compiling a small module (less than a few dozen functions).  It's also relatively easy to implement this yourself on top of the existing JIT: just install a stub function that calls out to something that triggers the JIT to compile that module and then jumps to the real function when it's called.  

The biggest limitation of the JIT for this kind of use (dynamic languages with lots of short-lived functions / closures) is that it doesn't provide any way of detecting that a function is in use.  This really requires stopping the world and walking all of the threads' stacks to ensure that an old function is not being called (you can do it lazily by doing a periodic trace, if you can have a barrier somewhere in every function.  Things like libdispatch complicate this significantly because then you have threads that are harder to find).  It would be nice if we could have some generic infrastructure for doing this: notify the JIT that you have removed all public references to a function and want it to be deleted when it is no longer on any stack.  Doing this in a portable way is impossible (POSIX doesn't provide the required functionality, although FreeBSD and OS X do provide everything needed to do it via non-portable extensions and presumably other systems do as well) so it would need a lot of code in the support library.

David