[LLVMdev] Lazy compilation in MCJIT (was RE: Old JIT Status (i.e., can we delete it?))

[Albert Graef]

On 11/26/2012 10:35 PM, Kaylor, Andrew wrote:
> So when you say 'lazy compilation' you mean 'deferred compilation of individual functions within a module until that function is needed,' right?  That's certainly what 'lazy compilation' meant in the legacy JIT.

Yes. Let's not forget what JIT actually means; this *is* what makes a
compiler a JIT compiler. In Pure you can add new functions and change
existing ones at any time, by just typing an equation at the command
prompt of Pure's interactive frontend. Other dynamic languages work in a
similar fashion. The old JIT makes this very easy to do; you just
(re)compile the function to IR on the fly, and the JIT will then take
care of lowering the function to native code when it's first needed. For
me this has always been one of the coolest features of LLVM, a feature
which really sets it apart from traditional compiler technology.

If the MCJIT lacks this feature then it's not a real JIT compiler for
me, it's an in-memory compiler/linker/loader. That's sufficient for many
applications, but if you're striving to eventually replace the old JIT
then you'll have to find a way to replicate the lazy compilation bit in
some way. Otherwise we'd need the old JIT to stick around forever to
support this kind of application (and then we'd also need a working ARM
backend for it).

> I don't think that will ever be possible in MCJIT.

That's bad news. :(

> I think the way to approach this problem is to deconstruct large modules into smaller modules which we can afford to re-JIT, and then optimize the process of linking modules together.  Currently, a single instance of MCJIT only supports handling of a single module, but I did recently add a mechanism to delay compilation of the module so that it doesn't happen when the MCJIT engine is constructed.

In Pure this would basically mean that each function would have to be in
its own module. (More precisely, each global Pure function. Pure also
has lexical closures, these could of course be in the same module as the
global functions they belong to.)

> Once MCJIT supports multiple modules, if it had a mechanism for determining dependencies between modules and managing lazy compilation of those dependencies would that be acceptable?

Yes, this might be an acceptable (if somewhat awkward) solution. But I'm
not really convinced that it's feasible. It would need a linker that's
able to cope with thousands of modules, add new modules and replace
existing ones at any time, and do this *fast*. Do you think that this
will be possible with MCJIT? Maintaining all these modules and the
linker tables in memory might add significant overhead compared to the
old JIT, right? Will it be at least as fast as the old JIT?

Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr.Graef at t-online.de, ag at muwiinfa.geschichte.uni-mainz.de
WWW:    http://www.musikinformatik.uni-mainz.de/ag