[LLVMdev] Dynamic updates of current executed code

[Chris Lattner]

On Tue, 20 Apr 2004, Anders Alexandersson wrote:

> Thanks!
>
> Problem is though, that the Ruby compiler is integrated in the
> compilation of the program being executed, to be able to parse & compile
> dynamic code at run-time. Therefore the calls to
> ExecutionEngine::getPointerToGlobal(F) need to be made in LLVM code.
> Here is a detailed simplistic example in pseudocode of what we want to
> do:

Okay, I think the problem is that you're trying to integrate the compiler
into the program being compiled.  Instead, you want to make a new "llvm
tool", say, "llvm-ruby" or something.  The input to llvm-ruby is a ruby
program, and the output is whatever the program produces, just like a
normal interpreter.

The key point of this is that you just make the compilation process
entirely JIT driven:  The first time a method or function is encountered,
you compile it from ruby, to llvm, to machine code.  There are two aspects
to this: ruby->LLVM (your part) and LLVM->machine code (our part), but
it's conceptually one ruby->machinecode step.  If you make it JIT driven,
dynamically loading code will not be a problem, and you don't need to
integrate the JIT into the program.

If you are interested in implementing this efficiently, I would *strongly*
recommend you design your ruby->LLVM converter to work with the C++
classes used to represent LLVM.  This will reduce the number of
translations and interfaces that the code will go through (and as Misha
pointed out, you will otherwise have to package up each function into it's
own module).  For examples of this, look at llvm/projects/ModuleMaker and
the Stacker front-end.

If this approach is acceptable to you, I would start with just a simple
*static* ruby -> LLVM compiler.  Doing a static compiler first will make
it easier to get up and running quickly and let you focus on the important
pieces (the ruby -> LLVM mapping, such as how method dispatch works).
However, when building this, you should keep an eye on making it modular
enough to allow function/method-at-a-time compilation.

Once you have a static compiler working reasonably well, you can integrate
the JIT into it.  Also, instead of parsing a compiling the ruby file from
top-to-bottom, you now start parsing and compiling on demand.  This is a
small change in the top-level flow of the compiler, but almost all of the
code you write should work unmodified.

This approach should allow you to dynamically load code into the
"ruby interpreter" and have it JIT compiled at the same time.  It has the
added advantage that you can do a purely static ruby compiler as well, so
long as it doesn't dynamically load code.

Please let me know if I'm not making any sense here. :)

-Chris

-- 
http://llvm.cs.uiuc.edu/
http://www.nondot.org/~sabre/Projects/