[LLVMdev] MC-JIT Design

[Török Edwin]

On Mon, 15 Nov 2010 10:15:09 -0800
Daniel Dunbar <daniel at zuster.org> wrote:

> Hi all,
> 
> As promised, here is the rough design of the upcoming MC-JIT*.
> Feedback appreciated!
> 
> (*) To be clear, we are only calling it the MC-JIT until we have
> finished killing the old one. When I say JIT below, I mean the MC-JIT.
> I basically am ignoring completely the existing JIT. I will keep
> things API compatible whenever possible, of course.
> 
> I see two main design directions for the JIT:
> 
> --
> 
> #1 (aka MCJIT) - We make a new MCJITStreamer which communicates with
> the JIT engine to arrange to plop code in the right place and update
> various state information.
> 
> This is the most obvious approach, is roughly similar to the way the
> existing JIT works, and this is the way the proposed MC-JIT patches
> work (see MCJITState object).
> 
> It also happens to not be the approach I want to take. :)
> 
> 
> #2 (aka FOOJIT) - MC grows a new "pure" backend, which is designed
> around representing everything that "can be run" on a target platform.
> This is very connected to the inherent capabilities of the hardware /
> OS, and is usually a superset** of what the native object format
> (Mach-O, ELF, COFF) can represent.
> 
> The "pure" backend defines a hard (but non-stable) object file format
> which is more or less a direct encoding of the native MC APIs (it is
> not stable, so it can directly encode things like FixupKind enum
> values).
> 
> I don't have a name for this format, so for now I will call it FOO.
> 
> The "MC-JIT" then becomes something more like a "FOO-JIT". It is
> architected as a consumer of "FOO" object files over time. The basic
> architecture is quite simple:
>  (a) Load a module, emit it as a "FOO" object.

While you are at it, can you support SELinux please?
SELinux only needs that you emit code to a writable memory region, and
execute it from a different executable memory region (both backed by an
unlinked tmpfile for example).
So you need to make your relocations "as if" you emitted the code at
the address of the executable region, but write them to the writable
map.

Right now in ClamAV we disable LLVM JIT when SELinux is in enforcing
mode (and fallback to our own interpreter).

>  (b) Load the object into a worklist, scan for undefined symbols,
> dynamically emit more "FOO" modules.

Will this allow symbol lookup to fail in a better way than
llvm_report_error? Like returning failure from getPointerToFunction, or
its equivalent.

Best regards,
--Edwin