[LLVMdev] Best way to use LLVM with byte code vm

[Eli Friedman]

On Thu, Sep 1, 2011 at 8:37 PM, Michael Clagett <mclagett at hotmail.com> wrote:
> So here are my questions (sorry to take so long to get to them).   If given
> the choice, am I generally better off taking my byte code level and
> translating it to LLVM intermediate code and then driving new LLVM-optimized
> intel code generation from there?  Or given that I already have each of
> these byte codes translated to intel assembler, am I better off just
> gathering all those assembler sequences together and optimizing this at the
> assembly language level.

LLVM can't optimize x86 assembly language in any meaningful way.  If
you already have a C implementation of your opcodes, you can use clang
to convert that to LLVM IR.

> I'm going to guess the former, since working at
> the byte code level, I may be able to substitute entire byte codes with more
> judicious use of registers and end up with a much shorter piece of
> aggregated assembler to work with.  But it could also be the case that there
> are no good rules of thumb and that I simply need to evaluate on a case by
> case basis.  But if anyone does have any wisdom to offer or sources they can
> direct me to, I would be most appreciative.

http://llvm.org/docs/tutorial/ is a good place to start for writing a
JIT'ing compiler with LLVM.  LLVM will usually be able to generate
substantially better code than just concatenating opcodes, but your
lightweight JIT probably spends much less time compiling the code.

> The other question is that a lot of my code isn't even written in byte code,
> but rather directly in assembly language using my built-in assembler.  For
> this code, does LLVM offer opportunities to optimize, even if the source
> assembler isn't something that has been generated from LLVM intermediate
> code?

Again, LLVM can't optimize x86 assembly language in any meaningful way.

> Also I'm imagining that if I succeed in optimize compiling a
> particular word into a sequence of assembly language, then any call to that
> word from another word (through my CALL op) can be translated to a jmp
> instruction, therby eliminating the substantial call overhead of my
> high-level forth code.  Once again any guidance would be greatly
> appreciated, even if it's just directing me to any good source material that
> might be helpful

LLVM can do tail call optimization, and there's an option to force all
calls in a tail call position to always be tail calls.  If you do
that, you'll never see a call instruction in the generated code, and
the state that gets passed around in registers will stay in registers.

-Eli