[LLVMdev] VMKit is retired (but you can help if you want!)

[Brian Faull]

Hello again Gaël, (et al)

More on rekindling work on VMKit! Thank you for your interactions thus far on- and off-list.

As you suggested in your VMKit-retirement email (to which I'm attempting to respond), I'm interested in producing a Java-to-LLVM compiler out of VMKit. I'd like to take you up on your offer to help understand the architecture.  If I can get the a Java-to-LLVM compiler working for my purposes, I'll be happy to maintain at least this part of the VMKit project.

To that end, I have exactly one fundamental question to start:


How do you suggest to perform Java-to-LLVM compilation using VMKit?


Here are my thoughts on this:

It looks like the `llcj` tool (VMKit/tools/llcj/llcj.cpp) was meant for this; it declares itself as "Java to native compiler" but it appears that it hasn't been maintained (e.g., deprecated interfaces, no-longer-existent command-line options and libraries). `llcj` is described in Appendix A of Geoffray's thesis, http://pagesperso-systeme.lip6.fr/Nicolas.Geoffray/these-geoffray.pdf linked from the VMKit site. 

`llcj` attempts the following to translate Java to native:
* vmjc (Java .class => LLVM bitcode .bc)
* opt  (LLVM bitcode optimizer)
* llc  (LLVM bitcode => target assembly .s)
* gcc  (assemble and link)

Is that "The Right Way"?

After some time hacking various changes, I can use basically that method to compile VMKit/tools/trainer/HelloWorld.java into a linked executable (but it barfs at runtime and I can't fix it).  

I'm doing roughly the following, on x86_64, using LLVM3.3, OpenJDK 1.6.0 build 30, a debug build of VMKit (required small build-hacks, which I could describe on request), and the (perhaps-incorrect) edit to `static-gc-printer.so` described in an earlier post: http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-August/076128.html :
    cd /path/to/vmkit/tools/trainer
    javac HelloWorld.java
    ../../Debug+Asserts/bin/vmjc -main=HelloWorld -print-aot-stats HelloWorld.class
    /path/to/llvm33/bin/llc -load=../../Debug+Asserts/lib/static-gc-printer.so HelloWorld.class.bc 

That produces for me a reasonable native assembly file.  

Next I assemble and link, using a set of libraries/objects inspired by `llcj` and refined by a semi-automated trial-and-error:
    /path/to/llvm33/bin/clang++ -g3 -O0 -o HelloWorld.class.exe HelloWorld.class.sed.s -L/path/to/vmkit_test-codechanges/Debug+Asserts/lib -L/path/to/llvm33/lib -pthread -lm -ldl -lz    -lJ3 -lClasspath -lJ3Compiler -lCommonThread -lVmkit -lVmkitCompiler -lPrecompiled -lFinalMMTk    -lLLVMX86Info -lLLVMX86CodeGen -lLLVMX86Desc -lLLVMX86Utils -lLLVMX86AsmPrinter    -lLLVMExecutionEngine -lLLVMScalarOpts -lLLVMipo -lLLVMipa -lLLVMAnalysis -lLLVMInstCombine -lLLVMInstrumentation -lLLVMTarget -lLLVMJIT -lLLVMRuntimeDyld -lLLVMObject -lLLVMObjCARCOpts -lLLVMVectorize -lLLVMTransformUtils -lLLVMSelectionDAG -lLLVMCodeGen -lLLVMMC -lLLVMCore -lLLVMSupport    -rdynamic

This links and executes, but here are my problems/hacks:
* `llc` generates an assembly file with symbols (e.g., ____Vstatic_buf() and friends) that conflict with one of the archives listed above (libPrecompiled.a, which I believe is necessary for java_lang_Object and others). I basically remove those symbols (sed '/_buf$/ s/globl/weak/' HelloWorld.class.s > HelloWorld.class.sed.s) before assembling.  I don't know if this is a reasonable hack or if it's causing problems.
* I get the following error at runtime (because some fields are NULL) and I can't make progress:
HelloWorld.class.exe: JavaRuntimeJIT.cpp:381: void *j3ResolveVirtualStub(j3::JavaObject *): Assertion `FI->Metadata != __null && "Wrong stack trace"' failed.

Am I on the right path, or should I be considering other methods?  Or wrong entirely?

Ultimately, I propose changing `llcj` into something like a Python script that executes the appropriate utilities to start from Java source and resulting in a linked target-architecture executable, using `clang++` as compiler-driver for assembling and linking. It would be essentially a work-alike to `gcj`.

Can you help point me in the right direction?

Thank you,
Brian





Date: Mon, 1 Sep 2014 21:34:58 +0200
From: Gaël Thomas <gael.thomas00 at gmail.com>
To: LLVM Developers Mailing List <llvmdev at cs.uiuc.edu>,	Nicolas
	Geoffray <nicolas.geoffray at gmail.com>
Subject: [LLVMdev] VMKit is retired (but you can help if you want!)
Message-ID:
	<CAOWuPDcZBpt_JJ5yo5YN=C+RWbtbneXB1UGd90d0mXdnrs8=RQ at mail.gmail.com>
Content-Type: text/plain; charset=UTF-8

Hi all,

So, as explained in the LLVM weekly, the VMKit project is retired. It
was a very fun project, but we don't have any manpower to maintain the
code since one year. If someone is interested by restarting the
project, just send me an email, I can help to understand the
architecture of the project and how it works. I'm pretty sure that we
can also extract a Java to LLVM compiler easily (I know that some
people are interested by that).

And I want to thank all the LLVM team for their support during these
last ten (yes ten:)) years! Without their help, it would have been
impossible to develop VMKit.

See you and thank you!
Gaël