[LLVMdev] VMKit is retired (but you can help if you want!)
Gaël Thomas
gael.thomas00 at gmail.com
Tue Sep 9 16:42:53 PDT 2014
Oups, sorry for the mistake, llcj (not llc:)) is not more maintained!
Gaël
Le 10 sept. 2014 00:27, "Gaël Thomas" <gael.thomas00 at gmail.com> a écrit :
> Hi Brian,
>
> So, I confirm, llc is not more maintained. And using vmjc is probably
> the good starting point to translate Java bytecode into llvm bitcode.
>
> However, I think that your hack (changing the way VMKitGCPrinter.cpp
> resolves symbols) is probably not the good way to solve the initial
> problem. And maybe that your current problem is simply a consequence
> of the mangling problem. So, let's start with the first problem :)
>
> So, I answer in your previous mail. As it's the beginning of the year
> (I'm teaching), I haven't found any time to install vmkit and make the
> tests. But we will proceed step by step to understand the problem. In
> parallel, I'm also writing a documentation to describe the internals
> of VMKit, it will be useful for you and other people interested by the
> AOT.
>
> 2014-09-02 18:13 GMT+02:00 Brian Faull <bfaull at cog-e.com>:
> > Hi Gaël,
> >
> > So as not to get too far into the "weeds" -- let's start with minimal
> configuration, then `vmjc` (.class -> .bc), then `llc` (.bc -> asm) to make
> sure I'm on the right path!
> >
> > 1- CONFIGURATION
> > I can configure out of the box with no problem. But perhaps I need to
> configure differently.
> >
> > I start with a clean checkout of the VMKit repository into
> /path/to/vmkit_test-clean . I'm on x86_64 using OpenJDK1.6 as follows:
> >
> > ==========
> > /path/to/vmkit_test-clean
> > ./configure --with-llvm-config-path=/path/to/llvm33/bin/llvm-config
> --with-classpath-impl=openjdk
> --with-openjdk-path=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64
> > ==========
> >
> > I have PATH set to provide `javac` as
> /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/bin/javac and
> LD_LIBRARY_PATH to include
> /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64 and
> ...amd64/server .
> >
> > This `configure` command works. I see two WARNINGs during this build,
> as a result of
> > ==========
> > [vmkit ./mmtk/java]: Linking module 'FinalMMTk.bc'
> > /path/to/llvm33/bin/llvm-link
> /path/to/vmkit_test-clean/mmtk/java/Release+Asserts/mmtk-vmkit.bc
> /path/to/vmkit_test-clean/Release+Asserts/lib/MMTKAlloc.bc
> /path/to/vmkit_test-clean/Release+Asserts/lib/MMTKRuntime.bc -o
> /path/to/vmkit_test-clean/mmtk/java/Release+Asserts/FinalMMTk.bc
> > WARNING: Linking two modules of different data layouts!
> > WARNING: Linking two modules of different data layouts!
> > ==========
> > I'm not sure if that matters.
>
> It does not matter :)
>
>
> >
> > I have also tried OpenJDK 1.7 and GNU Classpath, and I think I see the
> same results, but perhaps I didn't reset environment variables or something.
> >
> > Some fundamental questions: basically, how do *YOU* configure VMKit?
> > -> Which version of Java do you recommend? 1.5, 1.6, 1.7...? Is there
> any benefit of using one or another, or any restrictions?
> > -> Do you recommend OpenJDK or GNU Classpath? Is there any benefit of
> using one or the other, or any tradeoff?
>
> You only have to use openjdk (gnu classpath is not more maintained as
> the project is also retired:)). We have a bug with the new versions of
> openjdk, and we are using the 6u23. I don't know if all the previous
> versions will be ok. So, I suggest using this version.
>
>
> >
> > 2- COMPILATION: `vmjc` and `llc` to for Java class to native assembly.
> > I do the following, with the same OpenJDK 1.6.0 `javac` command:
> >
> > ==========
> > cd /path/to/vmkit_test-clean/tools/trainer
> > javac HelloWorld.java
> > ../../Release+Asserts/bin/vmjc -print-aot-stats HelloWorld.class
> > /path/to/llvm33/bin/llc
> -load=../../Release+Asserts/lib/static-gc-printer.so HelloWorld.class.bc
> > ==========
> >
> > The `javac` command succeeds, and produces Java classfile
> HelloWorld.class. `java HelloWorld` outputs "Hello World".
> > The `vmjc` command succeeds, and produces LLVM bitcode
> HelloWorld.class.bc. `/path/to/llvm33/bin/llvm-nm HelloWorld.class.bc`
> shows reasonable symbols from that bitcode.
> > The `llc` command fails exactly the same as Dave Brazdil reported on 08
> Mar; http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-March/070995.html .
> > => It seems that I need to point to that shared-object using
> -load=../../Release+Asserts/lib/static-gc-printer.so . I found this in a
> previous post to the LLVM list. Is this correct?
>
> Yes, it's mandatory.
>
> > => The output I see is as follows:
> > ==========
> > llc: VmkitGCPrinter.cpp:268: llvm::Constant *FindMetadata(const
> llvm::Function &): Assertion `0 && "Should have found a JavaMethod"' failed.
> > 0 llc 0x000000000154d5ee
> llvm::sys::PrintStackTrace(_IO_FILE*) + 46
> > 1 llc 0x000000000154d8ab
> > 2 llc 0x000000000154db1e
> > 3 libpthread.so.0 0x0000003f31a0f710
> > 4 libc.so.6 0x0000003ee9232925 gsignal + 53
> > 5 libc.so.6 0x0000003ee9234105 abort + 373
> > 6 libc.so.6 0x0000003ee922ba4e
> > 7 libc.so.6 0x0000003ee922bb10 __assert_perror_fail + 0
> > 8 static-gc-printer.so 0x00007febfc02b135 FindMetadata(llvm::Function
> const&) + 3813
> > 9 static-gc-printer.so 0x00007febfc02b892
> > 10 llc 0x0000000000e620a5
> llvm::AsmPrinter::doFinalization(llvm::Module&) + 2597
> > 11 llc 0x00000000014bf0dc
> llvm::FPPassManager::doFinalization(llvm::Module&) + 92
> > 12 llc 0x00000000014bf58e
> llvm::MPPassManager::runOnModule(llvm::Module&) + 1134
> > 13 llc 0x00000000014bfb4e
> llvm::PassManagerImpl::run(llvm::Module&) + 302
> > 14 llc 0x00000000014bfda1
> llvm::PassManager::run(llvm::Module&) + 33
> > 15 llc 0x00000000005ca548
> > 16 llc 0x00000000005c9242 main + 226
> > 17 libc.so.6 0x0000003ee921ed1d __libc_start_main + 253
> > 18 llc 0x00000000005c8f49
> > Stack dump:
> > 0. Program arguments: /path/to/llvm33/bin/llc
> -load=../../Release+Asserts/lib/static-gc-printer.so HelloWorld.class.bc
> > Aborted (core dumped)
> > ==========
> > => Do you see something similar? Or what could be the configuration
> difference?
> >
> > I fixed this situation by modifying the sources backing
> static-gc-printer.so, as shown below.
> >
> > Are we on the right path? Is this a good start?
>
> It's a very good start :) But your hack is not the good solution
> because you should return from FindMetaData at the line 229 (you
> should not reach the test at line 239).
>
> So, can you try this compilation scheme (without your hack in
> StaticGCPrinter.cpp:))?
>
> vmjc -print-aot-stats HelloWorld.class
> opt -load=../../Release+Asserts/lib/static-gc-pass.so -StaticGCPass
> HelloWorld.class.bc -o HelloWorld.class.gc.bc
> llc -load=../../Release+Asserts/lib/static-gc-printer.so
> HelloWorld.class.gc.bc
>
> If it does not change anything, can you send me the
> HelloWorld.class.bc file? I will quickly see if everything is ok for
> the GC and where is the problem.
>
> See you,
> Gaël
>
> >
> > Thank you again for your help. I really appreciate it.
> > =brian
>
>
>
>
>
>
>
>
> 2014-09-09 21:51 GMT+02:00 Brian Faull <bfaull at cog-e.com>:
> > Hello again Gaël, (et al)
> >
> > More on rekindling work on VMKit! Thank you for your interactions thus
> far on- and off-list.
> >
> > As you suggested in your VMKit-retirement email (to which I'm attempting
> to respond), I'm interested in producing a Java-to-LLVM compiler out of
> VMKit. I'd like to take you up on your offer to help understand the
> architecture. If I can get the a Java-to-LLVM compiler working for my
> purposes, I'll be happy to maintain at least this part of the VMKit project.
> >
> > To that end, I have exactly one fundamental question to start:
> >
> >
> > How do you suggest to perform Java-to-LLVM compilation using VMKit?
> >
> >
> > Here are my thoughts on this:
> >
> > It looks like the `llcj` tool (VMKit/tools/llcj/llcj.cpp) was meant for
> this; it declares itself as "Java to native compiler" but it appears that
> it hasn't been maintained (e.g., deprecated interfaces, no-longer-existent
> command-line options and libraries). `llcj` is described in Appendix A of
> Geoffray's thesis,
> http://pagesperso-systeme.lip6.fr/Nicolas.Geoffray/these-geoffray.pdf
> linked from the VMKit site.
> >
> > `llcj` attempts the following to translate Java to native:
> > * vmjc (Java .class => LLVM bitcode .bc)
> > * opt (LLVM bitcode optimizer)
> > * llc (LLVM bitcode => target assembly .s)
> > * gcc (assemble and link)
> >
> > Is that "The Right Way"?
> >
> > After some time hacking various changes, I can use basically that method
> to compile VMKit/tools/trainer/HelloWorld.java into a linked executable
> (but it barfs at runtime and I can't fix it).
> >
> > I'm doing roughly the following, on x86_64, using LLVM3.3, OpenJDK 1.6.0
> build 30, a debug build of VMKit (required small build-hacks, which I could
> describe on request), and the (perhaps-incorrect) edit to
> `static-gc-printer.so` described in an earlier post:
> http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-August/076128.html :
> > cd /path/to/vmkit/tools/trainer
> > javac HelloWorld.java
> > ../../Debug+Asserts/bin/vmjc -main=HelloWorld -print-aot-stats
> HelloWorld.class
> > /path/to/llvm33/bin/llc
> -load=../../Debug+Asserts/lib/static-gc-printer.so HelloWorld.class.bc
> >
> > That produces for me a reasonable native assembly file.
> >
> > Next I assemble and link, using a set of libraries/objects inspired by
> `llcj` and refined by a semi-automated trial-and-error:
> > /path/to/llvm33/bin/clang++ -g3 -O0 -o HelloWorld.class.exe
> HelloWorld.class.sed.s -L/path/to/vmkit_test-codechanges/Debug+Asserts/lib
> -L/path/to/llvm33/lib -pthread -lm -ldl -lz -lJ3 -lClasspath
> -lJ3Compiler -lCommonThread -lVmkit -lVmkitCompiler -lPrecompiled
> -lFinalMMTk -lLLVMX86Info -lLLVMX86CodeGen -lLLVMX86Desc -lLLVMX86Utils
> -lLLVMX86AsmPrinter -lLLVMExecutionEngine -lLLVMScalarOpts -lLLVMipo
> -lLLVMipa -lLLVMAnalysis -lLLVMInstCombine -lLLVMInstrumentation
> -lLLVMTarget -lLLVMJIT -lLLVMRuntimeDyld -lLLVMObject -lLLVMObjCARCOpts
> -lLLVMVectorize -lLLVMTransformUtils -lLLVMSelectionDAG -lLLVMCodeGen
> -lLLVMMC -lLLVMCore -lLLVMSupport -rdynamic
> >
> > This links and executes, but here are my problems/hacks:
> > * `llc` generates an assembly file with symbols (e.g., ____Vstatic_buf()
> and friends) that conflict with one of the archives listed above
> (libPrecompiled.a, which I believe is necessary for java_lang_Object and
> others). I basically remove those symbols (sed '/_buf$/ s/globl/weak/'
> HelloWorld.class.s > HelloWorld.class.sed.s) before assembling. I don't
> know if this is a reasonable hack or if it's causing problems.
> > * I get the following error at runtime (because some fields are NULL)
> and I can't make progress:
> > HelloWorld.class.exe: JavaRuntimeJIT.cpp:381: void
> *j3ResolveVirtualStub(j3::JavaObject *): Assertion `FI->Metadata != __null
> && "Wrong stack trace"' failed.
> >
> > Am I on the right path, or should I be considering other methods? Or
> wrong entirely?
> >
> > Ultimately, I propose changing `llcj` into something like a Python
> script that executes the appropriate utilities to start from Java source
> and resulting in a linked target-architecture executable, using `clang++`
> as compiler-driver for assembling and linking. It would be essentially a
> work-alike to `gcj`.
> >
> > Can you help point me in the right direction?
> >
> > Thank you,
> > Brian
> >
> >
> >
> >
> >
> > Date: Mon, 1 Sep 2014 21:34:58 +0200
> > From: Gaël Thomas <gael.thomas00 at gmail.com>
> > To: LLVM Developers Mailing List <llvmdev at cs.uiuc.edu>, Nicolas
> > Geoffray <nicolas.geoffray at gmail.com>
> > Subject: [LLVMdev] VMKit is retired (but you can help if you want!)
> > Message-ID:
> > <CAOWuPDcZBpt_JJ5yo5YN=C+RWbtbneXB1UGd90d0mXdnrs8=
> RQ at mail.gmail.com>
> > Content-Type: text/plain; charset=UTF-8
> >
> > Hi all,
> >
> > So, as explained in the LLVM weekly, the VMKit project is retired. It
> > was a very fun project, but we don't have any manpower to maintain the
> > code since one year. If someone is interested by restarting the
> > project, just send me an email, I can help to understand the
> > architecture of the project and how it works. I'm pretty sure that we
> > can also extract a Java to LLVM compiler easily (I know that some
> > people are interested by that).
> >
> > And I want to thank all the LLVM team for their support during these
> > last ten (yes ten:)) years! Without their help, it would have been
> > impossible to develop VMKit.
> >
> > See you and thank you!
> > Gaël
> >
> >
> >
> >
>
>
>
> --
> -------------------------------------------------------------------
> Gaël Thomas, Associate Professor, UPMC
> http://pagesperso-systeme.lip6.fr/Gael.Thomas/
> -------------------------------------------------------------------
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140910/1250d6c8/attachment.html>
More information about the llvm-dev
mailing list