[vmkit-commits] Breaking change for openjdk

[Nicolas Geoffray]

On Thu, Dec 29, 2011 at 1:06 AM, Will Dietz <wdietz2 at illinois.edu> wrote:

> On Thu, Dec 29, 2011 at 12:27 AM, Nicolas Geoffray
> <nicolas.geoffray at gmail.com> wrote:
> > Hi Will,
> >
> > Thanks very much for the instructions! I managed to get everything
> working
> > with two gotchas:
> > 1) libjvm.so could not be found. I had to set LD_LIBRARY_PATH to its
> > location.
>
> Ah, yes.  I should've mentioned that, hopefully that didn't give you
> too much trouble.
> Eventually should remove this (by creating our own libjvm of some
> kind, most likely), as requiring the setting of LD_LIBRARY_PATH is
> obnoxious if nothing else.
>
> > 2) Generating BootstrapClasses.s does not work on a machine with 1GB of
> RAM.
> > It did work on my 12GB work station. We might need to split it into
> mulitple
> > files.
> >
>
> Hmm, interesting.  This is just the class bytecodes emitted into
> assembly right?


Right.


> That should be split up easily enough, or perhaps
> when using OpenJDK we could limit the classes emitted in this manner,
> since we probably don't need to do this for all of them.
>

Yeah, both approaches could work.


>
> > Other than that, it worked like a charm! I did see a memory leak, but did
> > not investigate much. I also tried to use a generational GC but hit an
> MMTk
> > assertion quite soon. I will investigate.
> >
>
> Great to hear that it worked well for you!  As for memory issues, how
> do you find these leaks?  Valgrind?  I'm naturally interested in
> trying to tackle these myself but haven't found a particularly useful
> way to get a handle on the GC's behavior yet.


I found them by passing the option -X:gc:verbose=1 to j3. The memory always
grows when running multiple times the same benchmark.

Also, keep me posted on
> anything you find re:generational GC behavior, and thanks for looking
> into that :).
>
> > FYI, here are some numbers for the dacapo benchmarks. I can't wait to
> > see JVM_GetClassConstantPool implemented :) And hopefully, it's not as
> bad
> > as it looks. By the name, it looks like we need to have a common
> > representation of the constant pool between libjava.so and vmkit.
> >
>
> JVM_GetClassConstantPool as well as implementing Global Weak
> References (JNI) are the two big blockers at this point.
>
> The constant pool stuff seems fine, there's a class in OpenJDK
> "sun/reflect/ConstantPool" for this, with an Object field to be used
> to point to whatever representation we find useful, which is passed to
> the various JVM_* methods that operate on constant pools.
>

That's good news.


>
> I'm mid-implementing this, and was going the route of using this field
> to simply store the Class in question, which can be used in the other
> calls to find the appropriate constant pool (and means the GC will
> trace that ConstantPool object to the Class, preventing it from being
> unloaded while it's live).  I believe this should work, but we'll see
> as I finish implementing it :).
>
> I haven't tackled Global Weak References yet, but that shouldn't be
> too terrible given existing support for weak references and global
> references :).
>
> > avrora: 8490 (8103)
> > batik: segfault (java exception)
>
> For me this fails due to lack of Global Weak Reference support, and
> hacking those JNI calls to return global (non-weak) references was
> sufficient to make this benchmark complete.  Nothing that should be
> committed, but just FYI this benchmark should work once that support
> is in place.
>
> > eclipse: java exception (java exception)
>
> Fails here too with an invocation exception... strange since actual
> eclipse seems to work fine :).
>
> > fop: 1792 (java exception)
> > h2: java exception (gc error)
>
> Same here, not sure what's going on.
>
> > jython: unimplemented JVM_GetClassConstantPool (unimplemented
> > NewWeakGlobalRef)
> > luindex: 2185 (2135)
> > lusearch: 2131 (3683)
> > pmd: 3473 (3526)
> > sunflow: 5548 (4671)
> > tomcat: unimplemented JVM_GetClassConstantPool (segfault)
> > tradebeans: unimplemented JVM_GetClassConstantPool (java exception)
> > tradesoap: unimplemented JVM_GetClassConstantPool (java exception)
>
> Hopefully these fall into place once support for constant pools is added.
>
> > xalan: 21728 (2970)
> >
>
> Is the Classpath time a typo?  Locally I get 21919 (16550), which
> isn't good but isn't as dramatic as 10x either :).
>

No the classpath time was not a typo. FYI, I ran the benchmark on a 16 core
machine with 12GB of memory.


>
> > Nice work!
>
> Thanks, and I'm very glad you were able to get it up and running.
> Here's hoping we can get full dacapo 9-12 working soon-ish!
>
>
Can't wait for it! :)

Nicolas


> ~Will
>
> > Nicolas
> >
> > On Sun, Dec 18, 2011 at 7:55 AM, Will Dietz <wdietz2 at illinois.edu>
> wrote:
> >>
> >> On Fri, Dec 2, 2011 at 11:54 AM, Nicolas Geoffray
> >> <nicolas.geoffray at gmail.com> wrote:
> >> > Yes, I believe that's possible. Depends if openjdk needs fancy things
> >> > when
> >> > creating a WeakHashMap.
> >>
> >> How right you were (see commits involving hardcoding the
> >> java.lang.Reference's clinit as to not spawn its own thread....) but
> >> seems to be working fine now :).
> >>
> >> > This is great news! Does that mean precompilation and GC work on the
> >> > openjdk
> >> > port? That's awesome. Can't wait to try it out!
> >>
> >> Precompilation has worked for some time (the speedups it provides
> >> makes it a high priority :)), and I believe the GC works.  It
> >> certainly "mostly" works, but for example the limiting factor in
> >> specjvm2008 presently is that memory isn't being freed when it should,
> >> so I'm running out of memory when trying to do a full run--running
> >> individual benchmarks works much better.  In general the bugs I'm
> >> seeing seem to be more of a "memory not freed" variety not a "memory
> >> freed that shouldn't be", which is some comfort.
> >>
> >> Also, I've almost exclusively tested with the default MarkSweep mmtk
> >> plan, although the others seem to work in much more limited testing.
> >>
> >> As for trying this out yourself, here's a rough guide:
> >>
> >> ** Install an OpenJDK JDK6 (not Sun, not JRE, not 1.7).
> >> This should be available in your linux distribution's package manager,
> >> otherwise I've had great success with locally built copies of icedtea6
> >> (what I use for most my testing, actually).
> >>
> >> ** Replace the GNU Classpath-related configure arguments with
> >> "--with-classpath-impl=openjdk --with-openjdk-path=$JDK_HOME".
> >> JDK_HOME is in surprisingly different places on different
> >> distributions, but it should contain "bin/javac" as well as a "jre"
> >> subdirectory.  The configure script's checks on the path should tell
> >> you if you've got the right path.
> >>
> >> ** "make"
> >> Might want to cross your fingers first :).
> >>
> >>
> >> Some caveats when testing this on the various systems I have access to:
> >>
> >> * When precompiling, in particular building BootstrapClasses.s,
> >> sometimes gcc (at least with 4.4.5 on one particular system) barfs
> >> claiming invalid assembly.  Other versions of gcc build the same
> >> assembly fine, and "make CC=clang CXX=clang++" avoids this as well.
> >> FYI
> >> * Many recent OpenJDK versions seem to work (1.9 and later), although
> >> I've encountered one ("OpenJDK Runtime Environment (IcedTea6 1.9.10)
> >> (rhel-1.40.1.9.10.el6_1-x86_64)" from Scientific Linux 6.1) that has
> >> some dlopen/dlclose issue (of all things) that makes accessing fonts
> >> segfault.  Previous version from same distribution works as does
> >> building from scratch via IcedTea, so I've mostly given up on this.
> >>
> >> The abbreviated version of the above is that "some configurations
> >> don't work" and I don't have a particularly great handle on that due
> >> to lack of time and testing :).  So YMMV.
> >>
> >> If you do get to trying it please let me know how it goes/if you have
> >> problems/etc :).  Once it seems to work well for you, then I can work
> >> towards writing a more formal set of instructions for the website as
> >> you previously mentioned :).
> >>
> >> Good luck and happy holidays! :)
> >>
> >> ~Will
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/vmkit-commits/attachments/20111229/f8c92a80/attachment.html>