On Thu, Dec 29, 2011 at 1:06 AM, Will Dietz <span dir="ltr"><<a href="mailto:wdietz2@illinois.edu">wdietz2@illinois.edu</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im">On Thu, Dec 29, 2011 at 12:27 AM, Nicolas Geoffray<br>
<<a href="mailto:nicolas.geoffray@gmail.com">nicolas.geoffray@gmail.com</a>> wrote:<br>
> Hi Will,<br>
><br>
> Thanks very much for the instructions! I managed to get everything working<br>
> with two gotchas:<br>
> 1) libjvm.so could not be found. I had to set LD_LIBRARY_PATH to its<br>
> location.<br>
<br>
</div>Ah, yes. I should've mentioned that, hopefully that didn't give you<br>
too much trouble.<br>
Eventually should remove this (by creating our own libjvm of some<br>
kind, most likely), as requiring the setting of LD_LIBRARY_PATH is<br>
obnoxious if nothing else.<br>
<div class="im"><br>
> 2) Generating BootstrapClasses.s does not work on a machine with 1GB of RAM.<br>
> It did work on my 12GB work station. We might need to split it into mulitple<br>
> files.<br>
><br>
<br>
</div>Hmm, interesting. This is just the class bytecodes emitted into<br>
assembly right? </blockquote><div><br></div><div>Right.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">That should be split up easily enough, or perhaps<br>
when using OpenJDK we could limit the classes emitted in this manner,<br>
since we probably don't need to do this for all of them.<br></blockquote><div><br></div><div>Yeah, both approaches could work.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im"><br>
> Other than that, it worked like a charm! I did see a memory leak, but did<br>
> not investigate much. I also tried to use a generational GC but hit an MMTk<br>
> assertion quite soon. I will investigate.<br>
><br>
<br>
</div>Great to hear that it worked well for you! As for memory issues, how<br>
do you find these leaks? Valgrind? I'm naturally interested in<br>
trying to tackle these myself but haven't found a particularly useful<br>
way to get a handle on the GC's behavior yet. </blockquote><div><br></div><div>I found them by passing the option -X:gc:verbose=1 to j3. The memory always grows when running multiple times the same benchmark.</div><div>
<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Also, keep me posted on<br>
anything you find re:generational GC behavior, and thanks for looking<br>
into that :).<br>
<div class="im"><br>
> FYI, here are some numbers for the dacapo benchmarks. I can't wait to<br>
> see JVM_GetClassConstantPool implemented :) And hopefully, it's not as bad<br>
> as it looks. By the name, it looks like we need to have a common<br>
> representation of the constant pool between libjava.so and vmkit.<br>
><br>
<br>
</div>JVM_GetClassConstantPool as well as implementing Global Weak<br>
References (JNI) are the two big blockers at this point.<br>
<br>
The constant pool stuff seems fine, there's a class in OpenJDK<br>
"sun/reflect/ConstantPool" for this, with an Object field to be used<br>
to point to whatever representation we find useful, which is passed to<br>
the various JVM_* methods that operate on constant pools.<br></blockquote><div><br></div><div>That's good news.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
I'm mid-implementing this, and was going the route of using this field<br>
to simply store the Class in question, which can be used in the other<br>
calls to find the appropriate constant pool (and means the GC will<br>
trace that ConstantPool object to the Class, preventing it from being<br>
unloaded while it's live). I believe this should work, but we'll see<br>
as I finish implementing it :).<br>
<br>
I haven't tackled Global Weak References yet, but that shouldn't be<br>
too terrible given existing support for weak references and global<br>
references :).<br>
<div class="im"><br>
> avrora: 8490 (8103)<br>
> batik: segfault (java exception)<br>
<br>
</div>For me this fails due to lack of Global Weak Reference support, and<br>
hacking those JNI calls to return global (non-weak) references was<br>
sufficient to make this benchmark complete. Nothing that should be<br>
committed, but just FYI this benchmark should work once that support<br>
is in place.<br>
<div class="im"><br>
> eclipse: java exception (java exception)<br>
<br>
</div>Fails here too with an invocation exception... strange since actual<br>
eclipse seems to work fine :).<br>
<div class="im"><br>
> fop: 1792 (java exception)<br>
> h2: java exception (gc error)<br>
<br>
</div>Same here, not sure what's going on.<br>
<div class="im"><br>
> jython: unimplemented JVM_GetClassConstantPool (unimplemented<br>
> NewWeakGlobalRef)<br>
> luindex: 2185 (2135)<br>
> lusearch: 2131 (3683)<br>
> pmd: 3473 (3526)<br>
> sunflow: 5548 (4671)<br>
> tomcat: unimplemented JVM_GetClassConstantPool (segfault)<br>
> tradebeans: unimplemented JVM_GetClassConstantPool (java exception)<br>
> tradesoap: unimplemented JVM_GetClassConstantPool (java exception)<br>
<br>
</div>Hopefully these fall into place once support for constant pools is added.<br>
<br>
> xalan: 21728 (2970)<br>
><br>
<br>
Is the Classpath time a typo? Locally I get 21919 (16550), which<br>
isn't good but isn't as dramatic as 10x either :).<br></blockquote><div><br></div><div>No the classpath time was not a typo. FYI, I ran the benchmark on a 16 core machine with 12GB of memory.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
> Nice work!<br>
<br>
Thanks, and I'm very glad you were able to get it up and running.<br>
Here's hoping we can get full dacapo 9-12 working soon-ish!<br>
<span class="HOEnZb"><font color="#888888"><br></font></span></blockquote><div><br></div><div>Can't wait for it! :)</div><div><br></div><div>Nicolas</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<span class="HOEnZb"><font color="#888888">
~Will<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
> Nicolas<br>
><br>
> On Sun, Dec 18, 2011 at 7:55 AM, Will Dietz <<a href="mailto:wdietz2@illinois.edu">wdietz2@illinois.edu</a>> wrote:<br>
>><br>
>> On Fri, Dec 2, 2011 at 11:54 AM, Nicolas Geoffray<br>
>> <<a href="mailto:nicolas.geoffray@gmail.com">nicolas.geoffray@gmail.com</a>> wrote:<br>
>> > Yes, I believe that's possible. Depends if openjdk needs fancy things<br>
>> > when<br>
>> > creating a WeakHashMap.<br>
>><br>
>> How right you were (see commits involving hardcoding the<br>
>> java.lang.Reference's clinit as to not spawn its own thread....) but<br>
>> seems to be working fine now :).<br>
>><br>
>> > This is great news! Does that mean precompilation and GC work on the<br>
>> > openjdk<br>
>> > port? That's awesome. Can't wait to try it out!<br>
>><br>
>> Precompilation has worked for some time (the speedups it provides<br>
>> makes it a high priority :)), and I believe the GC works. It<br>
>> certainly "mostly" works, but for example the limiting factor in<br>
>> specjvm2008 presently is that memory isn't being freed when it should,<br>
>> so I'm running out of memory when trying to do a full run--running<br>
>> individual benchmarks works much better. In general the bugs I'm<br>
>> seeing seem to be more of a "memory not freed" variety not a "memory<br>
>> freed that shouldn't be", which is some comfort.<br>
>><br>
>> Also, I've almost exclusively tested with the default MarkSweep mmtk<br>
>> plan, although the others seem to work in much more limited testing.<br>
>><br>
>> As for trying this out yourself, here's a rough guide:<br>
>><br>
>> ** Install an OpenJDK JDK6 (not Sun, not JRE, not 1.7).<br>
>> This should be available in your linux distribution's package manager,<br>
>> otherwise I've had great success with locally built copies of icedtea6<br>
>> (what I use for most my testing, actually).<br>
>><br>
>> ** Replace the GNU Classpath-related configure arguments with<br>
>> "--with-classpath-impl=openjdk --with-openjdk-path=$JDK_HOME".<br>
>> JDK_HOME is in surprisingly different places on different<br>
>> distributions, but it should contain "bin/javac" as well as a "jre"<br>
>> subdirectory. The configure script's checks on the path should tell<br>
>> you if you've got the right path.<br>
>><br>
>> ** "make"<br>
>> Might want to cross your fingers first :).<br>
>><br>
>><br>
>> Some caveats when testing this on the various systems I have access to:<br>
>><br>
>> * When precompiling, in particular building BootstrapClasses.s,<br>
>> sometimes gcc (at least with 4.4.5 on one particular system) barfs<br>
>> claiming invalid assembly. Other versions of gcc build the same<br>
>> assembly fine, and "make CC=clang CXX=clang++" avoids this as well.<br>
>> FYI<br>
>> * Many recent OpenJDK versions seem to work (1.9 and later), although<br>
>> I've encountered one ("OpenJDK Runtime Environment (IcedTea6 1.9.10)<br>
>> (rhel-1.40.1.9.10.el6_1-x86_64)" from Scientific Linux 6.1) that has<br>
>> some dlopen/dlclose issue (of all things) that makes accessing fonts<br>
>> segfault. Previous version from same distribution works as does<br>
>> building from scratch via IcedTea, so I've mostly given up on this.<br>
>><br>
>> The abbreviated version of the above is that "some configurations<br>
>> don't work" and I don't have a particularly great handle on that due<br>
>> to lack of time and testing :). So YMMV.<br>
>><br>
>> If you do get to trying it please let me know how it goes/if you have<br>
>> problems/etc :). Once it seems to work well for you, then I can work<br>
>> towards writing a more formal set of instructions for the website as<br>
>> you previously mentioned :).<br>
>><br>
>> Good luck and happy holidays! :)<br>
>><br>
>> ~Will<br>
><br>
><br>
</div></div></blockquote></div><br>