[vmkit-commits] [PATCH] Build vmkit into libjvm.so for dynamic loading

[Will Dietz]

On Wed, Oct 26, 2011 at 4:18 PM, Nicolas Geoffray
<nicolas.geoffray at gmail.com> wrote:
> Hi Will,
> OK, I think I understand the problem, but I don't fully understand
> (especially with the patch you sent) the way you want to go. From what I
> read from you, we need to generate a libjvm.so file that will be loaded by
> libjava.so, itself being invoked by 'some' executable. Can this executable
> be some standard JNI code that uses JAVA_HOME or something to find
> libjvm.so? I think that's how HotSpot is bundled.

My apologies for my role in the confusion!

My primary goal is to get VMKit (which today means the set of tools
vmjc/j3/etc) using OpenJDK.  To that end, each of these tools needs to
use the runtime libraries to some extent (bootstrap them at least),
which for OpenJDK means being able to load libjava.so.

Turns out that you cannot load libjava.so without it loading some
libjvm.so--since it's a library dependency.  So what libjvm.so does it
load?

As a quick aside--the libjvm.so used during the mmtk compile phase,
for example, *cannot* be one that's fully compromised of VMKit, since
we're still building VMKit!

So this patch was an attempt to answer the "what libjvm.so does it
load" question with "our special libjvm.so", which was the result of
"tools/jvm" (perhaps not the best location, since it's not a tool at
all), and wasn't a libjvm.so that could be used externally, but a hack
that would meet libjava.so's dependency so that we could bootstrap
OpenJDK.

However, as mentioned this is a bit of a hack (it's not a "true"
libjvm, nor can it reasonably be), and presently has issues on 64bit
(something to do with link-once attribute on some
garbage-collection-related symbols)--on top of just being confusing
and lame to have in the codebase to begin with (see: this discussion).

The benefit is that we are always using a libjvm.so that to some
capacity compromises of VMKit code, which is conceptually nice, and is
the motivating reason for why this patch exists.

However, it turns out that we can actually "get away" with letting
libjava.so resolve its libjvm.so dependency with a libjvm.so from the
hotspot (which is done by not providing one of our own, aka by doing
nothing, since the JRE installation providing libjava.so also contains
a libjvm.so).  This is also hacky, since it counts on our tools
(j3/vmjc/etc) to already have defined all symbols from the hotspot
libjvm.so that "matter", so that while libjava.so loads a hotspot
libjvm.so to fix its dependency, it would never actually use any code
from it.  This requirement is already satisfied since we statically
link J3Classpath.a into our tools (and that's where the JVM_* stuff
lives in future patches).

That is to say, it turns out that while it's conceptually not the best
("why is the other libjvm.so loaded at all??"), it lets our tools work
with no changes which is nice.

Separate from this, later, we might find it useful to build all of
(everything that goes into j3 anyway) vmkit into a libjvm.so that can
be used with an OpenJDK JRE (like what's done here:
http://draenog.blogspot.com/2011/02/openjdkjamvm-git-repository.html
).  But this can be done much later after VMKit with OpenJDK actually
works :)

(Also, if the resulting VMKit works well enough, we could even
convince the OpenJDK guys to support building a vmkit-enabled JRE, as
they already do with cacao and jamvm, which would have numerous
benefits for us/VMKit/users)

In summary:
--VMKit needs a working runtime library to build itself (see MMTk and
precompiler)
--OpenJDK runtime library needs a libjvm.so, which ideally would be
something we create later using an already-built VMKit.  There might
be some engineering issues to accomplish this, but we cross that
bridge when we get to it.
--Proposal: When using our tools/launchers, let libjava.so load
whatever libjvm.so it wants, since loading it effectively does
nothing--it'll still use our symbols since they were defined first in
the j3/vmjc/etc process.  Ugly, but works, and we don't seem to have
many compelling options (although I'm happy to go a different way if
you think we can make it work/I'm missing something)
--Eventually we might want to build all of VMKit into a libjvm.so
suitable for use in an OpenJDK JRE, which would be great.  However
doing this doesn't help with the first point.  Chicken/egg, etc.

Hopefully this helps clarify things, and please let me know if you
have any questions/thoughts/etc :).

Thanks!

~Will