<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 01/14/2015 12:05 AM, Lang Hames
wrote:<br>
</div>
<blockquote
cite="mid:CALLttgr9Q_2JYGM8bVwy+cNzBJx64Zh_n3hNBYVb7Ayx1n5qKA@mail.gmail.com"
type="cite">
<div dir="ltr">
<div style="font-size:13px">Hi All,<br>
</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">The attached patch (against r225842)
contains some new JIT APIs that I've been working on. I'm
going to start breaking it up, tidying it up, and submitting
patches to llvm-commits soon, but while I'm working on that I
thought I'd put the whole patch out for the curious to start
playing around with and/or commenting on.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">The aim of these new APIs is to
cleanly support a wider range of JIT use cases in LLVM, and to
recover some of the functionality lost when the legacy JIT was
removed. In particular, I wanted to see if I could re-enable
lazy compilation while following MCJIT's design philosophy of
relying on the MC layer and module-at-a-time compilation. The
attached patch goes some way to addressing these aims, though
there's a lot still to do.</div>
</div>
</blockquote>
In terms of the overall idea, I like what your proposing. However,
I want to be very clear: you are not planning on removing any
functionality from the existing (fairly low level) MCJIT interface
right? We've built our own infrastructure around that and require a
few features it doesn't sounds like you're planning on supporting in
the new abstractions. (The biggest one is that we "install" code
into a different location from where it was compiled.) <br>
<br>
I really like the idea of having a low level JIT interface for
advanced users and an easy starting point for folks getting
started. <br>
<br>
<blockquote
cite="mid:CALLttgr9Q_2JYGM8bVwy+cNzBJx64Zh_n3hNBYVb7Ayx1n5qKA@mail.gmail.com"
type="cite">
<div dir="ltr">
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">The 20,000 ft overview, for those
who want to get straight to the code:</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">The new APIs are not built on top of
the MCJIT class, as I didn't want a single class trying to be
all things to all people. Instead, the new APIs consist of a
set of software components for building JITs. The idea is that
you should be able to take these off the shelf and compose
them reasonably easily to get the behavior that you want. In
the future I hope that people who are working on LLVM-based
JITs, if they find this approach useful, will contribute back
components that they've built locally and that they think
would be useful for a wider audience. As a demonstration of
the practicality of this approach the attached patch contains
a class, MCJITReplacement, that composes some of the
components to re-create the behavior of MCJIT. This works well
enough to pass all MCJIT regression and unit tests on Darwin,
and all but four regression tests on Linux. The patch also
contains the desired "new" feature: Function-at-a-time lazy
jitting in roughly the style of the legacy JIT. The attached
lazydemo.tgz file contains a program which composes the new
JIT components (including the lazy-jitting component) to
lazily execute bitcode. I've tested this program on Darwin and
it can run non-trivial benchmark programs, e.g. 401.bzip2 from
SPEC2006.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">These new APIs are named after the
motivating feature: On Request Compilation, or ORC. I believe
the logo potential is outstanding. I'm picturing an Orc riding
a Dragon. If I'm honest this was at least 45% of my motivation
for doing this project*.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">You'll find the new headers in
llvm/include/llvm/ExecutionEngine/OrcJIT/*.h, and the
implementation files in lib/ExecutionEngine/OrcJIT/*.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">I imagine there will be a number of
questions about the design and implementation. I've tried to
preempt a few below, but please fire away with anything I've
left out.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">Also, thanks to Jim Grosbach,
Michael Illseman, David Blaikie, Pete Cooper, Eric
Christopher, and Louis Gerbarg for taking time out to review,
discuss and test this thing as I've worked on it.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">Cheers,</div>
<div style="font-size:13px">Lang.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">Possible questions:</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">(1)</div>
<div style="font-size:13px">Q. Are you trying to kill off MCJIT?</div>
<div style="font-size:13px">A. There are no plans to remove
MCJIT. The new APIs are designed to live alongside it.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">(2)</div>
<div style="font-size:13px">Q. What do "JIT components" look
like, and how do you compose them?<br>
</div>
<div style="font-size:13px">A. The classes and functions you'll
find in OrcJIT/*.h fall into two rough categories: Layers and
Utilities. Layers are classes that implement a small common
interface that makes them easy to compose:<br>
</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">class SomeLayer {</div>
<div style="font-size:13px">private:</div>
<div style="font-size:13px"> // Implementation details</div>
<div style="font-size:13px">public:</div>
<div style="font-size:13px"> // Implementation details</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px"> typedef ??? Handle;</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px"> template <typename
ModuleSet></div>
<div style="font-size:13px"> Handle
addModuleSet(ModuleSet&& Ms);</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px"> void removeModuleSet(Handle H);</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px"> uint64_t
getSymbolAddress(StringRef Name, bool ExportedSymbolsOnly);</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px"> uint64_t
lookupSymbolAddressIn(Handle H, StringRef Name, bool
ExportedSymbolsOnly);</div>
<div style="font-size:13px">};</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">Layers are usually designed to sit
one-on-top-of-another, with each doing some sort of useful
work before handing off to the layer below it. The layers that
are currently included in the patch are the the
CompileOnDemandLayer, which breaks up modules and redirects
calls to not-yet-compiled functions back into the JIT; the
LazyEmitLayer, which defers adding modules to the layer below
until a symbol in the module is actually requested; the
IRCompilingLayer, which compiles bitcode to objects; and the
ObjectLinkingLayer, which links sets of objects in memory
using RuntimeDyld.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">Utilities are everything that's not
a layer. Ideally the heavy lifting is done by the utilities.
Layers just wrap certain uses-cases to make them easy to
compose.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">Clients are free to use utilities
directly, or compose layers, or implement new utilities or
layers.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">(3)</div>
<div style="font-size:13px">Q. Why "addModuleSet" rather than
"addModule"?</div>
<div style="font-size:13px">A. Allowing multiple modules to be
passed around together allows layers lower in the stack to
perform interesting optimizations. E.g. direct calls between
objects that are allocated sufficiently close in memory. To
add a single Module you just add a single-element set.</div>
</div>
</blockquote>
Please add a utility function for a single Module if you haven't
already. For a method based JIT use case, multiple Modules just
aren't that useful. <br>
<blockquote
cite="mid:CALLttgr9Q_2JYGM8bVwy+cNzBJx64Zh_n3hNBYVb7Ayx1n5qKA@mail.gmail.com"
type="cite">
<div dir="ltr">
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">(4)</div>
<div style="font-size:13px">Q. What happened to "finalize"?</div>
<div style="font-size:13px">A. In the Orc APIs, getSymbolAddress
automatically finalizes as necessary before returning
addresses to the client. When you get an address back from
getSymbolAddress, that address is ready to call.</div>
</div>
</blockquote>
As long as this is true for the high level API and *not* the low
level one (as is true today), this seems fine. I don't really like
the finalize mechanism we have, but we do need a mechanism to get at
the code before relocations have been applied. <br>
<blockquote
cite="mid:CALLttgr9Q_2JYGM8bVwy+cNzBJx64Zh_n3hNBYVb7Ayx1n5qKA@mail.gmail.com"
type="cite">
<div dir="ltr">
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">(5)</div>
<div style="font-size:13px">Q. What does "removeModuleSet" do?</div>
<div style="font-size:13px">A. It removes the modules
represented by the handle from the JIT. The meaning of this is
specific to each layer, but generally speaking it means that
any memory allocated for those modules (and their
corresponding Objects, linked sections, etc) has been freed,
and the symbols those modules provided are now undefined.
Calling getSymbolAddress for a symbol that was defined in a
module that has been removed is expected to return '0'.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">(5a)</div>
<div style="font-size:13px">Q. How are the linked sections
freed? RTDyldMemoryManager doesn't have any "free.*Section"
methods.</div>
<div style="font-size:13px">A. Each ModuleSet gets its own
RTDyldMemoryManager, and that is destroyed when the module set
is freed. The choice of RTDyldMemoryManager is up to the
client, but the standard memory managers will free the memory
allocated for the linked sections when they're destroyed.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">(6)</div>
<div style="font-size:13px">Q. How does the CompileOnDemand
layer redirect calls to the JIT?</div>
<div style="font-size:13px">A. It currently uses
double-indirection: Function bodies are extracted into new
modules, and the body of the original function is replaced
with an indirect call to the extracted body. The pointer for
the indirect call is initialized by the JIT to point at some
inline assembly which is injected into the module, and this
calls back in to the JIT to trigger compilation of the
extracted body. In the future I plan to make the redirection
strategy a parameter of the CompileOnDemand layer.
Double-indirection is the safest: It preserves
function-pointer equality and works with non-writable
executable memory, however there's no reason we couldn't use
single indirection (for extra speed where pointer-equality
isn't required), or patchpoints (for clients who can allocate
writable/executable memory), or any combination of the three.
My intent is that this should be up to the client.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">As a brief note: it's worth noting
that the CompileOnDemand layer doesn't handle lazy compilation
itself, just lazy symbol resolution (i.e. symbols are resolved
on first call, not when compiling). If you've put the
CompileOnDemand layer on top of the LazyEmitLayer then
deferring symbol lookup automatically defers compilation.
(E.g. You can remove the LazyEmitLayer in main.cpp of the
lazydemo and you'll get indirection and callbacks, but no lazy
compilation). </div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">(7)</div>
<div style="font-size:13px">Q. Do the new APIs support
cross-target JITing like MCJIT does?</div>
<div style="font-size:13px">A. Yes.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">(7.a)</div>
<div style="font-size:13px">Q. Do the new APIs support
cross-target (or cross process) lazy-jitting?</div>
<div style="font-size:13px">A. Not yet, but all that is required
is for us to add a small amount of runtime to the JIT'd
process to call back in to the JIT via some RPC mechanism.
There are no significant barriers to implementing this that
I'm aware of.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">(8)</div>
<div style="font-size:13px">Q. Do any of the components
implement the ExecutionEngine interface?</div>
<div style="font-size:13px">A. None of the components do, but
the MCJITReplacement class does.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">(9)</div>
<div style="font-size:13px">Q. Does this address any of the
long-standing issues with MCJIT - Stackmap parsing? Debugging?
Thread-local-storage?</div>
<div style="font-size:13px">A. No, but it doesn't get in the way
either. These features are still on the road-map (such as it
exists) and I'm hoping that the modular nature of Orc will us
to play around with new features like this without any risk of
disturbing existing clients, and so allow us to make faster
progress.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">(10)</div>
<div style="font-size:13px">Q. Why is part X of the patch (ugly
| buggy | in the wrong place) ?</div>
<div style="font-size:13px">A. I'm still tidying the patch up -
please save patch specific feedback for for llvm-commits,
otherwise we'll get cross-talk between the threads. The
patches should be coming soon.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">---</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">As mentioned above, I'm happy to
answer further general questions about what these APIs can do,
or where I see them going. Feedback on the patch itself should
be directed to the llvm-commits list when I start posting
patches there for discussion.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">* Marketing slogans abound: "Very
MachO". "Some warts". "Surprisingly friendly with ELF". "Not
yet on speaking terms with DWARF".</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
LLVM Developers mailing list
<a class="moz-txt-link-abbreviated" href="mailto:LLVMdev@cs.uiuc.edu">LLVMdev@cs.uiuc.edu</a> <a class="moz-txt-link-freetext" href="http://llvm.cs.uiuc.edu">http://llvm.cs.uiuc.edu</a>
<a class="moz-txt-link-freetext" href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a>
</pre>
</blockquote>
<br>
</body>
</html>