<font size=2 face="sans-serif">Hi Lang,</font>
<br>
<br><font size=2 face="sans-serif">Thank you very much for the detailed
reply!! I will take a closer </font>
<br><font size=2 face="sans-serif">look at it and hopefully could start
implementing my task </font>
<br><font size=2 face="sans-serif">based on Orc API.</font>
<br>
<br><font size=2 face="sans-serif">Btw, by code cache I meant to have the
ability to run the</font>
<br><font size=2 face="sans-serif">the executed code from a place where
I could later </font>
<br><font size=2 face="sans-serif">patch it -- redirect calls to a new
version of functions</font>
<br><font size=2 face="sans-serif">and store new versions of functions
in it as well.</font>
<br>
<br><font size=2 face="sans-serif">Thanks again,</font>
<br><font size=2 face="sans-serif">Revital</font>
<br>
<br>
<br>
<br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">Lang Hames <lhames@gmail.com></font>
<br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif">Revital1 Eres/Haifa/IBM@IBMIL</font>
<br><font size=1 color=#5f5f5f face="sans-serif">Cc:
</font><font size=1 face="sans-serif">LLVM Developers Mailing
List <llvmdev@cs.uiuc.edu></font>
<br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">28/07/2015 05:58 AM</font>
<br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">Re: [LLVMdev]
Help with using LLVM to re-compile hot functions at run-time</font>
<br>
<hr noshade>
<br>
<br>
<br><font size=3>Hi Revital,</font>
<br>
<br><font size=3>What do you mean by "code cache"? Orc (and MCJIT)
does have the concept of an ObjectCache, which is a long-lived, potentially
persistent, compiled version of some IR. It's not a key component of the
JIT though: Most clients run without a cache attached and just JIT their
code from scratch in each session.</font>
<br>
<br><font size=3>Recompilation is orthogonal to caching. There is no in-tree
support for recompilation yet. There are several ways that it could be
supported, depending on what security / performance trade-offs you're willing
to make, and how deep in to the LLVM code you want to get. As things stand
at the moment all function calls in the lazy JIT are indirected via function
pointers. We want to add support for patchable call-sites, but this hasn't
been implemented yet. The Indirect calls make recompilation reasonably
easy: You could add a transform layer on top of the CompileCallbackLayer
which would modify each function like this:</font>
<br>
<br><tt><font size=3>void foo$impl() { void
foo$impl() {</font></tt>
<br><tt><font size=3> // foo body ->
if (trigger_condition) {</font></tt>
<br><tt><font size=3>}
auto fooOpt = jit_recompile_hot(&foo);</font></tt>
<br><tt><font size=3>
fooOpt();</font></tt>
<br><tt><font size=3>
}</font></tt>
<br><tt><font size=3>
// foo body</font></tt>
<br><tt><font size=3>
}</font></tt>
<br>
<br><font size=3>You would implement the jit_recompile_hot function yourself
in your JIT and make it available to JIT'd code via the SymbolResolver.
When the trigger condition is met you'll get a call to recompile foo, at
which point you: (1) Add the IR for foo to a 2nd IRCompileLayer that has
been configured with a higher optimization level, (2) look up the address
of the optimized version of foo, and (3) update the function pointer for
foo to point at the optimized version. The process for patchable callsites
should be fairly similar once they're available, except that you'll trigger
a call-site update rather than rewriting a function pointer.</font>
<br>
<br><font size=3>This neglects all sorts of fun details (threading, garbage
collection of old function implementations), but hopefully it gives you
a place to start. </font>
<br>
<br>
<br><font size=3>Regarding laziness, as Hal mentioned you'll have to provide
some target support for PowerPC to support lazy compilation. For a rough
guide you can check out the X86_64 support code in llvm/include/llvm/ExecutionEngine/Orc/OrcTargetSupport.h
and llvm/lib/ExecutionEngine/Orc/OrcTargetSupport.cpp.</font>
<br>
<br><font size=3>There are two methods that you'll need to implement: insertCompileCallbackTrampoline
and insertResolverBlock. These work together to enable lazy compilation.
Both of these methods inject blobs of target specific code in to the JIT
process. To do this (at least for now) I make use of a handy feature of
LLVM IR: You can write raw assembly code directly into a bitcode module
("module-level asm"). If you look at the X86 implementation of
each of these methods you'll see they're written in terms of string-streams
building up a string of assembly which will be handed off to the JIT to
compile like any other code.</font>
<br>
<br><font size=3>The first blob that you need to be able to output is the
resolver block. The purpose of the resolver block is to save program state
and call back in to the JIT to trigger lazy compilation of a function.
When the JIT is done compiling the function it returns the address of the
compiled function to the resolver block, and the resolver block returns
to the compiled function (rather than its original return address).</font>
<br>
<br><font size=3>Because all functions share the same resolver block, the
JIT needs some way to distinguish them, which is where the trampolines
come in. The JIT emits one trampoline per function and each trampoline
just calls the resolver block. The return address of the call in each trampoline
provides the unique address that the JIT associates with the to-be-compiled
functions. The CompileCallbackManager manages this association between
trampolines and functions for you, you just need to provide the resolver/trampoline
primitives.</font>
<br>
<br><font size=3>In case it helps, here's what the output of all this looks
like on X86. Trampolines are trivial - they're emitted in blocks and proceeded
by a pointer to the resolver block:</font>
<br>
<br><tt><font size=3>module asm "Lorc_resolve_block_addr:"</font></tt>
<br><tt><font size=3>module asm " .quad 140439143575560"</font></tt>
<br><tt><font size=3>module asm "orc_jcc_0:"</font></tt>
<br><tt><font size=3>module asm " callq *Lorc_resolve_block_addr(%rip)"</font></tt>
<br><tt><font size=3>module asm "orc_jcc_1:"</font></tt>
<br><tt><font size=3>module asm " callq *Lorc_resolve_block_addr(%rip)"</font></tt>
<br><tt><font size=3>module asm "orc_jcc_2:"</font></tt>
<br><tt><font size=3>module asm " callq *Lorc_resolve_block_addr(%rip)"</font></tt>
<br><tt><font size=3>...</font></tt>
<br>
<br>
<br><font size=3>The resolver block is more complicated and I won't provide
the full code for it here. You can find it by running:</font>
<br>
<div align=center><tt><font size=3>lli -jit-kind=orc-lazy -orc-lazy-debug=mods-to-stderr
<hello_world.ll></font></tt></div>
<br>
<br><font size=3>and looking at the initial output. In pseudo-asm though,
it looks like this:</font>
<br>
<br><tt><font size=3>module asm "jit_callback_manager_addr:"</font></tt>
<br><tt><font size=3>module asm " .quad 0x46fc190" // <-
address of callback manager object</font></tt>
<br><tt><font size=3>module asm "orc_resolver_block:"</font></tt>
<br><tt><font size=3>module asm " // save register state."</font></tt>
<br><tt><font size=3>module asm " // load jit_callback_manager_addr
into %rdi</font></tt>
<br><tt><font size=3>module asm " // load the return address
(from the trampoline call) into %rsi</font></tt>
<br><tt><font size=3>module asm " // %rax = call jit(%rdi, %rsi)</font></tt>
<br><tt><font size=3>module asm " // save %rax over the return
address</font></tt>
<br><tt><font size=3>module asm " // restore register
state</font></tt>
<br><tt><font size=3>module asm " // retq"</font></tt>
<br>
<br><font size=3>So, that's a whirlwind intro to implementing lazy JITing
support for a new architecture in Orc. I'll try to answer any questions
you have on the topic, though I'm not familiar with PowerPC at all. If
you're comfortable with PowerPC assembly I think it should be possible
to implement once you grok the concepts.</font>
<br>
<br><font size=3>Hope this helps!</font>
<br>
<br><font size=3>Cheers,</font>
<br><font size=3>Lang.</font>
<br>
<br>
<br><font size=3>On Jul 26, 2015, at 11:17 PM, Revital1 Eres <</font><a href=mailto:ERES@il.ibm.com target=_blank><font size=3 color=blue><u>ERES@il.ibm.com</u></font></a><font size=3>>
wrote:<br>
</font>
<br><font size=2 face="sans-serif">Hi Again,</font><font size=3> <br>
</font><font size=2 face="sans-serif"><br>
I'm a little confused regarding what is the exact Orc's functions I should
use</font><font size=3> </font><font size=2 face="sans-serif"><br>
in order to save the functions code in a code cache so it could be later</font><font size=3>
</font><font size=2 face="sans-serif"><br>
replaced with different versions of it and I appreciate your help.</font><font size=3>
<br>
</font><font size=2 face="sans-serif"><br>
Just a reminder I want to dynamically recompile the program based on profile</font><font size=3>
</font><font size=2 face="sans-serif"><br>
collected at the run-time. I would like to start executing the program
from</font><font size=3> </font><font size=2 face="sans-serif"><br>
the code-cache and at some point be able to replace a function body with
it's</font><font size=3> </font><font size=2 face="sans-serif"><br>
new compiled version; this can be done by replacing the entry in the function</font><font size=3>
</font><font size=2 face="sans-serif"><br>
code with a trampoline to It's new version so that future calls to
it will</font><font size=3> </font><font size=2 face="sans-serif"><br>
call the new version code.</font><font size=3> <br>
</font><font size=2 face="sans-serif"><br>
Does the CompileOnDemandLayer executes the program from a code cache <br>
and holds pointers to the code of the functions it executes? I am compiling
for Power machine. <br>
Is there a target specific pieces that I should implement for making Orc
work on Power?</font><font size=3> <br>
</font><font size=2 face="sans-serif"><br>
Thanks again,</font><font size=3> </font><font size=2 face="sans-serif"><br>
Revital</font><font size=3> <br>
<br>
<br>
<br>
</font><font size=1 color=#5f5f5f face="sans-serif"><br>
From: </font><font size=1 face="sans-serif">Lang
Hames <</font><a href=mailto:lhames@gmail.com target=_blank><font size=1 color=blue face="sans-serif"><u>lhames@gmail.com</u></font></a><font size=1 face="sans-serif">></font><font size=3>
</font><font size=1 color=#5f5f5f face="sans-serif"><br>
To: </font><font size=1 face="sans-serif">Revital1
Eres/Haifa/IBM@IBMIL</font><font size=3> </font><font size=1 color=#5f5f5f face="sans-serif"><br>
Cc: </font><font size=1 face="sans-serif">LLVM
Developers Mailing List <</font><a href=mailto:llvmdev@cs.uiuc.edu target=_blank><font size=1 color=blue face="sans-serif"><u>llvmdev@cs.uiuc.edu</u></font></a><font size=1 face="sans-serif">></font><font size=3>
</font><font size=1 color=#5f5f5f face="sans-serif"><br>
Date: </font><font size=1 face="sans-serif">20/07/2015
08:41 PM</font><font size=3> </font><font size=1 color=#5f5f5f face="sans-serif"><br>
Subject: </font><font size=1 face="sans-serif">Re:
[LLVMdev] Help with using LLVM to re-compile hot functions at run-time</font><font size=3>
<br>
</font>
<hr noshade><font size=3><br>
<br>
<br>
Hi Revital, <br>
<br>
The CompileOnDemand layer is used by the lazy bitcode JIT in the lli tool.
You can find the code in llvm/tools/lli/OrcLazyJIT.* . <br>
<br>
Cheers, <br>
Lang. <br>
<br>
<br>
On Mon, Jul 20, 2015 at 2:32 AM, Revital1 Eres <</font><a href=mailto:ERES@il.ibm.com target=_blank><font size=3 color=blue><u>ERES@il.ibm.com</u></font></a><font size=3>>
wrote: </font><font size=2 face="sans-serif"><br>
Hello </font><font size=3>Lang</font><font size=2 face="sans-serif">,</font><font size=3>
</font><font size=2 face="sans-serif"><br>
<br>
Thanks for your answer.</font><font size=3> </font><font size=2 face="sans-serif"><br>
<br>
I am now looking for an example of the usage of CompileOnDemandLayer. Is
there an example available for that (could not find one in llvm/examples)?</font><font size=3>
</font><font size=2 face="sans-serif"><br>
<br>
Thanks,</font><font size=3> </font><font size=2 face="sans-serif"><br>
Revital</font><font size=3> <br>
<br>
</font><font size=1 color=#5f5f5f face="sans-serif"><br>
<br>
From: </font><font size=1 face="sans-serif">Lang
Hames <</font><a href=mailto:lhames@gmail.com target=_blank><font size=1 color=blue face="sans-serif"><u>lhames@gmail.com</u></font></a><font size=1 face="sans-serif">></font><font size=3>
</font><font size=1 color=#5f5f5f face="sans-serif"><br>
To: </font><font size=1 face="sans-serif">Revital1
Eres/Haifa/IBM@IBMIL</font><font size=3> </font><font size=1 color=#5f5f5f face="sans-serif"><br>
Cc: </font><font size=1 face="sans-serif">LLVM
Developers Mailing List <</font><a href=mailto:llvmdev@cs.uiuc.edu target=_blank><font size=1 color=blue face="sans-serif"><u>llvmdev@cs.uiuc.edu</u></font></a><font size=1 face="sans-serif">></font><font size=3>
</font><font size=1 color=#5f5f5f face="sans-serif"><br>
Date: </font><font size=1 face="sans-serif">10/07/2015
12:10 AM</font><font size=3> </font><font size=1 color=#5f5f5f face="sans-serif"><br>
Subject: </font><font size=1 face="sans-serif">Re:
[LLVMdev] Help with using LLVM to re-compile hot functions at run-time</font><font size=3>
<br>
</font>
<hr noshade><font size=3><br>
<br>
<br>
Hi Revital, <br>
<br>
LLVM does have an IR interpreter, but I don't think it's maintained well
(or possibly at all). The interpreter is also not designed to interact
with the LLVM JITs. <br>
<br>
We generally encourage people to just JIT LLVM IR, rather than interpreting
it. For the use-case you have described, you could JIT IR with no optimizations
to begin with, then re-JIT hot functions at a higher level. <br>
<br>
The Orc JIT APIs (LLVM's newer JIT APIs) were written with this kind of
use-case in mind, and are probably a better fit for this than MCJIT. There
is no built-in hot-function detection or recompilation yet, but I think
this would be *fairly* easy to write in terms of Orc's callback API. <br>
<br>
Cheers, <br>
Lang. <br>
<br>
<br>
On Thu, Jul 9, 2015 at 4:19 AM, Revital1 Eres <</font><a href=mailto:ERES@il.ibm.com target=_blank><font size=3 color=blue><u>ERES@il.ibm.com</u></font></a><font size=3>>
wrote: </font><font size=2 face="sans-serif"><br>
Hello,</font><font size=3> </font><font size=2 face="sans-serif"><br>
<br>
I am new to LLVM and a I appreciate your help with the following:</font><font size=3>
</font><font size=2 face="sans-serif"><br>
<br>
I want to run the LLVM IR through virtual machine (LLVM interpreter?) and
jit</font><font size=3> </font><font size=2 face="sans-serif"><br>
compile the hot functions (using MCJIT).</font><font size=3> </font><font size=2 face="sans-serif"><br>
<br>
This task will require amongst other identifying the hot functions and
having a</font><font size=3> </font><font size=2 face="sans-serif"><br>
code cache that should be patched with the native code of the functions
after</font><font size=3> </font><font size=2 face="sans-serif"><br>
they are jitted.</font><font size=3> </font><font size=2 face="sans-serif"><br>
<br>
I've read so far about MCJIT and lli however I have not seen that the LLVM
<br>
interpreter can be used as a VM the way I was looking for; meaning</font><font size=3>
</font><font size=2 face="sans-serif"><br>
execute the code one instruction at a time; have a profiling mode to <br>
identify hot functions and call jit to compile the hot functions.</font><font size=3>
</font><font size=2 face="sans-serif"><br>
<br>
I appreciate any advice/starting points for this project.</font><font size=3>
</font><font size=2 face="sans-serif"><br>
<br>
Thanks,</font><font size=3> </font><font size=2 face="sans-serif"><br>
Revital</font><font size=3> <br>
<br>
_______________________________________________<br>
LLVM Developers mailing list</font><font size=3 color=blue><u><br>
</u></font><a href=mailto:LLVMdev@cs.uiuc.edu target=_blank><font size=3 color=blue><u>LLVMdev@cs.uiuc.edu</u></font></a><font size=3>
</font><a href=http://llvm.cs.uiuc.edu/ target=_blank><font size=3 color=blue><u>http://llvm.cs.uiuc.edu</u></font></a><font size=3 color=blue><u><br>
</u></font><a href=http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev target=_blank><font size=3 color=blue><u>http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</u></font></a><font size=3><br>
<br>
<br>
<br>
<br>
</font>
<br>
<br>