<div dir="ltr"><div dir="auto"><div><span></span></div><div><div>Hi Revital,</div><div><br></div><div>What do you mean by "code cache"? Orc (and MCJIT) does have the concept of an ObjectCache, which is a long-lived, potentially persistent, compiled version of some IR. It's not a key component of the JIT though: Most clients run without a cache attached and just JIT their code from scratch in each session.</div><div><br></div><div>Recompilation is orthogonal to caching. There is no in-tree support for recompilation yet. There are several ways that it could be supported, depending on what security / performance trade-offs you're willing to make, and how deep in to the LLVM code you want to get. As things stand at the moment all function calls in the lazy JIT are indirected via function pointers. We want to add support for patchable call-sites, but this hasn't been implemented yet. The Indirect calls make recompilation reasonably easy: You could add a transform layer on top of the CompileCallbackLayer which would modify each function like this:</div><div><br></div><div><font face="monospace, monospace">void foo$impl() { void foo$impl() {</font></div><div><font face="monospace, monospace"> // foo body -> if (trigger_condition) {</font></div><div><font face="monospace, monospace">} auto fooOpt = jit_recompile_hot(&foo);</font></div><div><font face="monospace, monospace"> fooOpt();</font></div><div><font face="monospace, monospace"> }</font></div><div><font face="monospace, monospace"> // foo body</font></div><div><font face="monospace, monospace"> }</font></div><div><br></div><div>You would implement the jit_recompile_hot function yourself in your JIT and make it available to JIT'd code via the SymbolResolver. When the trigger condition is met you'll get a call to recompile foo, at which point you: (1) Add the IR for foo to a 2nd IRCompileLayer that has been configured with a higher optimization level, (2) look up the address of the optimized version of foo, and (3) update the function pointer for foo to point at the optimized version. The process for patchable callsites should be fairly similar once they're available, except that you'll trigger a call-site update rather than rewriting a function pointer.</div><div><br></div><div>This neglects all sorts of fun details (threading, garbage collection of old function implementations), but hopefully it gives you a place to start. </div><div><br></div><div><br></div><div>Regarding laziness, as Hal mentioned you'll have to provide some target support for PowerPC to support lazy compilation. For a rough guide you can check out the X86_64 support code in llvm/include/llvm/ExecutionEngine/Orc/OrcTargetSupport.h and llvm/lib/ExecutionEngine/Orc/OrcTargetSupport.cpp.</div><div><br></div><div>There are two methods that you'll need to implement: insertCompileCallbackTrampoline and insertResolverBlock. These work together to enable lazy compilation. Both of these methods inject blobs of target specific code in to the JIT process. To do this (at least for now) I make use of a handy feature of LLVM IR: You can write raw assembly code directly into a bitcode module ("module-level asm"). If you look at the X86 implementation of each of these methods you'll see they're written in terms of string-streams building up a string of assembly which will be handed off to the JIT to compile like any other code.</div><div><br></div><div>The first blob that you need to be able to output is the resolver block. The purpose of the resolver block is to save program state and call back in to the JIT to trigger lazy compilation of a function. When the JIT is done compiling the function it returns the address of the compiled function to the resolver block, and the resolver block returns to the compiled function (rather than its original return address).</div><div><br></div><div>Because all functions share the same resolver block, the JIT needs some way to distinguish them, which is where the trampolines come in. The JIT emits one trampoline per function and each trampoline just calls the resolver block. The return address of the call in each trampoline provides the unique address that the JIT associates with the to-be-compiled functions. The CompileCallbackManager manages this association between trampolines and functions for you, you just need to provide the resolver/trampoline primitives.</div><div><br></div><div>In case it helps, here's what the output of all this looks like on X86. Trampolines are trivial - they're emitted in blocks and proceeded by a pointer to the resolver block:</div><div><br></div><div><div><font face="monospace, monospace">module asm "Lorc_resolve_block_addr:"<br></font></div></div><div><div><font face="monospace, monospace">module asm " .quad 140439143575560"</font></div><div><font face="monospace, monospace">module asm "orc_jcc_0:"</font></div><div><font face="monospace, monospace">module asm " callq *Lorc_resolve_block_addr(%rip)"</font></div><div><font face="monospace, monospace">module asm "orc_jcc_1:"</font></div><div><font face="monospace, monospace">module asm " callq *Lorc_resolve_block_addr(%rip)"</font></div><div><font face="monospace, monospace">module asm "orc_jcc_2:"</font></div><div><font face="monospace, monospace">module asm " callq *Lorc_resolve_block_addr(%rip)"</font></div><div><font face="monospace, monospace">...</font></div></div><div><br></div><div><br></div><div>The resolver block is more complicated and I won't provide the full code for it here. You can find it by running:</div><div><br></div><div style="text-align:center"><font face="monospace, monospace">lli -jit-kind=orc-lazy -orc-lazy-debug=mods-to-stderr <hello_world.ll></font></div><div><br></div><div>and looking at the initial output. In pseudo-asm though, it looks like this:</div><div><br></div><div><div><font face="monospace, monospace">module asm "jit_callback_manager_addr:"</font></div><div><font face="monospace, monospace">module asm " .quad 0x46fc190" // <- address of callback manager object</font></div><div><font face="monospace, monospace">module asm "orc_resolver_block:"</font></div><div><font face="monospace, monospace">module asm " // save register state."</font></div></div><div><font face="monospace, monospace">module asm " // load jit_callback_manager_addr into %rdi</font></div><div><font face="monospace, monospace">module asm " // load the return address (from the trampoline call) into %rsi</font></div><div><font face="monospace, monospace">module asm " // %rax = call jit(%rdi, %rsi)</font></div><div><font face="monospace, monospace">module asm " // save %rax over the return address</font></div><div><font face="monospace, monospace">module asm " // restore register state</font></div><div><font face="monospace, monospace">module asm " // retq"</font></div><div><br></div><div><div>So, that's a whirlwind intro to implementing lazy JITing support for a new architecture in Orc. I'll try to answer any questions you have on the topic, though I'm not familiar with PowerPC at all. If you're comfortable with PowerPC assembly I think it should be possible to implement once you grok the concepts.</div><div><br></div><div>Hope this helps!</div><div><br></div><div>Cheers,</div><div>Lang.</div><div><br></div><div><br></div><div>On Jul 26, 2015, at 11:17 PM, Revital1 Eres <<a href="mailto:ERES@il.ibm.com" target="_blank">ERES@il.ibm.com</a>> wrote:<br><br></div><blockquote type="cite"><div><font size="2" face="sans-serif">Hi Again,</font>
<br>
<br><font size="2" face="sans-serif">I'm a little confused regarding what
is the exact Orc's functions I should use</font>
<br><font size="2" face="sans-serif">in order to save the functions code
in a code cache so it could be later</font>
<br><font size="2" face="sans-serif">replaced with different versions of
it and I appreciate your help.</font>
<br>
<br><font size="2" face="sans-serif">Just a reminder I want to dynamically
recompile the program based on profile</font>
<br><font size="2" face="sans-serif"> collected at the run-time. I would
like to start executing the program from</font>
<br><font size="2" face="sans-serif">the code-cache and at some point be
able to replace a function body with it's</font>
<br><font size="2" face="sans-serif">new compiled version; this can be done
by replacing the entry in the function</font>
<br><font size="2" face="sans-serif"> code with a trampoline to It's
new version so that future calls to it will</font>
<br><font size="2" face="sans-serif">call the new version code.</font>
<br>
<br><font size="2" face="sans-serif">Does the CompileOnDemandLayer executes
the program from a code cache </font>
<br><font size="2" face="sans-serif">and holds pointers to the code of the
functions it executes? I am compiling for Power machine. </font>
<br><font size="2" face="sans-serif">Is there a target specific pieces that
I should implement for making Orc work on Power?</font>
<br>
<br><font size="2" face="sans-serif">Thanks again,</font>
<br><font size="2" face="sans-serif">Revital</font>
<br>
<br>
<br>
<br>
<br><font size="1" color="#5f5f5f" face="sans-serif">From:
</font><font size="1" face="sans-serif">Lang Hames <<a href="mailto:lhames@gmail.com" target="_blank">lhames@gmail.com</a>></font>
<br><font size="1" color="#5f5f5f" face="sans-serif">To:
</font><font size="1" face="sans-serif">Revital1 Eres/Haifa/IBM@IBMIL</font>
<br><font size="1" color="#5f5f5f" face="sans-serif">Cc:
</font><font size="1" face="sans-serif">LLVM Developers Mailing
List <<a href="mailto:llvmdev@cs.uiuc.edu" target="_blank">llvmdev@cs.uiuc.edu</a>></font>
<br><font size="1" color="#5f5f5f" face="sans-serif">Date:
</font><font size="1" face="sans-serif">20/07/2015 08:41 PM</font>
<br><font size="1" color="#5f5f5f" face="sans-serif">Subject:
</font><font size="1" face="sans-serif">Re: [LLVMdev]
Help with using LLVM to re-compile hot functions at run-time</font>
<br>
<hr noshade>
<br>
<br>
<br><font size="3">Hi Revital,</font>
<br>
<br><font size="3">The CompileOnDemand layer is used by the lazy bitcode
JIT in the lli tool. You can find the code in llvm/tools/lli/OrcLazyJIT.*
.</font>
<br>
<br><font size="3">Cheers,</font>
<br><font size="3">Lang.</font>
<br>
<br>
<br><font size="3">On Mon, Jul 20, 2015 at 2:32 AM, Revital1 Eres <</font><a href="mailto:ERES@il.ibm.com" target="_blank"><font size="3" color="blue"><u>ERES@il.ibm.com</u></font></a><font size="3">>
wrote:</font>
<br><font size="2" face="sans-serif">Hello </font><font size="3">Lang</font><font size="2" face="sans-serif">,</font><font size="3">
<br>
</font><font size="2" face="sans-serif"><br>
Thanks for your answer.</font><font size="3"> <br>
</font><font size="2" face="sans-serif"><br>
I am now looking for an example of the usage of CompileOnDemandLayer. Is
there an example available for that (could not find one in llvm/examples)?</font><font size="3">
<br>
</font><font size="2" face="sans-serif"><br>
Thanks,</font><font size="3"> </font><font size="2" face="sans-serif"><br>
Revital</font><font size="3"> <br>
<br>
<br>
</font><font size="1" color="#5f5f5f" face="sans-serif"><br>
From: </font><font size="1" face="sans-serif">Lang
Hames <</font><a href="mailto:lhames@gmail.com" target="_blank"><font size="1" color="blue" face="sans-serif"><u>lhames@gmail.com</u></font></a><font size="1" face="sans-serif">></font><font size="3">
</font><font size="1" color="#5f5f5f" face="sans-serif"><br>
To: </font><font size="1" face="sans-serif">Revital1
Eres/Haifa/IBM@IBMIL</font><font size="3"> </font><font size="1" color="#5f5f5f" face="sans-serif"><br>
Cc: </font><font size="1" face="sans-serif">LLVM
Developers Mailing List <</font><a href="mailto:llvmdev@cs.uiuc.edu" target="_blank"><font size="1" color="blue" face="sans-serif"><u>llvmdev@cs.uiuc.edu</u></font></a><font size="1" face="sans-serif">></font><font size="3">
</font><font size="1" color="#5f5f5f" face="sans-serif"><br>
Date: </font><font size="1" face="sans-serif">10/07/2015
12:10 AM</font><font size="3"> </font><font size="1" color="#5f5f5f" face="sans-serif"><br>
Subject: </font><font size="1" face="sans-serif">Re:
[LLVMdev] Help with using LLVM to re-compile hot functions at run-time</font><font size="3">
<br>
</font>
<hr noshade><font size="3"><br>
<br>
<br>
Hi Revital, <br>
<br>
LLVM does have an IR interpreter, but I don't think it's maintained well
(or possibly at all). The interpreter is also not designed to interact
with the LLVM JITs. <br>
<br>
We generally encourage people to just JIT LLVM IR, rather than interpreting
it. For the use-case you have described, you could JIT IR with no optimizations
to begin with, then re-JIT hot functions at a higher level. <br>
<br>
The Orc JIT APIs (LLVM's newer JIT APIs) were written with this kind of
use-case in mind, and are probably a better fit for this than MCJIT. There
is no built-in hot-function detection or recompilation yet, but I think
this would be *fairly* easy to write in terms of Orc's callback API. <br>
<br>
Cheers, <br>
Lang. <br>
<br>
<br>
On Thu, Jul 9, 2015 at 4:19 AM, Revital1 Eres <</font><a href="mailto:ERES@il.ibm.com" target="_blank"><font size="3" color="blue"><u>ERES@il.ibm.com</u></font></a><font size="3">>
wrote: </font><font size="2" face="sans-serif"><br>
Hello,</font><font size="3"> </font><font size="2" face="sans-serif"><br>
<br>
I am new to LLVM and a I appreciate your help with the following:</font><font size="3">
</font><font size="2" face="sans-serif"><br>
<br>
I want to run the LLVM IR through virtual machine (LLVM interpreter?) and
jit</font><font size="3"> </font><font size="2" face="sans-serif"><br>
compile the hot functions (using MCJIT).</font><font size="3"> </font><font size="2" face="sans-serif"><br>
<br>
This task will require amongst other identifying the hot functions and
having a</font><font size="3"> </font><font size="2" face="sans-serif"><br>
code cache that should be patched with the native code of the functions
after</font><font size="3"> </font><font size="2" face="sans-serif"><br>
they are jitted.</font><font size="3"> </font><font size="2" face="sans-serif"><br>
<br>
I've read so far about MCJIT and lli however I have not seen that the LLVM
<br>
interpreter can be used as a VM the way I was looking for; meaning</font><font size="3">
</font><font size="2" face="sans-serif"><br>
execute the code one instruction at a time; have a profiling mode to <br>
identify hot functions and call jit to compile the hot functions.</font><font size="3">
</font><font size="2" face="sans-serif"><br>
<br>
I appreciate any advice/starting points for this project.</font><font size="3">
</font><font size="2" face="sans-serif"><br>
<br>
Thanks,</font><font size="3"> </font><font size="2" face="sans-serif"><br>
Revital</font><font size="3"> <br>
<br>
_______________________________________________<br>
LLVM Developers mailing list</font><font size="3" color="blue"><u><br>
</u></font><a href="mailto:LLVMdev@cs.uiuc.edu" target="_blank"><font size="3" color="blue"><u>LLVMdev@cs.uiuc.edu</u></font></a><font size="3">
</font><a href="http://llvm.cs.uiuc.edu/" target="_blank"><font size="3" color="blue"><u>http://llvm.cs.uiuc.edu</u></font></a><font size="3" color="blue"><u><br>
</u></font><a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev" target="_blank"><font size="3" color="blue"><u>http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</u></font></a><font size="3"><br>
<br>
<br>
</font>
<br>
<br>
<br></div></blockquote>
</div></div></div></div>