[LLVMdev] [RFC] MCJIT usage models
Vassil Vassilev
vasil.georgiev.vasilev at cern.ch
Tue Dec 10 02:11:24 PST 2013
Hi Andy,
Our use of MCJIT/old JIT in cling is very similar to what Yaron and
Keno described. We use clang to codegen C/C++ functions incrementally.
We need lazy compilation support and 'function-level' linking, too.
Remapping of GlobalValues is also important for our use case. We support
function removal, i.e we need to make LLVM IR and the JIT forget
functions that were executed. All that was possible with the old JIT
architecture and I hope it could be possible with the MCJIT.
That said, I think we mostly fit in the #1 case.
Vassil
On 12/10/2013 06:46 AM, Yaron Keren wrote:
> Hi Andy,
>
> My use case is quite similar to what Keno described. I am using clang
> + JIT to dynamically compile C++ functions generated in response to
> user interaction. Generated functions may be unloaded or modified.
>
> I would like to break down the old JIT code into three major parts.
>
> 1) The old JIT has its own code emitter, which duplicates code from
> lib/MC and does not generate debug info and other limitations.
> 2) The old JIT supports lazy compilation.
> 3) The old JIT has its own function-level "dynamic linker"/memory
> manager, supporting function replacement with low-overhead JMP stubs.
>
> Now 1) is clearly a problem of code duplication. I'm not sure why a
> different emitter was created for the JIT but it would seem it's
> possible to reuse the lib/MC code just like MCJIT does. 2) takes much
> code all over the JIT code and at least for my use case it could just
> be removed.
>
> If 1) and 2) are solved and removed we are left with the (relatively
> small) "dynamic linker"/hack code only. I'd say the way this
> linker/loader works is much better fit for use cases such as my and
> Keno's than a classic linker like the ELF loader.
>
> The JIT linker works at function level rather than module level,
> supports automatic stub generation and relinking management. We could
> wrap functions in modules for the ELF loader, handle the stubs
> ourselves etc but this is tricky code as Keno said, requires
> hard-learned knowledge and feels like teaching an elephant to dance.
> The ELF loader is really designed for very different requirements.
>
> I'd like to see the old JIT function-level "dynamic linker" code
> preserved somehow as a ready alternative to a "classic" linker,
> especially useful when used in combination clang to dynamically run
> C/C++ functions.
>
> Yaron
>
>
>
> 2013/12/10 Keno Fischer <kfischer at college.harvard.edu
> <mailto:kfischer at college.harvard.edu>>
>
> With Julia, we're obviously very much in the first use case. As
> you know, we pretty much have a working version of Julia on top of
> MCJIT, but there's still a few kinks to work out, which I'll talk
> about in a separate email.
>
> One think which I remember you asking at the BOF is what MCJIT
> currently can't do well that the old JIT did, so I'd like to offer
> up an example. With the old JIT, I used Clang to do dynamic code
> generation to interface to C++ easily. Now you might argue that is
> either a problem with clang or rather a misuse of clang, but I'd
> like to think that we should keep the tools as flexible as
> possible, so applications like this can emerge. With the old JIT,
> I'd incrementally compile functions as Clang added them to the
> Module, but with MCJIT that kind of stuff is rather tricky. What I
> ended up doing was having Clang emit into a shadow module that
> never gets codegen'd and when a function is requested, pulling
> that function and it's closure out of the shadow module into the
> current MCJIT module. Perhaps functionality like that should be
> more readily available in base LLVM to be able to use MCJIT with
> clients not necessarily designed for use with MCJIT.
>
>
>
>
> On Mon, Dec 9, 2013 at 1:08 PM, Kaylor, Andrew
> <andrew.kaylor at intel.com <mailto:andrew.kaylor at intel.com>> wrote:
>
> Below is an outline of various usage models for MCJIT that I
> put together based on conversations at last month's LLVM
> Developer Meeting. If you're using or thinking about using
> MCJIT and your use case doesn't seem to fit in one of the
> categories below then either I didn't talk to you or I didn't
> understand what you're doing.
>
> In any case, I'd like to see this get worked into a shape
> suitable for inclusion in the LLVM documentation. I imagine
> it serving as a guide both to those who are new to using MCJIT
> and to those who are developing and maintaining MCJIT. If
> you're using MCJIT the latter (yes, the latter) case is
> particularly important to you right now as having your use
> case properly represented in this document is the best way to
> ensure that it is adequately considered when changes are made
> to MCJIT and when the decision is made as to when we are ready
> to deprecate the old JIT engine (probably in the 3.5 release,
> BTW).
>
> So here's what I'm asking for: if you are currently using
> MCJIT or considering using MCJIT, can you please find the use
> case that best fits your program and comment on how well the
> outline describes it. If you understand what I'm saying below
> but you see something that is missing, please let me know. If
> you aren't sure what I'm saying or you don't know how MCJIT
> might address your particular issues, please let me know that
> too. If you think my outline is too sketchy and you need me
> to elaborate before you can provide meaningful feedback,
> please let me know about that. If you think it's the best
> piece of documentation you've read all year and you can't wait
> to read it again, that's good information too.
>
> Thanks in advance for any and all feedback.
>
> -Andy
>
> ------------------------------------------------------------------------------------------
>
> Models for MCJIT use
>
> 1. Interactive dynamic code generation
>
> - user types code which is compiled as needed for execution
>
> - example: Kaleidoscope
>
> - compilation speed probably isn't critical
>
> - use one MCJIT instance with many modules
>
> - create new modules on compilation
>
> - MCJIT handles linking between modules
>
> - external references still need prototypes
>
> - we can at least provide a module pass to automate it
>
> - memory overhead may be an issue but MCJIT can fix that
>
> - see model 2 for pre-defined library
>
> - if processing a large script pre-optimize before passing
> modules to MCJIT
>
> 2. Code generation for external target execution
>
> - client generates code to be injected into an external
> process
>
> - example: LLDB expression evaluation
>
> - target may be another local or remote
>
> - target architecture may not match host architecture
>
> - may use one or more instances of MCJIT (client preference)
>
> - MCJIT handles address remapping on request
>
> - custom memory manager handles code/data transfer
>
> - speed/memory requirements may vary
>
> 3. Large pre-defined module compilation and execution
>
> - code/IR is loaded from disk and prepared for execution
>
> - example: Intel(R) OpenCL SDK
>
> - compilation speed matters but isn't critical
>
> - initial startup time is somewhat important
>
> - execution speed is critical
>
> - memory consumption isn't an issue
>
> - tool integration may be important
>
> - use one MCJIT instance with multiple (but usually) few
> modules
>
> - use object caching for commonly used code
>
> - for very large, sparsely used libraries pre-link modules
>
> - object and archive support may be useful
>
> 4. Hot function replacement
>
> - client uses MCJIT to optimize frequently executed code
>
> - example: WebKit
>
> - compilation time is not critical
>
> - execution speed is critical
>
> - steady state memory consumption is very important
>
> - client handles pre-JIT interpretation/execution
>
> - MCJIT instances may be created as needed
>
> - custom memory manager transfers code memory ownership
> after compilation
>
> - MCJIT instance is deleted when no longer needed
>
> - client handles function replacement and lifetime management
>
> 5. On demand "one-time" execution
>
> - client provides a library of code which is used by
> small, disposable functions
>
> - example: database query?
>
> - initial load time isn't important
>
> - execution time is critical
>
> - if library code is fixed, load as shared library
>
> - if library code must be generated use a separate
> instance of MCJIT to hold the library
>
> - this instance can support multiple modules
>
> - use a custom memory manager to link with functions
> in this module
>
> - object caching and archive support may be useful in
> this case
>
> - if inlining/lto is more important than compile time keep
> library in an IR module and pre-link just before invoking MCJIT
>
> - create one instance of MCJIT as needed and destroy after
> execution
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu <mailto:LLVMdev at cs.uiuc.edu>
> http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu <mailto:LLVMdev at cs.uiuc.edu>
> http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20131210/eabbb188/attachment.html>
More information about the llvm-dev
mailing list