<div dir="ltr">Hi Andy, this looks great. I would echo what some of the other people are saying, that an all-llvm jit compiler (ie something for executing entire source programs, with no baseline interpreter/other jit engine) is missing from the list, and I think it's a common enough case to be worth including or mentioning as a non-goal. The main thing about that use case that feels not covered by the others is fast "-O0" compilation speed; I'm not saying that the burden of improving that should be on MCJIT or you, but it'd be nice to know to what extent it could be relied on.<div>
<br></div><div style>I'm not sure if this is a necessary consequence, but I think it's natural with an all-llvm jit to have your standard library be in llvm as well, which is maybe similar to #3/#5. I got a somewhat-hacky version of cross-module inlining working (it seems possible to make it non-hacky but would involve some refactoring of the LLVM inlining code), which means that the stdlib can stay fixed (you don't have to put new IR in it to do inlining), and thus can get cached, potentially lessening the importance of stdlib compile time.</div>
<div style><br></div><div style>About lazy compilation, I'm still of the opinion that that's better handled outside of MCJIT. For the people asking for it, would it be enough to have a wrapper around MCJIT that automatically splits modules and adds stubs to do lazy compilation? I don't think that would be too hard to add, though it could make the compilation speed situation worse. Anyway, I think it's just too restrictive to bake this kind of functionality into MCJIT itself, especially now that there's patchpoint which adds another dimension along which to customize function replacement. Plus, to truly compile things lazily, IR-generation should probably also be done lazily, which makes the situation even more complicated.</div>
<div class="gmail_extra"><br><div class="gmail_quote">On Mon, Dec 9, 2013 at 11:08 AM, Kaylor, Andrew <span dir="ltr"><<a href="mailto:andrew.kaylor@intel.com" target="_blank">andrew.kaylor@intel.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple">
<div>
<p class="MsoNormal">Below is an outline of various usage models for MCJIT that I put together based on conversations at last month’s LLVM Developer Meeting. If you’re using or thinking about using MCJIT and your use case doesn’t seem to fit in one of the
categories below then either I didn’t talk to you or I didn’t understand what you’re doing.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">In any case, I’d like to see this get worked into a shape suitable for inclusion in the LLVM documentation. I imagine it serving as a guide both to those who are new to using MCJIT and to those who are developing and maintaining MCJIT.
If you’re using MCJIT the latter (yes, the latter) case is particularly important to you right now as having your use case properly represented in this document is the best way to ensure that it is adequately considered when changes are made to MCJIT and when
the decision is made as to when we are ready to deprecate the old JIT engine (probably in the 3.5 release, BTW).<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">So here’s what I’m asking for: if you are currently using MCJIT or considering using MCJIT, can you please find the use case that best fits your program and comment on how well the outline describes it. If you understand what I’m saying
below but you see something that is missing, please let me know. If you aren’t sure what I’m saying or you don’t know how MCJIT might address your particular issues, please let me know that too. If you think my outline is too sketchy and you need me to elaborate
before you can provide meaningful feedback, please let me know about that. If you think it’s the best piece of documentation you’ve read all year and you can’t wait to read it again, that’s good information too.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Thanks in advance for any and all feedback.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">-Andy<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">------------------------------------------------------------------------------------------<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Models for MCJIT use<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">1. Interactive dynamic code generation<u></u><u></u></p>
<p class="MsoNormal"> - user types code which is compiled as needed for execution<u></u><u></u></p>
<p class="MsoNormal"> - example: Kaleidoscope<u></u><u></u></p>
<p class="MsoNormal"> - compilation speed probably isn't critical<u></u><u></u></p>
<p class="MsoNormal"> - use one MCJIT instance with many modules<u></u><u></u></p>
<p class="MsoNormal"> - create new modules on compilation<u></u><u></u></p>
<p class="MsoNormal"> - MCJIT handles linking between modules<u></u><u></u></p>
<p class="MsoNormal"> - external references still need prototypes<u></u><u></u></p>
<p class="MsoNormal"> - we can at least provide a module pass to automate it<u></u><u></u></p>
<p class="MsoNormal"> - memory overhead may be an issue but MCJIT can fix that<u></u><u></u></p>
<p class="MsoNormal"> - see model 2 for pre-defined library<u></u><u></u></p>
<p class="MsoNormal"> - if processing a large script pre-optimize before passing modules to MCJIT
<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">2. Code generation for external target execution<u></u><u></u></p>
<p class="MsoNormal"> - client generates code to be injected into an external process<u></u><u></u></p>
<p class="MsoNormal"> - example: LLDB expression evaluation<u></u><u></u></p>
<p class="MsoNormal"> - target may be another local or remote<u></u><u></u></p>
<p class="MsoNormal"> - target architecture may not match host architecture<u></u><u></u></p>
<p class="MsoNormal"> - may use one or more instances of MCJIT (client preference)<u></u><u></u></p>
<p class="MsoNormal"> - MCJIT handles address remapping on request<u></u><u></u></p>
<p class="MsoNormal"> - custom memory manager handles code/data transfer<u></u><u></u></p>
<p class="MsoNormal"> - speed/memory requirements may vary<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">3. Large pre-defined module compilation and execution<u></u><u></u></p>
<p class="MsoNormal"> - code/IR is loaded from disk and prepared for execution<u></u><u></u></p>
<p class="MsoNormal"> - example: Intel(R) OpenCL SDK<u></u><u></u></p>
<p class="MsoNormal"> - compilation speed matters but isn't critical<u></u><u></u></p>
<p class="MsoNormal"> - initial startup time is somewhat important<u></u><u></u></p>
<p class="MsoNormal"> - execution speed is critical<u></u><u></u></p>
<p class="MsoNormal"> - memory consumption isn't an issue<u></u><u></u></p>
<p class="MsoNormal"> - tool integration may be important<u></u><u></u></p>
<p class="MsoNormal"> - use one MCJIT instance with multiple (but usually) few modules<u></u><u></u></p>
<p class="MsoNormal"> - use object caching for commonly used code<u></u><u></u></p>
<p class="MsoNormal"> - for very large, sparsely used libraries pre-link modules<u></u><u></u></p>
<p class="MsoNormal"> - object and archive support may be useful<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">4. Hot function replacement<u></u><u></u></p>
<p class="MsoNormal"> - client uses MCJIT to optimize frequently executed code<u></u><u></u></p>
<p class="MsoNormal"> - example: WebKit<u></u><u></u></p>
<p class="MsoNormal"> - compilation time is not critical<u></u><u></u></p>
<p class="MsoNormal"> - execution speed is critical<u></u><u></u></p>
<p class="MsoNormal"> - steady state memory consumption is very important<u></u><u></u></p>
<p class="MsoNormal"> - client handles pre-JIT interpretation/execution<u></u><u></u></p>
<p class="MsoNormal"> - MCJIT instances may be created as needed<u></u><u></u></p>
<p class="MsoNormal"> - custom memory manager transfers code memory ownership after compilation<u></u><u></u></p>
<p class="MsoNormal"> - MCJIT instance is deleted when no longer needed<u></u><u></u></p>
<p class="MsoNormal"> - client handles function replacement and lifetime management<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">5. On demand "one-time" execution<u></u><u></u></p>
<p class="MsoNormal"> - client provides a library of code which is used by small, disposable functions<u></u><u></u></p>
<p class="MsoNormal"> - example: database query?<u></u><u></u></p>
<p class="MsoNormal"> - initial load time isn't important<u></u><u></u></p>
<p class="MsoNormal"> - execution time is critical<u></u><u></u></p>
<p class="MsoNormal"> - if library code is fixed, load as shared library<u></u><u></u></p>
<p class="MsoNormal"> - if library code must be generated use a separate instance of MCJIT to hold the library<u></u><u></u></p>
<p class="MsoNormal"> - this instance can support multiple modules<u></u><u></u></p>
<p class="MsoNormal"> - use a custom memory manager to link with functions in this module<u></u><u></u></p>
<p class="MsoNormal"> - object caching and archive support may be useful in this case<u></u><u></u></p>
<p class="MsoNormal"> - if inlining/lto is more important than compile time keep library in an IR module and pre-link just before invoking MCJIT<u></u><u></u></p>
<p class="MsoNormal"> - create one instance of MCJIT as needed and destroy after execution<u></u><u></u></p>
</div>
</div>
<br>_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:LLVMdev@cs.uiuc.edu">LLVMdev@cs.uiuc.edu</a> <a href="http://llvm.cs.uiuc.edu" target="_blank">http://llvm.cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a><br>
<br></blockquote></div><br></div></div>