[LLVMdev] [RFC] MCJIT usage models
Syoyo Fujita
syoyofujita at gmail.com
Tue Dec 10 02:10:26 PST 2013
Recently I've coded interactive C/C++ shader JIT using MCJIT feature
on ARM, and it works very well and mostly can do same thing as done in
old JIT.
https://vimeo.com/81260481
https://vimeo.com/78504044 (old JIT version. FYI)
This can be a good example of 1)
Compilation speed isn't a critical matter, but faster is better it if
could be faster.
--
Syoyo
On Tue, Dec 10, 2013 at 2:46 PM, Yaron Keren <yaron.keren at gmail.com> wrote:
> Hi Andy,
>
> My use case is quite similar to what Keno described. I am using clang + JIT
> to dynamically compile C++ functions generated in response to user
> interaction. Generated functions may be unloaded or modified.
>
> I would like to break down the old JIT code into three major parts.
>
> 1) The old JIT has its own code emitter, which duplicates code from lib/MC
> and does not generate debug info and other limitations.
> 2) The old JIT supports lazy compilation.
> 3) The old JIT has its own function-level "dynamic linker"/memory manager,
> supporting function replacement with low-overhead JMP stubs.
>
> Now 1) is clearly a problem of code duplication. I'm not sure why a
> different emitter was created for the JIT but it would seem it's possible to
> reuse the lib/MC code just like MCJIT does. 2) takes much code all over the
> JIT code and at least for my use case it could just be removed.
>
> If 1) and 2) are solved and removed we are left with the (relatively small)
> "dynamic linker"/hack code only. I'd say the way this linker/loader works is
> much better fit for use cases such as my and Keno's than a classic linker
> like the ELF loader.
>
> The JIT linker works at function level rather than module level, supports
> automatic stub generation and relinking management. We could wrap functions
> in modules for the ELF loader, handle the stubs ourselves etc but this is
> tricky code as Keno said, requires hard-learned knowledge and feels like
> teaching an elephant to dance. The ELF loader is really designed for very
> different requirements.
>
> I'd like to see the old JIT function-level "dynamic linker" code preserved
> somehow as a ready alternative to a "classic" linker, especially useful when
> used in combination clang to dynamically run C/C++ functions.
>
> Yaron
>
>
>
> 2013/12/10 Keno Fischer <kfischer at college.harvard.edu>
>>
>> With Julia, we're obviously very much in the first use case. As you know,
>> we pretty much have a working version of Julia on top of MCJIT, but there's
>> still a few kinks to work out, which I'll talk about in a separate email.
>>
>> One think which I remember you asking at the BOF is what MCJIT currently
>> can't do well that the old JIT did, so I'd like to offer up an example. With
>> the old JIT, I used Clang to do dynamic code generation to interface to C++
>> easily. Now you might argue that is either a problem with clang or rather a
>> misuse of clang, but I'd like to think that we should keep the tools as
>> flexible as possible, so applications like this can emerge. With the old
>> JIT, I'd incrementally compile functions as Clang added them to the Module,
>> but with MCJIT that kind of stuff is rather tricky. What I ended up doing
>> was having Clang emit into a shadow module that never gets codegen'd and
>> when a function is requested, pulling that function and it's closure out of
>> the shadow module into the current MCJIT module. Perhaps functionality like
>> that should be more readily available in base LLVM to be able to use MCJIT
>> with clients not necessarily designed for use with MCJIT.
>>
>>
>>
>>
>> On Mon, Dec 9, 2013 at 1:08 PM, Kaylor, Andrew <andrew.kaylor at intel.com>
>> wrote:
>>>
>>> Below is an outline of various usage models for MCJIT that I put together
>>> based on conversations at last month’s LLVM Developer Meeting. If you’re
>>> using or thinking about using MCJIT and your use case doesn’t seem to fit in
>>> one of the categories below then either I didn’t talk to you or I didn’t
>>> understand what you’re doing.
>>>
>>>
>>>
>>> In any case, I’d like to see this get worked into a shape suitable for
>>> inclusion in the LLVM documentation. I imagine it serving as a guide both
>>> to those who are new to using MCJIT and to those who are developing and
>>> maintaining MCJIT. If you’re using MCJIT the latter (yes, the latter) case
>>> is particularly important to you right now as having your use case properly
>>> represented in this document is the best way to ensure that it is adequately
>>> considered when changes are made to MCJIT and when the decision is made as
>>> to when we are ready to deprecate the old JIT engine (probably in the 3.5
>>> release, BTW).
>>>
>>>
>>>
>>> So here’s what I’m asking for: if you are currently using MCJIT or
>>> considering using MCJIT, can you please find the use case that best fits
>>> your program and comment on how well the outline describes it. If you
>>> understand what I’m saying below but you see something that is missing,
>>> please let me know. If you aren’t sure what I’m saying or you don’t know
>>> how MCJIT might address your particular issues, please let me know that too.
>>> If you think my outline is too sketchy and you need me to elaborate before
>>> you can provide meaningful feedback, please let me know about that. If you
>>> think it’s the best piece of documentation you’ve read all year and you
>>> can’t wait to read it again, that’s good information too.
>>>
>>>
>>>
>>> Thanks in advance for any and all feedback.
>>>
>>>
>>>
>>> -Andy
>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------------------
>>>
>>>
>>>
>>> Models for MCJIT use
>>>
>>>
>>>
>>> 1. Interactive dynamic code generation
>>>
>>> - user types code which is compiled as needed for execution
>>>
>>> - example: Kaleidoscope
>>>
>>> - compilation speed probably isn't critical
>>>
>>> - use one MCJIT instance with many modules
>>>
>>> - create new modules on compilation
>>>
>>> - MCJIT handles linking between modules
>>>
>>> - external references still need prototypes
>>>
>>> - we can at least provide a module pass to automate it
>>>
>>> - memory overhead may be an issue but MCJIT can fix that
>>>
>>> - see model 2 for pre-defined library
>>>
>>> - if processing a large script pre-optimize before passing modules to
>>> MCJIT
>>>
>>>
>>>
>>> 2. Code generation for external target execution
>>>
>>> - client generates code to be injected into an external process
>>>
>>> - example: LLDB expression evaluation
>>>
>>> - target may be another local or remote
>>>
>>> - target architecture may not match host architecture
>>>
>>> - may use one or more instances of MCJIT (client preference)
>>>
>>> - MCJIT handles address remapping on request
>>>
>>> - custom memory manager handles code/data transfer
>>>
>>> - speed/memory requirements may vary
>>>
>>>
>>>
>>> 3. Large pre-defined module compilation and execution
>>>
>>> - code/IR is loaded from disk and prepared for execution
>>>
>>> - example: Intel(R) OpenCL SDK
>>>
>>> - compilation speed matters but isn't critical
>>>
>>> - initial startup time is somewhat important
>>>
>>> - execution speed is critical
>>>
>>> - memory consumption isn't an issue
>>>
>>> - tool integration may be important
>>>
>>> - use one MCJIT instance with multiple (but usually) few modules
>>>
>>> - use object caching for commonly used code
>>>
>>> - for very large, sparsely used libraries pre-link modules
>>>
>>> - object and archive support may be useful
>>>
>>>
>>>
>>> 4. Hot function replacement
>>>
>>> - client uses MCJIT to optimize frequently executed code
>>>
>>> - example: WebKit
>>>
>>> - compilation time is not critical
>>>
>>> - execution speed is critical
>>>
>>> - steady state memory consumption is very important
>>>
>>> - client handles pre-JIT interpretation/execution
>>>
>>> - MCJIT instances may be created as needed
>>>
>>> - custom memory manager transfers code memory ownership after
>>> compilation
>>>
>>> - MCJIT instance is deleted when no longer needed
>>>
>>> - client handles function replacement and lifetime management
>>>
>>>
>>>
>>> 5. On demand "one-time" execution
>>>
>>> - client provides a library of code which is used by small,
>>> disposable functions
>>>
>>> - example: database query?
>>>
>>> - initial load time isn't important
>>>
>>> - execution time is critical
>>>
>>> - if library code is fixed, load as shared library
>>>
>>> - if library code must be generated use a separate instance of MCJIT
>>> to hold the library
>>>
>>> - this instance can support multiple modules
>>>
>>> - use a custom memory manager to link with functions in this
>>> module
>>>
>>> - object caching and archive support may be useful in this case
>>>
>>> - if inlining/lto is more important than compile time keep library in
>>> an IR module and pre-link just before invoking MCJIT
>>>
>>> - create one instance of MCJIT as needed and destroy after execution
>>>
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
More information about the llvm-dev
mailing list