[LLVMdev] EE, JIT, MCJIT class structure

[Yaron Keren]

We have discussed most of these issues, here is a summary and some more
issues I thought of -

1) Support dynamic linking of *C++* modules. I have tried it and seems to
work - but not reliably. I may have made some programming error, or maybe
C++ linking requires more than what the dynamic linker is doing now
(vtables related?).

2) Support function unloading, if possible. If not, a mockup: support
function hiding so that a function can be hidden and overridden.

3) Relink function callers to call the new address. Alternatively modify
the old function to be a forwarder to the new function, maybe this is
simpler. A combined smart "function replacer" function.

4) Support module unloading, including EH, debug information and fixing the
dynamic linker data structures.

5) Errors should return error codes instead of aborting.

6) Have everything scale to 1000s of modules - maybe more. This requires
correct datatypes and minimum memory usage, as far as possible.

7) Simplify and minimize the API. Eliminate all JIT and interpreter legacy
functions. Eliminate finalize* functions, they should happen automatically.
Backward compatibility with JIT is not important as the big issue when
transiting from JIT to MCJIT is not the interface but dealing with whole
modules instead of functions and no lazy function creation. This requires
program redesign.

8) Several MCJIT function are just forwarders to the dynamic linker, maybe
it should be directly exposed in the MCJIT interface.

9) Move as much out of the API as possible, runFunctions and maybe
replaceFunction should be helpers.

10) Construct MCJIT w/o module. Otherwise special case logic is required
between the first and later modules.

11) Maybe constructors and destructors should be called automatically
before function run and on module exit?

12) In other words, MCJIT + dynamic linker should ideally handle all the
details regular linker does, transparent to the programmer.

Yaron



2013/10/18 Kaylor, Andrew <andrew.kaylor at intel.com>

>  Yes, we’ve been planning to separate the execution engines for a while
> now, and we’ve made some small steps (such as breaking out the
> RTDyldMemoryManager interface) in preparation for doing so.  There are some
> serious entanglements, not least of which there are MCJIT clients (LLDB,
> for instance) which are using memory managers derived from the
> JITMemoryManager.  Eventually (and soon) we’re going to just have to bite
> the bullet and go through whatever pain is necessary to make the split.***
> *
>
> ** **
>
> There’s going to be a BoF session at the upcoming LLVM Developer Meeting
> to talk about MCJIT and JIT.  One of my goals for that session is to get
> input from people who are using these engines to form a plan for ultimate
> separation and (if the outcry isn’t too great) deprecation of the old
> engine.  Obviously we’ll continue the discussion on the mailing list for
> people who can’t be at the Dev Meeting, but I was thinking having a lot of
> people in one room to talk about it will be a good starting point to get
> some momentum.****
>
> ** **
>
> You’ve been providing a lot of good input and contributions to MCJIT
> recently.  If you aren’t going to be at the Dev Meeting, please consider
> giving some thought to what long term visions you might have for MCJIT and
> send me something so that your voice can be represented in the discussion.
> ****
>
> ** **
>
> Thanks,****
>
> Andy****
>
> ** **
>
> *From:* Yaron Keren [mailto:yaron.keren at gmail.com]
> *Sent:* Friday, October 18, 2013 3:10 AM
> *To:* <llvmdev at cs.uiuc.edu>; Kaylor, Andrew
> *Subject:* EE, JIT, MCJIT class structure****
>
> ** **
>
> The functionality of MCJIT vs JIT had diverged.****
>
> ** **
>
> Not only they have different operating models (lazy function vs.
> multi-module),  the API is almost completely different. There is a set of
> functions just for JIT, another set of functions just for MCJIT, with
> comments saying which ones work for JIT and which ones work for MCJIT.****
>
> ** **
>
> It would make sense for the EE to have only the shared functionality and
> the inheriting classes JIT and MCJIT exposing their specialized
> functionality.****
>
> ** **
>
> This requires exposing JIT and MCJIT headers, including a JITcreater and a
> MCJITcreator or suitable constructors.****
>
> ** **
>
> Yaron****
>
> ** **
>
> ** **
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20131018/d09a0d18/attachment.html>