[LLVMdev] Multiple module support in MCJIT

[Kaylor, Andrew]

It may be easy enough to support cross dependencies if MCJIT knows about both modules before either of them is required.  The reason that I said I wasn't going to do it now is that if there are any complications that will likely be something I would postpone in the interest of time.

Let's say Module A contains a function AF and needs an external symbol BF, and Module B contains a function BF and needs an external dependency AF.  When MCJIT is emitting code for Module A, it will try to resolve the external BF symbol during the "finalize" phase.  It's easy enough to determine that we need to emit code for Module B in order to resolve this symbol.  However, we'll need to actually load Module B to get the address and while loading Module B we'll encounter the external AF symbol which is in a Module that we've started to load but haven't finished.

This is further complicated by the fact that to support the remote case we may need to support sections being moved after they are loaded, which is potentially a problem for all of the module-to-module linking.  I'm not sure yet how that will need to be resolved.

So anyway, I'm just hedging my bets on mutual dependency resolution.  I'll keep it in mind with my initial implementation, but I may punt it until later.

-Andy


From: Eli Bendersky [mailto:eliben at google.com]
Sent: Wednesday, September 18, 2013 8:40 AM
To: Kaylor, Andrew
Cc: llvmdev at cs.uiuc.edu; rafael.espindola at gmail.com; ehsan at mozilla.com; NAKAMURA Takumi; lhames at gmail.com; atrick at apple.com; Filip Pizlo (fpizlo at apple.com); Jim Grosbach (grosbach at apple.com)
Subject: Re: [LLVMdev] Multiple module support in MCJIT

Hi Andy,

Sounds good in general. One question below:

On Tue, Sep 17, 2013 at 4:01 PM, Kaylor, Andrew <andrew.kaylor at intel.com<mailto:andrew.kaylor at intel.com>> wrote:
I'm about to start working on multiple module support in MCJIT.  I know a lot of people are interested in this so it seemed like a good idea to begin by soliciting input.

My immediate plan involves the following:

- Add something to the MCJIT interface to explicitly request module compilation
- Isolate module-specific state variables into a structure so that the state of each module can be tracked independently
- Identify intermediate objects (such as Target Machine) which have module-specific states and duplicate them as necessary
- Implement a mechanism to automatically attempt to resolve external symbols from a module being loaded against any modules that are associated with the same instance of MCJIT.

I do not intend (at this time) to address the case where modules have mutual dependencies on one another, though I imagine that will be a case that we'll want to handle in the future.

Would it be difficult to support it from the start?



There's a bit of ugliness that has crept in where RuntimeDyld maintains a module-related state.  That wasn't supposed to happen and now that it has I'll have to do something about it.  I'll need to do a bit of experimenting to determine the best/easiest approach to take at this time.

I expect that what I'm going to do will more or less follow the shape of what I wrote about in the MCJIT/Kaleidoscope blog posts a couple of months ago, moving whatever I can inside MCJIT.

If anyone has any work in progress in this area or requirements that you want to make sure get addressed, please let me know very soon.

 Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130918/7c660aee/attachment.html>