ExecutionEngine::finalizeModule()

[Yaron Keren]

Splitting into function-modules is useful as the first step to lazy
function compilation. Combined with function stubs to compile and add the
function-module as needed we'll have a real replacement for legacy JIT.

Two API would be required:

1) Compile and link whole modules as is today.
2) Lazy compile: automatically split the module to function-modules,
generate stubs only for the functions and lazy compile on the run.

Yaron



2013/10/19 Kaylor, Andrew <andrew.kaylor at intel.com>

>  > So the loaded state purpose is having modules in "standby" should they
> be required. OK.****
>
> ** **
>
> Not quite.  The purpose of the “added” state is having modules in standby.
> (added=known but not compiled)  The purpose of the “loaded” state is to
> have an intermediate state wherein clients can remap the addresses of
> sections, using MCJIT::mapSectionAddress, (in preparation for later copying
> to a new location or an external process) before relocations and page
> permissions are applied.  (loaded=compiled and required sections copied
> into local memory but not yet ready for execution)****
>
> ** **
>
> ** **
>
> > Maybe it's possible to split a module into function-modules
> automatically in a addModuleFunctions() function.****
>
> ** **
>
> That’s certainly possible.  I’m on the fence as to whether or not it’s
> desirable.  I could be convinced by a good argument either way.  It seems
> to me like it belongs more in a “helper” area.****
>
> ** **
>
> ** **
>
> > 1) Is the purpose of having loaded and finalized states to enable an
> external action on the module between loading and finalizing?****
>
> ** **
>
> Strictly speaking it is to enable externally initiated action on the
> object that was generated from the module between loading and finalizing.*
> ***
>
> ** **
>
> It might not be obvious at first glance, but the “remote-mcjit” case in
> the lli tool does this.  In that case, the intermediate action is taken by
> the RemoteMemoryManager in response to the notifyObjectLoaded function
> being called.****
>
> ** **
>
> ** **
>
> > 2) Is it OK to finalize a set of modules, add more modules and finalize
> again?****
>
> ** **
>
> Yes!****
>
> ** **
>
> ** **
>
> -Andy****
>
> ** **
>
> ** **
>
> *From:* Yaron Keren [mailto:yaron.keren at gmail.com]
> *Sent:* Friday, October 18, 2013 3:04 PM
>
> *To:* Kaylor, Andrew
> *Cc:* llvm-commits at cs.uiuc.edu
> *Subject:* Re: ExecutionEngine::finalizeModule()****
>
> ** **
>
> This data structure sounds good .****
>
> ** **
>
> I actually didn't consider lazy compilation with regard about module
> states because I knew that "MCJIT does not have lazy compilation".  So the
> loaded state purpose is having modules in "standby" should they be
> required. OK. ****
>
> ** **
>
> If you put one function into one module exactly (call this a
> function-module) you almost have the lazy compilation capability. Maybe
> it's possible to split a module into function-modules automatically in a
> addModuleFunctions() function.****
>
> ** **
>
> Some questions -****
>
> ** **
>
> 1) Is the purpose of having loaded and finalized states to enable an
> external action on the module between loading and finalizing?****
>
> ** **
>
> 2) Is it OK to finalize a set of modules, add more modules and finalize
> again?****
>
> ** **
>
> Yaron****
>
> ** **
>
> ** **
>
> 2013/10/19 Kaylor, Andrew <andrew.kaylor at intel.com>****
>
>  I’ve actually got an idea for this and am working on a patch today.
> I’ll send it to you for input when I have something working.****
>
>  ****
>
> Basically my idea is to replace the existing Modules vector that MCJIT
> inherits from ExecutionEngine with a smarter container class that will
> internally maintain separate lists/sets for added, loaded and finalized
> modules.   This class would provide iterator access that MCJIT could use to
> iterate through the entire combined set of modules (I think I can make that
> work reasonably) but also iterators to go through just the modules in a
> particular state.  This would let us get rid of the clumsy ModuleState
> thing completely and provide faster access to the subsets we need.****
>
>  ****
>
> The “loaded” state is needed because when relocations are processed
> (during the finalize handling) they are applied to all modules that have
> been loaded and the finalize operation on the memory manager (which sets
> page protection, invalidates code caches, etc.) is applied to the sections
> of all loaded modules.  It would require a significant amount of extra
> infrastructure to make this behave otherwise and I couldn’t think of a
> compelling reason why it would be needed.****
>
>  ****
>
> But maybe there’s some confusion here.  The emitted state is entirely
> useless because it is always immediately followed by object loading.  The
> state I’m currently calling “loaded” is probably what you’re referring to
> as “compiled”.  We definitely do not want to immediately compile modules
> when addModule is called because that would eliminate the only facsimile of
> lazy compilation MCJIT currently has.  The idea is that you can add a
> module to provide function definitions but if nothing in that module is
> referenced by a module in which a function is executed (note the wording
> here because it’s not exactly the ideal case) then the module never gets
> compiled.****
>
>  ****
>
> Eventually, we’ll want to add another level of lazy compilation in which
> we generate stubs to resolve external function during linking/finalization
> and only compile their dependent modules if the stub is called.  But that’s
> a job for another day.****
>
>  ****
>
> -Andy****
>
>  ****
>
> *From:* Yaron Keren [mailto:yaron.keren at gmail.com]
> *Sent:* Friday, October 18, 2013 1:41 PM****
>
>
> *To:* Kaylor, Andrew
> *Cc:* llvm-commits at cs.uiuc.edu
> *Subject:* Re: ExecutionEngine::finalizeModule()****
>
>  ****
>
> Ok, it's simpler than I thought since only three states are relevant.****
>
>  ****
>
> But first - do we need really all three states? what good is loaded state?
> is just means being on the list, and any operation that is possible on a
> loaded module should be possible before adding the module to MCJIT, right?
> ****
>
>  ****
>
> If we merge loading and emitted by having the addModule immediately emit
> the loaded module we'll need only two states:  "Compiled" and "Linked" (or
> "Finalized").****
>
>  ****
>
> addModule emits module and adds the module to an "Compiled" vector of
> modules.****
>
> Finalize loops over "Compiled" vector, each module is finalized, added to
> "Linked" list and then the "Compiled" list is cleared.****
>
>  ****
>
> If we still need three lists then just the same, three vectors would be
> required with add inserting the loaded vector, emit moving modules from
> loaded to compiled and finalize moving them from compiled to linked.****
>
>  ****
>
> Yaron****
>
>  ****
>
>  ****
>
>  ****
>
>  ****
>
> 2013/10/18 Kaylor, Andrew <andrew.kaylor at intel.com>****
>
>  Your suggestion to introduce an optimization within MCJIT to avoid
> inefficiency in finalizeLoadedModules is the correct direction.  I
> definitely prefer to improve the MCJIT implementation rather than
> increasing the surface of the interface and requiring clients to optimize
> their own use of MCJIT.****
>
>  ****
>
> We’ve talked about removing runFunction from the MCJIT interface.  It
> really doesn’t belong there and was just implemented as a convenience to
> ease the transition of legacy code to MCJIT.  As such, I think of it in the
> same way I think of finalizeObject.  That is, I’m hoping it will go away.
> It might make sense to provide an independent class or perhaps a static
> function that provides the signature decoding that’s contained in
> MCJIT::runFunction, which would ease the pain of deprecating it.  That
> method isn’t entirely complete as it stands, BTW.****
>
>  ****
>
> The Module state handling in MCJIT definitely isn’t mature.  The only
> states that really have any significance are “Added” (which needs a better
> name), “Loaded” and “Finalized”.  The current state tracking is just
> something I put in there to get the multiple module support up and
> running.  I would be happy to see it replaced with something better.****
>
>  ****
>
> -Andy****
>
>  ****
>
>  ****
>
> *From:* Yaron Keren [mailto:yaron.keren at gmail.com]
> *Sent:* Thursday, October 17, 2013 4:02 PM
> *To:* Kaylor, Andrew
> *Cc:* llvm-commits at cs.uiuc.edu
> *Subject:* Re: ExecutionEngine::finalizeModule()****
>
>  ****
>
> The changes you are making make sense. Return valid ready-to-run function
> address rather than depend upon the user calling finalize is right.****
>
>  ****
>
> The issue is with scalability of the current  implementation. Let's take
> your Kaliedoscope example (which was very helpful) as the use-case but
> every REPL will do the same.****
>
>  ****
>
> After adding  a new module it and only it needs to be finalized to run the
> new code. But finalizeLoadedModules loops over all modules to see which one
> should be finalized. It's not efficient. ****
>
>  ****
>
> I initially thought using finalizeModule would be the solution, calling it
> after creating a module, saving the loop.****
>
>  ****
>
> However this goes deeper and finalizeModule isn't the solution.  As you
> say getFunctionAddress calls finalizeLoadedModules() if a symbol found.
> finalizeLoadedModules always loops on all modules so it's wasteful. In an
> extreme case, we may know that all modules are already finalized but can't
> prevent the needless loop on module.****
>
>  ****
>
> The solution would be other data structures. ****
>
> Simplified case, instead of looping on all modules and checking module
> states  MCJIT could have a vector of un-finalized modules and a vector of
> finalized modules. New modules enter the un-finalized vector
> and finalizeLoadedModules() loops over the un-finalized vector, finalize
> every module, appends them to finalized vector and clears the unfinalized
> vector. ****
>
>  ****
>
> This is simplified since there are more ModuleStates so maybe other data
> structures should be used. I have not looked in depth the ModuleStates.
>
> Regarding getFunctionAddress, runFunction() does not use it but calls the
> older getPointerToFunction, so either runFunction() should be changed to
> the newer API or we still need to manually finalize modules.
>
> Yaron****
>
>  ****
>
> 2013/10/18 Kaylor, Andrew <andrew.kaylor at intel.com>****
>
>  Hi Yaron,****
>
>  ****
>
> What do you see as the use case for adding this?  I was thinking about
> removing finalizeObject() from the interface -- it’s mostly there to
> support code that was written when it was necessary.  It was probably a
> misstep to introduce finalizeObject in the EE interface in the first
> place.  It’s a case of leaking an implementation detail into the interface.
> ****
>
>  ****
>
> With the current trunk implementation of MCJIT, calling getFunctionAddress
> will implicitly finalize the module that contains the function (as well as
> any other modules that have been loaded).  The idea is that the address you
> get back is immediately useful (which was not true of the older
> getPointerToFunction method).****
>
>  ****
>
> Also note that within MCJIT calling finalizeModule has the effect of
> finalizing all loaded modules.  In fact, I plan to update the
> implementation of finalizeModule to call finalizeLoadedModules soon.****
>
>  ****
>
> -Andy****
>
>  ****
>
>  ****
>
> *From:* Yaron Keren [mailto:yaron.keren at gmail.com]
> *Sent:* Thursday, October 17, 2013 2:32 PM
> *To:* llvm-commits at cs.uiuc.edu; Kaylor, Andrew
> *Subject:* ExecutionEngine::finalizeModule()****
>
>  ****
>
> Hi,****
>
>  ****
>
> Make finalizeModule() accessible for MCJIT created through ExecutionEngine.
> ****
>
>  ****
>
> Maybe it would make sense to expose finalizeLoadedModules() as well.****
>
>  ****
>
> Yaron****
>
>  ****
>
>  ****
>
>   ****
>
>   ****
>
>  ** **
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20131019/c8dc6b13/attachment.html>