[LLVMdev] Memory clean for applications using LLVM for JIT compilation

[Kaylor, Andrew]

Hi Frank,

I'm not sure I completely understand what your delegating memory manager is doing, but I probably don't need to.  If your primary objection is that you can't derive from the DefaultJITMemoryManager then I personally wouldn't object to your submitting a patch to make that accessible as a base class.

The future path for JIT and MCJIT is under debate.  A few of us would really like to see MCJIT replace JIT entirely, but it can't do that until it meets all of the needs of the people currently using the JIT (which it doesn't yet).  I think we'll get there, but I don't have a clear timeline.

Currently the memory manager interfaces for JIT and MCJIT are overlaid on top of one another and have minimal overlap.  I'd like to separate them completely in the near future.  As long as the old JIT engine is still around the old JITMemoryManager (and its default implementation) will still be relevant, but neither one is particularly relevant to MCJIT even now.

In case you haven't been following the discussions, the primary shortcomings of the MCJIT as compared to the older JIT are that it doesn't allow lazy compilation at the function level, it has limited support for multiple modules and you can't recompile/relink functions.  There are some explorations in progress to figure out how to deal with these limitations, but nothing is imminent.

If these limitations aren't a concern for you, I suggest you give MCJIT a try.  It has some nice benefits and is being more actively maintained.

-Andy

-----Original Message-----
From: Frank Henigman [mailto:fjhenigman at google.com] 
Sent: Wednesday, March 20, 2013 9:14 AM
To: Kaylor, Andrew
Cc: Dirkjan Bussink; llvmdev at cs.uiuc.edu
Subject: Re: [LLVMdev] Memory clean for applications using LLVM for JIT compilation

It seems wrong to delegate a dozen or more methods when I only want to change the behavior in a couple small ways.  It would have been easier to derive a class, but the base class is inaccessible (in an anonymous namespace).  And I was afraid to roll my own memory manager because I didn't understand what all those methods do.
So MCJIT is future - will JITMemoryManager remain relevant?  If so I can send a patch to make it possible to derive from the default implementation.
In my case (compiling shaders in mesa, a graphics library) an engine is used once then thrown away so I don't think there's a risk of confusing it by moving the code.
Thanks for the tip about relocating with MCJIT.


On Mon, Mar 18, 2013 at 8:56 PM, Kaylor, Andrew <andrew.kaylor at intel.com> wrote:
> I'm not sure I see why the delegating memory manager feels wrong to both of you.  That's exactly the kind of usage model I would envision for clients that needed to handle multiple ExecutionEngines that had reason to share a memory manager.  I'm saying this as an invitation to discussion, not to be argumentative.  We certainly could change things if it would make the design better.
>
> Basically, the intention of the current design is to keep the ExecutionEngine ignorant of memory allocation details.  I'm not sure it always works that way, but that's the intent.  I believe that the decision to have the ExecutionEngine own the memory manager was made so that it would be self-contained and capable of cleaning up after itself.
>
> The JIT engine has a rather ad hoc history and is more complicated than anyone would want.  It allocates and links together more things than it directly exposes and the lifecycles of different bits of memory aren't always obvious, especially if you get into re-JITing code.  I'm not sure there's a lot that can be done in the general case to keep it clean other than to have the engine own everything.
>
> I don't believe that the JIT engine is designed to handle having its function memory moved, so even if you could work out the relocation issues I don't think it's a good idea.
>
> The MCJIT engine, on the other hand, is designed to have its memory moved on the fly (though at the granularity of sections, not functions).  The MCJIT engine does not work with the PIC relocation model on 32-bit x86 targets, but it will let you reapply relocations with other models after you move things.  There's some code in the lli utility that does this if you're interested.
>
> I'm not sure what the current maintenance model is for the older JIT engine.  Near as I can tell when it grows at all it is driven by people who have a need making specific contributions and as such it doesn't have a particularly coherent direction.
>
> I have more idealistic hopes for the MCJIT engine, and if you're interested in using it I'd be happy to talk to you about future design directions.
>
> -Andy
>
>
>
> -----Original Message-----
> From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] 
> On Behalf Of Frank Henigman
> Sent: Sunday, March 17, 2013 2:18 PM
> To: Dirkjan Bussink
> Cc: llvmdev at cs.uiuc.edu
> Subject: Re: [LLVMdev] Memory clean for applications using LLVM for 
> JIT compilation
>
> Thanks for the reply, nice to have some validation.  I thought of another approach which might be preferable:
> generate relocatable code, use a JITEventListener to grab each function and copy it to my own memory, let all the LLVM stuff die normally then use my copy of the code.
> However when I call setRelocationModel(Reloc::PIC_) on the engine builder I get code that seg faults.
> The assembly looks plausible at a glance, but I'm not really up to speed on x86 assembly.
> Is PIC supposed to work with JIT on X86-32?
>
> On Sat, Mar 16, 2013 at 11:35 AM, Dirkjan Bussink <d.bussink at gmail.com> wrote:
>>
>> On Mar 7, 2013, at 20:48 , Frank Henigman <fjhenigman at google.com> wrote:
>>
>>> I derived a class from JITMemoryManager which delegates everything 
>>> to an instance made with CreateDefaultMemManager().  ExecutionEngine 
>>> destroys the wrapper, but I keep the inner instance which did the 
>>> actual work.  Works, but seems a bit ugly.  Did you find any other 
>>> solutions?
>>
>> I ended up implementing the exact same thing, it feels dirty but it has worked great for us so far.
>>
>> --
>> Dirkjan
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev