[llvm-dev] MCJit and remove module memory leak?

[Lang Hames via llvm-dev]

Hi Koffie,

What you've described sounds like the KaleidoscopeJIT class from
http://llvm.org/docs/tutorial/BuildingAJIT1.html : Modules that are added
are compiled to executable code and the module immediately deleted. The
executable code persists until the JIT instance is destroyed, or you call
"removeModule" (passing the handle returned from addModule).

Cheers,
Lang.


On Thu, Nov 17, 2016 at 6:00 AM, koffie drinker <gekkekoe at gmail.com> wrote:

> Hi Lang,
>
> I'm starting to migrate the MCJIT to ORC.
> The ideal scenario for a jit would be to generate the code and release
> module related objects.
>
> I had this in mind:
> - Add module to jit
> - Generate code
> - Remove module
> - Retain memory section that was used to store the generated code.
>
> Using this scenario, memory usage will be kept to a minimum. (I'm seeing
> quite some mem usage with MCJIT, it allocates a lot of memory but only uses
> small portion of it).
> Do you have any pointers on how to  get this starting ? I'm willing to
> share the results once it's finished. I might be useful to others as well.
>
>
>
> On Thu, Nov 17, 2016 at 12:54 AM, Lang Hames <lhames at gmail.com> wrote:
>
>> Hi Kevin, Koffie,
>>
>> We will start migrating to ORC for next release, but for now, this
>>> release invoke delete after remove right?
>>
>>
>> MCJIT's removeModule method does not delete the module. You'll need to do
>> that manually. OrcMCJITReplacement is a bug-for-bug compatible
>> implementation of MCJIT using ORC components, so it does not free the
>> memory either.
>>
>> Does this mean MCJIT is dead/deprecated and projects using it should start
>>> migrating away now? If so, what's the time frame?
>>
>>
>> The short answer is yes: I expect MCJIT to be deprecated soon and
>> eventually killed off. I'll be sending an email with details and a
>> discussion of the timeline to the dev-list in the next few days. That will
>> contain suggestions on how to transition to the new APIs (which I expect to
>> be relatively painless for most people). I can CC you on it if that helps?
>>
>> Cheers,
>> Lang.
>>
>>
>> On Fri, Nov 11, 2016 at 12:00 AM, koffie drinker via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>> There's a orc mcjit drop in replacement in the source tree.
>>> Am I correct to assume that Orc is used (and emulating mcjit behaviour)
>>> when replacing
>>> LLVMLinkInOrcMCJITReplacement();
>>> //LLVMLinkInMCJIT();
>>> and linking with libOrcJit ?
>>> Does this replacement handle memory better than original mcjit ?
>>>
>>>
>>> On Mon, Nov 7, 2016 at 8:39 PM, Kevin P. Neal via llvm-dev <
>>> llvm-dev at lists.llvm.org> wrote:
>>>
>>>> On Fri, Nov 04, 2016 at 04:20:32PM -0700, Lang Hames via llvm-dev wrote:
>>>> >    Hi Koffie,
>>>> >    Kaleidoscope is no longer using MCJIT - it has been moved over to
>>>> the
>>>> >    ORC-based KaleidoscopeJIT class (see
>>>> >    llvm/examples/Kaleidoscope/include/KaleidoscopeJIT.h). The
>>>> removeModule
>>>> >    method there does not leak memory.
>>>> >    I've added documentation in r286026 to describe MCJIT
>>>> removeModule's
>>>> >    crazy ownership contract.
>>>> >    This will be fixed properly when we kill off ExecutionEngine. :)
>>>>
>>>> Does this mean MCJIT is dead/deprecated and projects using it should
>>>> start
>>>> migrating away now? If so, what's the time frame?
>>>>
>>>> --
>>>> Kevin P. Neal                                http://www.pobox.com/~kpn/
>>>>
>>>> "It sounded pretty good, but it's hard to tell how it will work out
>>>> in practice." -- Dennis Ritchie, ~1977, "Summary of a DEC 32-bit
>>>> machine"
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>
>>>
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20161117/779fbf28/attachment.html>