[llvm-dev] MCJit and remove module memory leak?

[koffie drinker via llvm-dev]

Hi Lang,

I actually meant, create execution engine, add module, jit, delete
module/execution engine and only keep the jitted memory parts

On Thu, Nov 17, 2016 at 8:10 PM, Lang Hames <lhames at gmail.com> wrote:

> Hi Koffie,
>
> What you've described sounds like the KaleidoscopeJIT class from
> http://llvm.org/docs/tutorial/BuildingAJIT1.html : Modules that are added
> are compiled to executable code and the module immediately deleted. The
> executable code persists until the JIT instance is destroyed, or you call
> "removeModule" (passing the handle returned from addModule).
>
> Cheers,
> Lang.
>
>
> On Thu, Nov 17, 2016 at 6:00 AM, koffie drinker <gekkekoe at gmail.com>
> wrote:
>
>> Hi Lang,
>>
>> I'm starting to migrate the MCJIT to ORC.
>> The ideal scenario for a jit would be to generate the code and release
>> module related objects.
>>
>> I had this in mind:
>> - Add module to jit
>> - Generate code
>> - Remove module
>> - Retain memory section that was used to store the generated code.
>>
>> Using this scenario, memory usage will be kept to a minimum. (I'm seeing
>> quite some mem usage with MCJIT, it allocates a lot of memory but only uses
>> small portion of it).
>> Do you have any pointers on how to  get this starting ? I'm willing to
>> share the results once it's finished. I might be useful to others as well.
>>
>>
>>
>> On Thu, Nov 17, 2016 at 12:54 AM, Lang Hames <lhames at gmail.com> wrote:
>>
>>> Hi Kevin, Koffie,
>>>
>>> We will start migrating to ORC for next release, but for now, this
>>>> release invoke delete after remove right?
>>>
>>>
>>> MCJIT's removeModule method does not delete the module. You'll need to
>>> do that manually. OrcMCJITReplacement is a bug-for-bug compatible
>>> implementation of MCJIT using ORC components, so it does not free the
>>> memory either.
>>>
>>> Does this mean MCJIT is dead/deprecated and projects using it should
>>>> start
>>>> migrating away now? If so, what's the time frame?
>>>
>>>
>>> The short answer is yes: I expect MCJIT to be deprecated soon and
>>> eventually killed off. I'll be sending an email with details and a
>>> discussion of the timeline to the dev-list in the next few days. That will
>>> contain suggestions on how to transition to the new APIs (which I expect to
>>> be relatively painless for most people). I can CC you on it if that helps?
>>>
>>> Cheers,
>>> Lang.
>>>
>>>
>>> On Fri, Nov 11, 2016 at 12:00 AM, koffie drinker via llvm-dev <
>>> llvm-dev at lists.llvm.org> wrote:
>>>
>>>> There's a orc mcjit drop in replacement in the source tree.
>>>> Am I correct to assume that Orc is used (and emulating mcjit behaviour)
>>>> when replacing
>>>> LLVMLinkInOrcMCJITReplacement();
>>>> //LLVMLinkInMCJIT();
>>>> and linking with libOrcJit ?
>>>> Does this replacement handle memory better than original mcjit ?
>>>>
>>>>
>>>> On Mon, Nov 7, 2016 at 8:39 PM, Kevin P. Neal via llvm-dev <
>>>> llvm-dev at lists.llvm.org> wrote:
>>>>
>>>>> On Fri, Nov 04, 2016 at 04:20:32PM -0700, Lang Hames via llvm-dev
>>>>> wrote:
>>>>> >    Hi Koffie,
>>>>> >    Kaleidoscope is no longer using MCJIT - it has been moved over to
>>>>> the
>>>>> >    ORC-based KaleidoscopeJIT class (see
>>>>> >    llvm/examples/Kaleidoscope/include/KaleidoscopeJIT.h). The
>>>>> removeModule
>>>>> >    method there does not leak memory.
>>>>> >    I've added documentation in r286026 to describe MCJIT
>>>>> removeModule's
>>>>> >    crazy ownership contract.
>>>>> >    This will be fixed properly when we kill off ExecutionEngine. :)
>>>>>
>>>>> Does this mean MCJIT is dead/deprecated and projects using it should
>>>>> start
>>>>> migrating away now? If so, what's the time frame?
>>>>>
>>>>> --
>>>>> Kevin P. Neal
>>>>> http://www.pobox.com/~kpn/
>>>>>
>>>>> "It sounded pretty good, but it's hard to tell how it will work out
>>>>> in practice." -- Dennis Ritchie, ~1977, "Summary of a DEC 32-bit
>>>>> machine"
>>>>> _______________________________________________
>>>>> LLVM Developers mailing list
>>>>> llvm-dev at lists.llvm.org
>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20161118/c6135a4a/attachment-0001.html>