[llvm-dev] Deleting function IR after codegen

Tue Mar 8 14:04:03 PST 2016

Outstanding. Maybe, then, it's best for me to skip straight to a fully modern LLVM that has ORC JIT, rather than trying to incrementally step through and support each LLVM release which requires me (for 3.6 at least) to figure it out for straight MCJIT.

I appreciate all the pointers here. This is immensely helpful.

> On Mar 8, 2016, at 1:45 PM, Lang Hames <lhames at gmail.com> wrote:
> 
> Hi Pete, Larry,
> 
> @Lang, would it be possible for one of the kaleidoscope tutorials to demonstrate how to hook up freeing the Module/Context but still run any binary bits?
> 
> The new ORC-based Kaleidoscope tutorials are actually already doing this. You get this behavior more-or-less for free if you use the ORC JIT APIs, because of an interplay between two features:
> 
> (1) All the ORC components try to destroy their module and object-file pointers as soon as they can, and
> (2) The module and object pointer types are template types.
> 
> So, if you pass your Modules in as 'unique_ptr<Module>'s, the JIT will delete them as soon as its done with them, freeing the memory in the process.
> The executable code for the JIT is owned by the user's memory manager which is managed using a similar scheme except that the memory manager pointer lives until the JIT is torn down, or the user explicitly asks the JIT to discard it.
> 
> Cheers,
> Lang.
> 
> 
> On Tue, Mar 8, 2016 at 12:28 PM, Pete Cooper via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
> 
>> On Mar 8, 2016, at 11:45 AM, Larry Gritz via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>> 
>> Thanks for the pointer, it's always helpful to be able to see how another project solved similar problems.
> CCed Lang, although I think Andy already answered all the questions you had.
> 
> @Lang, would it be possible for one of the kaleidoscope tutorials to demonstrate how to hook up freeing the Module/Context but still run any binary bits?  Or are some of them already doing so?
> 
> Cheers,
> Pete
>> 
>> 
>>> On Mar 8, 2016, at 11:24 AM, Andy Ayers <andya at microsoft.com <mailto:andya at microsoft.com>> wrote:
>>> 
>>> FWIW, LLILC (https://github.com/dotnet/llilc <https://github.com/dotnet/llilc>) uses MCJIT with a custom memory manager to hold onto the binary bits and discard the rest.
>>> 
>>> As far as I know it doesn't leak, though we don't blow away the context, so that grows a bit over time.
>>> 
>>> -----Original Message-----
>>> From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org <mailto:llvm-dev-bounces at lists.llvm.org>] On Behalf Of Larry Gritz via llvm-dev
>>> Sent: Tuesday, March 8, 2016 11:04 AM
>>> To: llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>>> Subject: Re: [llvm-dev] Deleting function IR after codegen
>>> 
>>> YES. My use of LLVM involves an app that JITs program after program and will quickly swamp memory if everything is retained. It is crucial to aggressively throw everything away but the functions we still need to execute.
>>> 
>>> I've been faking it with old JIT (llvm 3.4/3.5) by using a custom subclass of JITMemoryManager that squirrels away the jitted binary code so that when I free the Modules, ExecutionEngine, and MemoryManager, the real memory that got allocated for the binary code is hidden and does not get deallocated.
>>> 
>>> Currently I'm struggling with bringing my code base up to MCJIT without losing this ability, because the memory consumption is killing me.
>>> 
>>> Sometimes I think that clang as the canonical user of LLVM does not reflect the diversity of JIT-oriented LLVM use cases. An "offline" compiler like clang gets to exit after compiling a module, but other apps using LLVM may JIT module after module after module indefinitely.
>>> 
>>> For that kind of use case, it would be great to have as a first-class feature the ability to free the IR of a compiled module, and even better, to throw away the Module and EE entirely but keep the ability to call into the JITed binary function. Many apps would benefit from a stable API for doing this. 
>>> 
>>> 	-- lg
>>> 
>>> 
>>>> On Mar 7, 2016, at 4:55 PM, Pete Cooper via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>>>> 
>>>> 
>>>> After codegen for a given function, the IR should no longer be needed.  In the AsmPrinter we convert from MI->MCInstr, and then we never go back and look at the IR during the MC layer.
>>>> 
>>>> I’ve prototyped a simple pass which can be (optionally) scheduled to do just this.  It is added at the end of addPassesToEmitFile.  It is optional so that clang can continue to leak the IR with --disable-free, but i would propose that all other tools, and especially LTO, would enable it.  The savings are 20% of peak memory in LTO of clang itself.
>>>> 
>>>> I could attach a patch, but first i’d really like to know if anyone is fundamentally opposed to this.
>>>> 
>>>> I should note, a couple of issues have come up in the prototype.
>>>> - llvm::getDISubprogram was walking the function body to find the subprogram.  This is trivial to fix as functions now have !dbg on them.
>>>> - The AsmPrinter is calling canBeOmittedFromSymbolTable on GlobalValue’s which then walks all their uses.  I think this should be done earlier in codegen as an analysis whose results are available to the AsmPrinter.
>>>> - BB’s whose addresses are taken, i.e. jump tables, can’t be deleted.  Those functions will just keep their IR around so no changes there.
>>>> 
>>>> With the above issues fixed, I can run make check and pass all the tests.
>>>> 
>>>> Comments very welcome.
>>>> 
>>>> Cheers,
>>>> Pete
>>> 
>>> --
>>> Larry Gritz
>>> lg at larrygritz.com <mailto:lg at larrygritz.com>
>>> 
>>> 
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>>> https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2flists.llvm.org%2fcgi-bin%2fmailman%2flistinfo%2fllvm-dev%0a&data=01%7c01%7candya%40microsoft.com%7c3a126b157c9545a5070708d347847d3f%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=cmNfRJTEMpDrEfKReureGCmBwnyP1UNeX3lTYUbXDc8%3d <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2flists.llvm.org%2fcgi-bin%2fmailman%2flistinfo%2fllvm-dev%0a&data=01%7c01%7candya%40microsoft.com%7c3a126b157c9545a5070708d347847d3f%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=cmNfRJTEMpDrEfKReureGCmBwnyP1UNeX3lTYUbXDc8%3d>
>> 
>> --
>> Larry Gritz
>> lg at larrygritz.com <mailto:lg at larrygritz.com>
>> 
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
> 
> 

--
Larry Gritz
lg at larrygritz.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160308/307de9b8/attachment.html>