<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Outstanding. Maybe, then, it's best for me to skip straight to a fully modern LLVM that has ORC JIT, rather than trying to incrementally step through and support each LLVM release which requires me (for 3.6 at least) to figure it out for straight MCJIT.<div class=""><br class=""></div><div class="">I appreciate all the pointers here. This is immensely helpful.</div><div class=""><br class=""></div><div class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Mar 8, 2016, at 1:45 PM, Lang Hames <<a href="mailto:lhames@gmail.com" class="">lhames@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">Hi Pete, Larry,<div class=""><br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><span style="font-size:13px" class="">@Lang, would it be possible for one of the kaleidoscope tutorials to demonstrate how to hook up freeing the Module/Context but still run any binary bits?</span></blockquote><div class=""><br class=""></div><div class="">The new ORC-based Kaleidoscope tutorials are actually already doing this. You get this behavior more-or-less for free if you use the ORC JIT APIs, because of an interplay between two features:</div><div class=""><br class=""></div><div class="">(1) All the ORC components try to destroy their module and object-file pointers as soon as they can, and<br class=""></div><div class="">(2) The module and object pointer types are template types.</div><div class=""><br class=""></div><div class="">So, if you pass your Modules in as 'unique_ptr<Module>'s, the JIT will delete them as soon as its done with them, freeing the memory in the process.</div><div class="">The executable code for the JIT is owned by the user's memory manager which is managed using a similar scheme except that the memory manager pointer lives until the JIT is torn down, or the user explicitly asks the JIT to discard it.</div><div class=""><br class=""></div><div class="">Cheers,</div><div class="">Lang.</div><div class=""><br class=""></div></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On Tue, Mar 8, 2016 at 12:28 PM, Pete Cooper via llvm-dev <span dir="ltr" class=""><<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><br class=""><div class=""><span class=""><blockquote type="cite" class=""><div class="">On Mar 8, 2016, at 11:45 AM, Larry Gritz via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>> wrote:</div><br class=""><div class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">Thanks for the pointer, it's always helpful to be able to see how another project solved similar problems.</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""></div></blockquote></span>CCed Lang, although I think Andy already answered all the questions you had.</div><div class=""><br class=""></div><div class="">@Lang, would it be possible for one of the kaleidoscope tutorials to demonstrate how to hook up freeing the Module/Context but still run any binary bits? Or are some of them already doing so?</div><div class=""><br class=""></div><div class="">Cheers,</div><div class="">Pete</div><div class=""><div class="h5"><div class=""><blockquote type="cite" class=""><div class=""><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><blockquote type="cite" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class="">On Mar 8, 2016, at 11:24 AM, Andy Ayers <<a href="mailto:andya@microsoft.com" target="_blank" class="">andya@microsoft.com</a>> wrote:<br class=""><br class="">FWIW, LLILC (<a href="https://github.com/dotnet/llilc" target="_blank" class="">https://github.com/dotnet/llilc</a>) uses MCJIT with a custom memory manager to hold onto the binary bits and discard the rest.<br class=""><br class="">As far as I know it doesn't leak, though we don't blow away the context, so that grows a bit over time.<br class=""><br class="">-----Original Message-----<br class="">From: llvm-dev [<a href="mailto:llvm-dev-bounces@lists.llvm.org" target="_blank" class="">mailto:llvm-dev-bounces@lists.llvm.org</a>] On Behalf Of Larry Gritz via llvm-dev<br class="">Sent: Tuesday, March 8, 2016 11:04 AM<br class="">To: <a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a><br class="">Subject: Re: [llvm-dev] Deleting function IR after codegen<br class=""><br class="">YES. My use of LLVM involves an app that JITs program after program and will quickly swamp memory if everything is retained. It is crucial to aggressively throw everything away but the functions we still need to execute.<br class=""><br class="">I've been faking it with old JIT (llvm 3.4/3.5) by using a custom subclass of JITMemoryManager that squirrels away the jitted binary code so that when I free the Modules, ExecutionEngine, and MemoryManager, the real memory that got allocated for the binary code is hidden and does not get deallocated.<br class=""><br class="">Currently I'm struggling with bringing my code base up to MCJIT without losing this ability, because the memory consumption is killing me.<br class=""><br class="">Sometimes I think that clang as the canonical user of LLVM does not reflect the diversity of JIT-oriented LLVM use cases. An "offline" compiler like clang gets to exit after compiling a module, but other apps using LLVM may JIT module after module after module indefinitely.<br class=""><br class="">For that kind of use case, it would be great to have as a first-class feature the ability to free the IR of a compiled module, and even better, to throw away the Module and EE entirely but keep the ability to call into the JITed binary function. Many apps would benefit from a stable API for doing this.<span class=""> </span><br class=""><br class=""><span style="white-space:pre-wrap" class=""> </span>-- lg<br class=""><br class=""><br class=""><blockquote type="cite" class="">On Mar 7, 2016, at 4:55 PM, Pete Cooper via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>> wrote:<br class=""><br class=""><br class="">After codegen for a given function, the IR should no longer be needed. In the AsmPrinter we convert from MI->MCInstr, and then we never go back and look at the IR during the MC layer.<br class=""><br class="">I’ve prototyped a simple pass which can be (optionally) scheduled to do just this. It is added at the end of addPassesToEmitFile. It is optional so that clang can continue to leak the IR with --disable-free, but i would propose that all other tools, and especially LTO, would enable it. The savings are 20% of peak memory in LTO of clang itself.<br class=""><br class="">I could attach a patch, but first i’d really like to know if anyone is fundamentally opposed to this.<br class=""><br class="">I should note, a couple of issues have come up in the prototype.<br class="">- llvm::getDISubprogram was walking the function body to find the subprogram. This is trivial to fix as functions now have !dbg on them.<br class="">- The AsmPrinter is calling canBeOmittedFromSymbolTable on GlobalValue’s which then walks all their uses. I think this should be done earlier in codegen as an analysis whose results are available to the AsmPrinter.<br class="">- BB’s whose addresses are taken, i.e. jump tables, can’t be deleted. Those functions will just keep their IR around so no changes there.<br class=""><br class="">With the above issues fixed, I can run make check and pass all the tests.<br class=""><br class="">Comments very welcome.<br class=""><br class="">Cheers,<br class="">Pete<br class=""></blockquote><br class="">--<br class="">Larry Gritz<br class=""><a href="mailto:lg@larrygritz.com" target="_blank" class="">lg@larrygritz.com</a><br class=""><br class=""><br class="">_______________________________________________<br class="">LLVM Developers mailing list<br class=""><a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a><br class=""><a href="https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2flists.llvm.org%2fcgi-bin%2fmailman%2flistinfo%2fllvm-dev%0a&data=01%7c01%7candya%40microsoft.com%7c3a126b157c9545a5070708d347847d3f%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=cmNfRJTEMpDrEfKReureGCmBwnyP1UNeX3lTYUbXDc8%3d" target="_blank" class="">https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2flists.llvm.org%2fcgi-bin%2fmailman%2flistinfo%2fllvm-dev%0a&data=01%7c01%7candya%40microsoft.com%7c3a126b157c9545a5070708d347847d3f%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=cmNfRJTEMpDrEfKReureGCmBwnyP1UNeX3lTYUbXDc8%3d</a><br class=""></blockquote><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">--</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">Larry Gritz</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><a href="mailto:lg@larrygritz.com" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" target="_blank" class="">lg@larrygritz.com</a><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">_______________________________________________</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">LLVM Developers mailing list</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><a href="mailto:llvm-dev@lists.llvm.org" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" target="_blank" class="">llvm-dev@lists.llvm.org</a><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" target="_blank" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a></div></blockquote></div><br class=""></div></div></div><br class="">_______________________________________________<br class="">
LLVM Developers mailing list<br class="">
<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a><br class="">
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br class="">
<br class=""></blockquote></div><br class=""></div>
</div></blockquote></div><br class=""><div apple-content-edited="true" class="">
--<br class="">Larry Gritz<br class=""><a href="mailto:lg@larrygritz.com" class="">lg@larrygritz.com</a><br class=""><br class="">
</div>
<br class=""></div></body></html>