<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Apr 20, 2016, at 11:02 AM, Teresa Johnson <<a href="mailto:tejohnson@google.com" class="">tejohnson@google.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="gmail_extra" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br class="Apple-interchange-newline">+cc Artem, who added the LinkOnlyNeeded flag.</div><div class="gmail_extra" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br class=""><div class="gmail_quote">On Wed, Apr 20, 2016 at 9:18 AM, Mehdi Amini via llvm-dev<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>></span><span class="Apple-converted-space"> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div style="word-wrap: break-word;" class="">Hi Neil,<div class=""><br class=""><div class=""><span class=""><blockquote type="cite" class=""><div class="">On Apr 20, 2016, at 5:20 AM, Neil Henning via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>> wrote:</div><br class=""><div class=""><div text="#000000" bgcolor="#FFFFFF" class="">TL;DR - when linking from a lazily loaded module and using Linker::LinkOnlyNeeded, bodies of used functions aren't being copied during linking.<br class=""><br class="">Previously on one of our products, we would lazily load our runtime module (around 9000 functions), and link some user module into this (which is in all practical use cases much smaller).</div></div></blockquote><div class=""><br class=""></div></span><div class="">It sounds reverse to what I would intuitively do (i.e. load the runtime into my module).</div><span class=""><br class=""><blockquote type="cite" class=""><div class=""><div text="#000000" bgcolor="#FFFFFF" class="">Then, post linking, we have a pass that runs over the module and rips out all the un-materialized functions that weren't being used in the original user module.<br class=""><br class="">I only just noticed that LinkModules has a flags parameter that can take a LinkOnlyNeeded flag, which made me wonder if I could reverse the link order (EG. link from the lazily loaded runtime module into the user module), set the LinkOnlyNeeded flag, and hey presto, we wouldn't need to have a cleanup pass that ran afterwards ripping out functions that weren't used.<br class=""><br class="">So I tried it, and it failed. Basically any function that was still to be materialized wasn't getting its body copied over during linking.<br class=""><br class="">The only line of code that differs when you set LinkOnlyNeeded is in LinkModules.cpp -> ModuleLinker::linkIfNeeded:<br class=""><br class=""><pre class="">if (shouldLinkOnlyNeeded() && !(DGV && DGV->isDeclaration()))
return false;
</pre><br class="">The isDeclaration() for functions has a call to isMaterializable().<br class=""><br class="">Things I've tried:<br class=""><ul class=""><li class="">If I don't pass LinkOnlyNeeded but still link from the lazily loaded runtime module into the user module, it works (albeit it is orders of magnitude slower like we'd expect).</li><li class="">If I don't lazily load the runtime module, it works (but again, much slower).</li><li class="">I tried doing the linking and then materializing the newly linked module, but the runtime functions were still missing their bodies (which implies the information was lost during linking).<br class=""></li><li class="">If I hack the LinkModules.cpp code such that it checks if the DGV could be materialized, and if so materialize it, before checking for a declaration again, it works:</li></ul><pre class="">if (shouldLinkOnlyNeeded() && !(DGV && DGV->isDeclaration())) {
if (DGV && DGV->isMaterializable())
DGV->materialize();
if (!(DGV && DGV->isDeclaration()))
return false;
}
</pre></div></div></blockquote><div class=""><br class=""></div></span><div class="">DGV is the GlobalValue in the *destination* Module, it is not clear to me how materializing has an effect on the *source* Module.</div><div class="">I am probably missing something here...</div></div></div></div></blockquote><div class=""><br class=""></div><div class="">I think the difference here is that the destination module is being lazy loaded, whereas I typically see the source modules being lazily loaded. </div></div></div></div></blockquote><div><br class=""></div><div>I understood from his description that he reversed the destination and source so that destination is the user code.</div><div>I assumed it was not lazy loaded, but that would explain the question then :)</div><div><br class=""></div><div>Neil: can you clarify? If Teresa is right, why aren't you materializing the destination module entirely?</div><div><br class=""></div><blockquote type="cite" class=""><div class=""><div class="gmail_extra" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><div class="gmail_quote"><div class="">So it sounds like the issue is that DGV has not *yet* been materialized in the dest module, and therefore DGV->isDeclaration() is returning false, leading the linkIfNeeded to return false, despite the fact that if we did materialize DGV it would be a declaration and would decide to link in SGV.</div></div></div></div></blockquote><div><br class=""></div><div>Oh I thought only function definition could be materializable, not declaration?</div><div><br class=""></div><blockquote type="cite" class=""><div class=""><div class="gmail_extra" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><div class="gmail_quote"><div class="">Not sure that this usage mode of lazy loading has been tested before. As Mehdi says, Rafael may have more insight.</div></div></div></div></blockquote><div><br class=""></div><div>Even materializing functions from the source module on the fly isn't supported right now, is it?</div><div><br class=""></div><div>-- </div><div>Mehdi</div><div><br class=""></div><br class=""><blockquote type="cite" class=""><div class=""><div class="gmail_extra" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><div class="gmail_quote"><div class=""><br class=""></div><div class="">Teresa</div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div style="word-wrap: break-word;" class=""><div class=""><div class=""><span class=""><br class=""><blockquote type="cite" class=""><div class=""><div text="#000000" bgcolor="#FFFFFF" class=""><p class="">Even with the extra cost of the hack above - this has resulted in a 2x speedup in our total link time.<br class=""></p><p class="">So really here I am wondering - is this expected behaviour? A bug? And if so how best to go about fixing the issue would be some grand advice from people more in the know!<br class=""></p></div></div></blockquote></span><div class="">The linker was written before Module was lazy loaded I think. Many pieces in LLVM assume things their working on are materialized.</div><div class="">On a side note (a bit off-topic), I wonder if `isDeclaration()` should return false for materializable function?</div><div class=""><br class=""></div><div class="">CC Rafael, who knows this code better.</div><span class="HOEnZb"><font color="#888888" class=""><div class=""><br class=""></div><div class="">-- </div><div class="">Mehdi</div><div class=""><br class=""></div><div class=""><br class=""></div></font></span></div></div></div><br class="">_______________________________________________<br class="">LLVM Developers mailing list<br class=""><a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a><br class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br class=""><br class=""></blockquote></div><br class=""><br clear="all" class=""><div class=""><br class=""></div>--<span class="Apple-converted-space"> </span><br class=""><div class="gmail_signature"><span style="font-family: Times; font-size: inherit;" class=""><table cellspacing="0" cellpadding="0" class=""><tbody class=""><tr style="color: rgb(85, 85, 85); font-family: sans-serif; font-size: small;" class=""><td nowrap="" style="border-top-style: solid; border-top-color: rgb(213, 15, 37); border-top-width: 2px;" class="">Teresa Johnson |</td><td nowrap="" style="border-top-style: solid; border-top-color: rgb(51, 105, 232); border-top-width: 2px;" class=""> Software Engineer |</td><td nowrap="" style="border-top-style: solid; border-top-color: rgb(0, 153, 57); border-top-width: 2px;" class=""> <a href="mailto:tejohnson@google.com" target="_blank" class="">tejohnson@google.com</a> |</td><td nowrap="" style="border-top-style: solid; border-top-color: rgb(238, 178, 17); border-top-width: 2px;" class=""> 408-460-2413</td></tr></tbody></table></span></div></div></div></blockquote></div><br class=""></body></html>