<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">FWIW, support for recompilation of modules was a conscious intention of the design originally in order to support tiering up levels of optimization. The implementation since then has obviously diverged from that a bit, but it’s still a relevant design goal.<div><br></div><div>You make a good point on the section permissions. Supporting recompilation is more than just relocation handling and would require nontrivial juggling in a memory manager to do effectively.</div><div><br></div><div>-Jim</div><div><br><div><div>On May 7, 2014, at 4:43 PM, Kaylor, Andrew <<a href="mailto:andrew.kaylor@intel.com">andrew.kaylor@intel.com</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div lang="EN-US" link="blue" vlink="purple" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><div class="WordSection1" style="page: WordSection1;"><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">I’m not sure I’d agree that MCJIT is supposed to support recompilation of a module. I believe we’ve explicitly said that you can’t modify a module after it has been added to MCJIT, so why would you recompile it? In any event, if you do recompile a module, the relocation information will be recreated. <o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">But I take it that you’re actually addressing some case where a module is moved after it has been compiled and put into an executable state. I don’t understand this case either. Why would you move a module after it has been made executable?<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">Also, I would expect that you’ll run into problems with memory managers that set the permissions for code sections to RX after a module has been finalized. In those cases, if you try to reapply relocations (presumably to a module that didn’t move) it will cause a protection fault. Conversely, if you allow a module to be moved after it has been finalized, don’t you run into issues with the client having to keep track of whether or not the module is in an executable state?<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">-Andy<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><b><span style="font-size: 10pt; font-family: Tahoma, sans-serif;">From:</span></b><span style="font-size: 10pt; font-family: Tahoma, sans-serif;"><span class="Apple-converted-space"> </span>Lang Hames [<a href="mailto:lhames@gmail.com">mailto:lhames@gmail.com</a>]<span class="Apple-converted-space"> </span><br><b>Sent:</b><span class="Apple-converted-space"> </span>Wednesday, May 07, 2014 4:09 PM<br><b>To:</b><span class="Apple-converted-space"> </span>Kaylor, Andrew<br><b>Cc:</b><span class="Apple-converted-space"> </span><a href="mailto:llvm-commits@cs.uiuc.edu">llvm-commits@cs.uiuc.edu</a><br><b>Subject:</b><span class="Apple-converted-space"> </span>Re: [llvm] r208257 - [RuntimeDyld] Make RuntimeDyldImpl::resolveExternalSymbols preserve the<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><o:p> </o:p></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">Hi Andy,<o:p></o:p></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><o:p> </o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">MCJIT is supposed to support recompilation of a module, and linking multiple modules (thanks for making the latter work! :). So we need to keep external relocations around in case the module they point to is recompiled.<o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><o:p> </o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">Cheers,<o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">Lang.<o:p></o:p></div></div></div><div><p class="MsoNormal" style="margin: 0in 0in 12pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><o:p> </o:p></p><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">On Wed, May 7, 2014 at 3:56 PM, Kaylor, Andrew <<a href="mailto:andrew.kaylor@intel.com" target="_blank" style="color: purple; text-decoration: underline;">andrew.kaylor@intel.com</a>> wrote:<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">The discarding of processed relocations was an intentional choice. The relocation map can take up a pretty fair amount of memory, particularly in the case where you have a large number of modules that reference one another. Keeping the relocation map around will cause memory bloat issues for some clients.<br><br>My thinking was that once a module was finalized you can't remap its sections anymore. Do you have a use case where this is an unacceptable limitation?<br><br>-Andy<o:p></o:p></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><br>-----Original Message-----<br>From:<span class="Apple-converted-space"> </span><a href="mailto:llvm-commits-bounces@cs.uiuc.edu" style="color: purple; text-decoration: underline;">llvm-commits-bounces@cs.uiuc.edu</a><span class="Apple-converted-space"> </span>[mailto:<a href="mailto:llvm-commits-bounces@cs.uiuc.edu" style="color: purple; text-decoration: underline;">llvm-commits-bounces@cs.uiuc.edu</a>] On Behalf Of Lang Hames<br>Sent: Wednesday, May 07, 2014 3:34 PM<br>To:<span class="Apple-converted-space"> </span><a href="mailto:llvm-commits@cs.uiuc.edu" style="color: purple; text-decoration: underline;">llvm-commits@cs.uiuc.edu</a><br>Subject: [llvm] r208257 - [RuntimeDyld] Make RuntimeDyldImpl::resolveExternalSymbols preserve the<br><br>Author: lhames<br>Date: Wed May 7 17:34:08 2014<br>New Revision: 208257<br><br>URL:<span class="Apple-converted-space"> </span><a href="http://llvm.org/viewvc/llvm-project?rev=208257&view=rev" target="_blank" style="color: purple; text-decoration: underline;">http://llvm.org/viewvc/llvm-project?rev=208257&view=rev</a><br>Log:<br>[RuntimeDyld] Make RuntimeDyldImpl::resolveExternalSymbols preserve the relocation entries it applies.<br><br>Prior to this patch, RuntimeDyldImpl::resolveExternalSymbols discarded relocations for external symbols once they had been applied. This causes issues if the client calls MCJIT::finalizeLoadedModules more than once, and updates the location of any symbols in between (e.g. by calling MCJIT::mapSectionAddress).<br><br>No test case yet: None of our in-tree memory managers support moving sections around. I'll have to hack up a dummy memory manager before I can write a unit test.<br><br>Fixes <<a href="rdar://problem/16764378">rdar://problem/16764378</a>><br><br><br>Modified:<br> llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp<br><br>Modified: llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp<br>URL:<span class="Apple-converted-space"> </span><a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp?rev=208257&r1=208256&r2=208257&view=diff" target="_blank" style="color: purple; text-decoration: underline;">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp?rev=208257&r1=208256&r2=208257&view=diff</a><br>==============================================================================<br>--- llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp (original)<br>+++ llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp Wed May<o:p></o:p></div></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">+++ 7 17:34:08 2014<o:p></o:p></div><div><p class="MsoNormal" style="margin: 0in 0in 12pt; font-size: 12pt; font-family: 'Times New Roman', serif;">@@ -620,6 +620,8 @@ void RuntimeDyldImpl::resolveRelocationL<br> }<br><br> void RuntimeDyldImpl::resolveExternalSymbols() {<br>+ StringMap<RelocationList> ProcessedSymbols;<br>+<br> while (!ExternalSymbolRelocations.empty()) {<br> StringMap<RelocationList>::iterator i = ExternalSymbolRelocations.begin();<br><br>@@ -665,8 +667,20 @@ void RuntimeDyldImpl::resolveExternalSym<br> resolveRelocationList(Relocs, Addr);<br> }<br><br>+ ProcessedSymbols[i->first()] = i->second;<br> ExternalSymbolRelocations.erase(i);<br> }<br>+<br>+ // Restore the relocation entries that were consumed in the loop above:<br>+ //<br>+ // FIXME: Replace the following loop with:<br>+ // std::swap(ProcessedSymbols, ExternalSymbolRelocations)<br>+ // once StringMap has copy and move construction.<br>+ for (StringMap<RelocationList>::iterator I = ProcessedSymbols.begin(),<br>+ E = ProcessedSymbols.end();<br>+ I != E; ++I) {<br>+ ExternalSymbolRelocations[I->first()] = I->second; }<br> }<o:p></o:p></p></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"> //===----------------------------------------------------------------------===//<br><br><br>_______________________________________________<br>llvm-commits mailing list<br><a href="mailto:llvm-commits@cs.uiuc.edu" style="color: purple; text-decoration: underline;">llvm-commits@cs.uiuc.edu</a><br><a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits" target="_blank" style="color: purple; text-decoration: underline;">http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits</a><o:p></o:p></div></div></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><o:p> </o:p></div></div></div>_______________________________________________<br>llvm-commits mailing list<br><a href="mailto:llvm-commits@cs.uiuc.edu">llvm-commits@cs.uiuc.edu</a><br>http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits</div></blockquote></div><br></div></body></html>