<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">Existing clients (LLDB) deal with externals by resolving them to constant function pointers that are referenced in the IR. That’s obviously ugly as hell, but it gets things done.<div><br></div><div>The old JIT was able to simplify things because it assumed the JITed code was running in the same process as the JIT compiler. The MCJIT doesn’t assume that, so it has to handle more possibilities. For example, that the invoked function may be in a dylib which hasn’t yet been loaded into the target address space. I suggest having a look at the ld64, lld and dyld implementations to see what these relocation really imply. You won’t have to solve all of the potential scenarios to get anything done, but you will want to make sure to carefully check which use-case you’re in and error out when it’s one you haven’t handled yet. The worst case is if the code makes overly broad assumptions about possible inputs and resolves a relocation incorrectly in some subtle way. </div><div><br></div><div>-Jim</div><div><br><div><div>On Apr 10, 2013, at 11:49 AM, "Kaylor, Andrew" <<a href="mailto:andrew.kaylor@intel.com">andrew.kaylor@intel.com</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div lang="EN-US" link="blue" vlink="purple" style="letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><div class="WordSection1" style="page: WordSection1;"><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">The MachO handling isn’t always straightforward in that it has a lot of bit fields to handle while it’s also trying to accommodate the format abstractions baked into the interfaces. The actual relocation type gets extracted in the RuntimeDyldMachO::resolveRelocation function (RuntimeDyldMachO.cpp:32) before it is passed on to the architecture-specific handlers. So the mask you mentioned is probably OK.<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">The fact that this relocation type isn’t handled is, of course, the root of your problem.<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">MCJIT has grown on an as-needed basis up to now, with only the relocation types we were actually seeing being implemented. Unfortunately, I don’t know enough about MachO to help with this one. Maybe Jim Grosbach can help?<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">-Andy<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span></div><div><div style="border-style: solid none none; border-top-color: rgb(181, 196, 223); border-top-width: 1pt; padding: 3pt 0in 0in;"><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><b><span style="font-size: 10pt; font-family: Tahoma, sans-serif;">From:</span></b><span style="font-size: 10pt; font-family: Tahoma, sans-serif;"><span class="Apple-converted-space"> </span><a href="mailto:llvmdev-bounces@cs.uiuc.edu">llvmdev-bounces@cs.uiuc.edu</a> [<a href="mailto:llvmdev-bounces@cs.uiuc.edu">mailto:llvmdev-bounces@cs.uiuc.edu</a>]<span class="Apple-converted-space"> </span><b>On Behalf Of<span class="Apple-converted-space"> </span></b>Weiss, Eran<br><b>Sent:</b><span class="Apple-converted-space"> </span>Tuesday, April 09, 2013 11:51 PM<br><b>To:</b><span class="Apple-converted-space"> </span>Jiong Wang<br><b>Cc:</b><span class="Apple-converted-space"> </span><a href="mailto:llvmdev@cs.uiuc.edu">llvmdev@cs.uiuc.edu</a><br><b>Subject:</b><span class="Apple-converted-space"> </span>Re: [LLVMdev] Migration from JIT to MCJIT<o:p></o:p></span></div></div></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><o:p> </o:p></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;">Thank you for the help.<o:p></o:p></span></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;"> </span></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;">The relocation type value is anded with </span><span class="apple-style-span"><span style="font-size: 8.5pt; font-family: Menlo, serif; color: rgb(44, 46, 207);">0xffffffffL</span></span><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;">. (RuntimeDyldMachO.cpp:214)<o:p></o:p></span></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;">Maybe this mask should be different?<o:p></o:p></span></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;">Anyway, it seems like this relocation isn't implemented. (RuntimeDyldMachO.cpp:104)<o:p></o:p></span></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;"> </span></div></div><div style="border-style: solid none none; border-top-color: rgb(181, 196, 223); border-top-width: 1pt; padding: 3pt 0in 0in;"><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><b><span style="font-size: 11pt; font-family: Calibri, sans-serif;">From:<span class="Apple-converted-space"> </span></span></b><span style="font-size: 11pt; font-family: Calibri, sans-serif;">Jiong Wang <<a href="mailto:jiwang@tilera.com" style="color: purple; text-decoration: underline;">jiwang@tilera.com</a>><br><b>Date:<span class="Apple-converted-space"> </span></b>Tue, 9 Apr 2013 09:42:03 -0400<br><b>To:<span class="Apple-converted-space"> </span></b>Eran Weiss <<a href="mailto:eran.weiss@emc.com" style="color: purple; text-decoration: underline;">eran.weiss@emc.com</a>><br><b>Cc:<span class="Apple-converted-space"> </span></b>"<a href="mailto:llvmdev@cs.uiuc.edu" style="color: purple; text-decoration: underline;">llvmdev@cs.uiuc.edu</a>" <<a href="mailto:llvmdev@cs.uiuc.edu" style="color: purple; text-decoration: underline;">llvmdev@cs.uiuc.edu</a>><br><b>Subject:<span class="Apple-converted-space"> </span></b>Re: [LLVMdev] Migration from JIT to MCJIT<o:p></o:p></span></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;"> </span></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 10.5pt; font-family: 'MS Gothic';">于</span><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;"><span class="Apple-converted-space"> </span>2013/4/9 21:08, Weiss, Eran<span class="Apple-converted-space"> </span></span><span style="font-size: 10.5pt; font-family: 'MS Gothic';">写道</span><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;">:<o:p></o:p></span></div></div><blockquote style="margin-top: 5pt; margin-bottom: 5pt;"><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;">Hi,<o:p></o:p></span></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;"> </span></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;">I'm migrating my code (running on mac) from using JIT to MCJIT. My code generates in memory, mostly using the llvm-c api, and then runs the generated code.<o:p></o:p></span></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;">When I try to use MCJIT I encounter a problem with relocations of external symbols – functions compiled statically beforehand with gcc.<o:p></o:p></span></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;"> </span></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;">I get the following error:<o:p></o:p></span></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;"> </span></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;">Invalid relocation type!<o:p></o:p></span></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;">UNREACHABLE executed at /Users/weisse4/dev/llvm/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldMachO.cpp:89<o:p></o:p></span></div></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;"> </span></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;">While debugging, I see that the relocation type read in is 218103811, which seems corrupt to me.<o:p></o:p></span></div></div></blockquote><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;"><br>Hi Weiss,<br><br> I do not have any experience on Mach binary format, but the hex value of 218103811 is 0xd000003 (maybe the relocation type is RIT_Generic_PreboundLazyPointer = 3), something looks like a relocation entry composed of "symbol index" + "relocation type".<br> maybe something is wrong, that the relocation entry is not anded with a mask to get the final relocation type.<br><br>---<br>Regards,<br>Jiong<br><br><br><o:p></o:p></span></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;">Did someone encounter a similar error? Or can direct me to changes that I need to do while migrating from JIT to MCJIT?<o:p></o:p></span></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;"> </span></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;">Thanks.<o:p></o:p></span></div></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif;"><br><br><br><o:p></o:p></span></div><pre style="margin: 0in 0in 0.0001pt; font-size: 10pt; font-family: 'Courier New';"><span style="">_______________________________________________<o:p></o:p></span></pre><pre style="margin: 0in 0in 0.0001pt; font-size: 10pt; font-family: 'Courier New';"><span style="">LLVM Developers mailing list<o:p></o:p></span></pre><pre style="margin: 0in 0in 0.0001pt; font-size: 10pt; font-family: 'Courier New';"><span style=""><a href="mailto:LLVMdev@cs.uiuc.edu" style="color: purple; text-decoration: underline;">LLVMdev@cs.uiuc.edu</a> <a href="http://llvm.cs.uiuc.edu/" style="color: purple; text-decoration: underline;">http://llvm.cs.uiuc.edu</a><a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev" style="color: purple; text-decoration: underline;">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a></span></pre></div></div></div></blockquote></div><br></div></body></html>