<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif; "><div>Andrew,</div><div><br></div><div>I've attached a small reproduction of the issue.</div><div>Reproduce by:</div><div>$ /usr/bin/g++ `llvm-config --cxxflags` -g -m32 -c mcjit_external_symbol.cpp</div><div>$ /usr/bin/g++ `llvm-config --ldflags` -g -m32 -o mcjit_external_symbol mcjit_external_symbol.o `llvm-config --libs all`</div><div><div>$ ./mcjit_external_symbol </div><div>verifying... </div><div>LLVM ERROR: Program used external function '_external' which could not be resolved!</div></div><div><br></div><div>Note that using JIT the code works fine, in case it might help.</div><div><br></div><div>Eran.</div><div><br></div><span id="OLK_SRC_BODY_SECTION"><div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt"><span style="font-weight:bold">From: </span> "Kaylor, Andrew" <<a href="mailto:andrew.kaylor@intel.com">andrew.kaylor@intel.com</a>><br><span style="font-weight:bold">Date: </span> Wed, 10 Apr 2013 20:50:14 -0400<br><span style="font-weight:bold">To: </span> Eran Weiss <<a href="mailto:eran.weiss@emc.com">eran.weiss@emc.com</a>><br><span style="font-weight:bold">Cc: </span> "<a href="mailto:llvmdev@cs.uiuc.edu">llvmdev@cs.uiuc.edu</a>" <<a href="mailto:llvmdev@cs.uiuc.edu">llvmdev@cs.uiuc.edu</a>>, Jim Grosbach <<a href="mailto:grosbach@apple.com">grosbach@apple.com</a>>, Jiong Wang <<a href="mailto:jiwang@tilera.com">jiwang@tilera.com</a>><br><span style="font-weight:bold">Subject: </span> RE: [LLVMdev] Migration from JIT to MCJIT<br></div><div><br></div><div xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><meta http-equiv="Content-Type" content="text/html; charset=utf-8"><meta name="Generator" content="Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"MS Gothic";
panose-1:2 11 6 9 7 2 5 8 2 4;}
@font-face
{font-family:"MS Gothic";
panose-1:2 11 6 9 7 2 5 8 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
@font-face
{font-family:Menlo;
panose-1:0 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"\@MS Gothic";
panose-1:2 11 6 9 7 2 5 8 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.apple-style-span
{mso-style-name:apple-style-span;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
span.EmailStyle23
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--><div lang="EN-US" link="blue" vlink="purple"><div class="WordSection1"><p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125); font-family: Calibri, sans-serif; ">Eran,<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125); font-family: Calibri, sans-serif; "><o:p> </o:p></span></p><p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125); font-family: Calibri, sans-serif; ">Is there any chance you could boil this down to a small reproducer?<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125); font-family: Calibri, sans-serif; "><o:p> </o:p></span></p><p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125); font-family: Calibri, sans-serif; ">I’ve got a mid-to-long-term goal of getting rid of the ugliness in LLDB that Jim mentioned, and fixing your problem would be a good first step.<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125); font-family: Calibri, sans-serif; "><o:p> </o:p></span></p><p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125); font-family: Calibri, sans-serif; ">Thanks,<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125); font-family: Calibri, sans-serif; ">Andy<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125); font-family: Calibri, sans-serif; "><o:p> </o:p></span></p><div><div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in"><p class="MsoNormal"><b><span style="font-size: 10pt; font-family: Tahoma, sans-serif; ">From:</span></b><span style="font-size: 10pt; font-family: Tahoma, sans-serif; "> Jim Grosbach [<a href="mailto:grosbach@apple.com">mailto:grosbach@apple.com</a>]
<br><b>Sent:</b> Wednesday, April 10, 2013 4:19 PM<br><b>To:</b> Kaylor, Andrew; Jiong Wang<br><b>Cc:</b> Weiss, Eran; <a href="mailto:llvmdev@cs.uiuc.edu">llvmdev@cs.uiuc.edu</a><br><b>Subject:</b> Re: [LLVMdev] Migration from JIT to MCJIT<o:p></o:p></span></p></div></div><p class="MsoNormal"><o:p> </o:p></p><p class="MsoNormal">Existing clients (LLDB) deal with externals by resolving them to constant function pointers that are referenced in the IR. That’s obviously ugly as hell, but it gets things done.<o:p></o:p></p><div><p class="MsoNormal"><o:p> </o:p></p></div><div><p class="MsoNormal">The old JIT was able to simplify things because it assumed the JITed code was running in the same process as the JIT compiler. The MCJIT doesn’t assume that, so it has to handle more possibilities. For example, that the invoked function
may be in a dylib which hasn’t yet been loaded into the target address space. I suggest having a look at the ld64, lld and dyld implementations to see what these relocation really imply. You won’t have to solve all of the potential scenarios to get anything
done, but you will want to make sure to carefully check which use-case you’re in and error out when it’s one you haven’t handled yet. The worst case is if the code makes overly broad assumptions about possible inputs and resolves a relocation incorrectly in
some subtle way. <o:p></o:p></p></div><div><p class="MsoNormal"><o:p> </o:p></p></div><div><p class="MsoNormal">-Jim<o:p></o:p></p></div><div><p class="MsoNormal"><o:p> </o:p></p><div><div><p class="MsoNormal">On Apr 10, 2013, at 11:49 AM, "Kaylor, Andrew" <<a href="mailto:andrew.kaylor@intel.com">andrew.kaylor@intel.com</a>> wrote:<o:p></o:p></p></div><p class="MsoNormal"><br><br><o:p></o:p></p><div><div><p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125); font-family: Calibri, sans-serif; ">The MachO handling isn’t always straightforward in that it has a lot of bit fields to handle while it’s also trying to accommodate the format abstractions baked
into the interfaces. The actual relocation type gets extracted in the RuntimeDyldMachO::resolveRelocation function (RuntimeDyldMachO.cpp:32) before it is passed on to the architecture-specific handlers. So the mask you mentioned is probably OK.</span><o:p></o:p></p></div><div><p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125); font-family: Calibri, sans-serif; "> </span><o:p></o:p></p></div><div><p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125); font-family: Calibri, sans-serif; ">The fact that this relocation type isn’t handled is, of course, the root of your problem.</span><o:p></o:p></p></div><div><p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125); font-family: Calibri, sans-serif; "> </span><o:p></o:p></p></div><div><p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125); font-family: Calibri, sans-serif; ">MCJIT has grown on an as-needed basis up to now, with only the relocation types we were actually seeing being implemented. Unfortunately, I don’t know enough
about MachO to help with this one. Maybe Jim Grosbach can help?</span><o:p></o:p></p></div><div><p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125); font-family: Calibri, sans-serif; "> </span><o:p></o:p></p></div><div><p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125); font-family: Calibri, sans-serif; ">-Andy</span><o:p></o:p></p></div><div><p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125); font-family: Calibri, sans-serif; "> </span><o:p></o:p></p></div><div><div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in"><div><p class="MsoNormal"><b><span style="font-size: 10pt; font-family: Tahoma, sans-serif; ">From:</span></b><span class="apple-converted-space"><span style="font-size: 10pt; font-family: Tahoma, sans-serif; "> </span></span><span style="font-size: 10pt; font-family: Tahoma, sans-serif; "><a href="mailto:llvmdev-bounces@cs.uiuc.edu">llvmdev-bounces@cs.uiuc.edu</a>
[<a href="mailto:llvmdev-bounces@cs.uiuc.edu">mailto:llvmdev-bounces@cs.uiuc.edu</a>]<span class="apple-converted-space"> </span><b>On Behalf Of<span class="apple-converted-space"> </span></b>Weiss, Eran<br><b>Sent:</b><span class="apple-converted-space"> </span>Tuesday, April 09, 2013 11:51 PM<br><b>To:</b><span class="apple-converted-space"> </span>Jiong Wang<br><b>Cc:</b><span class="apple-converted-space"> </span><a href="mailto:llvmdev@cs.uiuc.edu">llvmdev@cs.uiuc.edu</a><br><b>Subject:</b><span class="apple-converted-space"> </span>Re: [LLVMdev] Migration from JIT to MCJIT</span><o:p></o:p></p></div></div></div><div><p class="MsoNormal"> <o:p></o:p></p></div><div><div><p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; ">Thank you for the help.</span><o:p></o:p></p></div></div><div><div><p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; "> </span><o:p></o:p></p></div></div><div><div><p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; ">The relocation type value is anded with </span><span class="apple-style-span"><span style="font-size: 8.5pt; color: rgb(44, 46, 207); font-family: Menlo, serif; ">0xffffffffL</span></span><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; ">.
(RuntimeDyldMachO.cpp:214)</span><o:p></o:p></p></div></div><div><div><p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; ">Maybe this mask should be different?</span><o:p></o:p></p></div></div><div><div><p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; ">Anyway, it seems like this relocation isn't implemented. (RuntimeDyldMachO.cpp:104)</span><o:p></o:p></p></div></div><div><div><p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; "> </span><o:p></o:p></p></div></div><div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in"><div><p class="MsoNormal"><b><span style="font-size: 11pt; font-family: Calibri, sans-serif; ">From:<span class="apple-converted-space"> </span></span></b><span style="font-size: 11pt; font-family: Calibri, sans-serif; ">Jiong Wang <<a href="mailto:jiwang@tilera.com"><span style="color:purple">jiwang@tilera.com</span></a>><br><b>Date:<span class="apple-converted-space"> </span></b>Tue, 9 Apr 2013 09:42:03 -0400<br><b>To:<span class="apple-converted-space"> </span></b>Eran Weiss <<a href="mailto:eran.weiss@emc.com"><span style="color:purple">eran.weiss@emc.com</span></a>><br><b>Cc:<span class="apple-converted-space"> </span></b>"<a href="mailto:llvmdev@cs.uiuc.edu"><span style="color:purple">llvmdev@cs.uiuc.edu</span></a>" <<a href="mailto:llvmdev@cs.uiuc.edu"><span style="color:purple">llvmdev@cs.uiuc.edu</span></a>><br><b>Subject:<span class="apple-converted-space"> </span></b>Re: [LLVMdev] Migration from JIT to MCJIT</span><o:p></o:p></p></div></div><div><div><p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; "> </span><o:p></o:p></p></div></div><div><div><div><p class="MsoNormal"><span style="font-size: 10.5pt; font-family: 'ＭＳ ゴシック', 'MS Gothic'; ">于</span><span class="apple-converted-space"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; "> </span></span><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; ">2013/4/9
21:08, Weiss, Eran<span class="apple-converted-space"> </span></span><span style="font-size: 10.5pt; font-family: 'ＭＳ ゴシック', 'MS Gothic'; ">写道</span><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; ">:</span><o:p></o:p></p></div></div><blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"><div><div><p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; ">Hi,</span><o:p></o:p></p></div></div><div><div><p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; "> </span><o:p></o:p></p></div></div><div><div><p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; ">I'm migrating my code (running on mac) from using JIT to MCJIT. My code generates in memory, mostly using the llvm-c api, and then runs the generated code.</span><o:p></o:p></p></div></div><div><div><p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; ">When I try to use MCJIT I encounter a problem with relocations of external symbols – functions compiled statically beforehand with gcc.</span><o:p></o:p></p></div></div><div><div><p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; "> </span><o:p></o:p></p></div></div><div><div><p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; ">I get the following error:</span><o:p></o:p></p></div></div><div><div><p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; "> </span><o:p></o:p></p></div></div><div><div><div><p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; ">Invalid relocation type!</span><o:p></o:p></p></div></div><div><div><p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; ">UNREACHABLE executed at /Users/weisse4/dev/llvm/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldMachO.cpp:89</span><o:p></o:p></p></div></div></div><div><div><p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; "> </span><o:p></o:p></p></div></div><div><div><p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; ">While debugging, I see that the relocation type read in is 218103811, which seems corrupt to me.</span><o:p></o:p></p></div></div></blockquote><div><p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; "><br>
Hi Weiss,<br><br>
I do not have any experience on Mach binary format, but the hex value of 218103811 is 0xd000003 (maybe the relocation type is RIT_Generic_PreboundLazyPointer = 3), something looks like a relocation entry composed of "symbol index" + "relocation type".<br>
maybe something is wrong, that the relocation entry is not anded with a mask to get the final relocation type.<br><br>
---<br>
Regards,<br>
Jiong<br><br><br><br></span><o:p></o:p></p></div><div><div><p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; ">Did someone encounter a similar error? Or can direct me to changes that I need to do while migrating from JIT to MCJIT?</span><o:p></o:p></p></div></div><div><div><p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; "> </span><o:p></o:p></p></div></div><div><div><p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; ">Thanks.</span><o:p></o:p></p></div></div><div><p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; "><br><br><br><br></span><o:p></o:p></p></div><pre>_______________________________________________<o:p></o:p></pre><pre>LLVM Developers mailing list<o:p></o:p></pre><pre><a href="mailto:LLVMdev@cs.uiuc.edu"><span style="color:purple">LLVMdev@cs.uiuc.edu</span></a> <a href="http://llvm.cs.uiuc.edu/"><span style="color:purple">http://llvm.cs.uiuc.edu</span></a><a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev"><span style="color:purple">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</span></a><o:p></o:p></pre></div></div></div><p class="MsoNormal"><o:p> </o:p></p></div></div></div></div></span></body></html>