<div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr">Hi Bjoern,<div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">That is really cool :D Is the creation of that table a Windows thingy or is this the way the LLVM handles it?</blockquote><div><br></div><div>This is a Windows ABI feature, similar to GOTs on MachO and ELF. RuntimeDyld just needed to be taught to build the table correctly.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">Also… since it is COFF related – the never ending story of “finding my global constructors” first of all: Yes! I tried using the “initialize” function of LLVMJIT – however this only worked when I was loading a Module. When I added the object file (from the same source) the constructors were not called at all. What also really bothers me, when I load the object from disk and iterate over the symbols I will find: “_GLOBAL__sub_I_TestModule.cpp” but when I do a lookup on it, the symbol will not be found…</blockquote><div><br></div><div>This is a known limitation. To enable running static constructors from a COFF object file we would need a COFF Platform, and a COFF version of JITLink. The Platform would identify the static constructors when they are added to the JIT, and the COFF JITLink implementation would make them accessible despite them being static.</div><div><br></div><div>Regards,</div><div>Lang.</div></div></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Aug 24, 2020 at 11:07 PM Gaier, Bjoern <<a href="mailto:Bjoern.Gaier@horiba.com">Bjoern.Gaier@horiba.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<div lang="DE">
<div class="gmail-m_-1669747118317628137WordSection1">
<p class="MsoNormal"><span lang="EN-GB">Hey Lang,<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB">That is really cool :D Is the creation of that table a Windows thingy or is this the way the LLVM handles it?<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB">Also… since it is COFF related – the never ending story of “finding my global constructors” first of all: Yes! I tried using the “initialize” function of LLVMJIT – however this only
worked when I was loading a Module. When I added the object file (from the same source) the constructors were not called at all. What also really bothers me, when I load the object from disk and iterate over the symbols I will find: “_GLOBAL__sub_I_TestModule.cpp”
but when I do a lookup on it, the symbol will not be found… <u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB">It's like being sooo close to the constructor but den someone takes it away from me :< Plus object files seem to be not passed to the TransformationFunction so no luck there either.
I solved this issue with Modules by either using the mentioned function or by changing the visibility of the symbol – it seems like both is not possible with an object file .w.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB">Kind greetings<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB">Björn<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Lang Hames <<a href="mailto:lhames@gmail.com" target="_blank">lhames@gmail.com</a>>
<br>
<b>Sent:</b> 24 August 2020 19:46<br>
<b>To:</b> Gaier, Bjoern <<a href="mailto:Bjoern.Gaier@horiba.com" target="_blank">Bjoern.Gaier@horiba.com</a>><br>
<b>Cc:</b> Stefan Gränitz <<a href="mailto:stefan.graenitz@gmail.com" target="_blank">stefan.graenitz@gmail.com</a>>; LLVM Developers Mailing List <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>><br>
<b>Subject:</b> Re: [llvm-dev] ORC JIT - Incorrect support for COFF files?<u></u><u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">Hi Bjoern,<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<blockquote style="border-style:none none none solid;border-left-width:1pt;border-left-color:rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal">Using dllimport on my “planschiValue” actually worked! But I have no idea why, because the relocation is still a REL32 if I use dumpbin…<u></u><u></u></p>
</blockquote>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">From memory dllimport is like a GOT access: You'll have a REL32 either way, but instead of a REL32 directly to the variable you'll end up with a REL32 to an entry in a pointer table containing the address of the variable, and the code sequence
will change to access the variable indirectly via the pointer you load. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">-- Lang.<u></u><u></u></p>
</div>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">On Mon, Aug 24, 2020 at 12:57 AM Gaier, Bjoern <<a href="mailto:Bjoern.Gaier@horiba.com" target="_blank">Bjoern.Gaier@horiba.com</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="border-style:none none none solid;border-left-width:1pt;border-left-color:rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal"><span lang="EN-GB">Hey Lang and Stefan,</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB">Using dllimport on my “planschiValue” actually worked! But I have no idea why, because the relocation is still a REL32 if I use dumpbin… So how is it possible
for that to work?</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB">However… when I load an COFF object file, am I able to change the relocations to dllimport somehow? I honestly can’t imagine how this would work since the machine
code is probably already adjusted to use a REL32… or something… Just wanted to ask…</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB"><a href="mailto:stefan.graenitz@gmail.com" target="_blank"><span style="text-decoration:none">@Stefan Gränitz</span></a> I remember your patch! How does it deal
with extern variables though? As far as I understood from the code – not that I understood much – it fixes function calls by having a trampoline right?</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB">Thank you guys!</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB">Kind greetings</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB">Björn</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB"> </span><u></u><u></u></p>
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Lang Hames <<a href="mailto:lhames@gmail.com" target="_blank">lhames@gmail.com</a>>
<br>
<b>Sent:</b> 22 August 2020 19:50<br>
<b>To:</b> Stefan Gränitz <<a href="mailto:stefan.graenitz@gmail.com" target="_blank">stefan.graenitz@gmail.com</a>><br>
<b>Cc:</b> Gaier, Bjoern <<a href="mailto:Bjoern.Gaier@horiba.com" target="_blank">Bjoern.Gaier@horiba.com</a>>; LLVM Developers Mailing List <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>><br>
<b>Subject:</b> Re: [llvm-dev] ORC JIT - Incorrect support for COFF files?</span><u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<div>
<p class="MsoNormal">Hi Stefan, Bjoern,<u></u><u></u></p>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">For calls across object boundaries __dllimport now supported in RuntimeDyld:
<a href="https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2fllvm%2fllvm%2dproject%2fcommit%2f337e131ca7de48072def7729df69434c37a66eb7&umid=9d785513-e9cd-44e5-a111-cb77401a12f6&auth=b6f66d00f8195cc5198eee21f0dbabe6af0a3180-2bd41aad154bd7e67313fcb36c401a0c65d55652" target="_blank">
https://github.com/llvm/llvm-project/commit/337e131ca7de48072def7729df69434c37a66eb7</a>. You may just be able to mark your externs as __dllimport for this to work. I don't recall whether __dllimport works for data symbols too, but I suspect so -- I think it's
basically an explicit GOT entry.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">If anyone is ever interested in writing a JITLink COFF implementation I will be happy to help out or review patches -- I'd love to get better COFF support in tree.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">-- Lang.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<div>
<p class="MsoNormal">On Fri, Aug 21, 2020 at 2:09 PM Stefan Gränitz <<a href="mailto:stefan.graenitz@gmail.com" target="_blank">stefan.graenitz@gmail.com</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="border-style:none none none solid;border-left-width:1pt;border-left-color:rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<div>
<p class="MsoNormal" style="margin-bottom:12pt">Hi Björn<br>
<br>
I made a workaround for this specific issue a long time ago for the Projucer C++ JIT Engine. It basically forwards the call to another stub that provides enough space to encode a full 64-bit address. The patch is based on LLVM 3.9, so I guess it won't work
out-of-the-box on a recent release, but it may give you enough hints to figure it out on your own:<br>
<a href="https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2fweliveindetail%2fpj%2dllvm%2fcommit%2f97cd336d458ae9c73232d1b539ceefcdb2f5eb0f&umid=9d785513-e9cd-44e5-a111-cb77401a12f6&auth=b6f66d00f8195cc5198eee21f0dbabe6af0a3180-dffa61903884c3a0176d5f66111470dc6e9b9d2b" target="_blank">https://github.com/weliveindetail/pj-llvm/commit/97cd336d458ae9c73232d1b539ceefcdb2f5eb0f</a><br>
<br>
Alternatively, you could consider a memory manager that makes sure all calls are within a 32-bit range. For the code you JIT yourself, this should be no problem. In case you have to link against an old MSVCRT though, this is not possible, because it has a fixed
load address that's far beyond the 32-bit range from any heap allocation you can make for your JITed code. That's been our issue back then and so this patch was the last resort.<br>
<br>
Please also note: this is for freestanding function only. I didn't consider member function calls, etc. (addend is always 0) because it was not relevant for the specific issue.<br>
<br>
Hope it helps<br>
Stefan<u></u><u></u></p>
<div>
<p class="MsoNormal">On 21/08/2020 13:22, Gaier, Bjoern via llvm-dev wrote:<u></u><u></u></p>
</div>
<blockquote style="margin-top:5pt;margin-bottom:5pt">
<div>
<p class="MsoNormal"><span lang="EN-GB">I figured out that this problem is caused because “planschiValue” has a REL32 relocation and the addresses between the code and the variable overflows 32bit.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB">Is there any workaround for this kind of issue?</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB"> </span><u></u><u></u></p>
<div>
<div style="border-style:solid none none;border-top-width:1pt;border-top-color:rgb(225,225,225);padding:3pt 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> llvm-dev
<a href="mailto:llvm-dev-bounces@lists.llvm.org" target="_blank"><llvm-dev-bounces@lists.llvm.org></a>
<b>On Behalf Of </b>Gaier, Bjoern via llvm-dev<br>
<b>Sent:</b> 20 August 2020 12:15<br>
<b>To:</b> LLVM Developers Mailing List <a href="mailto:llvm-dev@lists.llvm.org" target="_blank">
<llvm-dev@lists.llvm.org></a>; Lang Hames <a href="mailto:lhames@gmail.com" target="_blank">
<lhames@gmail.com></a><br>
<b>Subject:</b> [llvm-dev] ORC JIT - Incorrect support for COFF files?</span><u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB">Hey LLVM-Mailing-List and Lang,</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB">I’m still learning how to use the ORC JIT but I finally reached the point to JIT and execute some code. For this purpose I created a test file (TestModule.cpp)
and compiled it with Clang, generating two different files, one in the LLVM IR format and one in the Microsoft COFF format.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB"> </span><u></u><u></u></p>
<p class="MsoNormal" style="margin-bottom:12pt"><span lang="EN-GB">The JIT resolves all undefined references, including “</span><span lang="EN-GB" style="font-size:9.5pt;font-family:Consolas;color:blue">extern</span><span lang="EN-GB" style="font-size:9.5pt;font-family:Consolas;color:black">
</span><span lang="EN-GB" style="font-size:9.5pt;font-family:Consolas;color:blue">int</span><span lang="EN-GB" style="font-size:9.5pt;font-family:Consolas;color:black"> planschiValue;</span><span lang="EN-GB">” and “</span><span lang="EN-GB" style="font-size:9.5pt;font-family:Consolas;color:blue">void</span><span lang="EN-GB" style="font-size:9.5pt;font-family:Consolas;color:black">
externFunction();</span><span lang="EN-GB">”. Using the IR Module I will get the correct address for “planschiValue” and the correct value – however this is not the case for the object file.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB">For the object file I get (via printf from the module) the address 0x000002295D6B003C while the actual address is 0x00007FF71D9959A4.
</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB">I’m really surprised about this, because the IR module works with no problem. I attach the source code, the IR file and the resulting object file (including its
assembly file).</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB">Any ideas what I’m doing wrong?</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB">Thank you for the help in advance and kind greetings</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB">Björn</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-GB"> </span><u></u><u></u></p>
<p class="MsoNormal">Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert,
Takashi Nagano, Junichi Tajika, Ergin Cansiz. <u></u><u></u></p>
</div>
<p class="MsoNormal">Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert,
Takashi Nagano, Junichi Tajika, Ergin Cansiz. <u></u><u></u></p>
<pre>_______________________________________________<u></u><u></u></pre>
<pre>LLVM Developers mailing list<u></u><u></u></pre>
<pre><a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><u></u><u></u></pre>
<pre><a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><u></u><u></u></pre>
</blockquote>
<pre>-- <u></u><u></u></pre>
<pre><a href="https://flowcrypt.com/pub/stefan.graenitz@gmail.com" target="_blank">https://flowcrypt.com/pub/stefan.graenitz@gmail.com</a><u></u><u></u></pre>
</div>
</blockquote>
</div>
</div>
<p class="MsoNormal">Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin Cansiz.
<u></u><u></u></p>
</div>
</blockquote>
</div>
</div>
Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin Cansiz.
</div>
</blockquote></div>