<div dir="ltr"><div dir="ltr"><div dir="ltr">Hi Bjoern,<div><br></div>Thanks for your patience. The good news is that there is a neater way to do this: ExecutionSession's lookup methods take a orc::SymbolState parameter which describes the state that symbols must reach before a query can return (See <a href="https://github.com/llvm/llvm-project/blob/d1a7bfca74365c6523e647fcedd84f8fa1972dd3/llvm/include/llvm/ExecutionEngine/Orc/Core.h#L1273).In">https://github.com/llvm/llvm-project/blob/d1a7bfca74365c6523e647fcedd84f8fa1972dd3/llvm/include/llvm/ExecutionEngine/Orc/Core.h#L1273).In</a> your case you want to use orc::SymbolState::Resolved, which will cause the query to return as soon as the searched-for symbols have been assigned an address.<div><br></div><div>So if your process is generating a set of symbols that may need to be transmitted to other sessions then you would write something like:</div><div><br></div><div><span style="font-family:monospace">auto Syms = ES.lookup(SearchOrder, SymbolsToTransmit, LookupKind::Static, SymbolState::Resolved</span><span style="font-family:monospace">);</span></div><div><span style="font-family:monospace">if (!Syms)</span></div><div><span style="font-family:monospace"> reportError(Syms.takeError());</span></div><div><font face="monospace">sendSymbolsToRemote(*Syms);</font></div><div><br></div><div>Out of interest -- Is it necessary for your JIT itself to be split across two or more processes? I had always anticipated having a single ExecutionSession attached to the target process and then running just the compilers on other processes. That gives you a star-like IPC network with N + 1 connections: One between the execution session and each of the N compilers, and one between the execution session and the target processs. In your model it sounds like the risk is that you may end up with N(N <a class="gmail_plusreply" id="plusReplyChip-0">+</a> 1) IPC links with every execution session having to communicate with every other and also with the target process.</div><div><br></div><div>-- Lang.</div></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Mar 17, 2020 at 12:34 AM Gaier, Bjoern <<a href="mailto:Bjoern.Gaier@horiba.com" target="_blank">Bjoern.Gaier@horiba.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<div lang="DE">
<div>
<p class="MsoNormal"><span lang="EN-GB">Hello LLVM-Mailing-List and Lang,<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB">I played around with this problem a bit and found a way of doing this – however I’m not sure about the consequences and if this is really a good idea.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB">I went trough all of the global symbols and functions being defined in the llvm::Module and set for each of them a new section name (based on there name). With this approach the memorymanager could tell me already the
addresses of my symbols, which I could then share with the other process. However, this approach needed to load the module two times, first to get the future address and in a second run to use the addresses to resolve the symbols.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB">The only disadvantage I noticed so far was that the overall size of the emitted code was bigger then having the regular sections. Are there any other risks I might have overseen?
<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB">Kind greetings<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB">Björn<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB"><u></u> <u></u></span></p>
<div>
<div style="border-style:solid none none;border-top-width:1pt;border-top-color:rgb(225,225,225);padding:3pt 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> llvm-dev <<a href="mailto:llvm-dev-bounces@lists.llvm.org" target="_blank">llvm-dev-bounces@lists.llvm.org</a>>
<b>On Behalf Of </b>Gaier, Bjoern via llvm-dev<br>
<b>Sent:</b> 06 March 2020 14:24<br>
<b>To:</b> LLVM Developers Mailing List <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>>; Lang Hames <<a href="mailto:lhames@gmail.com" target="_blank">lhames@gmail.com</a>><br>
<b>Subject:</b> [llvm-dev] [ORC JIT] -Resolving cross references in a multi-process scenario<u></u><u></u></span></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><span lang="EN-GB">Hello LLVM-Mailing-List and Lang,<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB">I have a very weird (or strict?) scenario and was curious if the ORC JIT can help me with it. Soo here it comes:<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB">I have currently a windows process ("Runtime") which does nothing but creating a shared memory with read, write and execute rights - and also writing some function addresses like from printf to it.
<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB">Then I have two or more processes which are using the ORC JIT to load modules to this shared memory, in "perspective" of the process I mentioned above ("Runtime"). The sections are remapped from the other processes to
be correct for "Runtime", also they resolve the undefined references for "Runtime" - like for printf and so on.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB">This works quite well so far! Wuhuhu x3<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB">However, there is one issue about it. Given that those two (or more processes) are now loading modules that reference each other. Like Module A is using a function of Module B - but Module B also uses a function of Module
A. How could I resolve those modules when they are loaded from different processes? Normally I would use llvm-link to link those modules but in my current scenario this is sadly not possible.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB">Can the ORC JIT help me with that? Can I solve this problem differently? Like replacing all the functions of Module A that rely on Module B with function pointers? Is something like that possible?<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB">I know this situation is pretty uncommon but sadly I ran into that issue.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB">Kind greetings<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB">Björn<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span>Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Takeshi
Fukushima. Junichi Tajika <u></u><u></u></span></p>
</div>
Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Takeshi Fukushima. Junichi Tajika
</div>
</blockquote></div>