<div dir="ltr"><div>Hi Rajesh,</div><div>Quick Note : If you don't want to use Lazy Compilation, don't use CompileOnDemand Layer, instead just add Module to CompileLayer and do a lookup. <br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, 19 Jul 2019 at 00:20, Rajesh S R via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Thanks Stefan!</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jul 18, 2019 at 9:59 AM Stefan Gränitz <<a href="mailto:stefan.graenitz@gmail.com" target="_blank">stefan.graenitz@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
Hi Rajesh<br>
<br>
On 1.: ORC defers compilation of the module and on first request,
compiles the entire module. This will trigger recursive lookups for
external symbols. Eager compilation means that the modules which
provide these symbols will now be compiled too, while lazy
compilation should instead insert stubs that will trigger
compilation once they are reached by execution.<br></div></blockquote><div><br></div><div>ha I see! So looking up any symbol in a module will trigger compilation of a module. So one trick is to insert a dummy function that does nothing and look it up, so that the entire module will be compiled? </div><div><br></div><div>Is there some API/mode to turn off lazy compilation entirely and just compile each module on addModule call? My understanding is ORC is the new API which supports both lazy and eager computation as the old APIs have been made "legacy" APIs, so there must be some mode with ORC API to default to full eager compilation? Is that correct?</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF">
<br>
On 2.: I am maybe not perfectly up-to-date, but what I would expect
to happen is this. First, your remote process allocates memory for
the executable code and sends the base address to the compiler
process. Then the compiler process compiles and links with this base
address and sends the (position-dependent) code back. You may also
manage to go with position-independent code, but then your remote
needs a dynamic linker. Not sure if there's an example for it.<br></div></blockquote><div><br></div><div>hmm, interesting! Although for this work you need some lower bound on code size generated. My case though needs position-independent code as i want to just compile my module once and use it across multiple processes (same idea as shared library) even across different machines.</div><div><br></div><div>Yeah a dynamic linker should be fine for this case. Just curious, the tutorial uses this <a href="https://llvm.org/doxygen/classllvm_1_1orc_1_1RTDyldObjectLinkingLayer.html" target="_blank">linker</a>. Is this dynamic linking resolving any unknown symbol at first execution? In this case the symbols in the generated code will fall into 2 category:</div><div>1. Symbols are defined in the host process. This is common case.</div><div>2. Symbols are defined in another module which is similarly compiled into position-independent code. This is not likely and i don't see a use-case currently, but may have some use-case in the future.</div><div><br></div><div>My understanding is that the symbol resolver does exactly that:</div><div><br></div><div><pre style="overflow:auto hidden;font-family:Consolas,"Deja Vu Sans Mono","Bitstream Vera Sans Mono",monospace;font-size:0.95em;line-height:15.96px;padding:0.5em;border:1px solid rgb(204,204,204);background-color:rgb(248,248,248);color:rgb(0,0,0)"> <span class="gmail-m_754691230392583659m_8765610602658103445gmail-n">Expected</span><span class="gmail-m_754691230392583659m_8765610602658103445gmail-o" style="color:rgb(102,102,102)"><</span><span class="gmail-m_754691230392583659m_8765610602658103445gmail-n">JITEvaluatedSymbol</span><span class="gmail-m_754691230392583659m_8765610602658103445gmail-o" style="color:rgb(102,102,102)">></span> <span class="gmail-m_754691230392583659m_8765610602658103445gmail-n">lookup</span><span class="gmail-m_754691230392583659m_8765610602658103445gmail-p">(</span><span class="gmail-m_754691230392583659m_8765610602658103445gmail-n">StringRef</span> <span class="gmail-m_754691230392583659m_8765610602658103445gmail-n">Name</span><span class="gmail-m_754691230392583659m_8765610602658103445gmail-p">)</span> <span class="gmail-m_754691230392583659m_8765610602658103445gmail-p">{</span>
<span class="gmail-m_754691230392583659m_8765610602658103445gmail-k" style="color:rgb(0,112,32);font-weight:bold">return</span> <span class="gmail-m_754691230392583659m_8765610602658103445gmail-n">ES</span><span class="gmail-m_754691230392583659m_8765610602658103445gmail-p">.</span><span class="gmail-m_754691230392583659m_8765610602658103445gmail-n">lookup</span><span class="gmail-m_754691230392583659m_8765610602658103445gmail-p">({</span><span class="gmail-m_754691230392583659m_8765610602658103445gmail-o" style="color:rgb(102,102,102)">&</span><span class="gmail-m_754691230392583659m_8765610602658103445gmail-n">ES</span><span class="gmail-m_754691230392583659m_8765610602658103445gmail-p">.</span><span class="gmail-m_754691230392583659m_8765610602658103445gmail-n">getMainJITDylib</span><span class="gmail-m_754691230392583659m_8765610602658103445gmail-p">()},</span> <span class="gmail-m_754691230392583659m_8765610602658103445gmail-n">Mangle</span><span class="gmail-m_754691230392583659m_8765610602658103445gmail-p">(</span><span class="gmail-m_754691230392583659m_8765610602658103445gmail-n">Name</span><span class="gmail-m_754691230392583659m_8765610602658103445gmail-p">.</span><span class="gmail-m_754691230392583659m_8765610602658103445gmail-n">str</span><span class="gmail-m_754691230392583659m_8765610602658103445gmail-p">()));</span>
<span class="gmail-m_754691230392583659m_8765610602658103445gmail-p">}</span></pre></div><div><br></div><div>with symbols from hostporcess already exported to execution session.</div><div><br></div><div>In particular none of them needs to open a separate file and load the symbol definition from it. </div><div>So, may be that "<a href="https://llvm.org/doxygen/classllvm_1_1orc_1_1RTDyldObjectLinkingLayer.html" target="_blank">linker</a>" is sufficient for my use-case?<br></div><div><br></div><div>So the main open question is: For compiling to position independent code, wondering if there are some CompileLayer compiler which can be used to emit position-independent code?</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF">
<br>
Best<br>
Stefan<br>
<br>
On 18/07/2019 03:08, Rajesh S R via llvm-dev wrote:<br>
<blockquote type="cite">
<div dir="ltr">Hi all,
<div>I am following the LLVM JIT tutorial API based on: <a href="https://llvm.org/docs/tutorial/BuildingAJIT1.html" target="_blank">https://llvm.org/docs/tutorial/BuildingAJIT1.html</a></div>
<div><br>
</div>
<div>I have 2 independent questions:</div>
<div>1. The tutorial mentions that "<b>will defer compilation of
the module until any of its definitions is looked up</b>". </div>
<div><br>
</div>
<div>Does that mean to force eager compilation, i have to keep
track of all functions added to the Module and do a lookup? I
want the module to be compiled fully after I do add module to
compile_layer. What is the best way to achieve this? Or should
I not use ORC API at all?</div>
<div><br>
</div>
<div>2. I want to compile on one process and want to ship the
machine code to another process and load it for execution
(with no lazy compilation where an uncompiled function will
trigger compilation to remote process which as I understand is
what the tutorial talks about). This means we need to generate
position-independent code and may need dynamic linking to
resolve any unknown function call based on symbol name. What
is the best way to achieve this? Is there some code pointers?</div>
<div><br>
</div>
<div>Thanks!</div>
<div><br>
</div>
<div>Rajesh S R</div>
</div>
<br>
<fieldset class="gmail-m_754691230392583659m_8765610602658103445m_-848262785765260922gmail-m_8112184280108284443mimeAttachmentHeader"></fieldset>
<pre class="gmail-m_754691230392583659m_8765610602658103445m_-848262785765260922gmail-m_8112184280108284443moz-quote-pre">_______________________________________________
LLVM Developers mailing list
<a class="gmail-m_754691230392583659m_8765610602658103445m_-848262785765260922gmail-m_8112184280108284443moz-txt-link-abbreviated" href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>
<a class="gmail-m_754691230392583659m_8765610602658103445m_-848262785765260922gmail-m_8112184280108284443moz-txt-link-freetext" href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a>
</pre>
</blockquote>
<pre class="gmail-m_754691230392583659m_8765610602658103445m_-848262785765260922gmail-m_8112184280108284443moz-signature" cols="72">--
<a class="gmail-m_754691230392583659m_8765610602658103445m_-848262785765260922gmail-m_8112184280108284443moz-txt-link-freetext" href="https://flowcrypt.com/pub/stefan.graenitz@gmail.com" target="_blank">https://flowcrypt.com/pub/stefan.graenitz@gmail.com</a></pre>
</div>
</blockquote></div></div>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote></div>