<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1926182988;
mso-list-type:hybrid;
mso-list-template-ids:-490161934 -1721872162 67567641 67567643 67567631 67567641 67567643 67567631 67567641 67567643;}
@list l0:level1
{mso-level-text:"%1\.\)";
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="DE" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US">Hey Lang,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US">Woah! That mail contains a lot of information and things I never tried yet… Actually… the entire
</span><span lang="EN-GB" style="color:black">MaterializationUnit and MaterializationResponsibility part is… quite… overwhelming >O<<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="color:black">With “pop up” I mean… the process which is waiting for Module “Planschi” to “pop up” can not do a thing about it. It just waits until there is an table entry for it, indicating that the object file
was loaded by another process – or not.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="color:black">To understand the MU and MR things better…. I try now to explain the flow of program, when somebody would add an IR file to it – not caring about how that file comes to that central process.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<ol style="margin-top:0cm" start="1" type="1">
<li class="MsoListParagraph" style="margin-left:0cm;mso-list:l0 level1 lfo1"><span lang="EN-GB" style="mso-fareast-language:EN-US">I go over all symbols and grab the names I’m caring about for execution.<o:p></o:p></span></li><li class="MsoListParagraph" style="margin-left:0cm;mso-list:l0 level1 lfo1"><span lang="EN-GB" style="mso-fareast-language:EN-US">I create a DyLib with a unique name and stuff the IR module in it<o:p></o:p></span></li><li class="MsoListParagraph" style="margin-left:0cm;mso-list:l0 level1 lfo1"><span lang="EN-GB" style="mso-fareast-language:EN-US">I call lookup from LLJIT and pass the DyLib mentioned in 2. with my symbols, get the addresses and can execute them<o:p></o:p></span></li></ol>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US">This would be the case where everything went well.
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US">Okay…. So I use the “LLJITBuilder” to get my LLJIT instance, this would be before step 1.)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US">In step 2.) when I created the new DyLib, I would call addGenerator and pass a class to it that inherits from “DefinitionGenerator” right? Let’s call it “MyDefinitionGenerator" then.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US">“MyDefinitionGenerator" needs to provide an implementation for “tryToGenerate” which contains stuff which I ignore at this point.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US">Now I’m in step 3.) right? And oh nooo! Planschi_test is undefined.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US">Now the tryToGenerate function is executed and I realize there, that “Planschi_test” is something coming from a different module.
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US">Normally I would call the “define” function of the DyLib I got (in the tryToGenerate function) and providing an absolute value. But I can’t because “Planschi” didn’t popped up yet. So
I create instead a “My</span><span lang="EN-GB" style="color:black">TentativeDefinitionMaterializationUnit" and stuff it into the define function right?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="color:black">„MyTentativeDefinitionMaterializationUnit“ is something that inherits from „TentativeDefinitionMaterializationUnit" right? Now I have those overloaded functions where I need to provide implantation
for it. This is kinda shady to me, but this will make waaaay more sense when I looked at it I guess.</span><span lang="EN-GB" style="mso-fareast-language:EN-US"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US">However… Now I’m lost…<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US">What will happen now? So I gave
</span><span lang="EN-GB" style="color:black">„MyTentativeDefinitionMaterializationUnit“ To “Planschi_test” while doing the lookup and…. For some reason “Planschi” has not popped up yet. Where will the waiting for it happen? In one of the “MyTentativeDefinitionMaterializationUnit"
functions? Will the lookup I did in Step 3 return or will it be blocked until I used the MaterializationResponsibility To finally resolve the symbol?</span><span lang="EN-GB" style="mso-fareast-language:EN-US">
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US">I’m also not sure yet where exactly the code snippet you showed would go to…. But I guess it is in one of the functions of “</span><span lang="EN-GB" style="color:black">MyTentativeDefinitionMaterializationUnit"
right?</span><span lang="EN-GB" style="mso-fareast-language:EN-US"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US">Uh… Thank you again and sorry for any stupid question in this…
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US">Kind greetings<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US">Björn<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Lang Hames <lhames@gmail.com>
<br>
<b>Sent:</b> 30 September 2020 19:41<br>
<b>To:</b> Gaier, Bjoern <Bjoern.Gaier@horiba.com><br>
<b>Cc:</b> LLVM Developers Mailing List <llvm-dev@lists.llvm.org><br>
<b>Subject:</b> Re: [llvm-dev] ORC JIT - different behaviour of ExecutionSession.lookup?<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal">Hi Bjoern,<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal">In the current state we don’t have a JIT only an handcrafted object loader. All our object files are pre-compiled but will be loaded by different processes into the same shared memory for the main process to execute them.<o:p></o:p></p>
</blockquote>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Ok -- this helps a lot. As a general observation (not ORC specific): If you have multiple source processes targeting a single process then either everything needs to be lazy or your source processes will need to communicate via IPC to make
things safe (if modules in different processes can directly depend on one another). ORC is not a good fit for this model: It aims to solve the cross-module dependency problem with a dependency graph in one process, rather than IPC between multiple processes.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I think you could make ORC work for you here by changing your IPC just a little bit. Instead of:<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black">Source Process 1 (with JIT) -- Sends-linked-memory-to -\</span><span style="color:black"><o:p></o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">Source Process 2 (with JIT) -- Sends-linked-memory-to --> Target Process</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">Source Process 3 (with JIT) -- Sends-linked-memory-to -/</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">You would use:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black">Source Process 1 -- Sends-relocatable-object-to -\</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black">Source Process 2 -- Sends-relocatable-object-to --> Target Process (With JIT)</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black">Source Process 3 -- Sends-relocatable-object-to -/</span><span style="color:black"><o:p></o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal">In the second scenario you're sending relocatable objects via shared memory and then linking them with the JIT in the target process where we can fully track dependencies. You can link on multiple threads concurrently in the target process
so you needn't worry about this fully serializing your link steps. In fact it is likely to cut down on some of the IPC needed between source processes.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">If you don't want the JIT linked into your target process then you can add one extra step:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black">Source Process 1 -- Sends-relocatable-object-to -\</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black">Source Process 2 -- Sends-relocatable-object-to --> JIT Process --> Target Process</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black">Source Process 3 -- Sends-relocatable-object-to -/</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
</div>
</div>
<div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal"><span style="color:black">We would like to still have the files pre-compiled but we want to remove the handcrafted object loader. However, since we already create those files with Clang, it is not such a big change to compile them to IR.<o:p></o:p></span></p>
</blockquote>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">If you want to keep pre-compiling them to objects that should be fine too. The JIT can just load objects directly. The namespacing issue still needs to be solved, but that's doable at the object level.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal"><span style="color:black">> You're trying to do all this on Hard Mode. ;)<br>
Yeah it was a silly thought… sorry xO</span><o:p></o:p></p>
</blockquote>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">No worries -- Just want to make sure you don't do more work than you need to. :)</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal"><span style="color:black">You wouldn’t find a reference to “test”, you would find “Planschi_test”. We would first try resolving that with the standard C library functions. If this fails, then we will ask the object file “Planschi” if it
has “test” and resolve it. If “Planschi” is not loaded for some reason, then we go on with the next symbol and mark “Planschi_test” as undefined. So… we fail linking. However, our code allows to just try the linking again – so we do and do until it succeeds.
Only when the undefined references hit 0 we mark the object file as executable. However, if the module you depend on has still undefined references you can be screwed. Cause our code will not check the dependencies. Another reason why we want to move away.</span><o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">Great. I think I understand this now.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal"><span style="color:black">I think… but this is probably not thought through well:<br>
I need to do those steps on the IR level now – because the DefinitionGenerator needs to resolve all symbols, but I might have to wait for a different module to pop up first.<o:p></o:p></span></p>
</blockquote>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">What causes a module to "pop up"?<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">Either way I think you can solve this with a custom MaterializationUnit. Here's one scheme for solving this:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">If your definition generator receives a lookup for an external symbol (e.g. Planschi_test) then we'll call it "not-resolved-elsewhere" and create a TentativeDefinitionMaterializationUnit for it (this is something
you would need to write up). The TentativeDefinitionMaterializationUnit says "I provide Planschi_test" (by including it in its symbol flags map) but has no idea how it will actually do that yet. The emit method on your TentativeDefinitionMaterializationUnit
will be called immediately to satisfy the query, and be given a MaterializationResponsibility object covering Planschi_test. You hold on to this MaterializationResponsibility object while you do whatever it is that might cause the Planschi module to "pop up".
If/when Planschi does pop up you can add the object file to it to its own "Planschi" JITDylib, then do the following:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">(1) Look up the address of test in the Planschi JITDylib. This time you do want to use the Resolved state (rather than ready) because we don't want to create any cycles due to queries*.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">* E.g. "A" depends on "B", "B" depends on "A". If you try to produce each by looking up the other and waiting for it to be "Ready" then you'll deadlock. But you can look up the other and wait for it to be Resolved
and that will always succeed (addresses are assigned without waiting for dependencies to be resolved). Then you add "B" as a dependence of "A" and vice-versa and let Orc's dependence tracking take care of the rest. Anybody issuing a query looking for "A" or
"B" to be ready will find that their query doesn't return until both symbols are fully-materialized and safe to access.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">(2) Update the JIT state using your MaterializationResponsibility object:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black">if (auto TestSym = ES.lookup({&PlanschiJD}, ES.intern("test")), SymbolState::Resolved) {</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black"> // Tell the JIT that the address of PlanschiTest is the same</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black"> // as the address of "test" in the Planschi JITDylib.</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black"> MR.notifyResolved({</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black"> { ES.intern("Planschi_test"),</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black"> JITEvaluatedSymbol(TestSym.getAddress(),</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black"> JITSymbolFlags::Exported) }</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black"> });</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black"> </span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black"> // Tell the JIT that "Planschi_test" depends on the "test" symbol in</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black"> // the Planschi JITDylib:</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black"> MR.addDependencies(ES.intern("Planschi_test"),</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black"> {{&PlanschJD, { ES.intern("test") }}});</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black"> // Tell the JIT that "Planschi_test" has been emitted. We did not</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black"> // actually have to do any work for this because Planschi_test is</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black"> // just an alias.</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black"> MR.notifyEmitted();</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black"> // And you're done.</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black">} else {</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black"> // If we get here then the "test" symbol couldn't be resolved for some</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black"> // reason, so we need to report that error and notify everyone that</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black"> // "Planschi_test" failed too.</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black"> ES.reportError(TestSym.takeError());</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black"> MR.failMaterialization();</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black">}</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">The code above takes care of the case where the Planschi module can be found. The other possibility is that you get to the end of your process that causes modules to "pop up" (whatever this is) and there's still
no Planschi module. In this case you just report an error and call MR.failMaterialization() to tell everyone who needed Planschi_test that it really couldn't be found.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal"><span style="color:black">1.) When all symbols have such a unique name now, couldn’t I just add them all to the same DyLib and use the RessourceTracker to unload them later? This would spare me searching the right DyLibs.<o:p></o:p></span></p>
</blockquote>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">The scheme I just described uses JITDylibs to namespace things, but you could also (1) do everything at the IR level as you've described and rename them there, or (2) use a JITLink plugin to rename everything at
the object file level (this is doable, as long as you have a JITLink implementation for your target platform. So far we only have MachO and ELF on x86-64. I think you need COFF, right?).<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal"><span style="color:#222222">2.) Can I ask a DyLib if it has something for the “Parent_Child__Plansch_? test@@3HA" loaded?</span><span style="color:black"><o:p></o:p></span></p>
</blockquote>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal">Sort of. You can look up the symbol flags for “Parent_Child__Plansch_? test@@3HA". This will not trigger materialization, but will call your definition generator to try to generate a definition. In the scheme I described above you shouldn't
need to do this at all.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal">My biggest worry is, I do all those fancy renaming and such, but I still have no guarantee that the missing symbols are already present, but when I look up my symbols of interest it is to late… cause if something is missing because the
module was not loaded yet, then I get in an error state and I’m stuck there :c<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#222222">You shouldn't need to worry about this. By using a custom materialization unit as described above you can keep a query waiting as long as you like before either failing it or providing a definition. You don't
need to error out until you've determined that a symbol really can't be provided.</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal"><span style="color:#222222"> </span>3.) Isn’t the process of renaming those symbols like… really memory costly? Could I run into limitations of creating a too long function name?<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#222222">It depends on how long you expect your symbol names to get, and how memory constrained you are. Orc uses a string pool, so each string value is usually only held in one place in memory. This should keep the overhead
pretty low.</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#222222">-- Lang.</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">(</span><span style="color:#222222">n Wed, Sep 30, 2020 at 12:05 AM Gaier, Bjoern <</span><span style="color:black"><a href="mailto:Bjoern.Gaier@horiba.com" target="_blank">Bjoern.Gaier@horiba.com</a></span><span style="color:#222222">>
wrote:</span><span style="color:black"><o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Hey Lang,</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">> Do you mean that the object file is produced by another process and is being loaded into your JIT process for execution, or that you want your JIT to produce
code for several different processes? These are different problems with different solutions. I'll wait until I understand your use case to answer further.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">In the current state we don’t have a JIT only an handcrafted object loader. All our object files are pre-compiled but will be loaded by different processes into
the same shared memory for the main process to execute them.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">We would like to still have the files pre-compiled but we want to remove the handcrafted object loader. However, since we already create those files with Clang,
it is not such a big change to compile them to IR.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">> You're trying to do all this on Hard Mode. ;)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Yeah it was a silly thought… sorry xO</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">> The part of your use case that is the most opaque to me is the renaming. When you see a reference to "test" in some object, how do you decide that it should
resolve to the definition of "test" in, for example, Planschi, as opposed to some other module? Do you just have a list of modules that you check in-order until you find a matching symbol name?</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">You wouldn’t find a reference to “test”, you would find “Planschi_test”. We would first try resolving that with the standard C library functions. If this fails,
then we will ask the object file “Planschi” if it has “test” and resolve it. If “Planschi” is not loaded for some reason, then we go on with the next symbol and mark “Planschi_test” as undefined. So… we fail linking. However, our code allows to just try the
linking again – so we do and do until it succeeds. Only when the undefined references hit 0 we mark the object file as executable. However, if the module you depend on has still undefined references you can be screwed. Cause our code will not check the dependencies.
Another reason why we want to move away.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">I think… but this is probably not thought through well:</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">I need to do those steps on the IR level now – because the DefinitionGenerator needs to resolve all symbols, but I might have to wait for a different module to
pop up first.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">So I would iterate over the globals and the functions of the Module:</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">I would rename each symbol I encounter to have a unique name.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">This would be easy because when we load stuff, we give them a hierarchical name like “Parent_Child_Child_Dino”</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">So "?Sampler@@YAXXZ" Could be renamed to "Parent_Child_Child_Dino_?Sampler@@YAXXZ".<br>
<br>
If I encounter an undefined reference that is not part of the standard library like "?_Plansch_test@@3HA" Then I would extract the relative path and convert it to an absolute one, getting: “Parent_Child__Plansch_? test@@3HA”. When the “Plansch” module was loaded,
I would have give it an unique name as well so they would fit. I would then add them to there own DyLibs and would be happy… would I?</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">1.) When all symbols have such a unique name now, couldn’t I just add them all to the same DyLib and use the RessourceTracker to unload them later? This would
spare me searching the right DyLibs.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">2.) Can I ask a DyLib if it has something for the “Parent_Child__Plansch_? test@@3HA" loaded?</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-indent:35.4pt">
<span lang="EN-GB">My biggest worry is, I do all those fancy renaming and such, but I still have no guarantee that the missing symbols are already present, but when I look up my symbols of interest it is to late… cause if something is missing because the module
was not loaded yet, then I get in an error state and I’m stuck there :c</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">3.) Isn’t the process of renaming those symbols like… really memory costly? Could I run into limitations of creating a too long function name?</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">I hope that makes sense… cause I’m not good in explaining anything, especially not when it is in a different language :c</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Thank you again :D
</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Kind greetings</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Björn</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Lang Hames <<a href="mailto:lhames@gmail.com" target="_blank">lhames@gmail.com</a>>
<br>
<b>Sent:</b> 29 September 2020 18:55<br>
<b>To:</b> Gaier, Bjoern <<a href="mailto:Bjoern.Gaier@horiba.com" target="_blank">Bjoern.Gaier@horiba.com</a>><br>
<b>Cc:</b> LLVM Developers Mailing List <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>><br>
<b>Subject:</b> Re: [llvm-dev] ORC JIT - different behaviour of ExecutionSession.lookup?</span><o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Hi Bjoern,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">However, another thing of our system is, that each object file was loaded from a different process,<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Do you mean that the object file is produced by another process and is being loaded into your JIT process for execution, or that you want your JIT to produce code for several different
processes? These are different problems with different solutions. I'll wait until I understand your use case to answer further.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Writing this… I actually wondered about something else but this feels like a terrible approach… But now I’m curious xD<br>
So… if I load an IR-Module, could I use the static LLVM compiler to compile it to an object file and then use the source code of the LLD to load and resolve the symbols the same way/kind we did in the past?<br>
That sounds more like the “addObjectFile” function of the LLJIT… And then I guess I have to write a LinkLayer or something? That is where my knowledge ends…<br>
Disclaimer: I don’t like that approach but it would be interesting to know (also cause some people here would be happy with it .w.”)<o:p></o:p></p>
</blockquote>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">You're trying to do all this on Hard Mode. ;)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">ORC takes care of all this kind of stuff for you:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> Don't re-write IR. Leave references as symbolic -- they will be fixed up in the JIT linker.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> You don't need to write your own JIT linker. LLJIT has one built in.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">When you add things to the JIT:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> - If you have a program representation (module, object file, etc.) and you want to add it then just go ahead.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> - If your program representations contain external references then they must be resolvable or linking will fail (there's no getting around that, in a JIT or in a regular compile),
BUT...<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> - You can always add new definitions in response to a query by using a definition generator. If your definition generator can find/create a definition then great. If it can't
then the reference really is unresolved and linking really should fail.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">The part of your use case that is the most opaque to me is the renaming. When you see a reference to "test" in some object, how do you decide that it should resolve to the definition
of "test" in, for example, Planschi, as opposed to some other module? Do you just have a list of modules that you check in-order until you find a matching symbol name?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">-- Lang.<o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On Tue, Sep 29, 2020 at 2:30 AM Gaier, Bjoern <<a href="mailto:Bjoern.Gaier@horiba.com" target="_blank">Bjoern.Gaier@horiba.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Hey Lang,</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Thank you for your help and your patience – also for your answers in the “ORC JIT - Can modules independently managed with one LLJIT instance? + problems with
ExecutionSession.lookup” mail. Both problems have the same origin so I keep writing about it here, to avoid duplication.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">My big problem is still handling cross references between modules with “our” name scheme. Since our old loader loads object files, we resolved those references
with object files and since they were already compiled, we knew all addresses right away. With the LLJIT as I finally understand, I will only get the addresses when I have resolved every references, which makes the code way safer.
</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">However, another thing of our system is, that each object file was loaded from a different process, so sometimes not all symbols for ModuleA were present because
ModuleB was not loaded/requested yet. That was okay, so we kept resolving the undefined references of ModuleA until ModuleB was loaded and everything was fine.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">If I get it right… This would change now to having a single LLJIT representing the entire system. Each process would get it’s own DyLib for there module. However,
I would need to check on IR-Level now which symbols would be undefined – correct? Because if I wait until “DefinitionGenerator::tryToGenerate” is called and have to wait for a module that might never be loaded, then I’m stuck there forever.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">1.) If I find a symbol that is undefined – and it has our name scheme, then I would jump to for example ModuleB, which is also not jitted yet and would do a “replaceAllUsesWith”
on the Symbol of ModuleA to ModuleB – right? </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> - Would that mean, when I add ModuleA to DyLibA – is ModuleB then part of DyLibA as well?</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">1.1.) Alternatively I could rename the symbol</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">2.) If the ModuleB is already jitted, then I can take the address to do the “replaceAllUsesWith” right?</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">When I resolved all those references, then I can add the IR Module to my DyLib and compile it. However is it a good idea to use “replaceAllUsesWith” with addresses?
Seems like the DefinitionGenerator would be jobless…</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Writing this… I actually wondered about something else but this feels like a terrible approach… But now I’m curious xD</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">So… if I load an IR-Module, could I use the static LLVM compiler to compile it to an object file and then use the source code of the LLD to load and resolve the
symbols the same way/kind we did in the past?</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">That sounds more like the “addObjectFile” function of the LLJIT… And then I guess I have to write a LinkLayer or something? That is where my knowledge ends…<br>
Disclaimer: I don’t like that approach but it would be interesting to know (also cause some people here would be happy with it .w.”)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Thank you so far!</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Kind greetings</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Björn</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Lang Hames <<a href="mailto:lhames@gmail.com" target="_blank">lhames@gmail.com</a>>
<br>
<b>Sent:</b> 29 September 2020 01:47<br>
<b>To:</b> Gaier, Bjoern <<a href="mailto:Bjoern.Gaier@horiba.com" target="_blank">Bjoern.Gaier@horiba.com</a>><br>
<b>Cc:</b> LLVM Developers Mailing List <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>><br>
<b>Subject:</b> Re: [llvm-dev] ORC JIT - different behaviour of ExecutionSession.lookup?</span><o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Hi Bjoern,<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Even though the "tryToGenerate" function of my DefinitionGenerator returned a "llvm::orc::SymbolsNotFound" for the "?_Plansch_test@@3HA", I got an address for "?<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">That's because you're issuing the lookup with RequiredState == SymbolState::Resolved. This means that your query will return as soon as "?Sampler@@YAXXZ" is assigned an address.
In the JIT linker(s) addresses are assigned before external references are looked up. So after your lookup returns the linker attempts to find "?_Plansch_test@@3HA", fails, and so moves "?Sampler@@YAXXZ" to the error state.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">You almost always want to issue your lookups with RequiredState == SymbolState::Ready. This ensures that the query will not return until / unless the requested symbols (and all
their dependencies) are successfully linked into the target process and ready to execute.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Question 1.)<br>
Is there any way to reset the error state of "?Sampler@@YAXXZ" at this point?</span><o:p></o:p></p>
</blockquote>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">No. However, the removable code feature will allow you to remove failed materialization units once it lands in the mainline.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">- After my first call I used the "define" function of the JITDylib to define ?_Plansch_test@@3HA and then I tried calling the lookup function again and again,
however I only got the error: "Failed to materialize symbols" even though "?_Plansch_test@@3HA" was defined now...<br>
- Changing the order of the "define" and the "lookup" call works of course, but I'm interested in the case where I don't know the address yet.</span><o:p></o:p></p>
</blockquote>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">The JIT doesn't re-try linking. Once a symbol has failed to link it remains in the error state. In theory, once removable code is added you could choose to remove
and then re-add "?Sampler@@YAXXZ" after "?_Plansch_test@@3HA" is defined. The real solution though is just to make sure that "?_Plansch_test@@3HA" is defined (either directly or via a generator) before you look up "?Sampler@@YAXXZ".</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Out of curiosity I repeated the previous scenario - but added "?_Plansch_test@@3HA" to the "lookupSet" which changed things drasticly. When executing "lookup"
I now get the "llvm::orc::SymbolsNotFound" error from my DefinitionGenerator...</span><o:p></o:p></p>
</blockquote>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Yes. Because "?_Plansch_test@@3HA" is not defined. You should see a SymbolsNotFound error sent to your error reporter in the first scenario too, followed by a
failure-to-materialize error for "?Sampler@@YAXXZ".</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">... and "?Sampler@@YAXXZ" is stuck as a pending query in the MaterializingInfos entries.<o:p></o:p></p>
</blockquote>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Huh. That sounds like a bug: All references to the query should be removed from the state machine before it returns its result (in this case an error). I'll see
if I can reproduce this locally and fix it up, but it doesn't affect the discussion here.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">When I then add a definition for "?_Plansch_test@@3HA" and call "lookup" the second time, it will succeed and give me the addresses. Also I'm able to execute the code now. This
is great! However...<o:p></o:p></p>
</blockquote>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">When a lookup fails we try to restore the ExecutionSession state to what it was prior to the query. This is why the sequence "lookup -> symbols not found -> define -> lookup again"
worked.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Question 2.)<br>
Why did the first call to lookup not return the address of "?Sampler@@YAXXZ" like in the first scenario? I expected it would return an address for it.<o:p></o:p></p>
</blockquote>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">A lookup must match against all symbols before anything is JIT'd. When it failed to match "?_Plansch_test@@3HA" we immediately bailed out with an error. There was no further attempt
to compile "?Sampler@@YAXXZ".<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Question 3.)<br>
Can I somehow combine both behaviours? Getting the address for all the symbols (like in scenario 1) while still being able to provide definitions later (like in scenario 2)?<o:p></o:p></p>
</blockquote>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><i>Sort of.</i><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Definition generators allow you to provide a definition at the last minute (i.e. in response to a query). The best mental model though is: "All definitions that a generator can
generate are part of the interface of the dylib". E.g. if you use a DynamicLibrarySearchGenerator to mirror symbols from a dynamic library containing "foo", "bar" and "baz" then you should think of your JITDylib as containing definitions for "foo", "bar" and
"baz", even if the generator hasn't actually added them to the JITDylib yet. The reason is that it will add them in response to any query for them, so it's indistinguishable (except for timing and debug logging) from the case where they're already present.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">If you need to be able to defer adding a "real" definition beyond the initial lookup then your only option (and this only applies to functions) is a lazy-reexport. This allows you
to provide a definition for a function while deferring lookup until the first execution of the re-export at runtime. I wouldn't generally use this to break dependencies though: You want a definition of the real function body for "?_Plansch_test@@3HA" already
added to your JIT because (in general) you never know when JIT'd code will need it.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">My turn to ask a question: How is "?_Plansch_test@@3HA" created, and why not just add it up-front? :)<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">-- Lang.<o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On Mon, Sep 28, 2020 at 4:57 AM Gaier, Bjoern via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Hey everyone,</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">I felt this question is different from my other question - hope this is okay.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">So - I was playing around with the lookup function of the ExecutionSession and there are some things I don't understand.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">I have a .BC file with a function "?Sampler@@YAXXZ" referencing a value "?_Plansch_test@@3HA" that is not defined in that module itself. I first planed on not
providing an address for "?_Plansch_test@@3HA" but wanted to know the address of "?Sampler@@YAXXZ". So I issued something like that:</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> auto &ES = this->jit->getExecutionSession();</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> SymbolLookupSet lookupSet;</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">
</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> lookupSet.add("?Sampler@@YAXXZ", llvm::orc::SymbolLookupFlags::WeaklyReferencedSymbol);</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> ES.lookup({{&jit->getMainJITDylib(), llvm::orc::JITDylibLookupFlags::MatchAllSymbols}}, lookupSet, llvm::orc::LookupKind::Static, llvm::orc::SymbolState::Resolved);</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">
</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Even though the "tryToGenerate" function of my DefinitionGenerator returned a "llvm::orc::SymbolsNotFound" for the "?_Plansch_test@@3HA", I got an address for
"?Sampler@@YAXXZ". Dumping the "MainJITDylib" I saw, that the "?Sampler@@YAXXZ" was in an Error state. Which made sense - I guess.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Question 1.)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Is there any way to reset the error state of "?Sampler@@YAXXZ" at this point?</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">- After my first call I used the "define" function of the JITDylib to define ?_Plansch_test@@3HA and then I tried calling the lookup function again and again,
however I only got the error: "Failed to materialize symbols" even though "?_Plansch_test@@3HA" was defined now...</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">- Changing the order of the "define" and the "lookup" call works of course, but I'm interested in the case where I don't know the address yet.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Out of curiosity I repeated the previous scenario - but added "?_Plansch_test@@3HA" to the "lookupSet" which changed things drasticly. When executing "lookup"
I now get the "llvm::orc::SymbolsNotFound" error from my DefinitionGenerator and "?Sampler@@YAXXZ" is stuck as a pending query in the MaterializingInfos entries. When I then add a definition for "?_Plansch_test@@3HA" and call "lookup" the second time, it will
succeed and give me the addresses. Also I'm able to execute the code now. This is great! However...</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Question 2.)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Why did the first call to lookup not return the address of "?Sampler@@YAXXZ" like in the first scenario? I expected it would return an address for it.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Question 3.)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Can I somehow combine both behaviours? Getting the address for all the symbols (like in scenario 1) while still being able to provide definitions later (like
in scenario 2)?</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Thank you in advance and kind greetings,</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-GB">Björn</span><o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert,
Takashi Nagano, Junichi Tajika, Ergin Cansiz. <o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><o:p></o:p></p>
</blockquote>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert,
Takashi Nagano, Junichi Tajika, Ergin Cansiz. <o:p></o:p></p>
</div>
</blockquote>
</div>
</div>
<p class="MsoNormal">Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin Cansiz.
<o:p></o:p></p>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin Cansiz.
</body>
</html>