<div dir="ltr"><div dir="ltr"><div dir="auto"><div dir="ltr"><span></span></div><div dir="ltr">Hi Preejackie,<div><br></div><div>Sorry for the delayed reply. I was quite busy over the weekend.</div><div><br></div><div><div dir="ltr"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><font color="#000000"><span style="background-color:rgba(255,255,255,0)">1) Is there any hard contract that the JIT should compile a full module (basic unit) into native code, or it can extract only hot functions from the module and compile it to native code, and leave the client of JIT (maybe a interpreter) interpret the remains of that module? Or it's the client's job to pack all hot functions into module and transfer that to JIT layer? If the latter is case, how JIT can benefit from speculating (I'm bit lost here). </span></font></blockquote></div><br></div><div>It is possible to extract only hot functions. The mental model for doing this is slightly complicated though, because of the way ORC’s design has been influenced by a desire to integrate well with static compilers (like LLVM): The JIT always starts materialization of an entire MaterializationUnit (e.g. IR Module, object, etc.). This mirrors the way static compilers work — they always compiling a whole module at a time. In ORC however, clients are free to write custom MaterializationUnits that break up the underlying module representation and only continue compilation of part of it.</div><div><br></div><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><span style="background-color:rgba(255,255,255,0)">I have come up with some idea for the speculation task:<br></span><font color="#000000"><span style="background-color:rgba(255,255,255,0)">1) Construct the local(per-module) function ordering list based on the sequence of their call in control flow graphs and create stub for each symbol from other module that is referenced by the current function, put the module in waiting list.</span></font></blockquote><div> </div><div><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);border-right-width:1px;border-right-style:solid;border-right-color:rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote></div>You should not need to worry about stub creation directly. ORC manages this via the lazy re-exports utility. You configure two JITDylibs, one (libFoo in this example) that everyone will link to, and a second (libFoo.impl) that will contain the actual definitions. For each definition, the lazyReexports utility will build a stub in libFoo which, when it is executed, will trigger a lookup of the corresponding symbol in libFoo.impl, then jump to it.</div><div><br></div><div><font face="monospace, monospace"> stubs: implementations:</font></div><div><font face="monospace, monospace">+--------+ +-------------+<br></font></div><div><font face="monospace, monospace">| libFoo | | libFoo.impl |</font></div><div><font face="monospace, monospace">+========+ +-------------+</font></div><div><font face="monospace, monospace">| foo | | foo |</font></div><div><font face="monospace, monospace">| bar | -- on call, looks up -> | bar |</font></div><div><font face="monospace, monospace">| baz | | baz |</font></div><div><font face="monospace, monospace">+--------+ +-------------+<br></font></div><div><br></div><div>So for the speculation project you will want to track this association. When you see someone reference "bar" in libFoo, and you determine that it would be profitable to speculatively compile the body of "bar", you would consult your tracker and find that the implementation of this function is provided by "bar" in libFoo.impl. You would then issue a lookup for "bar" in libFoo.impl and discard the result. This will trigger compilation of the body of bar. ORC will manage synchronization so that it doesn't matter whether your lookup for speculation comes before, after, or during any other lookup of "bar": it will only be compiled once, and nobody will call into it until it is ready. </div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">These speculation actions should be buried into internals of concurrent compiler and should not be visible to clients right?</blockquote><div><br></div><div>Actually, ORC is designed as a library of components. The aim would be to build components that support speculation so that other people can optionally use your work to add speculation to their compiler.</div><div><br></div><div>A good test environment would be the LLI llvm-interpreter: This uses the JIT to execute LLVM IR, and would provide a good test platform for speculation.</div><div><br></div><div>Cheers,</div><div>Lang.<br><br><div id="gmail-m_-5612660878291636876AppleMailSignature" dir="ltr">Sent from my iPad</div><div dir="ltr"><br>On Mar 15, 2019, at 1:44 PM, preejackie <<a href="mailto:praveenvelliengiri@gmail.com" target="_blank">praveenvelliengiri@gmail.com</a>> wrote:<br><br></div><blockquote type="cite"><div dir="ltr">
<p><font face="Courier New, Courier, monospace">Hi Lang, <br>
</font></p>
<p><font face="Courier New, Courier, monospace">As I'm going through
the design & code of ORC JIT. Although I didn't completely
understand it. I have some few questions regarding it.<br>
</font></p>
<p><font face="Courier New, Courier, monospace">As from the demo in
your talk "Updating ORC for concurrency" I have noticed
currently the ORC has addModule(...) function, which gives the
jit access to compile the IR module into native code.</font></p>
<p><font face="Courier New, Courier, monospace">1) Is there any hard
contract that the JIT should compile a full module (basic unit)
into native code, or it can extract only hot functions from the
module and compile it to native code, and leave the client of
JIT (maybe a interpreter) interpret the remains of that module?
Or it's the client's job to pack all hot functions into module
and transfer that to JIT layer? If the latter is case, how JIT
can benefit from speculating (I'm bit lost here). <br>
</font></p>
<p><font face="Courier New, Courier, monospace">I have come up with
some idea for the speculation task:<br>
</font></p>
<p><font face="Courier New, Courier, monospace">1) Construct the
local(per-module) function ordering list based on the sequence
of their call in control flow graphs and create stub for each
symbol from other module that is referenced by the current
function, put the module in waiting list.<br>
</font></p>
<p><font face="Courier New, Courier, monospace">2) Replace the stub
with actual native code address and notify the modules in the
waiting list.</font></p>
<p><font face="Courier New, Courier, monospace">The creation of
function ordering list involves analysing the control flow
graphs and branch probability for conditional function calls.
I'm also trying to figure out whether the function atttributes
will fit in this picture + more like using module summaries from
thinlto builds. <br>
</font></p>
<p><font face="Courier New, Courier, monospace">These speculation
actions should be buried into internals of concurrent compiler
and should not be visible to clients right? <br>
</font></p>
<p><font face="Courier New, Courier, monospace">How I can proceed
with plan, I'm open to comments!<br>
</font></p>
<p><font face="Courier New, Courier, monospace">Also, I didn't get
any responses for the post I wrote in llvm-dev list yet. IMHO it
is difficult to get this project through GSoC without a mentor.
Do you have any thoughts over it?</font></p>
<p><font face="Courier New, Courier, monospace">I will highly
appreciate your kind help:)</font></p>
<p><font face="Courier New, Courier, monospace"></font><br>
</p>
<div class="gmail-m_-5612660878291636876moz-cite-prefix">On 14/03/19 4:15 AM, preejackie wrote:<br>
</div>
<blockquote type="cite">
<p><font face="Courier New, Courier, monospace">Hi Lang, </font><br>
</p>
<div class="gmail-m_-5612660878291636876moz-cite-prefix">On 13/03/19 4:55 AM, Lang Hames
wrote:<br>
</div>
<blockquote type="cite">
Hi Perejackie,
<div><br>
</div>
<blockquote type="cite">
<div><span style="background-color:rgba(255,255,255,0)">As
far as I know, many literature's speak about speculative
optimization for Ahead of time compilation (please correct
me if I'm wrong).</span></div>
</blockquote>
<br>
<div>I am actually not sure what speculative optimization would
mean in the context of ahead of time optimization. Possibly it
could refer to the use of profile guided optimization to
optimize for a certain set of assumptions (derived from the
profiling information) with a fallback for when those
assumptions do not hold.</div>
<div><br>
</div>
<div>The kind of speculative compilation that I am talking about
only makes sense in the context of a JIT compiler that
supports multiple threads of compilation. In that case the JIT
compiler is free to run ahead of the actual execution,
compiling functions that it has determined will be called, or
are likely to be called, in the future. The challenges to do
this in LLVM are</div>
<div><br>
</div>
<div>(1) Infrastructure: making it possible to speculate at all.
Right now all the foundational code is there, but the APIs and
glue code to make it work are not there yet, and indeed not
fully designed yet.</div>
<div><br>
</div>
<div>(2) Making good speculation decisions: Even once you *can*
speculate, you still want to speculate well. This would
involve program analysis on higher level program
representations (e.g. LLVM IR for a proof-of-concept), and
could also involve profiling data gathered for profile guided
optimizations.</div>
<div><br>
</div>
</blockquote>
I understand and working a finding heuristics for better
speculation.<br>
<blockquote type="cite">
<div>
<blockquote type="cite">
<div dir="ltr">
<p><font color="#000000"><span style="background-color:rgba(255,255,255,0)">It would be a great help if you point me to
some references that you found interesting &
relevant to speculative compilation. </span></font></p>
</div>
</blockquote>
</div>
<div>I have not actually done a literature search yet. The bulk
of my recent work focused on simply building the
infrastructure for safe multi-threaded JIT compilation in
LLVM.</div>
<div><br>
</div>
</blockquote>
<p>Okay, no probelm. <br>
</p>
<p>I have mailed the list with subject : Improving Speculative
compilation in concurrent orc jit. Please see that. <br>
</p>
<p>Thanks a lot<br>
</p>
<blockquote type="cite">
<div>Cheers,</div>
<div>Lang.</div>
<div><br>
<div id="gmail-m_-5612660878291636876AppleMailSignature" dir="ltr">Sent from my iPad</div>
<div dir="ltr"><br>
On Mar 12, 2019, at 2:27 PM, preejackie <<a href="mailto:praveenvelliengiri@gmail.com" target="_blank">praveenvelliengiri@gmail.com</a>>
wrote:<br>
<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<p><font face="Courier New, Courier, monospace">Hi Lang, <br>
</font></p>
<p><font face="Courier New, Courier, monospace">Thank you
very much for your reply. <br>
</font></p>
<p><font face="Courier New, Courier, monospace">Yeah, I
think it would be nice to refer to some existing
literature on the subject. I also started a thread in
Numba JIT project (which uses llvm) to see whether
they have anything implemented up-front. As far as I
know, many literature's speak about speculative
optimization for Ahead of time compilation (please
correct me if I'm wrong). <br>
</font></p>
<p><font face="Courier New, Courier, monospace">Of course,
I wish to pursue this and I will write this in the dev
list as soon as possible with some basic idea + plan
and seek for mentors.</font></p>
<p><font face="Courier New, Courier, monospace">It would
be a great help if you point me to some references
that you found interesting & relevant to
speculative compilation. <br>
</font></p>
<p><font face="Courier New, Courier, monospace">Thanks<br>
</font></p>
<div class="gmail-m_-5612660878291636876moz-cite-prefix"> On 13/03/19 12:11 AM, Lang
Hames wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi Preejackie,
<div><br>
</div>
<div>I would like to help, but am concerned that my
limited time and lack of prior experience with GSoC
will make me a less than ideal mentor. I would
encourage you to search for a mentor on list, but I
will try to answer as many of your questions as I
can.<br>
</div>
<div><br>
</div>
<div>Regarding speculative compilation: This has
definitely been done before, and there should be
existing literature on making good speculation
decisions. While that is worth looking at, I would
be inclined to start out by building a framework for
testing the ideas, starting with very basic
speculation decisions (e.g. only compiling
unconditionally called code) and then go from there.
That gives you a basis for comparing your
techniques. The part of the exercise that is of the
most immediate interest is coming up with an API to
fit speculation into the existing ORC
infrastructure. This is something I have basic ideas
about, but nothing detailed yet.</div>
<div><br>
</div>
<div>If you do wish to pursue this we should aim to
have a conversation on the LLVM dev list about how
to design it. That way the whole community will be
able to follow along, and it will serve as useful
documentation.</div>
<div><br>
</div>
<div>Cheers,</div>
<div>Lang.</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, Mar 12, 2019
at 10:41 AM preejackie <<a href="mailto:praveenvelliengiri@gmail.com" target="_blank">praveenvelliengiri@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>Hi Lang, <br>
</p>
<p>I'm following up on this mail. So that you
don't lost it in your schedule :) I have
started working on finding heuristics for
Speculative compilation support. <br>
</p>
<p>Are you available this summer to mentor this
project? Or I can take it to the list, to find
anyone interested. <br>
</p>
<p>Thank you</p>
<div class="gmail-m_-5612660878291636876gmail-m_104724167917977445gmail-m_5742397506994776812moz-cite-prefix"> <br>
</div>
<div class="gmail-m_-5612660878291636876gmail-m_104724167917977445gmail-m_5742397506994776812moz-cite-prefix">On
08/03/19 9:31 PM, preejackie wrote:<br>
</div>
<blockquote type="cite">
<p><tt>Dear Lang,</tt></p>
<p><tt>I forgot to ask you, whether you are
available this summer to mentor this
project? <br>
</tt></p>
<p><tt>Thanks<br>
</tt></p>
<p><tt><br>
</tt></p>
<div class="gmail-m_-5612660878291636876gmail-m_104724167917977445gmail-m_5742397506994776812moz-cite-prefix">On
08/03/19 1:59 AM, preejackie wrote:<br>
</div>
<blockquote type="cite">
<p><tt>Hi Lang, <br>
</tt></p>
<p><tt>Thank you very much for your reply.</tt></p>
<p><tt>Both better caching support and
speculative compilation support ideas are
very interesting and hope that they have a
very good scope within llvm community. I
will bit more investigate both the project
ideas in detail and choose one to write
the proposal :). I didn't gone through the
technical complexities in detail, so I
want to ask whether these projects can be
completed within 3 months with a realistic
timeline? <br>
</tt></p>
<p><tt>Once we get to concrete idea on design
and implementation, I would like to RFC on
the llvm-dev listing and I wish to do that
as soon as possible. I googled for docs on
llvm jit, but I can only able to find few
articles. Could you please point me to
some good (beginner friendly) articles :)
<br>
</tt></p>
<p><tt>In mean time, I will also checkout if
any compilers did something similar to
this. While working on this, If I get any
idea i will tell you.<br>
</tt></p>
<p><tt>And sorry for the delayed reply, I was
engaged in my university classes.<br>
</tt></p>
<p><tt><br>
</tt></p>
<div class="gmail-m_-5612660878291636876gmail-m_104724167917977445gmail-m_5742397506994776812moz-cite-prefix">On
07/03/19 2:53 AM, Lang Hames wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">Hi Praveen,
<div><br>
</div>
<div>Sorry for the delayed reply.</div>
<div><br>
</div>
<div>You're the first person to ask
about JIT specific projects in a
while. I will have a think about
it and see what I can come up
with.</div>
<div><br>
</div>
<div>Off the top of my head, I can
tell you a few of the things that
are on my to-do and wish lists.
Let me know if any of these
interest you:</div>
<div><br>
</div>
<div>TODO list:</div>
<div>(1) A C API for the new,
concurrent ORC (ORCv2) APIs. This
one might be a bit simple for
GSoC, but it's one that LLVM users
would appreciate, and is a good
high-level introduction to the new
APIs.</div>
<div><br>
</div>
<div>(2) Update Kaleidoscope
tutorials, including the "Building
A JIT" series, to use concurrent
ORC.<br>
</div>
<div><br>
</div>
<div>(3) Better caching support. ORC
can re-use relocatable object
files from previous compilations,
so if a JIT input hasn't changed
we can just re-use the old object
file (assuming we saved it
somewhere). LLVM already has a
basic form of support for this in
llvm::ObjectCache (<a href="http://llvm.org/doxygen/classllvm_1_1ObjectCache.html" target="_blank">http://llvm.org/doxygen/classllvm_1_1ObjectCache.html</a>)
but it would be nice to develop a
more general scheme, rather than
being tied to LLVM IR input. For
example, it would be nice if, when
JITing Swift, if we could
recognize Swift code (in the form
of text, ASTs, or SIL) that we've
already compiled, bypass LLVM
altogether and go straight to
the object file.</div>
<div><br>
</div>
<div>(4) Speculative compilation
support. One of the selling points
of the concurrent ORC APIs is that
you can start compiling a function
before you need it, in the hope
that by the time that you do need
it it is already compiled.
However, if we just speculatively
compile everything we're quickly
going to overload our memory/CPU
resources, which will be lousy for
performance. What we really want
to do is to take into account any
information we have (from high
level program representations,
e.g. ASTs or CFGs, or from
profiling data from previous
executions) to try to guess what
functions are worth speculatively
compiling next. </div>
<div><br>
</div>
<div>Wish list:</div>
<div>
<div>(1) JITLink support for ELF
and/or COFF. JITLink is a
super-new (it's still in review:
<a href="https://reviews.llvm.org/D58704" target="_blank">https://reviews.llvm.org/D58704</a>)
replacement for RuntimeDyld, the
JIT linker that currently
supports MCJIT and ORC. JITLink
aims to provide a cleaner API
that integrates better with the
concurrent ORC APIs,
dead-symbol-stripping support,
and small code model support.
I'm working on MachO support,
but would love a hand with ELF
and COFF support.</div>
</div>
<div><br>
</div>
<div>I'm sure more ideas will come
to me over time. :)</div>
<div><br>
</div>
<div>Do any of those projects
interest you?</div>
<div>Is there any aspect of the JIT
that you are particularly keen to
work on, or to learn more about?</div>
<div><br>
</div>
<div>Kind Regards,</div>
<div>Lang.</div>
<div><br>
</div>
</div>
</div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue,
Mar 5, 2019 at 12:48 PM preejackie <<a href="mailto:praveenvelliengiri@gmail.com" target="_blank">praveenvelliengiri@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">Hi
Lang,<br>
<br>
I'm Praveen Velliengiri masters student
from India. I would like to <br>
participate in GSoC19 with llvm. I'm
very much interested in <br>
contributing to the JIT infrastructure
of llvm. As a first step, I have <br>
gone through your talk titled "ORC- LLVM
next gen JIT API" and some docs <br>
on JIT infrastructure. I have followed
"Kaleidoscope , writing a llvm <br>
pass" tutorials and currently following
"Implementing a JIT in LLVM" to <br>
make myself familiar with LLVM JIT
Infrastructure.<br>
<br>
I find it really interesting and hope
that I can learn more about JIT <br>
compilation and LLVM in general. But
unfortunately I can't find any <br>
project ideas that is related to LLVM
JIT infrastructure in GSoC ideas <br>
list. I'm relatively new to LLVM, I find
it hard to frame a project idea <br>
myself. I would like to know is there
any improvements that can be made <br>
to LLVM JIT ? If so, could please point
me in right direction or bugs so <br>
that I can able to find if there is any
opportunities to improve JIT <br>
infrastructure. It will greatly help me
to propose that project idea for <br>
this GSoC 2019.<br>
<br>
I would highly appreciate your thoughts
on this!<br>
Thanks a lot for your time :)<br>
<br>
-- <br>
Have a great day!<br>
PreeJackie<br>
<br>
</blockquote>
</div>
</blockquote>
<pre class="gmail-m_-5612660878291636876gmail-m_104724167917977445gmail-m_5742397506994776812moz-signature" cols="72">--
Have a great day!
PreeJackie
</pre>
</blockquote>
<pre class="gmail-m_-5612660878291636876gmail-m_104724167917977445gmail-m_5742397506994776812moz-signature" cols="72">--
Have a great day!
PreeJackie
</pre>
</blockquote>
<pre class="gmail-m_-5612660878291636876gmail-m_104724167917977445gmail-m_5742397506994776812moz-signature" cols="72">--
Have a great day!
PreeJackie</pre>
</div>
</blockquote>
</div>
</blockquote>
<pre class="gmail-m_-5612660878291636876moz-signature" cols="72">--
Have a great day!
PreeJackie</pre>
</div>
</blockquote>
</div>
</blockquote>
<pre class="gmail-m_-5612660878291636876moz-signature" cols="72">--
Have a great day!
PreeJackie</pre>
</blockquote>
<pre class="gmail-m_-5612660878291636876moz-signature" cols="72">--
Have a great day!
PreeJackie</pre>
</div></blockquote></div></div></div></div></div>