<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
> I wonder about the motivation for this<br>
<br>
Prime motivation for our JIT is a desire to have a tight control on
behavior of individual compilation<br>
while leaving other compilations work in "default" mode.<br>
<br>
One major use is indeed debugging, say, opt-bisect for a specific
method compilation,<br>
or print-before/after for a specific method/pass.<br>
<br>
However, we also have other ideas on how to use this - say, we would
like to have different "optimization levels",<br>
and for that we need to tweak various defaults - thresholds or
off/on switches - that select exact amount of<br>
work for optimizer. In some cases we can handle that through
configuring a pass during its construction time,<br>
but for most cases there are no such controls.<br>
<br>
Similarly we would use command line optimizer tweaks in order to
guide performance analysis, but that is kinda<br>
on border with debugging.<br>
<br>
regards,<br>
Fedor.<br>
<br>
On 10/22/2018 12:56 AM, mbraun wrote:<br>
<blockquote type="cite"
cite="mid:9B0DD165-5021-4955-8EBA-7EC9008055A5@apple.com">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
As I just noted in the review: I wonder about the motivation for
this, if we find that cl::opts are not just used as debug flags
for users, then we really should rather find ways to expose proper
APIs through things like TargetOptions.h or function/module
attributes. It would certainly help the discussion if you could
describe what motivated you to do the patch in the first place.
<div class=""><br class="">
</div>
<div class="">We also have a system for options in LLVMContext
(see <a href="http://llvm.org/219854" class=""
moz-do-not-send="true">http://llvm.org/219854</a>) that
unfortunately was only ever used for a single options and was
not followed through to be used for all the other options we
have…</div>
<div class=""><br class="">
</div>
<div class="">- Matthias</div>
<div class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On Oct 20, 2018, at 10:09 AM, David Blaikie
via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org"
class="" moz-do-not-send="true">llvm-dev@lists.llvm.org</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" style="caret-color: rgb(0, 0, 0);
font-family: Helvetica; font-size: 12px; font-style:
normal; font-variant-caps: normal; font-weight: normal;
letter-spacing: normal; text-align: start; text-indent:
0px; text-transform: none; white-space: normal;
word-spacing: 0px; -webkit-text-stroke-width: 0px;
text-decoration: none;" class=""><br class="">
<br class="">
<div class="gmail_quote">
<div dir="ltr" class="">On Fri, Oct 19, 2018 at 11:15
AM Fedor Sergeev via llvm-dev <<a
href="mailto:llvm-dev@lists.llvm.org" class=""
moz-do-not-send="true">llvm-dev@lists.llvm.org</a>>
wrote:<br class="">
</div>
<blockquote class="gmail_quote" style="margin: 0px 0px
0px 0.8ex; border-left-width: 1px;
border-left-style: solid; border-left-color:
rgb(204, 204, 204); padding-left: 1ex;"><br class="">
<br class="">
On 10/19/2018 07:45 PM, David Blaikie via llvm-dev
wrote:<br class="">
> +Lang Hames <mailto:<a
href="mailto:lhames@gmail.com" target="_blank"
class="" moz-do-not-send="true">lhames@gmail.com</a>>
since he's playing with<span
class="Apple-converted-space"> </span><br class="">
> multithreaded compilation in the ORC JIT too.<br
class="">
One nit about terminology - there are two different
flavors of<span class="Apple-converted-space"> </span><br
class="">
"multithreaded compilation".<br class="">
Some people read it as "doing parallel processing of
a single<span class="Apple-converted-space"> </span><br
class="">
compilation job" and some as<br class="">
"doing parallel independent compilation jobs".<br
class="">
<br class="">
Azul's Falcon JIT compiler does the latter.<br
class="">
<br class="">
><br class="">
> My off-the-cuff thought (which is a lot of
work, I realize) would be<span
class="Apple-converted-space"> </span><br class="">
> that cl::opts that aren't either in drivers
(like opt.cpp, llc.cpp,<span
class="Apple-converted-space"> </span><br class="">
> etc) or developer options (dump-after-all,
things like that) shouldn't<span
class="Apple-converted-space"> </span><br class="">
> be cl::opts and should be migrated to options
structs and the like?<br class="">
+1<br class="">
It would be great to have a direct API
accessing/setting up these<span
class="Apple-converted-space"> </span><br class="">
"option structs" for in-process JIT clients<br
class="">
that start many different compilations.<br class="">
Having to parse option strings has always striked me
as something rather<span
class="Apple-converted-space"> </span><br class="">
clumsy.<br class="">
<br class="">
On other hand, ability to replay compilation with
standalone opt and<span
class="Apple-converted-space"> </span><br class="">
still have the same controls over functionality of
optimizer<br class="">
happens to be a great time saver. Thus having a way
to control these<span class="Apple-converted-space"> </span><br
class="">
non-cl::opt things from opt's command-line is also<br
class="">
a good thing to have.<br class="">
</blockquote>
<div class=""><br class="">
Oh, sure - that's true of lots of config options
passed through structs today & I believe
would/should continue to be true as these values are
migrated. That's necessary for testing those
configuration options from within LLVM lit tests as
we usually do.<br class="">
</div>
<blockquote class="gmail_quote" style="margin: 0px 0px
0px 0.8ex; border-left-width: 1px;
border-left-style: solid; border-left-color:
rgb(204, 204, 204); padding-left: 1ex;"><br class="">
(something along the line of a difference between
legacy PM's<span class="Apple-converted-space"> </span><br
class="">
command-line pass interface - where every pass
presents itself as an option,<br class="">
and new PM's -passes= single option).<br class="">
<br class="">
regards,<br class="">
<span class="Apple-converted-space"> </span>Fedor.<br
class="">
<br class="">
> I realize that's a ton of work, and we all sort
of cringe a little<span
class="Apple-converted-space"> </span><br class="">
> when we add another "backend option" (accessing
cl::opts via<span class="Apple-converted-space"> </span><br
class="">
> -backend-option in the Clang driver when
invoking clang cc1) & then do<span
class="Apple-converted-space"> </span><br class="">
> it anyway, etc... but would be pretty great to
clean it up and have a<span
class="Apple-converted-space"> </span><br class="">
> clear line about what cl::opts are for.<br
class="">
><br class="">
> (totally reasonable for you to push back and
say "that's not the hill<span
class="Apple-converted-space"> </span><br class="">
> I want to die on today", etc - and see what
everyone else thinks)<br class="">
><br class="">
> - Dave<br class="">
><br class="">
> On Fri, Oct 19, 2018 at 3:58 AM Yevgeny Rouban
via llvm-dev<span class="Apple-converted-space"> </span><br
class="">
> <<a href="mailto:llvm-dev@lists.llvm.org"
target="_blank" class="" moz-do-not-send="true">llvm-dev@lists.llvm.org</a><span
class="Apple-converted-space"> </span><mailto:<a
href="mailto:llvm-dev@lists.llvm.org"
target="_blank" class="" moz-do-not-send="true">llvm-dev@lists.llvm.org</a>>>
wrote:<br class="">
><br class="">
> Hello LLVM Developers.<br class="">
><br class="">
> We at Azul Systems are working on a
multi-threaded LLVM based<br class="">
> compiler. It can run several compilations
each of which compiles<br class="">
> its own module in its own thread.<br
class="">
><br class="">
> One of the limitation we face is that all
threads use the same<br class="">
> options (instances of cl::opt). In other
words, the options are<br class="">
> global and cannot be changed for one thread
and left unchanged for<br class="">
> the others.<br class="">
><br class="">
> One solution I propose in the patch<br
class="">
><br class="">
> <a href="https://reviews.llvm.org/D53424"
rel="noreferrer" target="_blank" class=""
moz-do-not-send="true">https://reviews.llvm.org/D53424</a><span
class="Apple-converted-space"> </span>Enable
thread specific cl::opt<br class="">
> values for multi-threaded support<br
class="">
><br class="">
> As the change affects many source files
(though slightly) I<br class="">
> decided to share it with wider audience.
Any less intrusive<br class="">
> solution is welcome.<br class="">
><br class="">
> Here is the patch description for your
convenience:<br class="">
><br class="">
> ===<br class="">
><br class="">
> When several threads compile different
modules the compiler<br class="">
> options (instances of cl::opt) cannot be
set individually for each<br class="">
> thread. That is because the options are
visible to all threads. In<br class="">
> other words all options are global.<br
class="">
><br class="">
> It would be convenient if the options were
specific to LLVMContext<br class="">
> and they were accessed through an instance
of LLVMContext. This<br class="">
> kind of change would need changes in all
source files where<br class="">
> options are used.<br class="">
><br class="">
> This patch proposes a solution that needs
minimal changes in LLVM<br class="">
> source base.<br class="">
><br class="">
> It is proposed to have a thread local set
of re-defined option<br class="">
> values mapped by pointers to options.<br
class="">
><br class="">
> Specifically, every time a program
gets/sets a value for an option<br class="">
> it is checked if the current thread local
context is set for the<br class="">
> current thread and the option has its local
copy in this context.<br class="">
> If so the local copy of the option is
accessed, otherwise the<br class="">
> global option is accessed. For all programs
that existed so far<br class="">
> the context is not set and they work with
the global options. For<br class="">
> new multi-threaded compilers (where every
thread compiles its own<br class="">
> module) every thread can be linked to its
own context (see<br class="">
> ContextValues) where any option can have
its thread specific value<br class="">
> that do not affect the other threads'
option values. See the<br class="">
> thread_routine() in the test
ContextSpecificValues2.<br class="">
><br class="">
> This feature allows a configuration
flexibility for multi-threaded<br class="">
> compilers that can compile every
compilation unit in its own<br class="">
> thread with different command line options.<br
class="">
><br class="">
> ===<br class="">
><br class="">
> Thanks.<br class="">
><br class="">
> -Yevgeny Rouban<br class="">
><br class="">
>
_______________________________________________<br
class="">
> LLVM Developers mailing list<br class="">
> <a href="mailto:llvm-dev@lists.llvm.org"
target="_blank" class="" moz-do-not-send="true">llvm-dev@lists.llvm.org</a><span
class="Apple-converted-space"> </span><mailto:<a
href="mailto:llvm-dev@lists.llvm.org"
target="_blank" class="" moz-do-not-send="true">llvm-dev@lists.llvm.org</a>><br
class="">
> <a
href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev"
rel="noreferrer" target="_blank" class=""
moz-do-not-send="true">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br
class="">
><br class="">
><br class="">
><br class="">
> _______________________________________________<br
class="">
> LLVM Developers mailing list<br class="">
><span class="Apple-converted-space"> </span><a
href="mailto:llvm-dev@lists.llvm.org"
target="_blank" class="" moz-do-not-send="true">llvm-dev@lists.llvm.org</a><br
class="">
><span class="Apple-converted-space"> </span><a
href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev"
rel="noreferrer" target="_blank" class=""
moz-do-not-send="true">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br
class="">
<br class="">
_______________________________________________<br
class="">
LLVM Developers mailing list<br class="">
<a href="mailto:llvm-dev@lists.llvm.org"
target="_blank" class="" moz-do-not-send="true">llvm-dev@lists.llvm.org</a><br
class="">
<a
href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev"
rel="noreferrer" target="_blank" class=""
moz-do-not-send="true">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br
class="">
</blockquote>
</div>
</div>
<span style="caret-color: rgb(0, 0, 0); font-family:
Helvetica; font-size: 12px; font-style: normal;
font-variant-caps: normal; font-weight: normal;
letter-spacing: normal; text-align: start; text-indent:
0px; text-transform: none; white-space: normal;
word-spacing: 0px; -webkit-text-stroke-width: 0px;
text-decoration: none; float: none; display: inline
!important;" class="">_______________________________________________</span><br
style="caret-color: rgb(0, 0, 0); font-family:
Helvetica; font-size: 12px; font-style: normal;
font-variant-caps: normal; font-weight: normal;
letter-spacing: normal; text-align: start; text-indent:
0px; text-transform: none; white-space: normal;
word-spacing: 0px; -webkit-text-stroke-width: 0px;
text-decoration: none;" class="">
<span style="caret-color: rgb(0, 0, 0); font-family:
Helvetica; font-size: 12px; font-style: normal;
font-variant-caps: normal; font-weight: normal;
letter-spacing: normal; text-align: start; text-indent:
0px; text-transform: none; white-space: normal;
word-spacing: 0px; -webkit-text-stroke-width: 0px;
text-decoration: none; float: none; display: inline
!important;" class="">LLVM Developers mailing list</span><br
style="caret-color: rgb(0, 0, 0); font-family:
Helvetica; font-size: 12px; font-style: normal;
font-variant-caps: normal; font-weight: normal;
letter-spacing: normal; text-align: start; text-indent:
0px; text-transform: none; white-space: normal;
word-spacing: 0px; -webkit-text-stroke-width: 0px;
text-decoration: none;" class="">
<a href="mailto:llvm-dev@lists.llvm.org"
style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-size-adjust: auto;
-webkit-text-stroke-width: 0px;" class=""
moz-do-not-send="true">llvm-dev@lists.llvm.org</a><br
style="caret-color: rgb(0, 0, 0); font-family:
Helvetica; font-size: 12px; font-style: normal;
font-variant-caps: normal; font-weight: normal;
letter-spacing: normal; text-align: start; text-indent:
0px; text-transform: none; white-space: normal;
word-spacing: 0px; -webkit-text-stroke-width: 0px;
text-decoration: none;" class="">
<a
href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev"
style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-size-adjust: auto;
-webkit-text-stroke-width: 0px;" class=""
moz-do-not-send="true">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a></div>
</blockquote>
</div>
<br class="">
</div>
</blockquote>
<br>
</body>
</html>