<div dir="ltr"><a class="GWVZpf gW" id="IloFPc-0" href="mailto:lhames@gmail.com" tabindex="-1">+Lang Hames</a> since he's playing with multithreaded compilation in the ORC JIT too.<br><br>My off-the-cuff thought (which is a lot of work, I realize) would be that cl::opts that aren't either in drivers (like opt.cpp, llc.cpp, etc) or developer options (dump-after-all, things like that) shouldn't be cl::opts and should be migrated to options structs and the like?<br><br>I realize that's a ton of work, and we all sort of cringe a little when we add another "backend option" (accessing cl::opts via -backend-option in the Clang driver when invoking clang cc1) & then do it anyway, etc... but would be pretty great to clean it up and have a clear line about what cl::opts are for.<br><br>(totally reasonable for you to push back and say "that's not the hill I want to die on today", etc - and see what everyone else thinks)<br><br>- Dave<br><br><div class="gmail_quote"><div dir="ltr">On Fri, Oct 19, 2018 at 3:58 AM Yevgeny Rouban via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>Hello LLVM Developers.<span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span><span> </span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>We at Azul Systems are
working on a multi-threaded LLVM based compiler. It can run several
compilations each of which compiles its own module in its own thread.<span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>One of the limitation we
face is that all threads use the same options (instances of cl::opt). In other
words, the options are global and cannot be changed for one thread and left
unchanged for the others.<span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span><span> </span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>One solution I propose in
the patch<span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span><a href="https://reviews.llvm.org/D53424" style="color:rgb(5,99,193);text-decoration:underline" target="_blank">https://reviews.llvm.org/D53424</a>
Enable thread specific cl::opt values for multi-threaded support<span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span><span> </span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>As the change affects many
source files (though slightly) I decided to share it with wider audience. Any
less intrusive solution is welcome.<span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span><span> </span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>Here is the patch
description for your convenience:<span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>===<span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>When several threads compile
different modules the compiler options (instances of cl::opt) cannot be set
individually for each thread. That is because the options are visible to all
threads. In other words all options are global.<span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span><span> </span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>It would be convenient if
the options were specific to LLVMContext and they were accessed through an
instance of LLVMContext. This kind of change would need changes in all source
files where options are used.<span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span><span> </span></span></p>
<p class="MsoNormal" style="margin:0in 0in 12pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>This patch proposes a solution that needs minimal changes in LLVM source
base.<span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>It is proposed to have a
thread local set of re-defined option values mapped by pointers to options.<span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>Specifically, every time a
program gets/sets a value for an option it is checked if the current thread
local context is set for the current thread and the option has its local copy
in this context. If so the local copy of the option is accessed, otherwise the
global option is accessed. For all programs that existed so far the context is
not set and they work with the global options. For new multi-threaded compilers
(where every thread compiles its own module) every thread can be linked to its
own context (see ContextValues) where any option can have its thread specific
value that do not affect the other threads' option values. See the
thread_routine() in the test ContextSpecificValues2.<span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span><span> </span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>This feature allows a
configuration flexibility for multi-threaded compilers that can compile every
compilation unit in its own thread with different command line options.<span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>===<span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span><span> </span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif">Thanks.<span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif">-Yevgeny Rouban<span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span> </span></p>
</div>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote></div></div>