<div dir="ltr">As another data point: I've been working on a multi-threaded compiler using LLVM in the past and I have patched passes to accommodate this, for instance by taking their parameters in a constructor, and only defaulting to the cl::opt. <div>The consensus at the time was that the cl::opt were only here for debugging / overriding default, but should not be the way any user of "LLVM as a library" set the options. <br><div><br></div><div>Best,</div><div><br></div><div>-- </div></div><div>Mehdi</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr">Le dim. 21 oct. 2018 à 14:56, mbraun via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>> a écrit :<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word;line-break:after-white-space">As I just noted in the review: I wonder about the motivation for this, if we find that cl::opts are not just used as debug flags for users, then we really should rather find ways to expose proper APIs through things like TargetOptions.h or function/module attributes. It would certainly help the discussion if you could describe what motivated you to do the patch in the first place.<div><br></div><div>We also have a system for options in LLVMContext (see <a href="http://llvm.org/219854" target="_blank">http://llvm.org/219854</a>) that unfortunately was only ever used for a single options and was not followed through to be used for all the other options we have…</div><div><br></div><div>- Matthias</div><div><div><br><blockquote type="cite"><div>On Oct 20, 2018, at 10:09 AM, David Blaikie via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:</div><br class="m_6799933490814106645Apple-interchange-newline"><div><div dir="ltr" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><br><br><div class="gmail_quote"><div dir="ltr">On Fri, Oct 19, 2018 at 11:15 AM Fedor Sergeev via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><br><br>On 10/19/2018 07:45 PM, David Blaikie via llvm-dev wrote:<br>> +Lang Hames <mailto:<a href="mailto:lhames@gmail.com" target="_blank">lhames@gmail.com</a>> since he's playing with<span class="m_6799933490814106645Apple-converted-space"> </span><br>> multithreaded compilation in the ORC JIT too.<br>One nit about terminology - there are two different flavors of<span class="m_6799933490814106645Apple-converted-space"> </span><br>"multithreaded compilation".<br>Some people read it as "doing parallel processing of a single<span class="m_6799933490814106645Apple-converted-space"> </span><br>compilation job" and some as<br>"doing parallel independent compilation jobs".<br><br>Azul's Falcon JIT compiler does the latter.<br><br>><br>> My off-the-cuff thought (which is a lot of work, I realize) would be<span class="m_6799933490814106645Apple-converted-space"> </span><br>> that cl::opts that aren't either in drivers (like opt.cpp, llc.cpp,<span class="m_6799933490814106645Apple-converted-space"> </span><br>> etc) or developer options (dump-after-all, things like that) shouldn't<span class="m_6799933490814106645Apple-converted-space"> </span><br>> be cl::opts and should be migrated to options structs and the like?<br>+1<br>It would be great to have a direct API accessing/setting up these<span class="m_6799933490814106645Apple-converted-space"> </span><br>"option structs" for in-process JIT clients<br>that start many different compilations.<br>Having to parse option strings has always striked me as something rather<span class="m_6799933490814106645Apple-converted-space"> </span><br>clumsy.<br><br>On other hand, ability to replay compilation with standalone opt and<span class="m_6799933490814106645Apple-converted-space"> </span><br>still have the same controls over functionality of optimizer<br>happens to be a great time saver. Thus having a way to control these<span class="m_6799933490814106645Apple-converted-space"> </span><br>non-cl::opt things from opt's command-line is also<br>a good thing to have.<br></blockquote><div><br>Oh, sure - that's true of lots of config options passed through structs today & I believe would/should continue to be true as these values are migrated. That's necessary for testing those configuration options from within LLVM lit tests as we usually do.<br> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><br>(something along the line of a difference between legacy PM's<span class="m_6799933490814106645Apple-converted-space"> </span><br>command-line pass interface - where every pass presents itself as an option,<br>and new PM's -passes= single option).<br><br>regards,<br> <span class="m_6799933490814106645Apple-converted-space"> </span>Fedor.<br><br>> I realize that's a ton of work, and we all sort of cringe a little<span class="m_6799933490814106645Apple-converted-space"> </span><br>> when we add another "backend option" (accessing cl::opts via<span class="m_6799933490814106645Apple-converted-space"> </span><br>> -backend-option in the Clang driver when invoking clang cc1) & then do<span class="m_6799933490814106645Apple-converted-space"> </span><br>> it anyway, etc... but would be pretty great to clean it up and have a<span class="m_6799933490814106645Apple-converted-space"> </span><br>> clear line about what cl::opts are for.<br>><br>> (totally reasonable for you to push back and say "that's not the hill<span class="m_6799933490814106645Apple-converted-space"> </span><br>> I want to die on today", etc - and see what everyone else thinks)<br>><br>> - Dave<br>><br>> On Fri, Oct 19, 2018 at 3:58 AM Yevgeny Rouban via llvm-dev<span class="m_6799933490814106645Apple-converted-space"> </span><br>> <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><span class="m_6799933490814106645Apple-converted-space"> </span><mailto:<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>>> wrote:<br>><br>> Hello LLVM Developers.<br>><br>> We at Azul Systems are working on a multi-threaded LLVM based<br>> compiler. It can run several compilations each of which compiles<br>> its own module in its own thread.<br>><br>> One of the limitation we face is that all threads use the same<br>> options (instances of cl::opt). In other words, the options are<br>> global and cannot be changed for one thread and left unchanged for<br>> the others.<br>><br>> One solution I propose in the patch<br>><br>> <a href="https://reviews.llvm.org/D53424" rel="noreferrer" target="_blank">https://reviews.llvm.org/D53424</a><span class="m_6799933490814106645Apple-converted-space"> </span>Enable thread specific cl::opt<br>> values for multi-threaded support<br>><br>> As the change affects many source files (though slightly) I<br>> decided to share it with wider audience. Any less intrusive<br>> solution is welcome.<br>><br>> Here is the patch description for your convenience:<br>><br>> ===<br>><br>> When several threads compile different modules the compiler<br>> options (instances of cl::opt) cannot be set individually for each<br>> thread. That is because the options are visible to all threads. In<br>> other words all options are global.<br>><br>> It would be convenient if the options were specific to LLVMContext<br>> and they were accessed through an instance of LLVMContext. This<br>> kind of change would need changes in all source files where<br>> options are used.<br>><br>> This patch proposes a solution that needs minimal changes in LLVM<br>> source base.<br>><br>> It is proposed to have a thread local set of re-defined option<br>> values mapped by pointers to options.<br>><br>> Specifically, every time a program gets/sets a value for an option<br>> it is checked if the current thread local context is set for the<br>> current thread and the option has its local copy in this context.<br>> If so the local copy of the option is accessed, otherwise the<br>> global option is accessed. For all programs that existed so far<br>> the context is not set and they work with the global options. For<br>> new multi-threaded compilers (where every thread compiles its own<br>> module) every thread can be linked to its own context (see<br>> ContextValues) where any option can have its thread specific value<br>> that do not affect the other threads' option values. See the<br>> thread_routine() in the test ContextSpecificValues2.<br>><br>> This feature allows a configuration flexibility for multi-threaded<br>> compilers that can compile every compilation unit in its own<br>> thread with different command line options.<br>><br>> ===<br>><br>> Thanks.<br>><br>> -Yevgeny Rouban<br>><br>> _______________________________________________<br>> LLVM Developers mailing list<br>> <a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><span class="m_6799933490814106645Apple-converted-space"> </span><mailto:<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>><br>> <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>><br>><br>><br>> _______________________________________________<br>> LLVM Developers mailing list<br>><span class="m_6799933490814106645Apple-converted-space"> </span><a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>><span class="m_6799933490814106645Apple-converted-space"> </span><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br><br>_______________________________________________<br>LLVM Developers mailing list<br><a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br></blockquote></div></div><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline!important">_______________________________________________</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline!important">LLVM Developers mailing list</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><a href="mailto:llvm-dev@lists.llvm.org" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" target="_blank">llvm-dev@lists.llvm.org</a><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a></div></blockquote></div><br></div></div>_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote></div>