<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">As I just noted in the review: I wonder about the motivation for this, if we find that cl::opts are not just used as debug flags for users, then we really should rather find ways to expose proper APIs through things like TargetOptions.h or function/module attributes. It would certainly help the discussion if you could describe what motivated you to do the patch in the first place.<div class=""><br class=""></div><div class="">We also have a system for options in LLVMContext (see <a href="http://llvm.org/219854" class="">http://llvm.org/219854</a>) that unfortunately was only ever used for a single options and was not followed through to be used for all the other options we have…</div><div class=""><br class=""></div><div class="">- Matthias</div><div class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Oct 20, 2018, at 10:09 AM, David Blaikie via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br class=""><br class=""><div class="gmail_quote"><div dir="ltr" class="">On Fri, Oct 19, 2018 at 11:15 AM Fedor Sergeev via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><br class=""><br class="">On 10/19/2018 07:45 PM, David Blaikie via llvm-dev wrote:<br class="">> +Lang Hames <mailto:<a href="mailto:lhames@gmail.com" target="_blank" class="">lhames@gmail.com</a>> since he's playing with<span class="Apple-converted-space"> </span><br class="">> multithreaded compilation in the ORC JIT too.<br class="">One nit about terminology - there are two different flavors of<span class="Apple-converted-space"> </span><br class="">"multithreaded compilation".<br class="">Some people read it as "doing parallel processing of a single<span class="Apple-converted-space"> </span><br class="">compilation job" and some as<br class="">"doing parallel independent compilation jobs".<br class=""><br class="">Azul's Falcon JIT compiler does the latter.<br class=""><br class="">><br class="">> My off-the-cuff thought (which is a lot of work, I realize) would be<span class="Apple-converted-space"> </span><br class="">> that cl::opts that aren't either in drivers (like opt.cpp, llc.cpp,<span class="Apple-converted-space"> </span><br class="">> etc) or developer options (dump-after-all, things like that) shouldn't<span class="Apple-converted-space"> </span><br class="">> be cl::opts and should be migrated to options structs and the like?<br class="">+1<br class="">It would be great to have a direct API accessing/setting up these<span class="Apple-converted-space"> </span><br class="">"option structs" for in-process JIT clients<br class="">that start many different compilations.<br class="">Having to parse option strings has always striked me as something rather<span class="Apple-converted-space"> </span><br class="">clumsy.<br class=""><br class="">On other hand, ability to replay compilation with standalone opt and<span class="Apple-converted-space"> </span><br class="">still have the same controls over functionality of optimizer<br class="">happens to be a great time saver. Thus having a way to control these<span class="Apple-converted-space"> </span><br class="">non-cl::opt things from opt's command-line is also<br class="">a good thing to have.<br class=""></blockquote><div class=""><br class="">Oh, sure - that's true of lots of config options passed through structs today & I believe would/should continue to be true as these values are migrated. That's necessary for testing those configuration options from within LLVM lit tests as we usually do.<br class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><br class="">(something along the line of a difference between legacy PM's<span class="Apple-converted-space"> </span><br class="">command-line pass interface - where every pass presents itself as an option,<br class="">and new PM's -passes= single option).<br class=""><br class="">regards,<br class=""> <span class="Apple-converted-space"> </span>Fedor.<br class=""><br class="">> I realize that's a ton of work, and we all sort of cringe a little<span class="Apple-converted-space"> </span><br class="">> when we add another "backend option" (accessing cl::opts via<span class="Apple-converted-space"> </span><br class="">> -backend-option in the Clang driver when invoking clang cc1) & then do<span class="Apple-converted-space"> </span><br class="">> it anyway, etc... but would be pretty great to clean it up and have a<span class="Apple-converted-space"> </span><br class="">> clear line about what cl::opts are for.<br class="">><br class="">> (totally reasonable for you to push back and say "that's not the hill<span class="Apple-converted-space"> </span><br class="">> I want to die on today", etc - and see what everyone else thinks)<br class="">><br class="">> - Dave<br class="">><br class="">> On Fri, Oct 19, 2018 at 3:58 AM Yevgeny Rouban via llvm-dev<span class="Apple-converted-space"> </span><br class="">> <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a><span class="Apple-converted-space"> </span><mailto:<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>>> wrote:<br class="">><br class="">> Hello LLVM Developers.<br class="">><br class="">> We at Azul Systems are working on a multi-threaded LLVM based<br class="">> compiler. It can run several compilations each of which compiles<br class="">> its own module in its own thread.<br class="">><br class="">> One of the limitation we face is that all threads use the same<br class="">> options (instances of cl::opt). In other words, the options are<br class="">> global and cannot be changed for one thread and left unchanged for<br class="">> the others.<br class="">><br class="">> One solution I propose in the patch<br class="">><br class="">> <a href="https://reviews.llvm.org/D53424" rel="noreferrer" target="_blank" class="">https://reviews.llvm.org/D53424</a><span class="Apple-converted-space"> </span>Enable thread specific cl::opt<br class="">> values for multi-threaded support<br class="">><br class="">> As the change affects many source files (though slightly) I<br class="">> decided to share it with wider audience. Any less intrusive<br class="">> solution is welcome.<br class="">><br class="">> Here is the patch description for your convenience:<br class="">><br class="">> ===<br class="">><br class="">> When several threads compile different modules the compiler<br class="">> options (instances of cl::opt) cannot be set individually for each<br class="">> thread. That is because the options are visible to all threads. In<br class="">> other words all options are global.<br class="">><br class="">> It would be convenient if the options were specific to LLVMContext<br class="">> and they were accessed through an instance of LLVMContext. This<br class="">> kind of change would need changes in all source files where<br class="">> options are used.<br class="">><br class="">> This patch proposes a solution that needs minimal changes in LLVM<br class="">> source base.<br class="">><br class="">> It is proposed to have a thread local set of re-defined option<br class="">> values mapped by pointers to options.<br class="">><br class="">> Specifically, every time a program gets/sets a value for an option<br class="">> it is checked if the current thread local context is set for the<br class="">> current thread and the option has its local copy in this context.<br class="">> If so the local copy of the option is accessed, otherwise the<br class="">> global option is accessed. For all programs that existed so far<br class="">> the context is not set and they work with the global options. For<br class="">> new multi-threaded compilers (where every thread compiles its own<br class="">> module) every thread can be linked to its own context (see<br class="">> ContextValues) where any option can have its thread specific value<br class="">> that do not affect the other threads' option values. See the<br class="">> thread_routine() in the test ContextSpecificValues2.<br class="">><br class="">> This feature allows a configuration flexibility for multi-threaded<br class="">> compilers that can compile every compilation unit in its own<br class="">> thread with different command line options.<br class="">><br class="">> ===<br class="">><br class="">> Thanks.<br class="">><br class="">> -Yevgeny Rouban<br class="">><br class="">> _______________________________________________<br class="">> LLVM Developers mailing list<br class="">> <a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a><span class="Apple-converted-space"> </span><mailto:<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>><br class="">> <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br class="">><br class="">><br class="">><br class="">> _______________________________________________<br class="">> LLVM Developers mailing list<br class="">><span class="Apple-converted-space"> </span><a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a><br class="">><span class="Apple-converted-space"> </span><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br class=""><br class="">_______________________________________________<br class="">LLVM Developers mailing list<br class=""><a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a><br class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br class=""></blockquote></div></div><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">_______________________________________________</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">LLVM Developers mailing list</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><a href="mailto:llvm-dev@lists.llvm.org" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">llvm-dev@lists.llvm.org</a><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a></div></blockquote></div><br class=""></div></body></html>