<div dir="ltr">Thanks for giving actual numbers. This does seem like a bad regression.<div><br></div><div>I don't really have a way to reproduce the same degree of slow down you're seeing, but I have what I think might be a "fix". It's pretty gross though. I'll mail it out. I'd like to make sure it actually addresses the performance problems you're seeing though. If it doesn't, things look more grim. =/ The only way it will effectively address them is if you have a very long lived set of tope level pass manager objects.</div><div><br></div><div>Anyways, I've mailed the patch here: <a href="http://reviews.llvm.org/D7213">http://reviews.llvm.org/D7213</a></div><div><br></div><div>Let me know if this seems like it might provide some resolution here.</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jan 27, 2015 at 2:08 PM, Owen Anderson <span dir="ltr"><<a href="mailto:resistor@mac.com" target="_blank">resistor@mac.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word">So we’re all on the same page, this is a release build with assertions disabled on Darwin x86_64.<div><br></div><div>—Owen</div><div><br><div><blockquote type="cite"><span class=""><div>On Jan 27, 2015, at 11:59 AM, Owen Anderson <<a href="mailto:resistor@mac.com" target="_blank">resistor@mac.com</a>> wrote:</div><br></span><div><div style="word-wrap:break-word"><span class="">To illustrate, the top of a profile. I’m running a single .ll file through LLC, with -time-compilations 10000. I’ve focused the profiler only on the call tree beneath llvm::legacy::PassManagerImpl::run(), and charged a few system library functions to their callers in order to make it more readable.<div><br></div></span><div><span><ThreadProfile.png></span></div><div><div class="h5"><div><br></div><div>Notice that we’re spending almost 13% of our time in reader/writer locks plus memory fences. I assume the remaining few percent of regression come from optimizations inhibited by the presence of the locks.</div><div><br></div><div>—Owen</div><div><br></div><div><br><div><br><div><blockquote type="cite"><div>On Jan 27, 2015, at 11:41 AM, Sean Silva <<a href="mailto:chisophugis@gmail.com" target="_blank">chisophugis@gmail.com</a>> wrote:</div><br><div><br><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><div class="gmail_quote" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">On Tue, Jan 27, 2015 at 7:13 PM, Owen Anderson<span> </span><span dir="ltr"><<a href="mailto:resistor@mac.com" target="_blank">resistor@mac.com</a>></span><span> </span>wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Hi Chandler,<br><br>I hate to revive an ancient thread, but our recent performance work has turned this up as a *massive* performance regression for compilation when the units of compilation are small. I’m able to consistently measure slowdowns just within PassManager::run() on the order of 15-20%, with read-writer locks at the top of the profile. Building without threading enabled is not an option, because some of our clients use LLVM in a multithreaded manner, while others do not.<br></blockquote><div><br></div><div>Maybe I'm just out of the loop, but do you know what locks PassManager::run() would end up running up against? Off the top of my head I can't think of any.</div><div><br></div><div>-- Sean Silva</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><span><font color="#888888"><br>—Owen<br></font></span><div><div><br><br>> On Jun 27, 2014, at 8:13 AM, Chandler Carruth <<a href="mailto:chandlerc@gmail.com" target="_blank">chandlerc@gmail.com</a>> wrote:<br>><br>> Author: chandlerc<br>> Date: Fri Jun 27 10:13:01 2014<br>> New Revision: 211900<br>><br>> URL:<span> </span><a href="http://llvm.org/viewvc/llvm-project?rev=211900&view=rev" target="_blank">http://llvm.org/viewvc/llvm-project?rev=211900&view=rev</a><br>> Log:<br>> Re-apply r211287: Remove support for LLVM runtime multi-threading.<br>><br>> I'll fix the problems in libclang and other projects in ways that don't<br>> require <mutex> until we sort out the cygwin situation.<br>><br>> Modified:<br>> llvm/trunk/docs/ProgrammersManual.rst<br>> llvm/trunk/include/llvm-c/Core.h<br>> llvm/trunk/include/llvm/Support/ManagedStatic.h<br>> llvm/trunk/include/llvm/Support/Threading.h<br>> llvm/trunk/lib/IR/Core.cpp<br>> llvm/trunk/lib/Support/Threading.cpp<br>> llvm/trunk/unittests/Support/ManagedStatic.cpp<br>><br>> Modified: llvm/trunk/docs/ProgrammersManual.rst<br>> URL:<span> </span><a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/ProgrammersManual.rst?rev=211900&r1=211899&r2=211900&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/ProgrammersManual.rst?rev=211900&r1=211899&r2=211900&view=diff</a><br>> ==============================================================================<br>> --- llvm/trunk/docs/ProgrammersManual.rst (original)<br>> +++ llvm/trunk/docs/ProgrammersManual.rst Fri Jun 27 10:13:01 2014<br>> @@ -2170,46 +2170,13 @@ compiler, consider compiling LLVM and LL<br>> using the resultant compiler to build a copy of LLVM with multithreading<br>> support.<br>><br>> -.. _startmultithreaded:<br>> -<br>> -Entering and Exiting Multithreaded Mode<br>> ----------------------------------------<br>> -<br>> -In order to properly protect its internal data structures while avoiding<br>> -excessive locking overhead in the single-threaded case, the LLVM must intialize<br>> -certain data structures necessary to provide guards around its internals. To do<br>> -so, the client program must invoke ``llvm_start_multithreaded()`` before making<br>> -any concurrent LLVM API calls. To subsequently tear down these structures, use<br>> -the ``llvm_stop_multithreaded()`` call. You can also use the<br>> -``llvm_is_multithreaded()`` call to check the status of multithreaded mode.<br>> -<br>> -Note that both of these calls must be made *in isolation*. That is to say that<br>> -no other LLVM API calls may be executing at any time during the execution of<br>> -``llvm_start_multithreaded()`` or ``llvm_stop_multithreaded``. It is the<br>> -client's responsibility to enforce this isolation.<br>> -<br>> -The return value of ``llvm_start_multithreaded()`` indicates the success or<br>> -failure of the initialization. Failure typically indicates that your copy of<br>> -LLVM was built without multithreading support, typically because GCC atomic<br>> -intrinsics were not found in your system compiler. In this case, the LLVM API<br>> -will not be safe for concurrent calls. However, it *will* be safe for hosting<br>> -threaded applications in the JIT, though :ref:`care must be taken<br>> -<jitthreading>` to ensure that side exits and the like do not accidentally<br>> -result in concurrent LLVM API calls.<br>> -<br>> .. _shutdown:<br>><br>> Ending Execution with ``llvm_shutdown()``<br>> -----------------------------------------<br>><br>> When you are done using the LLVM APIs, you should call ``llvm_shutdown()`` to<br>> -deallocate memory used for internal structures. This will also invoke<br>> -``llvm_stop_multithreaded()`` if LLVM is operating in multithreaded mode. As<br>> -such, ``llvm_shutdown()`` requires the same isolation guarantees as<br>> -``llvm_stop_multithreaded()``.<br>> -<br>> -Note that, if you use scope-based shutdown, you can use the<br>> -``llvm_shutdown_obj`` class, which calls ``llvm_shutdown()`` in its destructor.<br>> +deallocate memory used for internal structures.<br>><br>> .. _managedstatic:<br>><br>> @@ -2217,15 +2184,11 @@ Lazy Initialization with ``ManagedStatic<br>> ------------------------------------------<br>><br>> ``ManagedStatic`` is a utility class in LLVM used to implement static<br>> -initialization of static resources, such as the global type tables. Before the<br>> -invocation of ``llvm_shutdown()``, it implements a simple lazy initialization<br>> -scheme. Once ``llvm_start_multithreaded()`` returns, however, it uses<br>> +initialization of static resources, such as the global type tables. In a<br>> +single-threaded environment, it implements a simple lazy initialization scheme.<br>> +When LLVM is compiled with support for multi-threading, however, it uses<br>> double-checked locking to implement thread-safe lazy initialization.<br>><br>> -Note that, because no other threads are allowed to issue LLVM API calls before<br>> -``llvm_start_multithreaded()`` returns, it is possible to have<br>> -``ManagedStatic``\ s of ``llvm::sys::Mutex``\ s.<br>> -<br>> .. _llvmcontext:<br>><br>> Achieving Isolation with ``LLVMContext``<br>><br>> Modified: llvm/trunk/include/llvm-c/Core.h<br>> URL:<span> </span><a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm-c/Core.h?rev=211900&r1=211899&r2=211900&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm-c/Core.h?rev=211900&r1=211899&r2=211900&view=diff</a><br>> ==============================================================================<br>> --- llvm/trunk/include/llvm-c/Core.h (original)<br>> +++ llvm/trunk/include/llvm-c/Core.h Fri Jun 27 10:13:01 2014<br>> @@ -2848,16 +2848,13 @@ void LLVMDisposePassManager(LLVMPassMana<br>> * @{<br>> */<br>><br>> -/** Allocate and initialize structures needed to make LLVM safe for<br>> - multithreading. The return value indicates whether multithreaded<br>> - initialization succeeded. Must be executed in isolation from all<br>> - other LLVM api calls.<br>> - @see llvm::llvm_start_multithreaded */<br>> +/** Deprecated: Multi-threading can only be enabled/disabled with the compile<br>> + time define LLVM_ENABLE_THREADS. This function always returns<br>> + LLVMIsMultithreaded(). */<br>> LLVMBool LLVMStartMultithreaded(void);<br>><br>> -/** Deallocate structures necessary to make LLVM safe for multithreading.<br>> - Must be executed in isolation from all other LLVM api calls.<br>> - @see llvm::llvm_stop_multithreaded */<br>> +/** Deprecated: Multi-threading can only be enabled/disabled with the compile<br>> + time define LLVM_ENABLE_THREADS. */<br>> void LLVMStopMultithreaded(void);<br>><br>> /** Check whether LLVM is executing in thread-safe mode or not.<br>><br>> Modified: llvm/trunk/include/llvm/Support/ManagedStatic.h<br>> URL:<span> </span><a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Support/ManagedStatic.h?rev=211900&r1=211899&r2=211900&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Support/ManagedStatic.h?rev=211900&r1=211899&r2=211900&view=diff</a><br>> ==============================================================================<br>> --- llvm/trunk/include/llvm/Support/ManagedStatic.h (original)<br>> +++ llvm/trunk/include/llvm/Support/ManagedStatic.h Fri Jun 27 10:13:01 2014<br>> @@ -103,9 +103,6 @@ void llvm_shutdown();<br>> /// llvm_shutdown() when it is destroyed.<br>> struct llvm_shutdown_obj {<br>> llvm_shutdown_obj() { }<br>> - explicit llvm_shutdown_obj(bool multithreaded) {<br>> - if (multithreaded) llvm_start_multithreaded();<br>> - }<br>> ~llvm_shutdown_obj() { llvm_shutdown(); }<br>> };<br>><br>><br>> Modified: llvm/trunk/include/llvm/Support/Threading.h<br>> URL:<span> </span><a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Support/Threading.h?rev=211900&r1=211899&r2=211900&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Support/Threading.h?rev=211900&r1=211899&r2=211900&view=diff</a><br>> ==============================================================================<br>> --- llvm/trunk/include/llvm/Support/Threading.h (original)<br>> +++ llvm/trunk/include/llvm/Support/Threading.h Fri Jun 27 10:13:01 2014<br>> @@ -7,7 +7,8 @@<br>> //<br>> //===----------------------------------------------------------------------===//<br>> //<br>> -// TThis file defines llvm_start_multithreaded() and friends.<br>> +// This file declares helper functions for running LLVM in a multi-threaded<br>> +// environment.<br>> //<br>> //===----------------------------------------------------------------------===//<br>><br>> @@ -15,22 +16,8 @@<br>> #define LLVM_SUPPORT_THREADING_H<br>><br>> namespace llvm {<br>> - /// llvm_start_multithreaded - Allocate and initialize structures needed to<br>> - /// make LLVM safe for multithreading. The return value indicates whether<br>> - /// multithreaded initialization succeeded. LLVM will still be operational<br>> - /// on "failed" return, and will still be safe for hosting threading<br>> - /// applications in the JIT, but will not be safe for concurrent calls to the<br>> - /// LLVM APIs.<br>> - /// THIS MUST EXECUTE IN ISOLATION FROM ALL OTHER LLVM API CALLS.<br>> - bool llvm_start_multithreaded();<br>> -<br>> - /// llvm_stop_multithreaded - Deallocate structures necessary to make LLVM<br>> - /// safe for multithreading.<br>> - /// THIS MUST EXECUTE IN ISOLATION FROM ALL OTHER LLVM API CALLS.<br>> - void llvm_stop_multithreaded();<br>> -<br>> - /// llvm_is_multithreaded - Check whether LLVM is executing in thread-safe<br>> - /// mode or not.<br>> + /// Returns true if LLVM is compiled with support for multi-threading, and<br>> + /// false otherwise.<br>> bool llvm_is_multithreaded();<br>><br>> /// llvm_execute_on_thread - Execute the given \p UserFn on a separate<br>><br>> Modified: llvm/trunk/lib/IR/Core.cpp<br>> URL:<span> </span><a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/IR/Core.cpp?rev=211900&r1=211899&r2=211900&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/IR/Core.cpp?rev=211900&r1=211899&r2=211900&view=diff</a><br>> ==============================================================================<br>> --- llvm/trunk/lib/IR/Core.cpp (original)<br>> +++ llvm/trunk/lib/IR/Core.cpp Fri Jun 27 10:13:01 2014<br>> @@ -2708,11 +2708,10 @@ void LLVMDisposePassManager(LLVMPassMana<br>> /*===-- Threading ------------------------------------------------------===*/<br>><br>> LLVMBool LLVMStartMultithreaded() {<br>> - return llvm_start_multithreaded();<br>> + return LLVMIsMultithreaded();<br>> }<br>><br>> void LLVMStopMultithreaded() {<br>> - llvm_stop_multithreaded();<br>> }<br>><br>> LLVMBool LLVMIsMultithreaded() {<br>><br>> Modified: llvm/trunk/lib/Support/Threading.cpp<br>> URL:<span> </span><a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Support/Threading.cpp?rev=211900&r1=211899&r2=211900&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Support/Threading.cpp?rev=211900&r1=211899&r2=211900&view=diff</a><br>> ==============================================================================<br>> --- llvm/trunk/lib/Support/Threading.cpp (original)<br>> +++ llvm/trunk/lib/Support/Threading.cpp Fri Jun 27 10:13:01 2014<br>> @@ -7,7 +7,8 @@<br>> //<br>> //===----------------------------------------------------------------------===//<br>> //<br>> -// This file implements llvm_start_multithreaded() and friends.<br>> +// This file defines helper functions for running LLVM in a multi-threaded<br>> +// environment.<br>> //<br>> //===----------------------------------------------------------------------===//<br>><br>> @@ -19,38 +20,14 @@<br>><br>> using namespace llvm;<br>><br>> -static bool multithreaded_mode = false;<br>> -<br>> -bool llvm::llvm_start_multithreaded() {<br>> +bool llvm::llvm_is_multithreaded() {<br>> #if LLVM_ENABLE_THREADS != 0<br>> - assert(!multithreaded_mode && "Already multithreaded!");<br>> - multithreaded_mode = true;<br>> -<br>> - // We fence here to ensure that all initialization is complete BEFORE we<br>> - // return from llvm_start_multithreaded().<br>> - sys::MemoryFence();<br>> return true;<br>> #else<br>> return false;<br>> #endif<br>> }<br>><br>> -void llvm::llvm_stop_multithreaded() {<br>> -#if LLVM_ENABLE_THREADS != 0<br>> - assert(multithreaded_mode && "Not currently multithreaded!");<br>> -<br>> - // We fence here to insure that all threaded operations are complete BEFORE we<br>> - // return from llvm_stop_multithreaded().<br>> - sys::MemoryFence();<br>> -<br>> - multithreaded_mode = false;<br>> -#endif<br>> -}<br>> -<br>> -bool llvm::llvm_is_multithreaded() {<br>> - return multithreaded_mode;<br>> -}<br>> -<br>> #if LLVM_ENABLE_THREADS != 0 && defined(HAVE_PTHREAD_H)<br>> #include <pthread.h><br>><br>><br>> Modified: llvm/trunk/unittests/Support/ManagedStatic.cpp<br>> URL:<span> </span><a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/unittests/Support/ManagedStatic.cpp?rev=211900&r1=211899&r2=211900&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/unittests/Support/ManagedStatic.cpp?rev=211900&r1=211899&r2=211900&view=diff</a><br>> ==============================================================================<br>> --- llvm/trunk/unittests/Support/ManagedStatic.cpp (original)<br>> +++ llvm/trunk/unittests/Support/ManagedStatic.cpp Fri Jun 27 10:13:01 2014<br>> @@ -47,7 +47,6 @@ TEST(Initialize, MultipleThreads) {<br>> void *p1 = test1::allocate_stack(a1);<br>> void *p2 = test1::allocate_stack(a2);<br>><br>> - llvm_start_multithreaded();<br>> pthread_t t1, t2;<br>> pthread_create(&t1, &a1, test1::helper, nullptr);<br>> pthread_create(&t2, &a2, test1::helper, nullptr);<br>> @@ -55,7 +54,6 @@ TEST(Initialize, MultipleThreads) {<br>> pthread_join(t2, nullptr);<br>> free(p1);<br>> free(p2);<br>> - llvm_stop_multithreaded();<br>> }<br>> #endif<br>><br>><br>><br>> _______________________________________________<br>> llvm-commits mailing list<br>><span> </span><a href="mailto:llvm-commits@cs.uiuc.edu" target="_blank">llvm-commits@cs.uiuc.edu</a><br>><span> </span><a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits</a><br><br><br>_______________________________________________<br>llvm-commits mailing list<br><a href="mailto:llvm-commits@cs.uiuc.edu" target="_blank">llvm-commits@cs.uiuc.edu</a><br><a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits</a></div></div></blockquote></div></div></blockquote></div><br></div></div></div></div></div><div><div class="h5">_______________________________________________<br>llvm-commits mailing list<br><a href="mailto:llvm-commits@cs.uiuc.edu" target="_blank">llvm-commits@cs.uiuc.edu</a><br><a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits</a><br></div></div></div></blockquote></div><br></div></div><br>_______________________________________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@cs.uiuc.edu">llvm-commits@cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits</a><br>
<br></blockquote></div><br></div>