[libcxx-commits] [libcxx] 5b386ac - [libc++] Do not yield from __sp_mut::lock()

[Louis Dionne via libcxx-commits]

Author: Louis Dionne
Date: 2022-06-13T11:48:13-04:00
New Revision: 5b386ac912eae3fad455df71ce226963fc97cdeb

URL: https://github.com/llvm/llvm-project/commit/5b386ac912eae3fad455df71ce226963fc97cdeb
DIFF: https://github.com/llvm/llvm-project/commit/5b386ac912eae3fad455df71ce226963fc97cdeb.diff

LOG: [libc++] Do not yield from __sp_mut::lock()

Instead of trying to be clever and design our own locking primitive,
simply rely on the OS-provided implementation to do the right thing.

Indeed, manually yielding to the OS does not provide the necessary
information for it to make good prioritization decisions. For example,
if a thread with higher priority yields while waiting for a lock held
by a thread with lower priority but the system is contended, it is
possible for the thread with lower priority to not run until the higher
priority thread has yielded 16 times and goes for __libcpp_mutex_lock().
Once that happens, the OS can bump the priority of the thread that
currently holds the lock to unblock everyone. So instead, we might as
well give the system all the information from the start so it can make
appropriate decisions.

As a fly-by change, also increase the number of locks in the table.
The size increase is modest, but has the potential to half the amount
of contention on those locks.

rdar://93598606

Differential Revision: https://reviews.llvm.org/D126882

Added: 
    

Modified: 
    libcxx/src/memory.cpp

Removed: 
    


################################################################################
diff  --git a/libcxx/src/memory.cpp b/libcxx/src/memory.cpp
index faf3189e5ee27..bf96efa9ce6d4 100644
--- a/libcxx/src/memory.cpp
+++ b/libcxx/src/memory.cpp
@@ -138,9 +138,13 @@ __shared_weak_count::__get_deleter(const type_info&) const noexcept
 
 #if !defined(_LIBCPP_HAS_NO_THREADS)
 
-static constexpr std::size_t __sp_mut_count = 16;
+static constexpr std::size_t __sp_mut_count = 32;
 static constinit __libcpp_mutex_t mut_back[__sp_mut_count] =
 {
+    _LIBCPP_MUTEX_INITIALIZER, _LIBCPP_MUTEX_INITIALIZER, _LIBCPP_MUTEX_INITIALIZER, _LIBCPP_MUTEX_INITIALIZER,
+    _LIBCPP_MUTEX_INITIALIZER, _LIBCPP_MUTEX_INITIALIZER, _LIBCPP_MUTEX_INITIALIZER, _LIBCPP_MUTEX_INITIALIZER,
+    _LIBCPP_MUTEX_INITIALIZER, _LIBCPP_MUTEX_INITIALIZER, _LIBCPP_MUTEX_INITIALIZER, _LIBCPP_MUTEX_INITIALIZER,
+    _LIBCPP_MUTEX_INITIALIZER, _LIBCPP_MUTEX_INITIALIZER, _LIBCPP_MUTEX_INITIALIZER, _LIBCPP_MUTEX_INITIALIZER,
     _LIBCPP_MUTEX_INITIALIZER, _LIBCPP_MUTEX_INITIALIZER, _LIBCPP_MUTEX_INITIALIZER, _LIBCPP_MUTEX_INITIALIZER,
     _LIBCPP_MUTEX_INITIALIZER, _LIBCPP_MUTEX_INITIALIZER, _LIBCPP_MUTEX_INITIALIZER, _LIBCPP_MUTEX_INITIALIZER,
     _LIBCPP_MUTEX_INITIALIZER, _LIBCPP_MUTEX_INITIALIZER, _LIBCPP_MUTEX_INITIALIZER, _LIBCPP_MUTEX_INITIALIZER,
@@ -156,16 +160,7 @@ void
 __sp_mut::lock() noexcept
 {
     auto m = static_cast<__libcpp_mutex_t*>(__lx);
-    unsigned count = 0;
-    while (!__libcpp_mutex_trylock(m))
-    {
-        if (++count > 16)
-        {
-            __libcpp_mutex_lock(m);
-            break;
-        }
-        this_thread::yield();
-    }
+    __libcpp_mutex_lock(m);
 }
 
 void
@@ -181,7 +176,11 @@ __get_sp_mut(const void* p)
         &mut_back[ 0], &mut_back[ 1], &mut_back[ 2], &mut_back[ 3],
         &mut_back[ 4], &mut_back[ 5], &mut_back[ 6], &mut_back[ 7],
         &mut_back[ 8], &mut_back[ 9], &mut_back[10], &mut_back[11],
-        &mut_back[12], &mut_back[13], &mut_back[14], &mut_back[15]
+        &mut_back[12], &mut_back[13], &mut_back[14], &mut_back[15],
+        &mut_back[16], &mut_back[17], &mut_back[18], &mut_back[19],
+        &mut_back[20], &mut_back[21], &mut_back[22], &mut_back[23],
+        &mut_back[24], &mut_back[25], &mut_back[26], &mut_back[27],
+        &mut_back[28], &mut_back[29], &mut_back[30], &mut_back[31]
     };
     return muts[hash<const void*>()(p) & (__sp_mut_count-1)];
 }