<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On May 28, 2019, at 18:54, Olivier Giroux via libcxx-dev <<a href="mailto:libcxx-dev@lists.llvm.org" class="">libcxx-dev@lists.llvm.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class="">Hi everyone,<o:p class=""></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class=""><o:p class=""> </o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class="">To start the discussion of how libcxx should go about implementing this feature, I’ve prepared a cross-platform implementation of atomic_wait/atomic_notify_* for Mac/Linux/Windows/CUDA/unidentified-platform. I don’t claim that it’s fully tuned for your platform, nor do I claim that it’s perfect for every possible use, but it should not be terribly bad for any use either. It has various knobs to turn paths On/Off, so you can choose a different path on each platform, so long as it’s supported at all on that platform.<o:p class=""></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class=""><o:p class=""> </o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class="">You can find the implementation here:<span class="Apple-converted-space"> </span><a href="https://github.com/ogiroux/atomic_wait/" style="color: rgb(149, 79, 114); text-decoration: underline;" class="">https://github.com/ogiroux/atomic_wait/</a>.<o:p class=""></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class=""><o:p class=""> </o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class="">It has these strategies implemented:<o:p class=""></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class="">* Contention table. Used to optimize futex notify, or to hold CVs. Disable with __NO_TABLE.<o:p class=""></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class="">* Futex. Supported on Linux and Windows. For performance requires a table on Linux. Disable with __NO_FUTEX.<o:p class=""></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class="">* Condition variables. Supported on Linux and Mac. Requires table to function. Disable with __NO_CONDVAR.<o:p class=""></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class="">* Timed back-off. Supported on everything. Disable with __NO_SLEEP.<o:p class=""></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class="">* Spinlock. Supported on everything. Force with __NO_IDENT.<span class="Apple-converted-space"> </span><i class="">Note:</i><span class="Apple-converted-space"> </span><i class="">performance is too terrible to use</i>.<o:p class=""></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class=""><o:p class=""> </o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class="">The strategy is chosen this way, by platform:<o:p class=""></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class="">* Linux: default to futex (with table), fallback to futex (no table) -> CVs -> timed backoff -> spin.<o:p class=""></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class="">* Mac: default to CVs (table), fallback to timed backoff -> spin.<o:p class=""></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class="">* Windows: default to futex (no table), fallback to timed backoff -> spin.<o:p class=""></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class="">* CUDA: default to timed backoff, fallback to spin. (This is not all checked in in this tree.)<o:p class=""></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class="">* Unidentified platform: default to spin.<o:p class=""></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class=""><o:p class=""> </o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class="">The unidentified platform support could be better. For instance, we should probably assume that <thread> is implemented and use the sleeping/yielding facilities there. It should not fall all the way back to __NO_IDENT, it should instead fall back to about where CUDA is expected to be.<o:p class=""></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class=""><o:p class=""> </o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class="">One of the main discussion points I’d like to drive with this, is the design of the contention management table, to go along the sharded lock table that backs __atomic_* and __c11_atomic_* built-ins. Ideally this would be handled the same way, meaning that it should live in libatomic.a or your substitute, and be shared with other C++ standard libraries on your platform.</span></div></div></div></blockquote><div><br class=""></div><div>Just for the record, I gave some feedback to Olivier offline. The main thing we agreed on was that we can implement the contention table inside compiler-rt and provide function declarations in the headers that allow libc++ to get access to the associated functions (e.g. __cxx_atomic_notify_all). We wouldn't add builtins to Clang yet until we actually have a need for them.</div><div><br class=""></div><div>Louis</div></div><br class=""></body></html>