[libcxx-commits] [libcxx] [libc++][pstl] Introduce a "statpart" backend (PR #230540)
via libcxx-commits
libcxx-commits at lists.llvm.org
Wed Oct 14 04:14:06 PDT 2026
llvmorg-github-actions[bot] wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-libcxx
Author: Michael G. Kazakov (mikekazakov)
<details>
<summary>Changes</summary>
This PR introduces a simplified PSTL backend based on a static partitioning scheme and on a single "parallelism" function `__apply(size_t, void*, void (*)(void*, size_t))`. Currently the underlying parallelism is only provided via the `libdispatch` library, other mechanisms are assumed to be added later without changing the API+ABI surface.
This backend is mostly a re-packaging of the code existing in `libdispatch` and `cpu_algos`, currently placed in a dedicated `__pstl/statpart/...` directory. `__pstl/statpart.h` serves as a purely umbrella header.
The implementations of the algorithms with the STL APIs are identical to the existing implementations in `__pstl/cpu_algos/...` modulo these alterations:
- the naming is rebased to use the `__statpart` backend.
- the algorithms are presented directly under the `__statpart_backend_tag` specialisation, without a re-introduction via inheritance as currently done in `libdispatch` and `std_thread`.
- `#pragma omp` is not carried over.
There are a couple of differences in the backend basis as well:
- The partitioning logic is moved out of the ABI surface.
- The `__cpu_traits` API is not used, all backends' functions are structured as free functions.
- The partitioning logic is changed. Instead of capping the number of elements per partition, it now caps the maximum number of partitions per available parallel workers and allows to specify the minimum number of elements per partition. Partitioning that produces less than 2 sub-ranges is declared invalid, the caller should fall back to serial processing in this case.
- The basis algorithm implementations are copied as-is with minimal changes, with 2 exceptions:
- `__merge()` finds the cutpoints of each paired sub-ranges in parallel, instead of sequentially as before. This costs 2 binary searches per partition, comparing to 1 binary search per partition in the previous implementation, but all these searches are now done in parallel, instead being serial on a critical path.
- `__stable_sort()` no longer merges two the first partitions sequentially in case of an odd number of partitions. Instead the last span is carried over as-is in case of odd number of spans.
At the moment this backend is not enabled by default on any configuration and needs to be explicitly picked via `-DLIBCXX_PSTL_BACKEND=statpart`.
Fixes #<!-- -->229804.
---
Patch is 107.08 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/230540.diff
34 Files Affected:
- (modified) libcxx/CMakeLists.txt (+2)
- (modified) libcxx/include/CMakeLists.txt (+19)
- (modified) libcxx/include/__config_site.in (+1)
- (modified) libcxx/include/__pstl/backend.h (+6)
- (modified) libcxx/include/__pstl/backend_fwd.h (+3)
- (added) libcxx/include/__pstl/backends/statpart.h (+31)
- (added) libcxx/include/__pstl/backends/statpart/basis.h (+503)
- (added) libcxx/include/__pstl/backends/statpart/find_end.h (+102)
- (added) libcxx/include/__pstl/backends/statpart/find_if.h (+67)
- (added) libcxx/include/__pstl/backends/statpart/for_each.h (+63)
- (added) libcxx/include/__pstl/backends/statpart/is_heap_until.h (+105)
- (added) libcxx/include/__pstl/backends/statpart/merge.h (+88)
- (added) libcxx/include/__pstl/backends/statpart/min_element.h (+85)
- (added) libcxx/include/__pstl/backends/statpart/minmax_element.h (+90)
- (added) libcxx/include/__pstl/backends/statpart/mismatch.h (+91)
- (added) libcxx/include/__pstl/backends/statpart/reverse.h (+75)
- (added) libcxx/include/__pstl/backends/statpart/search.h (+102)
- (added) libcxx/include/__pstl/backends/statpart/search_n.h (+102)
- (added) libcxx/include/__pstl/backends/statpart/stable_sort.h (+61)
- (added) libcxx/include/__pstl/backends/statpart/swap_ranges.h (+70)
- (added) libcxx/include/__pstl/backends/statpart/transform.h (+102)
- (added) libcxx/include/__pstl/backends/statpart/transform_reduce.h (+126)
- (added) libcxx/include/__pstl/backends/statpart/uninitialized_copy.h (+69)
- (added) libcxx/include/__pstl/backends/statpart/uninitialized_move.h (+69)
- (modified) libcxx/include/module.modulemap.in (+57)
- (modified) libcxx/src/CMakeLists.txt (+6)
- (added) libcxx/src/pstl/statpart.cpp (+31)
- (added) libcxx/test/libcxx/algorithms/pstl.statpart.partition.pass.cpp (+181)
- (modified) libcxx/test/libcxx/transitive_includes/cxx17.csv (+4)
- (modified) libcxx/test/libcxx/transitive_includes/cxx20.csv (+4)
- (modified) libcxx/test/libcxx/transitive_includes/cxx23.csv (+4)
- (modified) libcxx/test/libcxx/transitive_includes/cxx26.csv (+4)
- (modified) libcxx/test/libcxx/transitive_includes/cxx29.csv (+4)
- (modified) libcxx/utils/libcxx/test/features/libcxx_macros.py (+1)
``````````diff
diff --git a/libcxx/CMakeLists.txt b/libcxx/CMakeLists.txt
index d163f0f579c395..5478c38afa781d 100644
--- a/libcxx/CMakeLists.txt
+++ b/libcxx/CMakeLists.txt
@@ -803,6 +803,8 @@ elseif(LIBCXX_PSTL_BACKEND STREQUAL "std_thread")
config_define(1 _LIBCPP_PSTL_BACKEND_STD_THREAD)
elseif(LIBCXX_PSTL_BACKEND STREQUAL "libdispatch")
config_define(1 _LIBCPP_PSTL_BACKEND_LIBDISPATCH)
+elseif(LIBCXX_PSTL_BACKEND STREQUAL "statpart")
+ config_define(1 _LIBCPP_PSTL_BACKEND_STATPART)
else()
message(FATAL_ERROR "LIBCXX_PSTL_BACKEND is set to ${LIBCXX_PSTL_BACKEND}, which is not a valid backend.
Valid backends are: serial, std_thread and libdispatch")
diff --git a/libcxx/include/CMakeLists.txt b/libcxx/include/CMakeLists.txt
index 9af3d00f36f7e6..91081893079c1b 100644
--- a/libcxx/include/CMakeLists.txt
+++ b/libcxx/include/CMakeLists.txt
@@ -695,6 +695,25 @@ set(files
__pstl/backends/default.h
__pstl/backends/libdispatch.h
__pstl/backends/serial.h
+ __pstl/backends/statpart.h
+ __pstl/backends/statpart/basis.h
+ __pstl/backends/statpart/find_end.h
+ __pstl/backends/statpart/find_if.h
+ __pstl/backends/statpart/for_each.h
+ __pstl/backends/statpart/is_heap_until.h
+ __pstl/backends/statpart/merge.h
+ __pstl/backends/statpart/min_element.h
+ __pstl/backends/statpart/minmax_element.h
+ __pstl/backends/statpart/mismatch.h
+ __pstl/backends/statpart/reverse.h
+ __pstl/backends/statpart/search.h
+ __pstl/backends/statpart/search_n.h
+ __pstl/backends/statpart/stable_sort.h
+ __pstl/backends/statpart/swap_ranges.h
+ __pstl/backends/statpart/transform.h
+ __pstl/backends/statpart/transform_reduce.h
+ __pstl/backends/statpart/uninitialized_copy.h
+ __pstl/backends/statpart/uninitialized_move.h
__pstl/backends/std_thread.h
__pstl/cpu_algos/any_of.h
__pstl/cpu_algos/cpu_traits.h
diff --git a/libcxx/include/__config_site.in b/libcxx/include/__config_site.in
index e6e33a2eeb2a9e..b0a2230c42c21d 100644
--- a/libcxx/include/__config_site.in
+++ b/libcxx/include/__config_site.in
@@ -36,6 +36,7 @@
#cmakedefine _LIBCPP_PSTL_BACKEND_SERIAL
#cmakedefine _LIBCPP_PSTL_BACKEND_STD_THREAD
#cmakedefine _LIBCPP_PSTL_BACKEND_LIBDISPATCH
+#cmakedefine _LIBCPP_PSTL_BACKEND_STATPART
// Hardening.
#cmakedefine _LIBCPP_HARDENING_MODE_DEFAULT @_LIBCPP_HARDENING_MODE_DEFAULT@
diff --git a/libcxx/include/__pstl/backend.h b/libcxx/include/__pstl/backend.h
index 5980b0708cd340..1626a929f9707d 100644
--- a/libcxx/include/__pstl/backend.h
+++ b/libcxx/include/__pstl/backend.h
@@ -30,8 +30,14 @@ _LIBCPP_PUSH_MACROS
# elif defined(_LIBCPP_PSTL_BACKEND_LIBDISPATCH)
# include <__pstl/backends/default.h>
# include <__pstl/backends/libdispatch.h>
+# elif defined(_LIBCPP_PSTL_BACKEND_STATPART)
+# include <__pstl/backends/default.h>
+# include <__pstl/backends/statpart.h>
# endif
+// statpart pulls <cerrno> in transitively via __thread/thread.h, make sure this include is not conditional/flaky.
+# include <cerrno>
+
#endif // _LIBCPP_STD_VER >= 17
_LIBCPP_POP_MACROS
diff --git a/libcxx/include/__pstl/backend_fwd.h b/libcxx/include/__pstl/backend_fwd.h
index e45943954bed98..41b74ad43c1055 100644
--- a/libcxx/include/__pstl/backend_fwd.h
+++ b/libcxx/include/__pstl/backend_fwd.h
@@ -51,6 +51,7 @@ struct __default_backend_tag;
struct __libdispatch_backend_tag;
struct __serial_backend_tag;
struct __std_thread_backend_tag;
+struct __statpart_backend_tag;
# if defined(_LIBCPP_PSTL_BACKEND_SERIAL)
using __current_configuration _LIBCPP_NODEBUG = __backend_configuration<__serial_backend_tag, __default_backend_tag>;
@@ -60,6 +61,8 @@ using __current_configuration _LIBCPP_NODEBUG =
# elif defined(_LIBCPP_PSTL_BACKEND_LIBDISPATCH)
using __current_configuration _LIBCPP_NODEBUG =
__backend_configuration<__libdispatch_backend_tag, __default_backend_tag>;
+# elif defined(_LIBCPP_PSTL_BACKEND_STATPART)
+using __current_configuration _LIBCPP_NODEBUG = __backend_configuration<__statpart_backend_tag, __default_backend_tag>;
# else
// ...New vendors can add parallel backends here...
diff --git a/libcxx/include/__pstl/backends/statpart.h b/libcxx/include/__pstl/backends/statpart.h
new file mode 100644
index 00000000000000..665d3d36d8a437
--- /dev/null
+++ b/libcxx/include/__pstl/backends/statpart.h
@@ -0,0 +1,31 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef _LIBCPP___PSTL_STATPART_H
+#define _LIBCPP___PSTL_STATPART_H
+
+#include <__pstl/backends/statpart/basis.h>
+#include <__pstl/backends/statpart/find_end.h>
+#include <__pstl/backends/statpart/find_if.h>
+#include <__pstl/backends/statpart/for_each.h>
+#include <__pstl/backends/statpart/is_heap_until.h>
+#include <__pstl/backends/statpart/merge.h>
+#include <__pstl/backends/statpart/min_element.h>
+#include <__pstl/backends/statpart/minmax_element.h>
+#include <__pstl/backends/statpart/mismatch.h>
+#include <__pstl/backends/statpart/reverse.h>
+#include <__pstl/backends/statpart/search.h>
+#include <__pstl/backends/statpart/search_n.h>
+#include <__pstl/backends/statpart/stable_sort.h>
+#include <__pstl/backends/statpart/swap_ranges.h>
+#include <__pstl/backends/statpart/transform.h>
+#include <__pstl/backends/statpart/transform_reduce.h>
+#include <__pstl/backends/statpart/uninitialized_copy.h>
+#include <__pstl/backends/statpart/uninitialized_move.h>
+
+#endif // _LIBCPP___PSTL_STATPART_H
diff --git a/libcxx/include/__pstl/backends/statpart/basis.h b/libcxx/include/__pstl/backends/statpart/basis.h
new file mode 100644
index 00000000000000..470785bd8b3f47
--- /dev/null
+++ b/libcxx/include/__pstl/backends/statpart/basis.h
@@ -0,0 +1,503 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef _LIBCPP___PSTL_STATPART_BASIS_H
+#define _LIBCPP___PSTL_STATPART_BASIS_H
+
+#include <__algorithm/lower_bound.h>
+#include <__algorithm/merge.h>
+#include <__algorithm/min.h>
+#include <__algorithm/move.h>
+#include <__algorithm/upper_bound.h>
+#include <__assert>
+#include <__atomic/atomic.h>
+#include <__config>
+#include <__functional/identity.h>
+#include <__functional/operations.h>
+#include <__iterator/concepts.h>
+#include <__iterator/distance.h>
+#include <__iterator/iterator_traits.h>
+#include <__iterator/move_iterator.h>
+#include <__memory/allocator.h>
+#include <__memory/construct_at.h>
+#include <__memory/destroy.h>
+#include <__memory/unique_ptr.h>
+#include <__numeric/reduce.h>
+#include <__optional/nullopt_t.h>
+#include <__optional/optional.h>
+#include <__thread/thread.h>
+#include <__type_traits/is_assignable.h>
+#include <__type_traits/is_constructible.h>
+#include <__type_traits/is_execution_policy.h>
+#include <__type_traits/is_trivially_copyable.h>
+#include <__utility/empty.h>
+#include <__utility/move.h>
+
+#if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
+# pragma GCC system_header
+#endif
+
+_LIBCPP_PUSH_MACROS
+#include <__undef_macros>
+
+#if _LIBCPP_STD_VER >= 17
+
+_LIBCPP_BEGIN_NAMESPACE_STD
+_LIBCPP_BEGIN_EXPLICIT_ABI_ANNOTATIONS
+namespace __pstl {
+
+namespace __statpart {
+
+// The backbone of the backend: applies the function to each of the indices [0, __chunk_count) in parallel.
+// The actual number of parallel workers and the order of application is unspecified.
+_LIBCPP_EXPORTED_FROM_ABI void
+__apply(size_t __chunk_count, void* __context, void (*__func)(void* __context, size_t __chunk)) noexcept;
+
+// The maximum number of partitions to be processed per CPU
+constexpr inline size_t __max_partitions_per_cpu = 8;
+
+// The minimum number of elements to sort serially in a single partition
+constexpr inline size_t __min_sort_partition_size = 256;
+
+enum class __partitioning_result {
+ // Returned when the partitioning result is not a nullopt.
+ __ok,
+ // Returned when there are not enough element to produce a single partition or the final number of partitions is fewer
+ // than 2, i.e. there's no reason to process the input in parallel.
+ __not_enough_elements,
+ // There's fewer than 2 processors available in the system, or their number is unknown, which is treated as having
+ // a single processor. In either case, fall back to serial processing.
+ __no_parallelism,
+ // Reserved for later: returned when memory could not be allocated for the stashed forward iterators.
+ __bad_alloc
+};
+
+template <class _Iterator>
+struct _LIBCPP_HIDE_FROM_ABI __iterator_range {
+ _Iterator __first;
+ _Iterator __last;
+};
+
+// __iterator_partitions<Iter> holds a finite pre-determined set of partitions and allows querying each iterator range
+// in O(1) time.
+// TODO: Support forward iterators later via a second implicit template argument and its specializations.
+template <class _Iterator>
+struct _LIBCPP_HIDE_FROM_ABI __iterator_partitions {
+ _LIBCPP_HIDE_FROM_ABI size_t __partitions_count() const { return __partitions_count_; }
+
+ _LIBCPP_HIDE_FROM_ABI __iterator_range<_Iterator> __partition(size_t __n) const {
+ _Iterator __first = __base_ + __n * __partition_size_ + (__n < __remainder_ ? __n : __remainder_);
+ _Iterator __last = __first + __partition_size_ + (__n < __remainder_ ? 1 : 0);
+ return __iterator_range<_Iterator>{__first, __last};
+ }
+
+ _LIBCPP_HIDE_FROM_ABI _Iterator __end() const { return __base_ + __count_; }
+
+private:
+ _Iterator __base_;
+ size_t __count_;
+ size_t __partition_size_;
+ size_t __remainder_;
+ size_t __partitions_count_;
+
+ template <class _Iter>
+ friend optional<__iterator_partitions<_Iter>> _LIBCPP_HIDE_FROM_ABI
+ __partition(_Iter, _Iter, __partitioning_result&, size_t, size_t, size_t, size_t) noexcept;
+};
+
+// Splits the input iterator range into a finite set of partitions, such that every iterator sub-range can be queried in
+// a O(1) time later.
+// If partitioning fails, returns nullopt and reports the reason in __result.
+// __min_partition_size specifies the minimal number of elements in every partition.
+// __prefix_to_skip and __suffix_to_skip allows optionally trimming the input range by the specified number of elements
+// before partitioning.
+// __workers_count is normally implicitly queried from hardware_concurrency() and should be explicitly specified only in
+// unit tests.
+template <class _ForwardIterator>
+_LIBCPP_HIDE_FROM_ABI optional<__iterator_partitions<_ForwardIterator>> __partition(
+ _ForwardIterator __first,
+ _ForwardIterator __last,
+ __partitioning_result& __result,
+ size_t __min_partition_size = 1,
+ size_t __prefix_to_skip = 0,
+ size_t __suffix_to_skip = 0,
+ size_t __workers_count = thread::hardware_concurrency()) noexcept {
+ // Forward iterators will be supported later.
+ static_assert(__has_random_access_iterator_category_or_concept<_ForwardIterator>::value);
+
+ _LIBCPP_ASSERT_INTERNAL(__min_partition_size >= 1, "minimum partition size must be at least 1 element");
+
+ if (__workers_count < 2) {
+ // There's no available parallelism in the system, no reason to perform partitioning.
+ __result = __partitioning_result::__no_parallelism;
+ return {};
+ }
+
+ size_t __count = static_cast<size_t>(std::distance(__first, __last));
+ if (__count <= __prefix_to_skip + __suffix_to_skip) {
+ __result = __partitioning_result::__not_enough_elements;
+ return {};
+ }
+
+ size_t __effective_count = __count - __prefix_to_skip - __suffix_to_skip;
+ size_t __max_partitions = __workers_count * __max_partitions_per_cpu;
+ size_t __partition_count = std::min(__max_partitions, __effective_count / __min_partition_size);
+
+ if (__partition_count < 2) {
+ __result = __partitioning_result::__not_enough_elements;
+ return {}; // not enough input data to process in parallel => no reason to build partitions
+ }
+
+ __iterator_partitions<_ForwardIterator> __p;
+ __p.__base_ = __first + __prefix_to_skip;
+ __p.__count_ = __effective_count;
+ __p.__partition_size_ = __effective_count / __partition_count;
+ __p.__remainder_ = __effective_count % __partition_count;
+ __p.__partitions_count_ = __partition_count;
+ __result = __partitioning_result::__ok;
+ return __p;
+}
+
+// for_each with explicit partitioning:
+// calls __func in parallel with every iterator range provided in __partitions.
+template <class _Iterator, class _Functor>
+_LIBCPP_HIDE_FROM_ABI void __for_each_partition(const __iterator_partitions<_Iterator>& __partitions, _Functor __func) {
+ struct __ctx_t {
+ const __iterator_partitions<_Iterator>& __partitions_;
+ _Functor& __func_;
+ } __ctx{__partitions, __func};
+ __statpart::__apply(__partitions.__partitions_count(), &__ctx, [](void* __context, size_t __partition) {
+ __ctx_t& __ctx = *static_cast<__ctx_t*>(__context);
+ auto __range = __ctx.__partitions_.__partition(__partition);
+ __ctx.__func_(__range.__first, __range.__last);
+ });
+}
+
+// Calls __func(index) for every index in [0, __count) in parallel.
+template <class _Functor>
+_LIBCPP_HIDE_FROM_ABI void __for_each_index(size_t __count, _Functor __func) {
+ __statpart::__apply(__count, &__func, [](void* __context, size_t __index) {
+ (*static_cast<_Functor*>(__context))(__index);
+ });
+}
+
+// for_each that calls __func in parallel for every partition index available in __partitions.
+template <class _Iterator, class _Functor>
+_LIBCPP_HIDE_FROM_ABI void
+__for_each_partition_index(const __iterator_partitions<_Iterator>& __partitions, _Functor __func) {
+ __statpart::__for_each_index(__partitions.__partitions_count(), std::move(__func));
+}
+
+// for_each with default implicit partitioning:
+// calls __func in parallel with every element in [__first, __last), provided as iterator ranges.
+template <class _Iterator, class _Functor>
+_LIBCPP_HIDE_FROM_ABI optional<__empty> __for_each(_Iterator __first, _Iterator __last, _Functor __func) {
+ if (__first == __last)
+ return __empty{};
+ __partitioning_result __partition_result;
+ auto __partitions = __statpart::__partition(__first, __last, __partition_result);
+ if (__partitions) {
+ __statpart::__for_each_partition(*__partitions, std::move(__func));
+ return __empty{};
+ } else if (__partition_result == __partitioning_result::__bad_alloc) {
+ return nullopt;
+ } else {
+ __func(__first, __last);
+ return __empty{};
+ }
+}
+
+// Applies the 3-legged function to sub-ranges of the input ranges in parallel:
+// f(sub_range1_first, sub_range1_last, sub_range2_first)
+template <class _RandomAccessIterator1, class _RandomAccessIterator2, class _BrickFunction>
+_LIBCPP_HIDE_FROM_ABI optional<__empty> __for_each_iter_pair(
+ _RandomAccessIterator1 __first1,
+ _RandomAccessIterator1 __last1,
+ _RandomAccessIterator2 __first2,
+ _BrickFunction __f) {
+ return __for_each(
+ __first1,
+ __last1,
+ [__first1, __first2, __f = std::move(__f)](
+ _RandomAccessIterator1 __brick_first1, _RandomAccessIterator1 __brick_last1) {
+ _RandomAccessIterator2 __brick_first2 = __first2 + (__brick_first1 - __first1);
+ __f(__brick_first1, __brick_last1, __brick_first2);
+ });
+}
+
+template <class _Index, class _Brick, class _Compare>
+_LIBCPP_HIDE_FROM_ABI optional<_Index>
+__find(_Index __first, _Index __last, _Brick __f, _Compare __comp, bool __b_first) {
+ typedef typename std::iterator_traits<_Index>::difference_type _DifferenceType;
+ const _DifferenceType __n = __last - __first;
+ _DifferenceType __initial_dist = __b_first ? __n : -1;
+ std::atomic<_DifferenceType> __extremum(__initial_dist);
+ // TODO: find out what is better here: parallel_for or parallel_reduce
+ auto __res = __statpart::__for_each(__first, __last, [__comp, __f, __first, &__extremum](_Index __i, _Index __j) {
+ // See "Reducing Contention Through Priority Updates", PPoPP '13, for discussion of
+ // why using a shared variable scales fairly well in this situation.
+ if (__comp(__i - __first, __extremum)) {
+ _Index __result = __f(__i, __j);
+ // If not '__last' returned then we found what we want so put this to extremum
+ if (__result != __j) {
+ const _DifferenceType __k = __result - __first;
+ for (_DifferenceType __old = __extremum; __comp(__k, __old); __old = __extremum) {
+ __extremum.compare_exchange_weak(__old, __k);
+ }
+ }
+ }
+ });
+ if (!__res)
+ return nullopt;
+ return __extremum.load() != __initial_dist ? __first + __extremum.load() : __last;
+}
+
+template <class _RandomAccessIterator, class _Transform, class _Value, class _Combiner, class _Reduction>
+_LIBCPP_HIDE_FROM_ABI optional<_Value> __transform_reduce(
+ _RandomAccessIterator __first,
+ _RandomAccessIterator __last,
+ _Transform __transform,
+ _Value __init,
+ _Combiner __combiner,
+ _Reduction __reduction) {
+ if (__first == __last)
+ return __init;
+
+ // Split into partitions with at least 3 elements in each
+ __partitioning_result __partitioning_result;
+ auto __partitions = __statpart::__partition(__first, __last, __partitioning_result, 3);
+ if (!__partitions) {
+ if (__partitioning_result == __partitioning_result::__bad_alloc) {
+ // Failed to allocate memory for the partitions, propagate the error up.
+ return nullopt;
+ } else {
+ // Not enough elements to run in parallel, switch to serial implementation.
+ return __reduction(std::move(__first), std::move(__last), std::move(__init));
+ }
+ }
+
+ auto __destroy = [__count = __partitions->__partitions_count()](_Value* __ptr) {
+ std::destroy_n(__ptr, __count);
+ std::allocator<_Value>().deallocate(__ptr, __count);
+ };
+
+ // TODO: use __uninitialized_buffer
+ // TODO: allocate one element per worker instead of one element per chunk
+ unique_ptr<_Value[], decltype(__destroy)> __values(
+ std::allocator<_Value>().allocate(__partitions->__partitions_count()), __destroy);
+
+ // __dispatch_apply is noexcept
+ __statpart::__for_each_partition_index(*__partitions, [&](size_t __index) {
+ auto __range = __partitions->__partition(__index);
+ std::__construct_at(
+ __values.get() + __index,
+ __reduction(__range.__first + 2,
+ __range.__last,
+ __combiner(__transform(__range.__first), __transform(__range.__first + 1))));
+ });
+
+ return std::reduce(
+ std::make_move_iterator(__values.get()),
+ std::make_move_iterator(__values.get() + __partitions->__partitions_count()),
+ std::move(__init),
+ __combiner);
+}
+
+template <class _RandomAccessIterator1,
+ class _RandomAccessIterator2,
+ class _RandomAccessIterator3,
+ class _Compare,
+ class _LeafMerge>
+_LIBCPP_HIDE_FROM_ABI optional<__empty>
+__merge(_RandomAccessIterator1 __first1,
+ _RandomAccessIterator1 __last1,
+ _RandomAccessIterator2 __first2,
+ _RandomAccessIterator2 __last2,
+ _RandomAccessIterator3 __result,
+ _Compare __comp,
+ _LeafMerge __leaf_merge) noexcept {
+ // Run the merge such that the longest range is partitioned.
+ // __merge_sub_ranges swaps the ranges implicitly in case the second range was the longest one.
+ auto __run =
+ [__result](auto __long_first,
+ auto __long_last,
+ auto __short_first,
+ auto __short_last,
+ auto __find_cutpoint,
+ auto __merge_sub_ranges) -> optional<__empty> {
+ // Partition the longest input range evenly.
+ __partitioning_result __partitioning_result;
+ auto __partitions = __statpart::__partition(__long_first, __long_last, __partitioning_result);
+ if (!__partitions) {
+ if (__partitioning_result == __partitioning_result::__bad_alloc)
+ return nullopt;
+ // Not enough ...
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/230540
More information about the libcxx-commits
mailing list