[libcxx-commits] [libcxx] d96aac4 - Optimize 'construct at end' loops in vector

[Martijn Vels via libcxx-commits]

Author: Martijn Vels
Date: 2020-06-18T13:51:12-04:00
New Revision: d96aac435423b550840afae79315bda98953214e

URL: https://github.com/llvm/llvm-project/commit/d96aac435423b550840afae79315bda98953214e
DIFF: https://github.com/llvm/llvm-project/commit/d96aac435423b550840afae79315bda98953214e.diff

LOG: Optimize 'construct at end' loops in vector

Summary:
This change adds local 'end' and 'pos' variables for the main loop inmstead of using the ConstructTransaction variables directly.

We observed that not all vector initialization and resize operations got properly vectorized, i.e., (partially) unrolled into XMM stores for floats.

For example, `vector<int32_t> v(n, 1)` gets vectorized, but `vector<float> v(n, 1)`. It looks like the compiler assumes the state is leaked / aliased in the latter case (unclear how/why for float, but not for int32), and because of this fails to see vectorization optimization?

See https://gcc.godbolt.org/z/UWhiie

By using a local `__new_end_` (fixed), and local `__pos` (copied into __tx.__pos_ per iteration), we offer the compiler a clean loop for unrolling.

A demonstration can be seen in the isolated logic in https://gcc.godbolt.org/z/KoCNWv

The com

Reviewers: EricWF, #libc!

Subscribers: libcxx-commits

Tags: #libc

Differential Revision: https://reviews.llvm.org/D82111

Added: 
    

Modified: 
    libcxx/include/vector

Removed: 
    


################################################################################
diff  --git a/libcxx/include/vector b/libcxx/include/vector
index 7d0fec87c4fa..1007beeaafd0 100644
--- a/libcxx/include/vector
+++ b/libcxx/include/vector
@@ -1043,8 +1043,9 @@ void
 vector<_Tp, _Allocator>::__construct_at_end(size_type __n)
 {
     _ConstructTransaction __tx(*this, __n);
-    for (; __tx.__pos_ != __tx.__new_end_; ++__tx.__pos_) {
-      __alloc_traits::construct(this->__alloc(), _VSTD::__to_address(__tx.__pos_));
+    const_pointer __new_end = __tx.__new_end_;
+    for (pointer __pos = __tx.__pos_; __pos != __new_end; ++__pos, __tx.__pos_ = __pos) {
+        __alloc_traits::construct(this->__alloc(), _VSTD::__to_address(__pos));
     }
 }
 
@@ -1060,8 +1061,9 @@ void
 vector<_Tp, _Allocator>::__construct_at_end(size_type __n, const_reference __x)
 {
     _ConstructTransaction __tx(*this, __n);
-    for (; __tx.__pos_ != __tx.__new_end_; ++__tx.__pos_) {
-        __alloc_traits::construct(this->__alloc(), _VSTD::__to_address(__tx.__pos_), __x);
+    const_pointer __new_end = __tx.__new_end_;
+    for (pointer __pos = __tx.__pos_; __pos != __new_end; ++__pos, __tx.__pos_ = __pos) {
+        __alloc_traits::construct(this->__alloc(), _VSTD::__to_address(__pos), __x);
     }
 }
 
@@ -1753,9 +1755,10 @@ vector<_Tp, _Allocator>::__move_range(pointer __from_s, pointer __from_e, pointe
     {
       pointer __i = __from_s + __n;
       _ConstructTransaction __tx(*this, __from_e - __i);
-      for (; __i < __from_e; ++__i, ++__tx.__pos_) {
+      for (pointer __pos = __tx.__pos_; __i < __from_e;
+           ++__i, ++__pos, __tx.__pos_ = __pos) {
           __alloc_traits::construct(this->__alloc(),
-                                    _VSTD::__to_address(__tx.__pos_),
+                                    _VSTD::__to_address(__pos),
                                     _VSTD::move(*__i));
       }
     }