[PATCH] D70288: [ADT][Expensive checks] Create a /dev/random seed only once when shuffling the range before sorting it to avoid bot timeouts

[Alex Lorenz via Phabricator via llvm-commits]

arphaman created this revision.
arphaman added reviewers: dexonsmith, RKSimon, mgrang.
Herald added subscribers: ributzka, jkorous.
Herald added a project: LLVM.

The expensive checks enabled bot on green dragon (http://lab.llvm.org:8080/green/job/clang-stage1-cmake-RA-expensive/) has been failing to pass the `libFuzzer :: large.test` ever since the test was enabled on Darwin (i.e. REQUIRES: linux was removed). It's failing because it's timing out, and it's timing out because the compiler is too slow. I sampled the clang invocations and was shocked to see that the majority of the time was spent in calls to `std::random_device` inside of `llvm::sort`.

This patch changes the expensive checking behavior to compute a single `std::random_device` shuffle seed, per an invocation of clang, rather than computing it for every sort. This fixes the test failure on the bot. On my machine, this improved the performance of building compiler-rt with clang that has expensive checks enabled by 50-100x!


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D70288

Files:
  llvm/include/llvm/ADT/STLExtras.h


Index: llvm/include/llvm/ADT/STLExtras.h
===================================================================
--- llvm/include/llvm/ADT/STLExtras.h
+++ llvm/include/llvm/ADT/STLExtras.h
@@ -1053,6 +1053,23 @@
   return array_pod_sort_comparator<T>;
 }
 
+#ifdef EXPENSIVE_CHECKS
+namespace detail {
+
+inline unsigned presortShuffleEntropy() {
+  static unsigned Result(std::random_device{}());
+  return Result;
+}
+
+template <class IteratorTy>
+inline void presortShuffle(IteratorTy Start, IteratorTy End) {
+  std::mt19937 Generator(presortShuffleEntropy());
+  std::shuffle(Start, End, Generator);
+}
+
+} // end namespace detail
+#endif
+
 /// array_pod_sort - This sorts an array with the specified start and end
 /// extent.  This is just like std::sort, except that it calls qsort instead of
 /// using an inlined template.  qsort is slightly slower than std::sort, but
@@ -1074,8 +1091,7 @@
   auto NElts = End - Start;
   if (NElts <= 1) return;
 #ifdef EXPENSIVE_CHECKS
-  std::mt19937 Generator(std::random_device{}());
-  std::shuffle(Start, End, Generator);
+  detail::presortShuffle<IteratorTy>(Start, End);
 #endif
   qsort(&*Start, NElts, sizeof(*Start), get_array_pod_sort_comparator(*Start));
 }
@@ -1091,8 +1107,7 @@
   auto NElts = End - Start;
   if (NElts <= 1) return;
 #ifdef EXPENSIVE_CHECKS
-  std::mt19937 Generator(std::random_device{}());
-  std::shuffle(Start, End, Generator);
+  detail::presortShuffle<IteratorTy>(Start, End);
 #endif
   qsort(&*Start, NElts, sizeof(*Start),
         reinterpret_cast<int (*)(const void *, const void *)>(Compare));
@@ -1103,8 +1118,7 @@
 template <typename IteratorTy>
 inline void sort(IteratorTy Start, IteratorTy End) {
 #ifdef EXPENSIVE_CHECKS
-  std::mt19937 Generator(std::random_device{}());
-  std::shuffle(Start, End, Generator);
+  detail::presortShuffle<IteratorTy>(Start, End);
 #endif
   std::sort(Start, End);
 }
@@ -1116,8 +1130,7 @@
 template <typename IteratorTy, typename Compare>
 inline void sort(IteratorTy Start, IteratorTy End, Compare Comp) {
 #ifdef EXPENSIVE_CHECKS
-  std::mt19937 Generator(std::random_device{}());
-  std::shuffle(Start, End, Generator);
+  detail::presortShuffle<IteratorTy>(Start, End);
 #endif
   std::sort(Start, End, Comp);
 }


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D70288.229439.patch
Type: text/x-patch
Size: 2245 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20191115/0b68c9db/attachment.bin>