[PATCH] D138919: llvm-reduce: Fix tsan failures

[Matt Arsenault via Phabricator via llvm-commits]

arsenm created this revision.
arsenm added reviewers: fhahn, aeubanks, regehr, lebedev.ri, Meinersbur.
Herald added a project: All.
arsenm requested review of this revision.
Herald added a subscriber: wdng.
Herald added a project: LLVM.

There's a data race on the UninterestingChunks set. The code seems to
be operating on the assumption that all the tasks completed, so ensure
the unused results do complete. This started showing up about 50% of
the time when running operands-skip-parallel.ll after the recent
switch to use DenseSet; previously it failed much less frequently with
std::set.

      

We should introduce a mechanism to early terminate unused
results. Alternatively, I've been thinking about ways to to make the
reduction order smarter. I frequently have tests that take multiple
minutes to compile and hit the failure. It may be helpful to see which
chunks took the least time and prefer those over just taking the first
result.


https://reviews.llvm.org/D138919

Files:
  llvm/tools/llvm-reduce/deltas/Delta.cpp


Index: llvm/tools/llvm-reduce/deltas/Delta.cpp
===================================================================
--- llvm/tools/llvm-reduce/deltas/Delta.cpp
+++ llvm/tools/llvm-reduce/deltas/Delta.cpp
@@ -210,6 +210,16 @@
   return Result;
 }
 
+using SharedTaskQueue = std::deque<std::shared_future<SmallString<0>>>;
+
+static void waitAndDiscardResultsBarrier(SharedTaskQueue &TaskQueue) {
+  while (!TaskQueue.empty()) {
+    auto &Future = TaskQueue.front();
+    Future.wait();
+    TaskQueue.pop_front();
+  }
+}
+
 /// Runs the Delta Debugging algorithm, splits the code into chunks and
 /// reduces the amount of chunks that are considered interesting by the
 /// given test. The number of chunks is determined by a preliminary run of the
@@ -280,7 +290,7 @@
       writeBitcode(Test.getProgram(), BCOS);
     }
 
-    std::deque<std::shared_future<SmallString<0>>> TaskQueue;
+    SharedTaskQueue TaskQueue;
     for (auto I = ChunksStillConsideredInteresting.rbegin(),
               E = ChunksStillConsideredInteresting.rend();
          I != E; ++I) {
@@ -350,6 +360,14 @@
                       Test.getToolName());
           break;
         }
+
+        // If we broke out of the loop, we still need to wait for everything to
+        // avoid race access to the chunk set.
+        //
+        // TODO: Create a way to remaining items we're ignoring; they could take
+        // a long time.
+        waitAndDiscardResultsBarrier(TaskQueue);
+
         // Forward I to the last chunk processed in parallel.
         I += NumChunksProcessed - 1;
       } else {


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D138919.478563.patch
Type: text/x-patch
Size: 1579 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20221129/4bf36712/attachment.bin>