[compiler-rt] 614b8ba - [libFuzzer] Deflake entropic exec-time test.

[Matt Morehouse via llvm-commits]

Author: Dokyung Song
Date: 2021-11-10T07:43:21-08:00
New Revision: 614b8bac01f6e9b877bd9a3a379c3bb8074f1f4a

URL: https://github.com/llvm/llvm-project/commit/614b8bac01f6e9b877bd9a3a379c3bb8074f1f4a
DIFF: https://github.com/llvm/llvm-project/commit/614b8bac01f6e9b877bd9a3a379c3bb8074f1f4a.diff

LOG: [libFuzzer] Deflake entropic exec-time test.

Entropic scheduling with exec-time option can be misled, if inputs
on the right path to become crashing inputs accidentally take more
time to execute before it's added to the corpus. This patch, by letting
more of such inputs added to the corpus (four inputs of size 7 to 10,
instead of a single input of size 2), reduces possibilities of being
influenced by timing flakiness.

A longer-term fix could be to reduce timing flakiness in the fuzzer;
one way could be to execute inputs multiple times and take average of
their execution time before they are added to the corpus.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D113544

Added: 
    

Modified: 
    compiler-rt/test/fuzzer/EntropicScalePerExecTimeTest.cpp

Removed: 
    


################################################################################
diff  --git a/compiler-rt/test/fuzzer/EntropicScalePerExecTimeTest.cpp b/compiler-rt/test/fuzzer/EntropicScalePerExecTimeTest.cpp
index 0cab7b7035f95..44d58a0874539 100644
--- a/compiler-rt/test/fuzzer/EntropicScalePerExecTimeTest.cpp
+++ b/compiler-rt/test/fuzzer/EntropicScalePerExecTimeTest.cpp
@@ -3,11 +3,11 @@
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 
 // Tests whether scaling the Entropic scheduling weight based on input execution
-// time is effective or not. Inputs of size 10 will take at least 100
-// microseconds more than any input of size 1-9. The input of size 2 in the
-// corpus should be favored by the exec-time-scaled Entropic scheduling policy
-// than the input of size 10 in the corpus, eventually finding the crashing
-// input {0xab, 0xcd} with less executions.
+// time is effective or not. Inputs of size less than 7 will take at least 100
+// microseconds more than inputs of size greater than or equal to 7. Inputs of
+// size greater than 7 in the corpus should be favored by the exec-time-scaled
+// Entropic scheduling policy than the input of size less than 7 in the corpus,
+// eventually finding the crashing input with less executions.
 #include <chrono>
 #include <cstdint>
 #include <thread>
@@ -19,17 +19,32 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
   if (Size > 10)
     return 0; // To make the test quicker.
 
-  if (Size != 2) {
+  if (Size < 7) {
     // execute a lot slower than the crashing input below.
     size_t ExecTimeUSec = 100;
     std::this_thread::sleep_for(std::chrono::microseconds(ExecTimeUSec));
+    Sink = 7;
+
     if (Size > 0 && Data[0] == 0xaa && Size > 1 && Data[1] == 0xbb &&
         Size > 2 && Data[2] == 0xcc && Size > 3 && Data[3] == 0xdd &&
         Size > 4 && Data[4] == 0xee && Size > 5 && Data[5] == 0xff)
       Sink += 7;
   }
 
-  if (Size == 2 && Data[0] == 0xab && Data[1] == 0xcd)
+  // Give unique coverage for each input of size (7, 8, 9, 10)
+  if (Size == 7)
+    Sink = -7;
+
+  if (Size == 8)
+    Sink = -8;
+
+  if (Size == 9)
+    Sink = -9;
+
+  if (Size == 10)
+    Sink = -10;
+
+  if (Sink < 0 && Data[0] == 0xab && Data[1] == 0xcd)
     *Nil = 42; // crash.
 
   return 0;