[PATCH] D42414: [llvm-opt-fuzzer] Avoid adding incorrect inputs to the fuzzer corpus

[Igor Laevsky via Phabricator via llvm-commits]

igor-laevsky added a comment.

With regards to the performance concerns - I will keep an eye on the fuzzer statistics and revert the change if the slowdown becomes too much,



================
Comment at: tools/llvm-opt-fuzzer/llvm-opt-fuzzer.cpp:68-73
   if (verifyModule(*M, &errs())) {
     errs() << "mutation result doesn't pass verification\n";
     M->dump();
-    abort();
+    // Avoid adding incorrect test cases to the corpus.
+    return 0;
+  }
----------------
bogner wrote:
> igor-laevsky wrote:
> > bogner wrote:
> > > Can we drop this part and only verify after the reload?
> > I'm not sure how bitcode writer will behave on the invalid module. I would suspect that this verification is fast compared to the save/reload part. 
> Are you set up to measure this? It'd be nice to qualify that claim.
I based it mainly on the fact that verification is faster than save/reload + verification. In any case I don't think there is a way to avoid the first verification since bitcode writer may be unfriendly to the invalid modules.


================
Comment at: tools/llvm-opt-fuzzer/llvm-opt-fuzzer.cpp:83-85
+  // There are some invariants which are not checked by the verifier in favor
+  // of having them checked by the parser. They may be considered as bugs in the
+  // verifier and should be fixed there. However until all of those are covered
----------------
bogner wrote:
> Interestingly, it would be trivial to write a fuzzer that finds these verifier bugs at this point. Maybe we should do that as a way to move towards a state where we can remove the redundant checks?
Yes, that's an interesting idea. Actually we can even write more general fuzzer for the FuzzMutate itself which will cover this kinds of issues and all other possible bugs in the FuzzMutate.


Repository:
  rL LLVM

https://reviews.llvm.org/D42414