[llvm] [DevPolicy] Add guidelines for fuzzer-generated issue reports (PR #112372)

Philip Reames via llvm-commits llvm-commits at lists.llvm.org
Tue Oct 29 14:54:47 PDT 2024


================
@@ -853,6 +853,69 @@ their patch with every possible configuration.
 * 3rd step: If Galina could not help you, please escalate to the
   `Infrastructure Working Group <mailto:iwg at llvm.org>`_.
 
+Guidelines for fuzzer-generated issues
+--------------------------------------
+
+Fuzzing is a valuable tool for finding compiler bugs, and the LLVM project
+welcomes fuzzer-generated test cases. However, some additional guidelines
+should be followed to make such reports maximally useful.
+
+Fuzzer-generated issues should indicate that they are such, either in the
+issue description, or (for organization members) by applying the
+``fuzzer-generated`` label.
+
+Issues should include a minimized reproducer (including both the necessary code
+and command line arguments) both as part of the issue description and as a
+godbolt.org link. An effort should be made to deduplicate issues that likely
+have the same root cause, and check whether a similar issue has already been
+reported. Reports should always be submitted against current LLVM ``main``,
+not a released version.
+
+If possible, provide information on when an issue was introduced (e.g. by
+checking older versions on godbolt). A regression from the last LLVM release
+has higher priority than an issue that existed for decades.
+
+The remaining guidelines depend on the type of issue the fuzzer detects.
----------------
preames wrote:

Could you add something like this summary comment that appears to address many of your specific points below?  You could leave the details if you want, but the high level point seems useful as well.  

The person filing an issue is responsible for justifying why the reported behavior is in fact wrong.  In many cases, this justification is distinctly non-trivial.  For fuzzer campaigns, our expectations are higher than for average user reported bugs, and you may need to either structure a fuzzer campaign carefully to ensure valid bugs or spend a bunch of time analyzing each before filing.  

https://github.com/llvm/llvm-project/pull/112372


More information about the llvm-commits mailing list