[llvm] [DevPolicy] Add guidelines for fuzzer-generated issue reports (PR #112372)

Philip Reames via llvm-commits llvm-commits at lists.llvm.org
Tue Oct 29 14:54:47 PDT 2024


https://github.com/preames commented:

I think I understand the intent here, but I worry this is too negative framed.  I've been involved with several fuzzing efforts focused on regression tracking, and believe they've been hugely beneficial to the project.

At a macro level, I'd suggest a framing something along the following:

Fuzzing is useful when it can identify small actionable test cases, but can also be harmful if the volume of issues reported costs more to analyze than the benefit obtained from fixes.  For this reason, we ask anyone fuzzing LLVM to focus on ensuring that issue filed are small and actionable.  The most important metric to monitor is what fraction of issues filed get promptly fixed (not "closed", "fixed"). 

Another idea might be to reframe this from fuzzer specific rules to rules which apply to anyone filing a large number of bugs.  Most of the specific advice given would seem to apply to e.g. a downstream test suite with regressions being reported upstream.  

https://github.com/llvm/llvm-project/pull/112372


More information about the llvm-commits mailing list