[llvm] r227252 - Add a Fuzzer library
Kostya Serebryany
kcc at google.com
Thu Jan 29 08:31:59 PST 2015
On Thu, Jan 29, 2015 at 8:28 AM, Aaron Ballman <aaron at aaronballman.com>
wrote:
> On Thu, Jan 29, 2015 at 11:27 AM, Kostya Serebryany <kcc at google.com>
> wrote:
> >
> >
> > On Thu, Jan 29, 2015 at 8:18 AM, Aaron Ballman <aaron at aaronballman.com>
> > wrote:
> >>
> >> On Thu, Jan 29, 2015 at 11:11 AM, Kostya Serebryany <kcc at google.com>
> >> wrote:
> >> >
> >> >
> >> > On Thu, Jan 29, 2015 at 7:28 AM, Aaron Ballman <
> aaron at aaronballman.com>
> >> > wrote:
> >> >>
> >> >> Actually, this is completely broken on Windows and should be reverted
> >> >> until it is fixed up. Comments below.
> >> >
> >> >
> >> >
> >> >
> >> >>
> >> >>
> >> >> On Thu, Jan 29, 2015 at 10:11 AM, Aaron Ballman
> >> >> <aaron at aaronballman.com>
> >> >> wrote:
> >> >> > This commit puts the fuzzer targets on the top-level for MSVC
> >> >> > solutions when they should be in a separate folder instead. Picture
> >> >> > attached is worth 1000 words. ;-)
> >> >> >
> >> >> > ~Aaron
> >> >> >
> >> >> > On Tue, Jan 27, 2015 at 5:08 PM, Kostya Serebryany <kcc at google.com
> >
> >> >> > wrote:
> >> >> >> Author: kcc
> >> >> >> Date: Tue Jan 27 16:08:41 2015
> >> >> >> New Revision: 227252
> >> >> >>
> >> >> >> URL: http://llvm.org/viewvc/llvm-project?rev=227252&view=rev
> >> >> >> Log:
> >> >> >> Add a Fuzzer library
> >> >> >>
> >> >> >> Summary:
> >> >> >> A simple genetic in-process coverage-guided fuzz testing library.
> >> >> >>
> >> >> >> I've used this fuzzer to test clang-format
> >> >> >> (it found 12+ bugs, thanks djasper@ for the fixes!)
> >> >> >> and it may also help us test other parts of LLVM.
> >> >> >> So why not keep it in the LLVM repository?
> >> >> >>
> >> >> >> I plan to add the cmake build rules later (in a separate patch, if
> >> >> >> that's ok)
> >> >> >> and also add a clang-format-fuzzer target.
> >> >> >>
> >> >> >> See README.txt for details.
> >> >> >>
> >> >> >> Test Plan: Tests will follow separately.
> >> >> >>
> >> >> >> Reviewers: djasper, chandlerc, rnk
> >> >> >>
> >> >> >> Reviewed By: rnk
> >> >> >>
> >> >> >> Subscribers: majnemer, ygribov, dblaikie, llvm-commits
> >> >> >>
> >> >> >> Differential Revision: http://reviews.llvm.org/D7184
> >> >> >>
> >> >> >> Added:
> >> >> >> llvm/trunk/lib/Fuzzer/
> >> >> >> llvm/trunk/lib/Fuzzer/CMakeLists.txt
> >> >> >> llvm/trunk/lib/Fuzzer/FuzzerCrossOver.cpp
> >> >> >> llvm/trunk/lib/Fuzzer/FuzzerFlags.def
> >> >> >> llvm/trunk/lib/Fuzzer/FuzzerIO.cpp
> >> >> >> llvm/trunk/lib/Fuzzer/FuzzerInternal.h
> >> >> >> llvm/trunk/lib/Fuzzer/FuzzerLoop.cpp
> >> >> >> llvm/trunk/lib/Fuzzer/FuzzerMain.cpp
> >> >> >> llvm/trunk/lib/Fuzzer/FuzzerMutate.cpp
> >> >> >> llvm/trunk/lib/Fuzzer/FuzzerUtil.cpp
> >> >> >> llvm/trunk/lib/Fuzzer/README.txt
> >> >> >> llvm/trunk/lib/Fuzzer/test/
> >> >> >> llvm/trunk/lib/Fuzzer/test/ExactTest.cpp
> >> >> >> llvm/trunk/lib/Fuzzer/test/InfiniteTest.cpp
> >> >> >> llvm/trunk/lib/Fuzzer/test/NullDerefTest.cpp
> >> >> >> llvm/trunk/lib/Fuzzer/test/SimpleTest.cpp
> >> >> >> llvm/trunk/lib/Fuzzer/test/TestFuzzerCrossOver.cpp
> >> >> >> llvm/trunk/lib/Fuzzer/test/TimeoutTest.cpp
> >> >> >> Modified:
> >> >> >> llvm/trunk/lib/CMakeLists.txt
> >> >> >>
> >> >> >> Modified: llvm/trunk/lib/CMakeLists.txt
> >> >> >> URL:
> >> >> >>
> >> >> >>
> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CMakeLists.txt?rev=227252&r1=227251&r2=227252&view=diff
> >> >> >>
> >> >> >>
> >> >> >>
> ==============================================================================
> >> >> >> --- llvm/trunk/lib/CMakeLists.txt (original)
> >> >> >> +++ llvm/trunk/lib/CMakeLists.txt Tue Jan 27 16:08:41 2015
> >> >> >> @@ -17,3 +17,4 @@ add_subdirectory(Target)
> >> >> >> add_subdirectory(AsmParser)
> >> >> >> add_subdirectory(LineEditor)
> >> >> >> add_subdirectory(ProfileData)
> >> >> >> +add_subdirectory(Fuzzer)
> >> >> >>
> >> >> >> Added: llvm/trunk/lib/Fuzzer/CMakeLists.txt
> >> >> >> URL:
> >> >> >>
> >> >> >>
> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/CMakeLists.txt?rev=227252&view=auto
> >> >> >>
> >> >> >>
> >> >> >>
> ==============================================================================
> >> >> >> --- llvm/trunk/lib/Fuzzer/CMakeLists.txt (added)
> >> >> >> +++ llvm/trunk/lib/Fuzzer/CMakeLists.txt Tue Jan 27 16:08:41 2015
> >> >> >> @@ -0,0 +1,9 @@
> >> >> >> +add_library(LLVMFuzzer STATIC
> >> >> >> + EXCLUDE_FROM_ALL # Do not build if you are not building
> fuzzers.
> >> >> >> + FuzzerCrossOver.cpp
> >> >> >> + FuzzerIO.cpp
> >> >> >> + FuzzerLoop.cpp
> >> >> >> + FuzzerMain.cpp
> >> >> >> + FuzzerMutate.cpp
> >> >> >> + FuzzerUtil.cpp
> >> >> >> + )
> >> >> >>
> >> >> >> Added: llvm/trunk/lib/Fuzzer/FuzzerCrossOver.cpp
> >> >> >> URL:
> >> >> >>
> >> >> >>
> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerCrossOver.cpp?rev=227252&view=auto
> >> >> >>
> >> >> >>
> >> >> >>
> ==============================================================================
> >> >> >> --- llvm/trunk/lib/Fuzzer/FuzzerCrossOver.cpp (added)
> >> >> >> +++ llvm/trunk/lib/Fuzzer/FuzzerCrossOver.cpp Tue Jan 27 16:08:41
> >> >> >> 2015
> >> >> >> @@ -0,0 +1,46 @@
> >> >> >> +//===- FuzzerCrossOver.cpp - Cross over two test inputs
> >> >> >> -------------------===//
> >> >> >> +//
> >> >> >> +// The LLVM Compiler Infrastructure
> >> >> >> +//
> >> >> >> +// This file is distributed under the University of Illinois Open
> >> >> >> Source
> >> >> >> +// License. See LICENSE.TXT for details.
> >> >> >> +//
> >> >> >>
> >> >> >>
> >> >> >>
> +//===----------------------------------------------------------------------===//
> >> >> >> +// Cross over test inputs.
> >> >> >>
> >> >> >>
> >> >> >>
> +//===----------------------------------------------------------------------===//
> >> >> >> +
> >> >> >> +#include "FuzzerInternal.h"
> >> >> >> +
> >> >> >> +namespace fuzzer {
> >> >> >> +
> >> >> >> +// Cross A and B, store the result (ap to MaxLen bytes) in U.
> >> >> >> +void CrossOver(const Unit &A, const Unit &B, Unit *U, size_t
> >> >> >> MaxLen) {
> >> >> >> + size_t Size = rand() % MaxLen + 1;
> >> >> >> + U->clear();
> >> >> >> + const Unit *V = &A;
> >> >> >> + size_t PosA = 0;
> >> >> >> + size_t PosB = 0;
> >> >> >> + size_t *Pos = &PosA;
> >> >> >> + while (U->size() < Size && (PosA < A.size() || PosB <
> B.size()))
> >> >> >> {
> >> >> >> + // Merge a part of V into U.
> >> >> >> + size_t SizeLeftU = Size - U->size();
> >> >> >> + if (*Pos < V->size()) {
> >> >> >> + size_t SizeLeftV = V->size() - *Pos;
> >> >> >> + size_t MaxExtraSize = std::min(SizeLeftU, SizeLeftV);
> >> >> >> + size_t ExtraSize = rand() % MaxExtraSize + 1;
> >> >> >> + U->insert(U->end(), V->begin() + *Pos, V->begin() + *Pos +
> >> >> >> ExtraSize);
> >> >> >> + (*Pos) += ExtraSize;
> >> >> >> + }
> >> >> >> +
> >> >> >> + // Use the other Unit on the next iteration.
> >> >> >> + if (Pos == &PosA) {
> >> >> >> + Pos = &PosB;
> >> >> >> + V = &B;
> >> >> >> + } else {
> >> >> >> + Pos = &PosA;
> >> >> >> + V = &A;
> >> >> >> + }
> >> >> >> + }
> >> >> >> +}
> >> >> >> +
> >> >> >> +} // namespace fuzzer
> >> >> >>
> >> >> >> Added: llvm/trunk/lib/Fuzzer/FuzzerFlags.def
> >> >> >> URL:
> >> >> >>
> >> >> >>
> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerFlags.def?rev=227252&view=auto
> >> >> >>
> >> >> >>
> >> >> >>
> ==============================================================================
> >> >> >> --- llvm/trunk/lib/Fuzzer/FuzzerFlags.def (added)
> >> >> >> +++ llvm/trunk/lib/Fuzzer/FuzzerFlags.def Tue Jan 27 16:08:41 2015
> >> >> >> @@ -0,0 +1,24 @@
> >> >> >> +//===- FuzzerFlags.def - Run-time flags
> -------------------------*-
> >> >> >> C++ -* ===//
> >> >> >> +//
> >> >> >> +// The LLVM Compiler Infrastructure
> >> >> >> +//
> >> >> >> +// This file is distributed under the University of Illinois Open
> >> >> >> Source
> >> >> >> +// License. See LICENSE.TXT for details.
> >> >> >> +//
> >> >> >>
> >> >> >>
> >> >> >>
> +//===----------------------------------------------------------------------===//
> >> >> >> +// Flags. FUZZER_FLAG macro should be defined at the point of
> >> >> >> inclusion.
> >> >> >> +// We are not using any flag parsing library for better
> portability
> >> >> >> and
> >> >> >> +// independence.
> >> >> >>
> >> >> >>
> >> >> >>
> +//===----------------------------------------------------------------------===//
> >> >> >> +FUZZER_FLAG(int, verbosity, 1, "Verbosity level.")
> >> >> >> +FUZZER_FLAG(int, seed, 0, "Random seed. If 0, seed is
> generated.")
> >> >> >> +FUZZER_FLAG(int, iterations, -1,
> >> >> >> + "Number of iterations of the fuzzer (-1 for infinite
> >> >> >> runs).")
> >> >> >> +FUZZER_FLAG(int, max_len, 64, "Maximal length of the test
> input.")
> >> >> >> +FUZZER_FLAG(int, cross_over, 1, "If 1, cross over inputs.")
> >> >> >> +FUZZER_FLAG(int, mutate_depth, 10,
> >> >> >> + "Apply this number of consecutive mutations to each
> >> >> >> input.")
> >> >> >> +FUZZER_FLAG(int, exit_on_first, 0,
> >> >> >> + "If 1, exit after the first new interesting input is
> >> >> >> found.")
> >> >> >> +FUZZER_FLAG(int, timeout, -1, "Timeout in seconds (if
> positive).")
> >> >> >> +FUZZER_FLAG(int, help, 0, "Print help.")
> >> >> >>
> >> >> >> Added: llvm/trunk/lib/Fuzzer/FuzzerIO.cpp
> >> >> >> URL:
> >> >> >>
> >> >> >>
> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerIO.cpp?rev=227252&view=auto
> >> >> >>
> >> >> >>
> >> >> >>
> ==============================================================================
> >> >> >> --- llvm/trunk/lib/Fuzzer/FuzzerIO.cpp (added)
> >> >> >> +++ llvm/trunk/lib/Fuzzer/FuzzerIO.cpp Tue Jan 27 16:08:41 2015
> >> >> >> @@ -0,0 +1,44 @@
> >> >> >> +//===- FuzzerIO.cpp - IO utils.
> >> >> >> -------------------------------------------===//
> >> >> >> +//
> >> >> >> +// The LLVM Compiler Infrastructure
> >> >> >> +//
> >> >> >> +// This file is distributed under the University of Illinois Open
> >> >> >> Source
> >> >> >> +// License. See LICENSE.TXT for details.
> >> >> >> +//
> >> >> >>
> >> >> >>
> >> >> >>
> +//===----------------------------------------------------------------------===//
> >> >> >> +// IO functions.
> >> >> >>
> >> >> >>
> >> >> >>
> +//===----------------------------------------------------------------------===//
> >> >> >> +#include "FuzzerInternal.h"
> >> >> >> +#include <fstream>
> >> >> >> +#include <dirent.h>
> >> >>
> >> >> dirent.h does not exist on Windows with MSVC.
> >> >>
> >> >> >> +namespace fuzzer {
> >> >> >> +
> >> >> >> +std::vector<std::string> ListFilesInDir(const std::string &Dir) {
> >> >> >> + std::vector<std::string> V;
> >> >> >> + DIR *D = opendir(Dir.c_str());
> >> >> >> + if (!D) return V;
> >> >> >> + while (auto E = readdir(D)) {
> >> >> >> + if (E->d_type == DT_REG || E->d_type == DT_LNK)
> >> >> >> + V.push_back(E->d_name);
> >> >> >> + }
> >> >> >> + closedir(D);
> >> >> >> + return V;
> >> >>
> >> >> We have filesystem APIs to do this, don't we?
> >> >>
> >> >
> >> > As the README.txt explains, this library can not use any of the great
> >> > LLVM
> >> > support.
> >> > It has to remain independent. I amy even have to remove the use of STL
> >> > from
> >> > it later.
> >> >
> >> >>
> >> >> >> +}
> >> >> >> +
> >> >> >> +Unit FileToVector(const std::string &Path) {
> >> >> >> + std::ifstream T(Path);
> >> >> >> + return Unit((std::istreambuf_iterator<char>(T)),
> >> >> >> + std::istreambuf_iterator<char>());
> >> >> >> +}
> >> >> >> +
> >> >> >> +void WriteToFile(const Unit &U, const std::string &Path) {
> >> >> >> + std::ofstream OF(Path);
> >> >> >> + OF.write((const char*)U.data(), U.size());
> >> >> >> +}
> >> >> >> +
> >> >> >> +void ReadDirToVectorOfUnits(const char *Path, std::vector<Unit>
> *V)
> >> >> >> {
> >> >> >> + for (auto &X : ListFilesInDir(Path))
> >> >> >> + V->push_back(FileToVector(std::string(Path) + "/" + X));
> >> >> >> +}
> >> >> >> +
> >> >> >> +} // namespace fuzzer
> >> >> >>
> >> >> >> Added: llvm/trunk/lib/Fuzzer/FuzzerInternal.h
> >> >> >> URL:
> >> >> >>
> >> >> >>
> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerInternal.h?rev=227252&view=auto
> >> >> >>
> >> >> >>
> >> >> >>
> ==============================================================================
> >> >> >> --- llvm/trunk/lib/Fuzzer/FuzzerInternal.h (added)
> >> >> >> +++ llvm/trunk/lib/Fuzzer/FuzzerInternal.h Tue Jan 27 16:08:41
> 2015
> >> >> >> @@ -0,0 +1,77 @@
> >> >> >> +//===- FuzzerInternal.h - Internal header for the Fuzzer
> --------*-
> >> >> >> C++ -* ===//
> >> >> >> +//
> >> >> >> +// The LLVM Compiler Infrastructure
> >> >> >> +//
> >> >> >> +// This file is distributed under the University of Illinois Open
> >> >> >> Source
> >> >> >> +// License. See LICENSE.TXT for details.
> >> >> >> +//
> >> >> >>
> >> >> >>
> >> >> >>
> +//===----------------------------------------------------------------------===//
> >> >> >> +// Define the main class fuzzer::Fuzzer and most functions.
> >> >> >>
> >> >> >>
> >> >> >>
> +//===----------------------------------------------------------------------===//
> >> >> >> +#include <cassert>
> >> >> >> +#include <chrono>
> >> >> >> +#include <cstddef>
> >> >> >> +#include <cstdlib>
> >> >> >> +#include <string>
> >> >> >> +#include <vector>
> >> >> >> +
> >> >> >> +namespace fuzzer {
> >> >> >> +typedef std::vector<uint8_t> Unit;
> >> >> >> +using namespace std::chrono;
> >> >> >> +
> >> >> >> +Unit ReadFile(const char *Path);
> >> >> >> +std::vector<std::string> ListFilesInDir(const std::string &Dir);
> >> >> >> +void ReadDirToVectorOfUnits(const char *Path, std::vector<Unit>
> >> >> >> *V);
> >> >> >> +void WriteToFile(const Unit &U, const std::string &Path);
> >> >> >> +
> >> >> >> +void Mutate(Unit *U, size_t MaxLen);
> >> >> >> +
> >> >> >> +void CrossOver(const Unit &A, const Unit &B, Unit *U, size_t
> >> >> >> MaxLen);
> >> >> >> +
> >> >> >> +void Print(const Unit &U, const char *PrintAfter = "");
> >> >> >> +void PrintASCII(const Unit &U, const char *PrintAfter = "");
> >> >> >> +std::string Hash(const Unit &U);
> >> >> >> +void SetTimer(int Seconds);
> >> >> >> +
> >> >> >> +class Fuzzer {
> >> >> >> + public:
> >> >> >> + struct FuzzingOptions {
> >> >> >> + int Verbosity = 1;
> >> >> >> + int MaxLen = 0;
> >> >> >> + bool DoCrossOver = true;
> >> >> >> + bool MutateDepth = 10;
> >> >> >> + bool ExitOnFirst = false;
> >> >> >> + std::string OutputCorpus;
> >> >> >> + };
> >> >> >> + Fuzzer(FuzzingOptions Options) : Options(Options) {
> >> >> >> + SetDeathCallback();
> >> >> >> + }
> >> >> >> + void AddToCorpus(const Unit &U) { Corpus.push_back(U); }
> >> >> >> + size_t Loop(size_t NumIterations);
> >> >> >> + void ShuffleAndMinimize();
> >> >> >> + size_t CorpusSize() const { return Corpus.size(); }
> >> >> >> + void ReadDir(const std::string &Path) {
> >> >> >> + ReadDirToVectorOfUnits(Path.c_str(), &Corpus);
> >> >> >> + }
> >> >> >> +
> >> >> >> + static void AlarmCallback();
> >> >> >> +
> >> >> >> + private:
> >> >> >> + size_t MutateAndTestOne(Unit *U);
> >> >> >> + size_t RunOne(const Unit &U);
> >> >> >> + void WriteToOutputCorpus(const Unit &U);
> >> >> >> + static void WriteToCrash(const Unit &U, const char *Prefix);
> >> >> >> +
> >> >> >> + void SetDeathCallback();
> >> >> >> + static void DeathCallback();
> >> >> >> + static Unit CurrentUnit;
> >> >> >> +
> >> >> >> + size_t TotalNumberOfRuns = 0;
> >> >> >> +
> >> >> >> + std::vector<Unit> Corpus;
> >> >> >> + FuzzingOptions Options;
> >> >> >> + system_clock::time_point ProcessStartTime =
> system_clock::now();
> >> >> >> + static system_clock::time_point UnitStartTime;
> >> >> >> +};
> >> >> >> +
> >> >> >> +}; // namespace fuzzer
> >> >> >>
> >> >> >> Added: llvm/trunk/lib/Fuzzer/FuzzerLoop.cpp
> >> >> >> URL:
> >> >> >>
> >> >> >>
> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerLoop.cpp?rev=227252&view=auto
> >> >> >>
> >> >> >>
> >> >> >>
> ==============================================================================
> >> >> >> --- llvm/trunk/lib/Fuzzer/FuzzerLoop.cpp (added)
> >> >> >> +++ llvm/trunk/lib/Fuzzer/FuzzerLoop.cpp Tue Jan 27 16:08:41 2015
> >> >> >> @@ -0,0 +1,161 @@
> >> >> >> +//===- FuzzerLoop.cpp - Fuzzer's main loop
> >> >> >> --------------------------------===//
> >> >> >> +//
> >> >> >> +// The LLVM Compiler Infrastructure
> >> >> >> +//
> >> >> >> +// This file is distributed under the University of Illinois Open
> >> >> >> Source
> >> >> >> +// License. See LICENSE.TXT for details.
> >> >> >> +//
> >> >> >>
> >> >> >>
> >> >> >>
> +//===----------------------------------------------------------------------===//
> >> >> >> +// Fuzzer's main loop.
> >> >> >>
> >> >> >>
> >> >> >>
> +//===----------------------------------------------------------------------===//
> >> >> >> +
> >> >> >> +#include "FuzzerInternal.h"
> >> >> >> +#include <sanitizer/asan_interface.h>
> >> >>
> >> >> Please do not assume everyone has asan checked out. Also, this is
> >> >> using a system include path on Windows instead of a project include
> >> >> path.
> >> >
> >> >
> >> >
> >> > I do not. This is what EXCLUDE_FROM_ALL was for.
> >> > Apparently it did not help.
> >> > Suggestions?
> >>
> >> I'm not certain how EXCLUDE_FROM_ALL would help for this #include?
> >
> >
> > Not sure what you mean here.
> > My understanding of EXCLUDE_FROM_ALL is that a target will not be built
> > unless it is explicitly requested from the build system,
> > at which point the requester makes sure there is proper sanitizer
> support.
> > This library will not function w/o sanitizers, so there is no way to
> have it
> > available in all builds.
>
> Ah, I understand your logic now. You want CMake to only include the
> entire fuzzer library when sanitizers are present, and otherwise not
> build it. So then it would be safe for this #include. That makes far
> more sense to me. :-)
>
Correct. This is why I used EXCLUDE_FROM_ALL.
Now that you've made me think of it more, maybe the proper guard will be
if (LLVM_USE_SANITIZE_COVERAGE) from
http://llvm.org/viewvc/llvm-project?view=revision&revision=227216
wdyt?
>
> ~Aaron
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20150129/484d0a2f/attachment.html>
More information about the llvm-commits
mailing list