[llvm] r227252 - Add a Fuzzer library
Aaron Ballman
aaron at aaronballman.com
Thu Jan 29 07:28:40 PST 2015
Actually, this is completely broken on Windows and should be reverted
until it is fixed up. Comments below.
On Thu, Jan 29, 2015 at 10:11 AM, Aaron Ballman <aaron at aaronballman.com> wrote:
> This commit puts the fuzzer targets on the top-level for MSVC
> solutions when they should be in a separate folder instead. Picture
> attached is worth 1000 words. ;-)
>
> ~Aaron
>
> On Tue, Jan 27, 2015 at 5:08 PM, Kostya Serebryany <kcc at google.com> wrote:
>> Author: kcc
>> Date: Tue Jan 27 16:08:41 2015
>> New Revision: 227252
>>
>> URL: http://llvm.org/viewvc/llvm-project?rev=227252&view=rev
>> Log:
>> Add a Fuzzer library
>>
>> Summary:
>> A simple genetic in-process coverage-guided fuzz testing library.
>>
>> I've used this fuzzer to test clang-format
>> (it found 12+ bugs, thanks djasper@ for the fixes!)
>> and it may also help us test other parts of LLVM.
>> So why not keep it in the LLVM repository?
>>
>> I plan to add the cmake build rules later (in a separate patch, if that's ok)
>> and also add a clang-format-fuzzer target.
>>
>> See README.txt for details.
>>
>> Test Plan: Tests will follow separately.
>>
>> Reviewers: djasper, chandlerc, rnk
>>
>> Reviewed By: rnk
>>
>> Subscribers: majnemer, ygribov, dblaikie, llvm-commits
>>
>> Differential Revision: http://reviews.llvm.org/D7184
>>
>> Added:
>> llvm/trunk/lib/Fuzzer/
>> llvm/trunk/lib/Fuzzer/CMakeLists.txt
>> llvm/trunk/lib/Fuzzer/FuzzerCrossOver.cpp
>> llvm/trunk/lib/Fuzzer/FuzzerFlags.def
>> llvm/trunk/lib/Fuzzer/FuzzerIO.cpp
>> llvm/trunk/lib/Fuzzer/FuzzerInternal.h
>> llvm/trunk/lib/Fuzzer/FuzzerLoop.cpp
>> llvm/trunk/lib/Fuzzer/FuzzerMain.cpp
>> llvm/trunk/lib/Fuzzer/FuzzerMutate.cpp
>> llvm/trunk/lib/Fuzzer/FuzzerUtil.cpp
>> llvm/trunk/lib/Fuzzer/README.txt
>> llvm/trunk/lib/Fuzzer/test/
>> llvm/trunk/lib/Fuzzer/test/ExactTest.cpp
>> llvm/trunk/lib/Fuzzer/test/InfiniteTest.cpp
>> llvm/trunk/lib/Fuzzer/test/NullDerefTest.cpp
>> llvm/trunk/lib/Fuzzer/test/SimpleTest.cpp
>> llvm/trunk/lib/Fuzzer/test/TestFuzzerCrossOver.cpp
>> llvm/trunk/lib/Fuzzer/test/TimeoutTest.cpp
>> Modified:
>> llvm/trunk/lib/CMakeLists.txt
>>
>> Modified: llvm/trunk/lib/CMakeLists.txt
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CMakeLists.txt?rev=227252&r1=227251&r2=227252&view=diff
>> ==============================================================================
>> --- llvm/trunk/lib/CMakeLists.txt (original)
>> +++ llvm/trunk/lib/CMakeLists.txt Tue Jan 27 16:08:41 2015
>> @@ -17,3 +17,4 @@ add_subdirectory(Target)
>> add_subdirectory(AsmParser)
>> add_subdirectory(LineEditor)
>> add_subdirectory(ProfileData)
>> +add_subdirectory(Fuzzer)
>>
>> Added: llvm/trunk/lib/Fuzzer/CMakeLists.txt
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/CMakeLists.txt?rev=227252&view=auto
>> ==============================================================================
>> --- llvm/trunk/lib/Fuzzer/CMakeLists.txt (added)
>> +++ llvm/trunk/lib/Fuzzer/CMakeLists.txt Tue Jan 27 16:08:41 2015
>> @@ -0,0 +1,9 @@
>> +add_library(LLVMFuzzer STATIC
>> + EXCLUDE_FROM_ALL # Do not build if you are not building fuzzers.
>> + FuzzerCrossOver.cpp
>> + FuzzerIO.cpp
>> + FuzzerLoop.cpp
>> + FuzzerMain.cpp
>> + FuzzerMutate.cpp
>> + FuzzerUtil.cpp
>> + )
>>
>> Added: llvm/trunk/lib/Fuzzer/FuzzerCrossOver.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerCrossOver.cpp?rev=227252&view=auto
>> ==============================================================================
>> --- llvm/trunk/lib/Fuzzer/FuzzerCrossOver.cpp (added)
>> +++ llvm/trunk/lib/Fuzzer/FuzzerCrossOver.cpp Tue Jan 27 16:08:41 2015
>> @@ -0,0 +1,46 @@
>> +//===- FuzzerCrossOver.cpp - Cross over two test inputs -------------------===//
>> +//
>> +// The LLVM Compiler Infrastructure
>> +//
>> +// This file is distributed under the University of Illinois Open Source
>> +// License. See LICENSE.TXT for details.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +// Cross over test inputs.
>> +//===----------------------------------------------------------------------===//
>> +
>> +#include "FuzzerInternal.h"
>> +
>> +namespace fuzzer {
>> +
>> +// Cross A and B, store the result (ap to MaxLen bytes) in U.
>> +void CrossOver(const Unit &A, const Unit &B, Unit *U, size_t MaxLen) {
>> + size_t Size = rand() % MaxLen + 1;
>> + U->clear();
>> + const Unit *V = &A;
>> + size_t PosA = 0;
>> + size_t PosB = 0;
>> + size_t *Pos = &PosA;
>> + while (U->size() < Size && (PosA < A.size() || PosB < B.size())) {
>> + // Merge a part of V into U.
>> + size_t SizeLeftU = Size - U->size();
>> + if (*Pos < V->size()) {
>> + size_t SizeLeftV = V->size() - *Pos;
>> + size_t MaxExtraSize = std::min(SizeLeftU, SizeLeftV);
>> + size_t ExtraSize = rand() % MaxExtraSize + 1;
>> + U->insert(U->end(), V->begin() + *Pos, V->begin() + *Pos + ExtraSize);
>> + (*Pos) += ExtraSize;
>> + }
>> +
>> + // Use the other Unit on the next iteration.
>> + if (Pos == &PosA) {
>> + Pos = &PosB;
>> + V = &B;
>> + } else {
>> + Pos = &PosA;
>> + V = &A;
>> + }
>> + }
>> +}
>> +
>> +} // namespace fuzzer
>>
>> Added: llvm/trunk/lib/Fuzzer/FuzzerFlags.def
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerFlags.def?rev=227252&view=auto
>> ==============================================================================
>> --- llvm/trunk/lib/Fuzzer/FuzzerFlags.def (added)
>> +++ llvm/trunk/lib/Fuzzer/FuzzerFlags.def Tue Jan 27 16:08:41 2015
>> @@ -0,0 +1,24 @@
>> +//===- FuzzerFlags.def - Run-time flags -------------------------*- C++ -* ===//
>> +//
>> +// The LLVM Compiler Infrastructure
>> +//
>> +// This file is distributed under the University of Illinois Open Source
>> +// License. See LICENSE.TXT for details.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +// Flags. FUZZER_FLAG macro should be defined at the point of inclusion.
>> +// We are not using any flag parsing library for better portability and
>> +// independence.
>> +//===----------------------------------------------------------------------===//
>> +FUZZER_FLAG(int, verbosity, 1, "Verbosity level.")
>> +FUZZER_FLAG(int, seed, 0, "Random seed. If 0, seed is generated.")
>> +FUZZER_FLAG(int, iterations, -1,
>> + "Number of iterations of the fuzzer (-1 for infinite runs).")
>> +FUZZER_FLAG(int, max_len, 64, "Maximal length of the test input.")
>> +FUZZER_FLAG(int, cross_over, 1, "If 1, cross over inputs.")
>> +FUZZER_FLAG(int, mutate_depth, 10,
>> + "Apply this number of consecutive mutations to each input.")
>> +FUZZER_FLAG(int, exit_on_first, 0,
>> + "If 1, exit after the first new interesting input is found.")
>> +FUZZER_FLAG(int, timeout, -1, "Timeout in seconds (if positive).")
>> +FUZZER_FLAG(int, help, 0, "Print help.")
>>
>> Added: llvm/trunk/lib/Fuzzer/FuzzerIO.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerIO.cpp?rev=227252&view=auto
>> ==============================================================================
>> --- llvm/trunk/lib/Fuzzer/FuzzerIO.cpp (added)
>> +++ llvm/trunk/lib/Fuzzer/FuzzerIO.cpp Tue Jan 27 16:08:41 2015
>> @@ -0,0 +1,44 @@
>> +//===- FuzzerIO.cpp - IO utils. -------------------------------------------===//
>> +//
>> +// The LLVM Compiler Infrastructure
>> +//
>> +// This file is distributed under the University of Illinois Open Source
>> +// License. See LICENSE.TXT for details.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +// IO functions.
>> +//===----------------------------------------------------------------------===//
>> +#include "FuzzerInternal.h"
>> +#include <fstream>
>> +#include <dirent.h>
dirent.h does not exist on Windows with MSVC.
>> +namespace fuzzer {
>> +
>> +std::vector<std::string> ListFilesInDir(const std::string &Dir) {
>> + std::vector<std::string> V;
>> + DIR *D = opendir(Dir.c_str());
>> + if (!D) return V;
>> + while (auto E = readdir(D)) {
>> + if (E->d_type == DT_REG || E->d_type == DT_LNK)
>> + V.push_back(E->d_name);
>> + }
>> + closedir(D);
>> + return V;
We have filesystem APIs to do this, don't we?
>> +}
>> +
>> +Unit FileToVector(const std::string &Path) {
>> + std::ifstream T(Path);
>> + return Unit((std::istreambuf_iterator<char>(T)),
>> + std::istreambuf_iterator<char>());
>> +}
>> +
>> +void WriteToFile(const Unit &U, const std::string &Path) {
>> + std::ofstream OF(Path);
>> + OF.write((const char*)U.data(), U.size());
>> +}
>> +
>> +void ReadDirToVectorOfUnits(const char *Path, std::vector<Unit> *V) {
>> + for (auto &X : ListFilesInDir(Path))
>> + V->push_back(FileToVector(std::string(Path) + "/" + X));
>> +}
>> +
>> +} // namespace fuzzer
>>
>> Added: llvm/trunk/lib/Fuzzer/FuzzerInternal.h
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerInternal.h?rev=227252&view=auto
>> ==============================================================================
>> --- llvm/trunk/lib/Fuzzer/FuzzerInternal.h (added)
>> +++ llvm/trunk/lib/Fuzzer/FuzzerInternal.h Tue Jan 27 16:08:41 2015
>> @@ -0,0 +1,77 @@
>> +//===- FuzzerInternal.h - Internal header for the Fuzzer --------*- C++ -* ===//
>> +//
>> +// The LLVM Compiler Infrastructure
>> +//
>> +// This file is distributed under the University of Illinois Open Source
>> +// License. See LICENSE.TXT for details.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +// Define the main class fuzzer::Fuzzer and most functions.
>> +//===----------------------------------------------------------------------===//
>> +#include <cassert>
>> +#include <chrono>
>> +#include <cstddef>
>> +#include <cstdlib>
>> +#include <string>
>> +#include <vector>
>> +
>> +namespace fuzzer {
>> +typedef std::vector<uint8_t> Unit;
>> +using namespace std::chrono;
>> +
>> +Unit ReadFile(const char *Path);
>> +std::vector<std::string> ListFilesInDir(const std::string &Dir);
>> +void ReadDirToVectorOfUnits(const char *Path, std::vector<Unit> *V);
>> +void WriteToFile(const Unit &U, const std::string &Path);
>> +
>> +void Mutate(Unit *U, size_t MaxLen);
>> +
>> +void CrossOver(const Unit &A, const Unit &B, Unit *U, size_t MaxLen);
>> +
>> +void Print(const Unit &U, const char *PrintAfter = "");
>> +void PrintASCII(const Unit &U, const char *PrintAfter = "");
>> +std::string Hash(const Unit &U);
>> +void SetTimer(int Seconds);
>> +
>> +class Fuzzer {
>> + public:
>> + struct FuzzingOptions {
>> + int Verbosity = 1;
>> + int MaxLen = 0;
>> + bool DoCrossOver = true;
>> + bool MutateDepth = 10;
>> + bool ExitOnFirst = false;
>> + std::string OutputCorpus;
>> + };
>> + Fuzzer(FuzzingOptions Options) : Options(Options) {
>> + SetDeathCallback();
>> + }
>> + void AddToCorpus(const Unit &U) { Corpus.push_back(U); }
>> + size_t Loop(size_t NumIterations);
>> + void ShuffleAndMinimize();
>> + size_t CorpusSize() const { return Corpus.size(); }
>> + void ReadDir(const std::string &Path) {
>> + ReadDirToVectorOfUnits(Path.c_str(), &Corpus);
>> + }
>> +
>> + static void AlarmCallback();
>> +
>> + private:
>> + size_t MutateAndTestOne(Unit *U);
>> + size_t RunOne(const Unit &U);
>> + void WriteToOutputCorpus(const Unit &U);
>> + static void WriteToCrash(const Unit &U, const char *Prefix);
>> +
>> + void SetDeathCallback();
>> + static void DeathCallback();
>> + static Unit CurrentUnit;
>> +
>> + size_t TotalNumberOfRuns = 0;
>> +
>> + std::vector<Unit> Corpus;
>> + FuzzingOptions Options;
>> + system_clock::time_point ProcessStartTime = system_clock::now();
>> + static system_clock::time_point UnitStartTime;
>> +};
>> +
>> +}; // namespace fuzzer
>>
>> Added: llvm/trunk/lib/Fuzzer/FuzzerLoop.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerLoop.cpp?rev=227252&view=auto
>> ==============================================================================
>> --- llvm/trunk/lib/Fuzzer/FuzzerLoop.cpp (added)
>> +++ llvm/trunk/lib/Fuzzer/FuzzerLoop.cpp Tue Jan 27 16:08:41 2015
>> @@ -0,0 +1,161 @@
>> +//===- FuzzerLoop.cpp - Fuzzer's main loop --------------------------------===//
>> +//
>> +// The LLVM Compiler Infrastructure
>> +//
>> +// This file is distributed under the University of Illinois Open Source
>> +// License. See LICENSE.TXT for details.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +// Fuzzer's main loop.
>> +//===----------------------------------------------------------------------===//
>> +
>> +#include "FuzzerInternal.h"
>> +#include <sanitizer/asan_interface.h>
Please do not assume everyone has asan checked out. Also, this is
using a system include path on Windows instead of a project include
path.
>> +#include <algorithm>
>> +#include <string>
>> +#include <iostream>
>> +#include <stdlib.h>
Why are we including a C header instead of cstdlib?
>> +
>> +// This function should be defined by the user.
>> +extern "C" void TestOneInput(const uint8_t *Data, size_t Size);
>> +
>> +namespace fuzzer {
>> +
>> +// static
>> +Unit Fuzzer::CurrentUnit;
>> +system_clock::time_point Fuzzer::UnitStartTime;
>> +
>> +void Fuzzer::SetDeathCallback() {
>> + __sanitizer_set_death_callback(DeathCallback);
>> +}
>> +
>> +void Fuzzer::DeathCallback() {
>> + std::cerr << "DEATH: " << std::endl;
>> + Print(CurrentUnit, "\n");
>> + PrintASCII(CurrentUnit, "\n");
>> + WriteToCrash(CurrentUnit, "crash-");
>> +}
>> +
>> +void Fuzzer::AlarmCallback() {
>> + size_t Seconds =
>> + duration_cast<seconds>(system_clock::now() - UnitStartTime).count();
>> + std::cerr << "ALARM: working on the last Unit for " << Seconds << " seconds"
>> + << std::endl;
>> + if (Seconds > 60) {
>> + Print(CurrentUnit, "\n");
>> + PrintASCII(CurrentUnit, "\n");
>> + WriteToCrash(CurrentUnit, "timeout-");
>> + }
>> + abort();
>> +}
>> +
>> +void Fuzzer::ShuffleAndMinimize() {
>> + if (Options.Verbosity)
>> + std::cerr << "Shuffle: " << Corpus.size() << "\n";
>> + std::vector<Unit> NewCorpus;
>> + random_shuffle(Corpus.begin(), Corpus.end());
>> + size_t MaxCov = 0;
>> + Unit &U = CurrentUnit;
>> + for (const auto &C : Corpus) {
>> + for (size_t First = 0; First < 1; First++) {
>> + U.clear();
>> + size_t Last = std::min(First + Options.MaxLen, C.size());
>> + U.insert(U.begin(), C.begin() + First, C.begin() + Last);
>> + size_t NewCoverage = RunOne(U);
>> + if (NewCoverage) {
>> + MaxCov = NewCoverage;
>> + NewCorpus.push_back(U);
>> + if (Options.Verbosity >= 2)
>> + std::cerr << "NEW0: " << NewCoverage << "\n";
>> + }
>> + }
>> + }
>> + Corpus = NewCorpus;
>> + if (Options.Verbosity)
>> + std::cerr << "Shuffle done: " << Corpus.size() << " IC: " << MaxCov << "\n";
>> +}
>> +
>> +size_t Fuzzer::RunOne(const Unit &U) {
>> + UnitStartTime = system_clock::now();
>> + TotalNumberOfRuns++;
>> + size_t OldCoverage = __sanitizer_get_total_unique_coverage();
>> + TestOneInput(U.data(), U.size());
>> + size_t NewCoverage = __sanitizer_get_total_unique_coverage();
>> + if (!(TotalNumberOfRuns & (TotalNumberOfRuns - 1)) && Options.Verbosity) {
>> + size_t Seconds =
>> + duration_cast<seconds>(system_clock::now() - ProcessStartTime).count();
>> + std::cerr
>> + << "#" << TotalNumberOfRuns
>> + << "\tcov: " << NewCoverage
>> + << "\texec/s: " << (Seconds ? TotalNumberOfRuns / Seconds : 0) << "\n";
>> + }
>> + if (NewCoverage > OldCoverage)
>> + return NewCoverage;
>> + return 0;
>> +}
>> +
>> +void Fuzzer::WriteToOutputCorpus(const Unit &U) {
>> + if (Options.OutputCorpus.empty()) return;
>> + std::string Path = Options.OutputCorpus + "/" + Hash(U);
>> + WriteToFile(U, Path);
>> + if (Options.Verbosity >= 2)
>> + std::cerr << "Written to " << Path << std::endl;
>> +}
>> +
>> +void Fuzzer::WriteToCrash(const Unit &U, const char *Prefix) {
>> + std::string Path = Prefix + Hash(U);
>> + WriteToFile(U, Path);
>> + std::cerr << "CRASHED; file written to " << Path << std::endl;
>> +}
>> +
>> +size_t Fuzzer::MutateAndTestOne(Unit *U) {
>> + size_t NewUnits = 0;
>> + for (size_t i = 0; i < Options.MutateDepth; i++) {
>> + Mutate(U, Options.MaxLen);
>> + if (U->empty()) continue;
>> + size_t NewCoverage = RunOne(*U);
>> + if (NewCoverage) {
>> + Corpus.push_back(*U);
>> + NewUnits++;
>> + if (Options.Verbosity) {
>> + std::cerr << "#" << TotalNumberOfRuns
>> + << "\tNEW: " << NewCoverage
>> + << " L: " << U->size()
>> + << "\t";
>> + if (U->size() < 30) {
>> + PrintASCII(*U);
>> + std::cerr << "\t";
>> + Print(*U);
>> + }
>> + std::cerr << "\n";
>> + }
>> + WriteToOutputCorpus(*U);
>> + if (Options.ExitOnFirst)
>> + exit(0);
>> + }
>> + }
>> + return NewUnits;
>> +}
>> +
>> +size_t Fuzzer::Loop(size_t NumIterations) {
>> + size_t NewUnits = 0;
>> + for (size_t i = 1; i <= NumIterations; i++) {
>> + if (Options.DoCrossOver) {
>> + for (size_t J1 = 0; J1 < Corpus.size(); J1++) {
>> + for (size_t J2 = 0; J2 < Corpus.size(); J2++) {
>> + CurrentUnit.clear();
>> + CrossOver(Corpus[J1], Corpus[J2], &CurrentUnit, Options.MaxLen);
>> + NewUnits += MutateAndTestOne(&CurrentUnit);
>> + }
>> + }
>> + } else { // No CrossOver
>> + for (size_t J = 0; J < Corpus.size(); J++) {
>> + CurrentUnit = Corpus[J];
>> + NewUnits += MutateAndTestOne(&CurrentUnit);
>> + }
>> + }
>> + }
>> + return NewUnits;
>> +}
>> +
>> +} // namespace fuzzer
>>
>> Added: llvm/trunk/lib/Fuzzer/FuzzerMain.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerMain.cpp?rev=227252&view=auto
>> ==============================================================================
>> --- llvm/trunk/lib/Fuzzer/FuzzerMain.cpp (added)
>> +++ llvm/trunk/lib/Fuzzer/FuzzerMain.cpp Tue Jan 27 16:08:41 2015
>> @@ -0,0 +1,143 @@
>> +//===- FuzzerMain.cpp - main() function and flags -------------------------===//
>> +//
>> +// The LLVM Compiler Infrastructure
>> +//
>> +// This file is distributed under the University of Illinois Open Source
>> +// License. See LICENSE.TXT for details.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +// main() and flags.
>> +//===----------------------------------------------------------------------===//
>> +
>> +#include "FuzzerInternal.h"
>> +
>> +#include <climits>
>> +#include <cstring>
>> +#include <unistd.h>
unistd.h does not exist on Windows with MSVC.
>> +#include <iostream>
>> +
>> +// ASAN options:
>> +// * don't dump the coverage to disk.
>> +extern "C" const char* __asan_default_options() { return "coverage_pcs=0"; }
>> +
>> +// Program arguments.
>> +struct FlagDescription {
>> + const char *Name;
>> + const char *Description;
>> + int Default;
>> + int *Flag;
>> +};
>> +
>> +struct {
>> +#define FUZZER_FLAG(Type, Name, Default, Description) Type Name;
>> +#include "FuzzerFlags.def"
>> +#undef FUZZER_FLAG
>> +} Flags;
>> +
>> +static FlagDescription FlagDescriptions [] {
>> +#define FUZZER_FLAG(Type, Name, Default, Description) {#Name, Description, Default, &Flags.Name},
>> +#include "FuzzerFlags.def"
>> +#undef FUZZER_FLAG
>> +};
>> +
>> +static const size_t kNumFlags =
>> + sizeof(FlagDescriptions) / sizeof(FlagDescriptions[0]);
>> +
>> +static std::vector<std::string> inputs;
>> +static const char *ProgName;
>> +
>> +static void PrintHelp() {
>> + std::cerr << "Usage: " << ProgName
>> + << " [-flag1=val1 [-flag2=val2 ...] ] [dir1 [dir2 ...] ]\n";
>> + std::cerr << "\nFlags: (strictly in form -flag=value)\n";
>> + size_t MaxFlagLen = 0;
>> + for (size_t F = 0; F < kNumFlags; F++)
>> + MaxFlagLen = std::max(strlen(FlagDescriptions[F].Name), MaxFlagLen);
>> +
>> + for (size_t F = 0; F < kNumFlags; F++) {
>> + const auto &D = FlagDescriptions[F];
>> + std::cerr << " " << D.Name;
>> + for (size_t i = 0, n = MaxFlagLen - strlen(D.Name); i < n; i++)
>> + std::cerr << " ";
>> + std::cerr << "\t";
>> + std::cerr << D.Default << "\t" << D.Description << "\n";
>> + }
>> +}
>> +
>> +static const char *FlagValue(const char *Param, const char *Name) {
>> + size_t Len = strlen(Name);
>> + if (Param[0] == '-' && strstr(Param + 1, Name) == Param + 1 &&
>> + Param[Len + 1] == '=')
>> + return &Param[Len + 2];
>> + return nullptr;
>> +}
>> +
>> +static bool ParseOneFlag(const char *Param) {
>> + if (Param[0] != '-') return false;
>> + for (size_t F = 0; F < kNumFlags; F++) {
>> + const char *Name = FlagDescriptions[F].Name;
>> + const char *Str = FlagValue(Param, Name);
>> + if (Str) {
>> + int Val = std::stol(Str);
>> + *FlagDescriptions[F].Flag = Val;
>> + if (Flags.verbosity >= 2)
>> + std::cerr << "Flag: " << Name << " " << Val << "\n";
>> + return true;
>> + }
>> + }
>> + PrintHelp();
>> + exit(1);
>> +}
>> +
>> +// We don't use any library to minimize dependencies.
>> +static void ParseFlags(int argc, char **argv) {
>> + for (size_t F = 0; F < kNumFlags; F++)
>> + *FlagDescriptions[F].Flag = FlagDescriptions[F].Default;
>> + for (int A = 1; A < argc; A++) {
>> + if (ParseOneFlag(argv[A])) continue;
>> + inputs.push_back(argv[A]);
>> + }
>> +}
>> +
>> +int main(int argc, char **argv) {
>> + using namespace fuzzer;
>> +
>> + ProgName = argv[0];
>> + ParseFlags(argc, argv);
>> + if (Flags.help) {
>> + PrintHelp();
>> + return 0;
>> + }
>> + Fuzzer::FuzzingOptions Options;
>> + Options.Verbosity = Flags.verbosity;
>> + Options.MaxLen = Flags.max_len;
>> + Options.DoCrossOver = Flags.cross_over;
>> + Options.MutateDepth = Flags.mutate_depth;
>> + Options.ExitOnFirst = Flags.exit_on_first;
>> + if (!inputs.empty())
>> + Options.OutputCorpus = inputs[0];
>> + Fuzzer F(Options);
>> +
>> + unsigned seed = Flags.seed;
>> + // Initialize seed.
>> + if (seed == 0)
>> + seed = time(0) * 10000 + getpid();
>> + if (Flags.verbosity)
>> + std::cerr << "Seed: " << seed << "\n";
>> + srand(seed);
>> +
>> + // Timer
>> + if (Flags.timeout > 0)
>> + SetTimer(Flags.timeout);
>> +
>> + for (auto &inp : inputs)
>> + F.ReadDir(inp);
>> +
>> + if (F.CorpusSize() == 0)
>> + F.AddToCorpus(Unit()); // Can't fuzz empty corpus, so add an empty input.
>> + F.ShuffleAndMinimize();
>> + F.Loop(Flags.iterations < 0 ? INT_MAX : Flags.iterations);
>> + if (Flags.verbosity)
>> + std::cerr << "Done\n";
>> + return 1;
>> +}
>>
>> Added: llvm/trunk/lib/Fuzzer/FuzzerMutate.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerMutate.cpp?rev=227252&view=auto
>> ==============================================================================
>> --- llvm/trunk/lib/Fuzzer/FuzzerMutate.cpp (added)
>> +++ llvm/trunk/lib/Fuzzer/FuzzerMutate.cpp Tue Jan 27 16:08:41 2015
>> @@ -0,0 +1,62 @@
>> +//===- FuzzerMutate.cpp - Mutate a test input -----------------------------===//
>> +//
>> +// The LLVM Compiler Infrastructure
>> +//
>> +// This file is distributed under the University of Illinois Open Source
>> +// License. See LICENSE.TXT for details.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +// Mutate a test input.
>> +//===----------------------------------------------------------------------===//
>> +
>> +#include "FuzzerInternal.h"
>> +
>> +namespace fuzzer {
>> +
>> +static char FlipRandomBit(char X) {
>> + int Bit = rand() % 8;
>> + char Mask = 1 << Bit;
>> + char R;
>> + if (X & (1 << Bit))
>> + R = X & ~Mask;
>> + else
>> + R = X | Mask;
>> + assert(R != X);
>> + return R;
>> +}
>> +
>> +static char RandCh() {
>> + if (rand() % 2) return rand();
>> + const char *Special = "!*'();:@&=+$,/?%#[]123ABCxyz-`~.";
>> + return Special[rand() % (sizeof(Special) - 1)];
>> +}
>> +
>> +void Mutate(Unit *U, size_t MaxLen) {
>> + assert(MaxLen > 0);
>> + assert(U->size() <= MaxLen);
>> + switch (rand() % 3) {
>> + case 0:
>> + if (U->size())
>> + U->erase(U->begin() + rand() % U->size());
>> + break;
>> + case 1:
>> + if (U->empty()) {
>> + U->push_back(RandCh());
>> + } else if (U->size() < MaxLen) {
>> + U->insert(U->begin() + rand() % U->size(), RandCh());
>> + } else { // At MaxLen.
>> + uint8_t Ch = RandCh();
>> + size_t Idx = rand() % U->size();
>> + (*U)[Idx] = Ch;
>> + }
>> + break;
>> + default:
>> + if (!U->empty()) {
>> + size_t idx = rand() % U->size();
>> + (*U)[idx] = FlipRandomBit((*U)[idx]);
>> + }
>> + break;
>> + }
>> +}
>> +
>> +} // namespace fuzzer
>>
>> Added: llvm/trunk/lib/Fuzzer/FuzzerUtil.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerUtil.cpp?rev=227252&view=auto
>> ==============================================================================
>> --- llvm/trunk/lib/Fuzzer/FuzzerUtil.cpp (added)
>> +++ llvm/trunk/lib/Fuzzer/FuzzerUtil.cpp Tue Jan 27 16:08:41 2015
>> @@ -0,0 +1,61 @@
>> +//===- FuzzerUtil.cpp - Misc utils ----------------------------------------===//
>> +//
>> +// The LLVM Compiler Infrastructure
>> +//
>> +// This file is distributed under the University of Illinois Open Source
>> +// License. See LICENSE.TXT for details.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +// Misc utils.
>> +//===----------------------------------------------------------------------===//
>> +
>> +#include "FuzzerInternal.h"
>> +#include <iostream>
>> +#include <sys/time.h>
sys/time.h does not exist on Windows with MSVC.
>> +#include <cassert>
>> +#include <cstring>
>> +#include <signal.h>
>> +
>> +namespace fuzzer {
>> +
>> +void Print(const Unit &v, const char *PrintAfter) {
>> + std::cerr << v.size() << ": ";
>> + for (auto x : v)
>> + std::cerr << (unsigned) x << " ";
>> + std::cerr << PrintAfter;
>> +}
>> +
>> +void PrintASCII(const Unit &U, const char *PrintAfter) {
>> + for (auto X : U)
>> + std::cerr << (char)((isascii(X) && X >= ' ') ? X : '?');
>> + std::cerr << PrintAfter;
>> +}
>> +
>> +std::string Hash(const Unit &in) {
>> + size_t h1 = 0, h2 = 0;
>> + for (auto x : in) {
>> + h1 += x;
>> + h1 *= 5;
>> + h2 += x;
>> + h2 *= 7;
>> + }
>> + return std::to_string(h1) + std::to_string(h2);
>> +}
>> +
>> +static void AlarmHandler(int, siginfo_t *, void *) {
>> + Fuzzer::AlarmCallback();
>> +}
>> +
>> +void SetTimer(int Seconds) {
>> + struct itimerval T {{Seconds, 0}, {Seconds, 0}};
>> + std::cerr << "SetTimer " << Seconds << "\n";
>> + int Res = setitimer(ITIMER_REAL, &T, nullptr);
>> + assert(Res == 0);
>> + struct sigaction sigact;
>> + memset(&sigact, 0, sizeof(sigact));
>> + sigact.sa_sigaction = AlarmHandler;
>> + Res = sigaction(SIGALRM, &sigact, 0);
>> + assert(Res == 0);
>> +}
>> +
>> +} // namespace fuzzer
>>
>> Added: llvm/trunk/lib/Fuzzer/README.txt
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/README.txt?rev=227252&view=auto
>> ==============================================================================
>> --- llvm/trunk/lib/Fuzzer/README.txt (added)
>> +++ llvm/trunk/lib/Fuzzer/README.txt Tue Jan 27 16:08:41 2015
>> @@ -0,0 +1,56 @@
>> +===============================
>> +Fuzzer -- a library for coverage-guided fuzz testing.
>> +===============================
>> +
>> +This library is intended primarily for in-process coverage-guided fuzz testing
>> +(fuzzing) of other libraries. The typical workflow looks like this:
>> +
>> + * Build the Fuzzer library as a static archive (or just a set of .o files).
>> + Note that the Fuzzer contains the main() function.
>> + Preferably do *not* use sanitizers while building the Fuzzer.
>> + * Build the library you are going to test with -fsanitize-coverage=[234]
>> + and one of the sanitizers. We recommend to build the library in several
>> + different modes (e.g. asan, msan, lsan, ubsan, etc) and even using different
>> + optimizations options (e.g. -O0, -O1, -O2) to diversify testing.
>> + * Build a test driver using the same options as the library.
>> + The test driver is a C/C++ file containing interesting calls to the library
>> + inside a single function:
>> + extern "C" void TestOneInput(const uint8_t *Data, size_t Size);
>> + * Link the Fuzzer, the library and the driver together into an executable
>> + using the same sanitizer options as for the library.
>> + * Collect the initial corpus of inputs for the
>> + fuzzer (a directory with test inputs, one file per input).
>> + The better your inputs are the faster you will find something interesting.
>> + Also try to keep your inputs small, otherwise the Fuzzer will run too slow.
>> + * Run the fuzzer with the test corpus. As new interesting test cases are
>> + discovered they will be added to the corpus. If a bug is discovered by
>> + the sanitizer (asan, etc) it will be reported as usual and the reproducer
>> + will be written to disk.
>> + Each Fuzzer process is single-threaded (unless the library starts its own
>> + threads). You can run the Fuzzer on the same corpus in multiple processes.
>> + in parallel. For run-time options run the Fuzzer binary with '-help=1'.
>> +
>> +
>> +The Fuzzer is similar in concept to AFL (http://lcamtuf.coredump.cx/afl/),
>> +but uses in-process Fuzzing, which is more fragile, more restrictive, but
>> +potentially much faster as it has no overhead for process start-up.
>> +It uses LLVM's "Sanitizer Coverage" instrumentation to get in-process
>> +coverage-feedback https://code.google.com/p/address-sanitizer/wiki/AsanCoverage
>> +
>> +The code resides in the LLVM repository and is (or will be) used by various
>> +parts of LLVM, but the Fuzzer itself does not (and should not) depend on any
>> +part of LLVM and can be used for other projects. Ideally, the Fuzzer's code
>> +should not have any external dependencies. Right now it uses STL, which may need
>> +to be fixed later.
>> +
>> +Examples of usage in LLVM:
>> + * clang-format-fuzzer. The inputs are random pieces of C++-like text.
>> + * TODO: add more
>> +
>> +Toy example (see SimpleTest.cpp):
>> +a simple function that does something interesting if it receives bytes "Hi!".
>> + # Build the Fuzzer with asan:
>> + % clang++ -std=c++11 -fsanitize=address -fsanitize-coverage=3 -O1 -g \
>> + Fuzzer*.cpp test/SimpleTest.cpp
>> + # Run the fuzzer with no corpus (assuming on empty input)
>> + % ./a.out
>>
>> Added: llvm/trunk/lib/Fuzzer/test/ExactTest.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/test/ExactTest.cpp?rev=227252&view=auto
>> ==============================================================================
>> --- llvm/trunk/lib/Fuzzer/test/ExactTest.cpp (added)
>> +++ llvm/trunk/lib/Fuzzer/test/ExactTest.cpp Tue Jan 27 16:08:41 2015
>> @@ -0,0 +1,21 @@
>> +// Simple test for a fuzzer. The fuzzer must find the string "FUZZER".
>> +#include <cstdlib>
>> +#include <cstddef>
>> +#include <iostream>
>> +
>> +static volatile int Sink;
>> +
>> +extern "C" void TestOneInput(const uint8_t *Data, size_t Size) {
>> + int bits = 0;
>> + if (Size > 0 && Data[0] == 'F') bits |= 1;
>> + if (Size > 1 && Data[1] == 'U') bits |= 2;
>> + if (Size > 2 && Data[2] == 'Z') bits |= 4;
>> + if (Size > 3 && Data[3] == 'Z') bits |= 8;
>> + if (Size > 4 && Data[4] == 'E') bits |= 16;
>> + if (Size > 5 && Data[5] == 'R') bits |= 32;
>> + if (bits == 63) {
>> + std::cerr << "BINGO!\n";
>> + abort();
>> + }
>> +}
>> +
>>
>> Added: llvm/trunk/lib/Fuzzer/test/InfiniteTest.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/test/InfiniteTest.cpp?rev=227252&view=auto
>> ==============================================================================
>> --- llvm/trunk/lib/Fuzzer/test/InfiniteTest.cpp (added)
>> +++ llvm/trunk/lib/Fuzzer/test/InfiniteTest.cpp Tue Jan 27 16:08:41 2015
>> @@ -0,0 +1,19 @@
>> +// Simple test for a fuzzer. The fuzzer must find the string "Hi!".
>> +#include <cstdlib>
>> +#include <cstddef>
>> +#include <iostream>
>> +
>> +static volatile int Sink;
>> +
>> +extern "C" void TestOneInput(const uint8_t *Data, size_t Size) {
>> + if (Size > 0 && Data[0] == 'H') {
>> + Sink = 1;
>> + if (Size > 1 && Data[1] == 'i') {
>> + Sink = 2;
>> + if (Size > 2 && Data[2] == '!') {
>> + Size = 2;
>> + }
>> + }
>> + }
>> +}
>> +
>>
>> Added: llvm/trunk/lib/Fuzzer/test/NullDerefTest.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/test/NullDerefTest.cpp?rev=227252&view=auto
>> ==============================================================================
>> --- llvm/trunk/lib/Fuzzer/test/NullDerefTest.cpp (added)
>> +++ llvm/trunk/lib/Fuzzer/test/NullDerefTest.cpp Tue Jan 27 16:08:41 2015
>> @@ -0,0 +1,21 @@
>> +// Simple test for a fuzzer. The fuzzer must find the string "Hi!".
>> +#include <cstdlib>
>> +#include <cstddef>
>> +#include <iostream>
>> +
>> +static volatile int Sink;
>> +static volatile int *Null = 0;
>> +
>> +extern "C" void TestOneInput(const uint8_t *Data, size_t Size) {
>> + if (Size > 0 && Data[0] == 'H') {
>> + Sink = 1;
>> + if (Size > 1 && Data[1] == 'i') {
>> + Sink = 2;
>> + if (Size > 2 && Data[2] == '!') {
>> + std::cout << "Found the target, dereferencing NULL\n";
>> + *Null = 1;
>> + }
>> + }
>> + }
>> +}
>> +
>>
>> Added: llvm/trunk/lib/Fuzzer/test/SimpleTest.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/test/SimpleTest.cpp?rev=227252&view=auto
>> ==============================================================================
>> --- llvm/trunk/lib/Fuzzer/test/SimpleTest.cpp (added)
>> +++ llvm/trunk/lib/Fuzzer/test/SimpleTest.cpp Tue Jan 27 16:08:41 2015
>> @@ -0,0 +1,20 @@
>> +// Simple test for a fuzzer. The fuzzer must find the string "Hi!".
>> +#include <cstdlib>
>> +#include <cstddef>
>> +#include <iostream>
>> +
>> +static volatile int Sink;
>> +
>> +extern "C" void TestOneInput(const uint8_t *Data, size_t Size) {
>> + if (Size > 0 && Data[0] == 'H') {
>> + Sink = 1;
>> + if (Size > 1 && Data[1] == 'i') {
>> + Sink = 2;
>> + if (Size > 2 && Data[2] == '!') {
>> + std::cout << "Found the target, exiting\n";
>> + exit(0);
>> + }
>> + }
>> + }
>> +}
>> +
>>
>> Added: llvm/trunk/lib/Fuzzer/test/TestFuzzerCrossOver.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/test/TestFuzzerCrossOver.cpp?rev=227252&view=auto
>> ==============================================================================
>> --- llvm/trunk/lib/Fuzzer/test/TestFuzzerCrossOver.cpp (added)
>> +++ llvm/trunk/lib/Fuzzer/test/TestFuzzerCrossOver.cpp Tue Jan 27 16:08:41 2015
>> @@ -0,0 +1,13 @@
>> +#include "FuzzerInternal.h"
>> +
>> +int main() {
>> + using namespace fuzzer;
>> + Unit A({0, 1, 2, 3, 4}), B({5, 6, 7, 8, 9});
>> + Unit C;
>> + for (size_t Len = 1; Len < 15; Len++) {
>> + for (int Iter = 0; Iter < 1000; Iter++) {
>> + CrossOver(A, B, &C, Len);
>> + Print(C);
>> + }
>> + }
>> +}
>>
>> Added: llvm/trunk/lib/Fuzzer/test/TimeoutTest.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/test/TimeoutTest.cpp?rev=227252&view=auto
>> ==============================================================================
>> --- llvm/trunk/lib/Fuzzer/test/TimeoutTest.cpp (added)
>> +++ llvm/trunk/lib/Fuzzer/test/TimeoutTest.cpp Tue Jan 27 16:08:41 2015
>> @@ -0,0 +1,21 @@
>> +// Simple test for a fuzzer. The fuzzer must find the string "Hi!".
>> +#include <cstdlib>
>> +#include <cstddef>
>> +#include <iostream>
>> +
>> +static volatile int Sink;
>> +
>> +extern "C" void TestOneInput(const uint8_t *Data, size_t Size) {
>> + if (Size > 0 && Data[0] == 'H') {
>> + Sink = 1;
>> + if (Size > 1 && Data[1] == 'i') {
>> + Sink = 2;
>> + if (Size > 2 && Data[2] == '!') {
>> + Size = 2;
>> + while (Sink)
>> + ;
>> + }
>> + }
>> + }
>> +}
>> +
Aside from the obvious errors for MSVC, it does not compile cleanly:
Warning 1 warning C4530: C++ exception handler used, but unwind
semantics are not enabled. Specify /EHsc
(E:\llvm\llvm\lib\Fuzzer\FuzzerLoop.cpp) D:\Program Files Warning 5
warning C4305: 'initializing' : truncation from 'int' to 'bool'
(E:\llvm\llvm\lib\Fuzzer\FuzzerIO.cpp)
e:\llvm\llvm\lib\fuzzer\FuzzerInternal.h 44
Warning 13 warning C4530: C++ exception handler used, but unwind
semantics are not enabled. Specify /EHsc D:\Program Files
(x86)\Microsoft Visual Studio
(And some variation of these repeat dozens of times.)
~Aaron
>>
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
More information about the llvm-commits
mailing list