[all-commits] [llvm/llvm-project] c4a41c: Refactor mutation strategies into a standalone lib...

[Aaron via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: c4a41cd77c15c2905ac74beeec09f8343a65a549
      https://github.com/llvm/llvm-project/commit/c4a41cd77c15c2905ac74beeec09f8343a65a549
  Author: Aaron Green <aarongreen at google.com>
  Date:   2021-05-26 (Wed, 26 May 2021)

  Changed paths:
    M compiler-rt/lib/fuzzer/CMakeLists.txt
    R compiler-rt/lib/fuzzer/FuzzerCrossOver.cpp
    M compiler-rt/lib/fuzzer/FuzzerDefs.h
    R compiler-rt/lib/fuzzer/FuzzerDictionary.h
    M compiler-rt/lib/fuzzer/FuzzerDriver.cpp
    M compiler-rt/lib/fuzzer/FuzzerInternal.h
    M compiler-rt/lib/fuzzer/FuzzerLoop.cpp
    M compiler-rt/lib/fuzzer/FuzzerMutate.cpp
    M compiler-rt/lib/fuzzer/FuzzerMutate.h
    M compiler-rt/lib/fuzzer/FuzzerRandom.h
    M compiler-rt/lib/fuzzer/FuzzerTracePC.cpp
    M compiler-rt/lib/fuzzer/FuzzerTracePC.h
    M compiler-rt/lib/fuzzer/FuzzerUtil.h
    M compiler-rt/lib/fuzzer/FuzzerUtilFuchsia.cpp
    M compiler-rt/lib/fuzzer/FuzzerUtilPosix.cpp
    M compiler-rt/lib/fuzzer/FuzzerUtilWindows.cpp
    M compiler-rt/lib/fuzzer/build.sh
    A compiler-rt/lib/fuzzer/mutagen/CMakeLists.txt
    A compiler-rt/lib/fuzzer/mutagen/Mutagen.cpp
    A compiler-rt/lib/fuzzer/mutagen/Mutagen.h
    A compiler-rt/lib/fuzzer/mutagen/MutagenCrossOver.cpp
    A compiler-rt/lib/fuzzer/mutagen/MutagenDictionary.h
    A compiler-rt/lib/fuzzer/mutagen/MutagenDispatcher.cpp
    A compiler-rt/lib/fuzzer/mutagen/MutagenDispatcher.h
    A compiler-rt/lib/fuzzer/mutagen/MutagenSequence.h
    A compiler-rt/lib/fuzzer/mutagen/MutagenUtil.h
    A compiler-rt/lib/fuzzer/mutagen/MutagenUtilPosix.cpp
    A compiler-rt/lib/fuzzer/mutagen/MutagenUtilWindows.cpp
    A compiler-rt/lib/fuzzer/mutagen/build.sh
    M compiler-rt/lib/fuzzer/tests/CMakeLists.txt
    M compiler-rt/lib/fuzzer/tests/FuzzerUnittest.cpp
    A compiler-rt/lib/fuzzer/tests/MutagenUnittest.cpp
    M compiler-rt/test/fuzzer/CMakeLists.txt

  Log Message:
  -----------
  Refactor mutation strategies into a standalone library

This change introduces libMutagen/libclang_rt.mutagen.a as a subset of libFuzzer/libclang_rt.fuzzer.a. This library contains only the fuzzing strategies used by libFuzzer to produce new test inputs from provided inputs, dictionaries, and SanitizerCoverage feedback.

Most of this change is simply moving sections of code to one side or the other of the library boundary. The only meaningful new code is:

* The Mutagen.h interface and its implementation in Mutagen.cpp.
* The following methods in MutagenDispatcher.cpp:
  * UseCmp
  * UseMemmem
  * SetCustomMutator
  * SetCustomCrossOver
  * LateInitialize (similar to the MutationDispatcher's original constructor)
  * Mutate_AddWordFromTORC (uses callbacks instead of accessing TPC directly)
  * StartMutationSequence
  * MutationSequence
  * DictionaryEntrySequence
  * RecommendDictionary
  * RecommendDictionaryEntry
* FuzzerMutate.cpp (which now justs sets callbacks and handles printing)
* MutagenUnittest.cpp (which adds tests of Mutagen.h)

A note on performance: This change was tested with a 100 passes of test/fuzzer/LargeTest.cpp with 1000 runs per pass, both with and without the change. The running time distribution was qualitatively similar both with and without the change, and the average difference was within 30 microseconds (2.240 ms/run vs 2.212 ms/run, respectively). Both times were much higher than observed with the fully optimized system clang (~0.38 ms/run), most likely due to the combination of CMake "dev mode" settings (e.g. CMAKE_BUILD_TYPE="Debug", LLVM_ENABLE_LTO=OFF, etc.). The difference between the two versions built similarly seems to be "in the noise" and suggests no meaningful performance degradation.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D102447