r310408 - Integrate Kostya's clang-proto-fuzzer with LLVM.
Kostya Serebryany via cfe-commits
cfe-commits at lists.llvm.org
Thu Aug 10 12:13:36 PDT 2017
On Thu, Aug 10, 2017 at 12:01 PM, Nico Weber <thakis at chromium.org> wrote:
> On Thu, Aug 10, 2017 at 2:04 PM, Kostya Serebryany <kcc at google.com> wrote:
>
>>
>>
>> On Thu, Aug 10, 2017 at 10:56 AM, Nico Weber via cfe-commits <
>> cfe-commits at lists.llvm.org> wrote:
>>
>>> I really believe this has way too many deps to live in the clang repro,
>>> as said on the review already.
>>>
>>
>> I don't have a very strong opinion here and would be happy to move if I
>> see more support for Nico's opinion
>> (I haven't seen it on the review, and you didn't object further, so we
>> proceeded).
>> Again, my rational is that the simpler it is to use the more likely other
>> researchers will extend this work.
>>
>> BTW, I am going to commit a Dockerfile that will make experimenting with
>> this trivial.
>> My current (dirty) version looks like this. Not too much trouble.
>>
>> FROM ubuntu:16.04
>> RUN apt-get update -y && apt-get install -y autoconf automake libtool
>> curl make g++ unzip
>> RUN apt-get install -y wget
>> RUN apt-get install -y git binutils liblzma-dev libz-dev
>> RUN apt-get install -y python-all
>> RUN apt-get install -y cmake ninja-build
>> RUN apt-get install -y subversion
>>
>> WORKDIR /root
>> RUN wget -qO- https://github.com/google/prot
>> obuf/releases/download/v3.3.0/protobuf-cpp-3.3.0.tar.gz | tar zxf -
>> RUN cd protobuf-3.3.0 && ./autogen.sh && ./configure && make -j $(nproc)
>> && make check -j $(nproc) && make install && ldconfig
>> RUN apt-get install -y pkg-config
>> RUN svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm
>> RUN cd llvm/tools && svn co http://llvm.org/svn/llvm-project/cfe/trunk
>> clang -r $(cd ../ && svn info | grep Revision | awk '{print $2}')
>> RUN cd llvm/projects && svn co http://llvm.org/svn/llvm-proje
>> ct/compiler-rt/trunk clang -r $(cd ../ && svn info | grep Revision | awk
>> '{print $2}')
>> RUN mkdir build0 && cd build0 && cmake -GNinja -DCMAKE_BUILD_TYPE=Release
>> ../llvm && ninja
>> RUN mkdir build1 && cd build1 && cmake -GNinja -DCMAKE_BUILD_TYPE=Release
>> ../llvm -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_C_COMPILER=`pwd`/../build0/bin/clang
>> -DCMAKE_CXX_COMPILER=`pwd`/../build0/bin/clang++
>> -DLLVM_USE_SANITIZE_COVERAGE=YES -DLLVM_USE_SANITIZER=Address
>> -DCLANG_ENABLE_PROTO_FUZZER=ON
>> RUN cd build1 && ninja clang-fuzzer
>> RUN cd build1 && ninja clang-proto-fuzzer
>> #RUN cd build1 && ninja clang-proto-to-cxx
>>
>>
>>
>>> Maybe this could live in clang-extra instead?
>>>
>>
>> clang-extra?
>>
>
> clang-tools-extra, sorry.
>
>
>> That's a separate repo, right?
>>
>
> Yes.
>
>
>> It may require more cmake trickery, and we'll also have to share the
>> clang-fuzzer-specific code between two repos.
>>
>
> We could move the whole thing. I'd imagine that at most 3% of people who
> use clang will use this fuzzer, so having it elsewhere seems reasonable.
> (I'd imagine many more people to use clang-tidy for example, and that's in
> the other repro.)
>
The clang-tidy argument doesn't work for me.
clang-tidy is a separate tool.
clang*fuzzer are ways to test clang, and so they have more reasons to stay
closer to clang (for the same reason that the clang tests stay with clang).
--kcc
> Also see the "Contributing Extensions to Clang" section on
> http://clang.llvm.org/get_involved.html
>
>
>> I do want the original clang-fuzzer to remain where it was, and both
>> (clang-fuzzer and clang-proto-fuzzer) share the code.
>>
>>
>>
>>
>>>
>>> On Aug 8, 2017 4:15 PM, "Matt Morehouse via cfe-commits" <
>>> cfe-commits at lists.llvm.org> wrote:
>>>
>>>> Author: morehouse
>>>> Date: Tue Aug 8 13:15:04 2017
>>>> New Revision: 310408
>>>>
>>>> URL: http://llvm.org/viewvc/llvm-project?rev=310408&view=rev
>>>> Log:
>>>> Integrate Kostya's clang-proto-fuzzer with LLVM.
>>>>
>>>> Summary:
>>>> The clang-proto-fuzzer models a subset of C++ as a protobuf and
>>>> uses libprotobuf-mutator to generate interesting mutations of C++
>>>> programs. Clang-proto-fuzzer has already found several bugs in
>>>> Clang (e.g., https://bugs.llvm.org/show_bug.cgi?id=33747,
>>>> https://bugs.llvm.org/show_bug.cgi?id=33749).
>>>>
>>>> As with clang-fuzzer, clang-proto-fuzzer requires the following
>>>> cmake flags:
>>>> - CMAKE_C_COMPILER=clang
>>>> - CMAKE_CXX_COMPILER=clang++
>>>> - LLVM_USE_SANITIZE_COVERAGE=YES // needed for libFuzzer
>>>> - LLVM_USE_SANITIZER=Address // needed for libFuzzer
>>>>
>>>> In addition, clang-proto-fuzzer requires:
>>>> - CLANG_ENABLE_PROTO_FUZZER=ON
>>>>
>>>> clang-proto-fuzzer also requires the following dependencies:
>>>> - binutils // needed for libprotobuf-mutator
>>>> - liblzma-dev // needed for libprotobuf-mutator
>>>> - libz-dev // needed for libprotobuf-mutator
>>>> - docbook2x // needed for libprotobuf-mutator
>>>> - Recent version of protobuf [3.3.0 is known to work]
>>>>
>>>> A working version of libprotobuf-mutator will automatically be
>>>> downloaded and built as an external project.
>>>>
>>>> Implementation of clang-proto-fuzzer provided by Kostya
>>>> Serebryany.
>>>>
>>>> https://bugs.llvm.org/show_bug.cgi?id=33829
>>>>
>>>> Reviewers: kcc, vitalybuka, bogner
>>>>
>>>> Reviewed By: kcc, vitalybuka
>>>>
>>>> Subscribers: thakis, mgorny, cfe-commits
>>>>
>>>> Differential Revision: https://reviews.llvm.org/D36324
>>>>
>>>> Added:
>>>> cfe/trunk/cmake/modules/ProtobufMutator.cmake
>>>> cfe/trunk/tools/clang-fuzzer/ExampleClangProtoFuzzer.cpp
>>>> cfe/trunk/tools/clang-fuzzer/README.txt
>>>> cfe/trunk/tools/clang-fuzzer/cxx_proto.proto
>>>> cfe/trunk/tools/clang-fuzzer/handle-cxx/
>>>> cfe/trunk/tools/clang-fuzzer/handle-cxx/CMakeLists.txt
>>>> cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.cpp
>>>> cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.h
>>>> cfe/trunk/tools/clang-fuzzer/proto-to-cxx/
>>>> cfe/trunk/tools/clang-fuzzer/proto-to-cxx/CMakeLists.txt
>>>> cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.cpp
>>>> cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.h
>>>> cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx_main.cpp
>>>> Modified:
>>>> cfe/trunk/CMakeLists.txt
>>>> cfe/trunk/tools/clang-fuzzer/CMakeLists.txt
>>>> cfe/trunk/tools/clang-fuzzer/ClangFuzzer.cpp
>>>>
>>>> Modified: cfe/trunk/CMakeLists.txt
>>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/CMakeLists.txt
>>>> ?rev=310408&r1=310407&r2=310408&view=diff
>>>> ============================================================
>>>> ==================
>>>> --- cfe/trunk/CMakeLists.txt (original)
>>>> +++ cfe/trunk/CMakeLists.txt Tue Aug 8 13:15:04 2017
>>>> @@ -377,6 +377,8 @@ option(CLANG_ENABLE_STATIC_ANALYZER "Bui
>>>> option(CLANG_ANALYZER_BUILD_Z3
>>>> "Build the static analyzer with the Z3 constraint manager." OFF)
>>>>
>>>> +option(CLANG_ENABLE_PROTO_FUZZER "Build Clang protobuf fuzzer." OFF)
>>>> +
>>>> if(NOT CLANG_ENABLE_STATIC_ANALYZER AND (CLANG_ENABLE_ARCMT OR
>>>> CLANG_ANALYZER_BUILD_Z3))
>>>> message(FATAL_ERROR "Cannot disable static analyzer while enabling
>>>> ARCMT or Z3")
>>>> endif()
>>>>
>>>> Added: cfe/trunk/cmake/modules/ProtobufMutator.cmake
>>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/cmake/modules/
>>>> ProtobufMutator.cmake?rev=310408&view=auto
>>>> ============================================================
>>>> ==================
>>>> --- cfe/trunk/cmake/modules/ProtobufMutator.cmake (added)
>>>> +++ cfe/trunk/cmake/modules/ProtobufMutator.cmake Tue Aug 8 13:15:04
>>>> 2017
>>>> @@ -0,0 +1,24 @@
>>>> +set(PBM_PREFIX protobuf_mutator)
>>>> +set(PBM_PATH ${CMAKE_CURRENT_BINARY_DIR}/${
>>>> PBM_PREFIX}/src/${PBM_PREFIX})
>>>> +set(PBM_LIB_PATH ${PBM_PATH}/src/libprotobuf-mutator.a)
>>>> +set(PBM_FUZZ_LIB_PATH ${PBM_PATH}/src/libfuzzer/libp
>>>> rotobuf-mutator-libfuzzer.a)
>>>> +
>>>> +ExternalProject_Add(${PBM_PREFIX}
>>>> + PREFIX ${PBM_PREFIX}
>>>> + GIT_REPOSITORY https://github.com/google/libprotobuf-mutator.git
>>>> + GIT_TAG 34287f8
>>>> + CONFIGURE_COMMAND ${CMAKE_COMMAND} -G${CMAKE_GENERATOR}
>>>> + -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
>>>> + -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
>>>> + -DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}
>>>> + BUILD_COMMAND ${CMAKE_MAKE_PROGRAM}
>>>> + BUILD_BYPRODUCTS ${PBM_LIB_PATH} ${PBM_FUZZ_LIB_PATH}
>>>> + BUILD_IN_SOURCE 1
>>>> + INSTALL_COMMAND ""
>>>> + LOG_DOWNLOAD 1
>>>> + LOG_CONFIGURE 1
>>>> + LOG_BUILD 1
>>>> + )
>>>> +
>>>> +set(ProtobufMutator_INCLUDE_DIRS ${PBM_PATH})
>>>> +set(ProtobufMutator_LIBRARIES ${PBM_FUZZ_LIB_PATH} ${PBM_LIB_PATH})
>>>>
>>>> Modified: cfe/trunk/tools/clang-fuzzer/CMakeLists.txt
>>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu
>>>> zzer/CMakeLists.txt?rev=310408&r1=310407&r2=310408&view=diff
>>>> ============================================================
>>>> ==================
>>>> --- cfe/trunk/tools/clang-fuzzer/CMakeLists.txt (original)
>>>> +++ cfe/trunk/tools/clang-fuzzer/CMakeLists.txt Tue Aug 8 13:15:04
>>>> 2017
>>>> @@ -1,21 +1,60 @@
>>>> if( LLVM_USE_SANITIZE_COVERAGE )
>>>> set(LLVM_LINK_COMPONENTS ${LLVM_TARGETS_TO_BUILD})
>>>>
>>>> + if(CLANG_ENABLE_PROTO_FUZZER)
>>>> + # Create protobuf .h and .cc files, and put them in a library for
>>>> use by
>>>> + # clang-proto-fuzzer components.
>>>> + find_package(Protobuf REQUIRED)
>>>> + add_definitions(-DGOOGLE_PROTOBUF_NO_RTTI)
>>>> + include_directories(${PROTOBUF_INCLUDE_DIRS})
>>>> + include_directories(${CMAKE_CURRENT_BINARY_DIR})
>>>> + protobuf_generate_cpp(PROTO_SRCS PROTO_HDRS cxx_proto.proto)
>>>> + # Hack to bypass LLVM's cmake sources check and allow multiple
>>>> libraries and
>>>> + # executables from this directory.
>>>> + set(LLVM_OPTIONAL_SOURCES
>>>> + ClangFuzzer.cpp
>>>> + ExampleClangProtoFuzzer.cpp
>>>> + ${PROTO_SRCS}
>>>> + )
>>>> + add_clang_library(clangCXXProto
>>>> + ${PROTO_SRCS}
>>>> + ${PROTO_HDRS}
>>>> +
>>>> + LINK_LIBS
>>>> + ${PROTOBUF_LIBRARIES}
>>>> + )
>>>> +
>>>> + # Build and include libprotobuf-mutator
>>>> + include(ProtobufMutator)
>>>> + include_directories(${ProtobufMutator_INCLUDE_DIRS})
>>>> +
>>>> + # Build the protobuf->C++ translation library and driver.
>>>> + add_clang_subdirectory(proto-to-cxx)
>>>> +
>>>> + # Build the protobuf fuzzer
>>>> + add_clang_executable(clang-proto-fuzzer
>>>> ExampleClangProtoFuzzer.cpp)
>>>> + target_link_libraries(clang-proto-fuzzer
>>>> + ${ProtobufMutator_LIBRARIES}
>>>> + clangCXXProto
>>>> + clangHandleCXX
>>>> + clangProtoToCXX
>>>> + LLVMFuzzer
>>>> + )
>>>> + else()
>>>> + # Hack to bypass LLVM's cmake sources check and allow multiple
>>>> libraries and
>>>> + # executables from this directory.
>>>> + set(LLVM_OPTIONAL_SOURCES ClangFuzzer.cpp
>>>> ExampleClangProtoFuzzer.cpp)
>>>> + endif()
>>>> +
>>>> + add_clang_subdirectory(handle-cxx)
>>>> +
>>>> add_clang_executable(clang-fuzzer
>>>> EXCLUDE_FROM_ALL
>>>> ClangFuzzer.cpp
>>>> )
>>>>
>>>> target_link_libraries(clang-fuzzer
>>>> - ${CLANG_FORMAT_LIB_DEPS}
>>>> - clangAST
>>>> - clangBasic
>>>> - clangCodeGen
>>>> - clangDriver
>>>> - clangFrontend
>>>> - clangRewriteFrontend
>>>> - clangStaticAnalyzerFrontend
>>>> - clangTooling
>>>> + clangHandleCXX
>>>> LLVMFuzzer
>>>> )
>>>> endif()
>>>>
>>>> Modified: cfe/trunk/tools/clang-fuzzer/ClangFuzzer.cpp
>>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu
>>>> zzer/ClangFuzzer.cpp?rev=310408&r1=310407&r2=310408&view=diff
>>>> ============================================================
>>>> ==================
>>>> --- cfe/trunk/tools/clang-fuzzer/ClangFuzzer.cpp (original)
>>>> +++ cfe/trunk/tools/clang-fuzzer/ClangFuzzer.cpp Tue Aug 8 13:15:04
>>>> 2017
>>>> @@ -13,43 +13,12 @@
>>>> ///
>>>> //===------------------------------------------------------
>>>> ----------------===//
>>>>
>>>> -#include "clang/Tooling/Tooling.h"
>>>> -#include "clang/CodeGen/CodeGenAction.h"
>>>> -#include "clang/Frontend/CompilerInstance.h"
>>>> -#include "clang/Lex/PreprocessorOptions.h"
>>>> -#include "llvm/Option/Option.h"
>>>> -#include "llvm/Support/TargetSelect.h"
>>>> +#include "handle-cxx/handle_cxx.h"
>>>>
>>>> -using namespace clang;
>>>> +using namespace clang_fuzzer;
>>>>
>>>> extern "C" int LLVMFuzzerTestOneInput(uint8_t *data, size_t size) {
>>>> std::string s((const char *)data, size);
>>>> - llvm::InitializeAllTargets();
>>>> - llvm::InitializeAllTargetMCs();
>>>> - llvm::InitializeAllAsmPrinters();
>>>> - llvm::InitializeAllAsmParsers();
>>>> -
>>>> - llvm::opt::ArgStringList CC1Args;
>>>> - CC1Args.push_back("-cc1");
>>>> - CC1Args.push_back("./test.cc");
>>>> - CC1Args.push_back("-O2");
>>>> - llvm::IntrusiveRefCntPtr<FileManager> Files(
>>>> - new FileManager(FileSystemOptions()));
>>>> - IgnoringDiagConsumer Diags;
>>>> - IntrusiveRefCntPtr<DiagnosticOptions> DiagOpts = new
>>>> DiagnosticOptions();
>>>> - DiagnosticsEngine Diagnostics(
>>>> - IntrusiveRefCntPtr<clang::DiagnosticIDs>(new DiagnosticIDs()),
>>>> &*DiagOpts,
>>>> - &Diags, false);
>>>> - std::unique_ptr<clang::CompilerInvocation> Invocation(
>>>> - tooling::newInvocation(&Diagnostics, CC1Args));
>>>> - std::unique_ptr<llvm::MemoryBuffer> Input =
>>>> - llvm::MemoryBuffer::getMemBuffer(s);
>>>> - Invocation->getPreprocessorOpts().addRemappedFile("./test.cc",
>>>> Input.release());
>>>> - std::unique_ptr<tooling::ToolAction> action(
>>>> - tooling::newFrontendActionFactory<clang::EmitObjAction>());
>>>> - std::shared_ptr<PCHContainerOperations> PCHContainerOps =
>>>> - std::make_shared<PCHContainerOperations>();
>>>> - action->runInvocation(std::move(Invocation), Files.get(),
>>>> PCHContainerOps,
>>>> - &Diags);
>>>> + HandleCXX(s, {"-O2"});
>>>> return 0;
>>>> }
>>>>
>>>> Added: cfe/trunk/tools/clang-fuzzer/ExampleClangProtoFuzzer.cpp
>>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu
>>>> zzer/ExampleClangProtoFuzzer.cpp?rev=310408&view=auto
>>>> ============================================================
>>>> ==================
>>>> --- cfe/trunk/tools/clang-fuzzer/ExampleClangProtoFuzzer.cpp (added)
>>>> +++ cfe/trunk/tools/clang-fuzzer/ExampleClangProtoFuzzer.cpp Tue Aug
>>>> 8 13:15:04 2017
>>>> @@ -0,0 +1,28 @@
>>>> +//===-- ExampleClangProtoFuzzer.cpp - Fuzz Clang
>>>> --------------------------===//
>>>> +//
>>>> +// The LLVM Compiler Infrastructure
>>>> +//
>>>> +// This file is distributed under the University of Illinois Open
>>>> Source
>>>> +// License. See LICENSE.TXT for details.
>>>> +//
>>>> +//===------------------------------------------------------
>>>> ----------------===//
>>>> +///
>>>> +/// \file
>>>> +/// \brief This file implements a function that runs Clang on a single
>>>> +/// input and uses libprotobuf-mutator to find new inputs. This
>>>> function is
>>>> +/// then linked into the Fuzzer library.
>>>> +///
>>>> +//===------------------------------------------------------
>>>> ----------------===//
>>>> +
>>>> +#include "cxx_proto.pb.h"
>>>> +#include "handle-cxx/handle_cxx.h"
>>>> +#include "proto-to-cxx/proto_to_cxx.h"
>>>> +
>>>> +#include "src/libfuzzer/libfuzzer_macro.h"
>>>> +
>>>> +using namespace clang_fuzzer;
>>>> +
>>>> +DEFINE_BINARY_PROTO_FUZZER(const Function& input) {
>>>> + auto S = FunctionToString(input);
>>>> + HandleCXX(S, {"-O2"});
>>>> +}
>>>>
>>>> Added: cfe/trunk/tools/clang-fuzzer/README.txt
>>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu
>>>> zzer/README.txt?rev=310408&view=auto
>>>> ============================================================
>>>> ==================
>>>> --- cfe/trunk/tools/clang-fuzzer/README.txt (added)
>>>> +++ cfe/trunk/tools/clang-fuzzer/README.txt Tue Aug 8 13:15:04 2017
>>>> @@ -0,0 +1,73 @@
>>>> +This directory contains two utilities for fuzzing Clang: clang-fuzzer
>>>> and
>>>> +clang-proto-fuzzer. Both use libFuzzer to generate inputs to clang via
>>>> +coverage-guided mutation.
>>>> +
>>>> +The two utilities differ, however, in how they structure inputs to
>>>> Clang.
>>>> +clang-fuzzer makes no attempt to generate valid C++ programs and is
>>>> therefore
>>>> +primarily useful for stressing the surface layers of Clang (i.e.
>>>> lexer, parser).
>>>> +clang-proto-fuzzer uses a protobuf class to describe a subset of the
>>>> C++
>>>> +language and then uses libprotobuf-mutator to mutate instantiations of
>>>> that
>>>> +class, producing valid C++ programs in the process. As a result,
>>>> +clang-proto-fuzzer is better at stressing deeper layers of Clang and
>>>> LLVM.
>>>> +
>>>> +===================================
>>>> + Building clang-fuzzer
>>>> +===================================
>>>> +Within your LLVM build directory, run CMake with the following variable
>>>> +definitions:
>>>> +- CMAKE_C_COMPILER=clang
>>>> +- CMAKE_CXX_COMPILER=clang++
>>>> +- LLVM_USE_SANITIZE_COVERAGE=YES
>>>> +- LLVM_USE_SANITIZER=Address
>>>> +
>>>> +Then build the clang-fuzzer target.
>>>> +
>>>> +Example:
>>>> + cd $LLVM_SOURCE_DIR
>>>> + mkdir build && cd build
>>>> + cmake .. -GNinja -DCMAKE_C_COMPILER=clang
>>>> -DCMAKE_CXX_COMPILER=clang++ \
>>>> + -DLLVM_USE_SANITIZE_COVERAGE=YES -DLLVM_USE_SANITIZER=Address
>>>> + ninja clang-fuzzer
>>>> +
>>>> +
>>>> +=======================================================
>>>> + Building clang-proto-fuzzer (Linux-only instructions)
>>>> +=======================================================
>>>> +Install the necessary dependencies:
>>>> +- binutils // needed for libprotobuf-mutator
>>>> +- liblzma-dev // needed for libprotobuf-mutator
>>>> +- libz-dev // needed for libprotobuf-mutator
>>>> +- docbook2x // needed for libprotobuf-mutator
>>>> +- Recent version of protobuf [3.3.0 is known to work]
>>>> +
>>>> +Within your LLVM build directory, run CMake with the following variable
>>>> +definitions:
>>>> +- CMAKE_C_COMPILER=clang
>>>> +- CMAKE_CXX_COMPILER=clang++
>>>> +- LLVM_USE_SANITIZE_COVERAGE=YES
>>>> +- LLVM_USE_SANITIZER=Address
>>>> +- CLANG_ENABLE_PROTO_FUZZER=ON
>>>> +
>>>> +Then build the clang-proto-fuzzer and clang-proto-to-cxx targets.
>>>> Optionally,
>>>> +you may also build clang-fuzzer with this setup.
>>>> +
>>>> +Example:
>>>> + cd $LLVM_SOURCE_DIR
>>>> + mkdir build && cd build
>>>> + cmake .. -GNinja -DCMAKE_C_COMPILER=clang
>>>> -DCMAKE_CXX_COMPILER=clang++ \
>>>> + -DLLVM_USE_SANITIZE_COVERAGE=YES -DLLVM_USE_SANITIZER=Address \
>>>> + -DCLANG_ENABLE_PROTO_FUZZER=ON
>>>> + ninja clang-proto-fuzzer clang-proto-to-cxx
>>>> +
>>>> +
>>>> +=====================
>>>> + Running the fuzzers
>>>> +=====================
>>>> +clang-fuzzer:
>>>> + bin/clang-fuzzer CORPUS_DIR
>>>> +
>>>> +clang-proto-fuzzer:
>>>> + bin/clang-proto-fuzzer CORPUS_DIR
>>>> +
>>>> +Translating a clang-proto-fuzzer corpus output to C++:
>>>> + bin/clang-proto-to-cxx CORPUS_OUTPUT_FILE
>>>>
>>>> Added: cfe/trunk/tools/clang-fuzzer/cxx_proto.proto
>>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu
>>>> zzer/cxx_proto.proto?rev=310408&view=auto
>>>> ============================================================
>>>> ==================
>>>> --- cfe/trunk/tools/clang-fuzzer/cxx_proto.proto (added)
>>>> +++ cfe/trunk/tools/clang-fuzzer/cxx_proto.proto Tue Aug 8 13:15:04
>>>> 2017
>>>> @@ -0,0 +1,93 @@
>>>> +//===-- cxx_proto.proto - Protobuf description of C++
>>>> ---------------------===//
>>>> +//
>>>> +// The LLVM Compiler Infrastructure
>>>> +//
>>>> +// This file is distributed under the University of Illinois Open
>>>> Source
>>>> +// License. See LICENSE.TXT for details.
>>>> +//
>>>> +//===------------------------------------------------------
>>>> ----------------===//
>>>> +///
>>>> +/// \file
>>>> +/// \brief This file describes a subset of C++ as a protobuf. It is
>>>> used to
>>>> +/// more easily find interesting inputs for fuzzing Clang.
>>>> +///
>>>> +//===------------------------------------------------------
>>>> ----------------===//
>>>> +
>>>> +syntax = "proto2";
>>>> +
>>>> +message VarRef {
>>>> + required int32 varnum = 1;
>>>> +}
>>>> +
>>>> +message Lvalue {
>>>> + required VarRef varref = 1;
>>>> +}
>>>> +
>>>> +message Const {
>>>> + required int32 val = 1;
>>>> +}
>>>> +
>>>> +message BinaryOp {
>>>> + enum Op {
>>>> + PLUS = 0;
>>>> + MINUS = 1;
>>>> + MUL = 2;
>>>> + DIV = 3;
>>>> + MOD = 4;
>>>> + XOR = 5;
>>>> + AND = 6;
>>>> + OR = 7;
>>>> + EQ = 8;
>>>> + NE = 9;
>>>> + LE = 10;
>>>> + GE = 11;
>>>> + LT = 12;
>>>> + GT = 13;
>>>> + };
>>>> + required Op op = 1;
>>>> + required Rvalue left = 2;
>>>> + required Rvalue right = 3;
>>>> +}
>>>> +
>>>> +message Rvalue {
>>>> + oneof rvalue_oneof {
>>>> + VarRef varref = 1;
>>>> + Const cons = 2;
>>>> + BinaryOp binop = 3;
>>>> + }
>>>> +}
>>>> +
>>>> +message AssignmentStatement {
>>>> + required Lvalue lvalue = 1;
>>>> + required Rvalue rvalue = 2;
>>>> +}
>>>> +
>>>> +
>>>> +message IfElse {
>>>> + required Rvalue cond = 1;
>>>> + required StatementSeq if_body = 2;
>>>> + required StatementSeq else_body = 3;
>>>> +}
>>>> +
>>>> +message While {
>>>> + required Rvalue cond = 1;
>>>> + required StatementSeq body = 2;
>>>> +}
>>>> +
>>>> +message Statement {
>>>> + oneof stmt_oneof {
>>>> + AssignmentStatement assignment = 1;
>>>> + IfElse ifelse = 2;
>>>> + While while_loop = 3;
>>>> + }
>>>> +}
>>>> +
>>>> +message StatementSeq {
>>>> + repeated Statement statements = 1;
>>>> +}
>>>> +
>>>> +message Function {
>>>> + required StatementSeq statements = 1;
>>>> +}
>>>> +
>>>> +package clang_fuzzer;
>>>>
>>>> Added: cfe/trunk/tools/clang-fuzzer/handle-cxx/CMakeLists.txt
>>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu
>>>> zzer/handle-cxx/CMakeLists.txt?rev=310408&view=auto
>>>> ============================================================
>>>> ==================
>>>> --- cfe/trunk/tools/clang-fuzzer/handle-cxx/CMakeLists.txt (added)
>>>> +++ cfe/trunk/tools/clang-fuzzer/handle-cxx/CMakeLists.txt Tue Aug 8
>>>> 13:15:04 2017
>>>> @@ -0,0 +1,11 @@
>>>> +set(LLVM_LINK_COMPONENTS ${LLVM_TARGETS_TO_BUILD})
>>>> +
>>>> +add_clang_library(clangHandleCXX
>>>> + handle_cxx.cpp
>>>> +
>>>> + LINK_LIBS
>>>> + clangCodeGen
>>>> + clangFrontend
>>>> + clangLex
>>>> + clangTooling
>>>> + )
>>>>
>>>> Added: cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.cpp
>>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu
>>>> zzer/handle-cxx/handle_cxx.cpp?rev=310408&view=auto
>>>> ============================================================
>>>> ==================
>>>> --- cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.cpp (added)
>>>> +++ cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.cpp Tue Aug 8
>>>> 13:15:04 2017
>>>> @@ -0,0 +1,58 @@
>>>> +//==-- handle_cxx.cpp - Helper function for Clang fuzzers
>>>> ------------------==//
>>>> +//
>>>> +// The LLVM Compiler Infrastructure
>>>> +//
>>>> +// This file is distributed under the University of Illinois Open
>>>> Source
>>>> +// License. See LICENSE.TXT for details.
>>>> +//
>>>> +//===------------------------------------------------------
>>>> ----------------===//
>>>> +//
>>>> +// Implements HandleCXX for use by the Clang fuzzers.
>>>> +//
>>>> +//===------------------------------------------------------
>>>> ----------------===//
>>>> +
>>>> +#include "handle_cxx.h"
>>>> +
>>>> +#include "clang/CodeGen/CodeGenAction.h"
>>>> +#include "clang/Frontend/CompilerInstance.h"
>>>> +#include "clang/Lex/PreprocessorOptions.h"
>>>> +#include "clang/Tooling/Tooling.h"
>>>> +#include "llvm/Option/Option.h"
>>>> +#include "llvm/Support/TargetSelect.h"
>>>> +
>>>> +using namespace clang;
>>>> +
>>>> +void clang_fuzzer::HandleCXX(const std::string &S,
>>>> + const std::vector<const char *>
>>>> &ExtraArgs) {
>>>> + llvm::InitializeAllTargets();
>>>> + llvm::InitializeAllTargetMCs();
>>>> + llvm::InitializeAllAsmPrinters();
>>>> + llvm::InitializeAllAsmParsers();
>>>> +
>>>> + llvm::opt::ArgStringList CC1Args;
>>>> + CC1Args.push_back("-cc1");
>>>> + for (auto &A : ExtraArgs)
>>>> + CC1Args.push_back(A);
>>>> + CC1Args.push_back("./test.cc");
>>>> +
>>>> + llvm::IntrusiveRefCntPtr<FileManager> Files(
>>>> + new FileManager(FileSystemOptions()));
>>>> + IgnoringDiagConsumer Diags;
>>>> + IntrusiveRefCntPtr<DiagnosticOptions> DiagOpts = new
>>>> DiagnosticOptions();
>>>> + DiagnosticsEngine Diagnostics(
>>>> + IntrusiveRefCntPtr<clang::DiagnosticIDs>(new DiagnosticIDs()),
>>>> &*DiagOpts,
>>>> + &Diags, false);
>>>> + std::unique_ptr<clang::CompilerInvocation> Invocation(
>>>> + tooling::newInvocation(&Diagnostics, CC1Args));
>>>> + std::unique_ptr<llvm::MemoryBuffer> Input =
>>>> + llvm::MemoryBuffer::getMemBuffer(S);
>>>> + Invocation->getPreprocessorOpts().addRemappedFile("./test.cc",
>>>> + Input.release());
>>>> + std::unique_ptr<tooling::ToolAction> action(
>>>> + tooling::newFrontendActionFactory<clang::EmitObjAction>());
>>>> + std::shared_ptr<PCHContainerOperations> PCHContainerOps =
>>>> + std::make_shared<PCHContainerOperations>();
>>>> + action->runInvocation(std::move(Invocation), Files.get(),
>>>> PCHContainerOps,
>>>> + &Diags);
>>>> +}
>>>> +
>>>>
>>>> Added: cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.h
>>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu
>>>> zzer/handle-cxx/handle_cxx.h?rev=310408&view=auto
>>>> ============================================================
>>>> ==================
>>>> --- cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.h (added)
>>>> +++ cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.h Tue Aug 8
>>>> 13:15:04 2017
>>>> @@ -0,0 +1,25 @@
>>>> +//==-- handle_cxx.h - Helper function for Clang fuzzers
>>>> --------------------==//
>>>> +//
>>>> +// The LLVM Compiler Infrastructure
>>>> +//
>>>> +// This file is distributed under the University of Illinois Open
>>>> Source
>>>> +// License. See LICENSE.TXT for details.
>>>> +//
>>>> +//===------------------------------------------------------
>>>> ----------------===//
>>>> +//
>>>> +// Defines HandleCXX for use by the Clang fuzzers.
>>>> +//
>>>> +//===------------------------------------------------------
>>>> ----------------===//
>>>> +
>>>> +#ifndef LLVM_CLANG_TOOLS_CLANG_FUZZER_HANDLE_CXX_HANDLECXX_H
>>>> +#define LLVM_CLANG_TOOLS_CLANG_FUZZER_HANDLE_CXX_HANDLECXX_H
>>>> +
>>>> +#include <string>
>>>> +#include <vector>
>>>> +
>>>> +namespace clang_fuzzer {
>>>> +void HandleCXX(const std::string &S,
>>>> + const std::vector<const char *> &ExtraArgs);
>>>> +} // namespace clang_fuzzer
>>>> +
>>>> +#endif
>>>>
>>>> Added: cfe/trunk/tools/clang-fuzzer/proto-to-cxx/CMakeLists.txt
>>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu
>>>> zzer/proto-to-cxx/CMakeLists.txt?rev=310408&view=auto
>>>> ============================================================
>>>> ==================
>>>> --- cfe/trunk/tools/clang-fuzzer/proto-to-cxx/CMakeLists.txt (added)
>>>> +++ cfe/trunk/tools/clang-fuzzer/proto-to-cxx/CMakeLists.txt Tue Aug
>>>> 8 13:15:04 2017
>>>> @@ -0,0 +1,10 @@
>>>> +set(LLVM_LINK_COMPONENTS ${LLVM_TARGETS_TO_BUILD})
>>>> +
>>>> +# Hack to bypass LLVM's CMake source checks so we can have both a
>>>> library and
>>>> +# an executable built from this directory.
>>>> +set(LLVM_OPTIONAL_SOURCES proto_to_cxx.cpp proto_to_cxx_main.cpp)
>>>> +
>>>> +add_clang_library(clangProtoToCXX proto_to_cxx.cpp LINK_LIBS
>>>> clangCXXProto)
>>>> +
>>>> +add_clang_executable(clang-proto-to-cxx proto_to_cxx_main.cpp)
>>>> +target_link_libraries(clang-proto-to-cxx clangProtoToCXX)
>>>>
>>>> Added: cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.cpp
>>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu
>>>> zzer/proto-to-cxx/proto_to_cxx.cpp?rev=310408&view=auto
>>>> ============================================================
>>>> ==================
>>>> --- cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.cpp (added)
>>>> +++ cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.cpp Tue
>>>> Aug 8 13:15:04 2017
>>>> @@ -0,0 +1,102 @@
>>>> +//==-- proto_to_cxx.cpp - Protobuf-C++ conversion
>>>> --------------------------==//
>>>> +//
>>>> +// The LLVM Compiler Infrastructure
>>>> +//
>>>> +// This file is distributed under the University of Illinois Open
>>>> Source
>>>> +// License. See LICENSE.TXT for details.
>>>> +//
>>>> +//===------------------------------------------------------
>>>> ----------------===//
>>>> +//
>>>> +// Implements functions for converting between protobufs and C++.
>>>> +//
>>>> +//===------------------------------------------------------
>>>> ----------------===//
>>>> +
>>>> +#include "proto_to_cxx.h"
>>>> +#include "cxx_proto.pb.h"
>>>> +
>>>> +#include <ostream>
>>>> +#include <sstream>
>>>> +
>>>> +namespace clang_fuzzer {
>>>> +
>>>> +// Forward decls.
>>>> +std::ostream &operator<<(std::ostream &os, const BinaryOp &x);
>>>> +std::ostream &operator<<(std::ostream &os, const StatementSeq &x);
>>>> +
>>>> +// Proto to C++.
>>>> +std::ostream &operator<<(std::ostream &os, const Const &x) {
>>>> + return os << "(" << x.val() << ")";
>>>> +}
>>>> +std::ostream &operator<<(std::ostream &os, const VarRef &x) {
>>>> + return os << "a[" << (static_cast<uint32_t>(x.varnum()) % 100) <<
>>>> "]";
>>>> +}
>>>> +std::ostream &operator<<(std::ostream &os, const Lvalue &x) {
>>>> + return os << x.varref();
>>>> +}
>>>> +std::ostream &operator<<(std::ostream &os, const Rvalue &x) {
>>>> + if (x.has_varref()) return os << x.varref();
>>>> + if (x.has_cons()) return os << x.cons();
>>>> + if (x.has_binop()) return os << x.binop();
>>>> + return os << "1";
>>>> +}
>>>> +std::ostream &operator<<(std::ostream &os, const BinaryOp &x) {
>>>> + os << "(" << x.left();
>>>> + switch (x.op()) {
>>>> + case BinaryOp::PLUS: os << "+"; break;
>>>> + case BinaryOp::MINUS: os << "-"; break;
>>>> + case BinaryOp::MUL: os << "*"; break;
>>>> + case BinaryOp::DIV: os << "/"; break;
>>>> + case BinaryOp::MOD: os << "%"; break;
>>>> + case BinaryOp::XOR: os << "^"; break;
>>>> + case BinaryOp::AND: os << "&"; break;
>>>> + case BinaryOp::OR: os << "|"; break;
>>>> + case BinaryOp::EQ: os << "=="; break;
>>>> + case BinaryOp::NE: os << "!="; break;
>>>> + case BinaryOp::LE: os << "<="; break;
>>>> + case BinaryOp::GE: os << ">="; break;
>>>> + case BinaryOp::LT: os << "<"; break;
>>>> + case BinaryOp::GT: os << ">"; break;
>>>> + }
>>>> + return os << x.right() << ")";
>>>> +}
>>>> +std::ostream &operator<<(std::ostream &os, const AssignmentStatement
>>>> &x) {
>>>> + return os << x.lvalue() << "=" << x.rvalue() << ";\n";
>>>> +}
>>>> +std::ostream &operator<<(std::ostream &os, const IfElse &x) {
>>>> + return os << "if (" << x.cond() << "){\n"
>>>> + << x.if_body() << "} else { \n"
>>>> + << x.else_body() << "}\n";
>>>> +}
>>>> +std::ostream &operator<<(std::ostream &os, const While &x) {
>>>> + return os << "while (" << x.cond() << "){\n" << x.body() << "}\n";
>>>> +}
>>>> +std::ostream &operator<<(std::ostream &os, const Statement &x) {
>>>> + if (x.has_assignment()) return os << x.assignment();
>>>> + if (x.has_ifelse()) return os << x.ifelse();
>>>> + if (x.has_while_loop()) return os << x.while_loop();
>>>> + return os << "(void)0;\n";
>>>> +}
>>>> +std::ostream &operator<<(std::ostream &os, const StatementSeq &x) {
>>>> + for (auto &st : x.statements()) os << st;
>>>> + return os;
>>>> +}
>>>> +std::ostream &operator<<(std::ostream &os, const Function &x) {
>>>> + return os << "void foo(int *a) {\n" << x.statements() << "}\n";
>>>> +}
>>>> +
>>>> +// ---------------------------------
>>>> +
>>>> +std::string FunctionToString(const Function &input) {
>>>> + std::ostringstream os;
>>>> + os << input;
>>>> + return os.str();
>>>> +
>>>> +}
>>>> +std::string ProtoToCxx(const uint8_t *data, size_t size) {
>>>> + Function message;
>>>> + if (!message.ParseFromArray(data, size))
>>>> + return "#error invalid proto\n";
>>>> + return FunctionToString(message);
>>>> +}
>>>> +
>>>> +} // namespace clang_fuzzer
>>>>
>>>> Added: cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.h
>>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu
>>>> zzer/proto-to-cxx/proto_to_cxx.h?rev=310408&view=auto
>>>> ============================================================
>>>> ==================
>>>> --- cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.h (added)
>>>> +++ cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.h Tue Aug
>>>> 8 13:15:04 2017
>>>> @@ -0,0 +1,22 @@
>>>> +//==-- proto_to_cxx.h - Protobuf-C++ conversion
>>>> ----------------------------==//
>>>> +//
>>>> +// The LLVM Compiler Infrastructure
>>>> +//
>>>> +// This file is distributed under the University of Illinois Open
>>>> Source
>>>> +// License. See LICENSE.TXT for details.
>>>> +//
>>>> +//===------------------------------------------------------
>>>> ----------------===//
>>>> +//
>>>> +// Defines functions for converting between protobufs and C++.
>>>> +//
>>>> +//===------------------------------------------------------
>>>> ----------------===//
>>>> +
>>>> +#include <cstdint>
>>>> +#include <cstddef>
>>>> +#include <string>
>>>> +
>>>> +namespace clang_fuzzer {
>>>> +class Function;
>>>> +std::string FunctionToString(const Function &input);
>>>> +std::string ProtoToCxx(const uint8_t *data, size_t size);
>>>> +}
>>>>
>>>> Added: cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx_main.cpp
>>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu
>>>> zzer/proto-to-cxx/proto_to_cxx_main.cpp?rev=310408&view=auto
>>>> ============================================================
>>>> ==================
>>>> --- cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx_main.cpp
>>>> (added)
>>>> +++ cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx_main.cpp
>>>> Tue Aug 8 13:15:04 2017
>>>> @@ -0,0 +1,30 @@
>>>> +//==-- proto_to_cxx_main.cpp - Driver for protobuf-C++ conversion
>>>> ----------==//
>>>> +//
>>>> +// The LLVM Compiler Infrastructure
>>>> +//
>>>> +// This file is distributed under the University of Illinois Open
>>>> Source
>>>> +// License. See LICENSE.TXT for details.
>>>> +//
>>>> +//===------------------------------------------------------
>>>> ----------------===//
>>>> +//
>>>> +// Implements a simple driver to print a C++ program from a protobuf.
>>>> +//
>>>> +//===------------------------------------------------------
>>>> ----------------===//
>>>> +#include <fstream>
>>>> +#include <iostream>
>>>> +#include <streambuf>
>>>> +#include <string>
>>>> +
>>>> +#include "proto_to_cxx.h"
>>>> +
>>>> +int main(int argc, char **argv) {
>>>> + for (int i = 1; i < argc; i++) {
>>>> + std::fstream in(argv[i]);
>>>> + std::string str((std::istreambuf_iterator<char>(in)),
>>>> + std::istreambuf_iterator<char>());
>>>> + std::cout << "// " << argv[i] << std::endl;
>>>> + std::cout << clang_fuzzer::ProtoToCxx(
>>>> + reinterpret_cast<const uint8_t *>(str.data()), str.size());
>>>> + }
>>>> +}
>>>> +
>>>>
>>>>
>>>> _______________________________________________
>>>> cfe-commits mailing list
>>>> cfe-commits at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
>>>>
>>>
>>> _______________________________________________
>>> cfe-commits mailing list
>>> cfe-commits at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20170810/b2dfb1b5/attachment-0001.html>
More information about the cfe-commits
mailing list