r310408 - Integrate Kostya's clang-proto-fuzzer with LLVM.
Nico Weber via cfe-commits
cfe-commits at lists.llvm.org
Thu Aug 10 12:01:47 PDT 2017
On Thu, Aug 10, 2017 at 2:04 PM, Kostya Serebryany <kcc at google.com> wrote:
>
>
> On Thu, Aug 10, 2017 at 10:56 AM, Nico Weber via cfe-commits <
> cfe-commits at lists.llvm.org> wrote:
>
>> I really believe this has way too many deps to live in the clang repro,
>> as said on the review already.
>>
>
> I don't have a very strong opinion here and would be happy to move if I
> see more support for Nico's opinion
> (I haven't seen it on the review, and you didn't object further, so we
> proceeded).
> Again, my rational is that the simpler it is to use the more likely other
> researchers will extend this work.
>
> BTW, I am going to commit a Dockerfile that will make experimenting with
> this trivial.
> My current (dirty) version looks like this. Not too much trouble.
>
> FROM ubuntu:16.04
> RUN apt-get update -y && apt-get install -y autoconf automake libtool curl
> make g++ unzip
> RUN apt-get install -y wget
> RUN apt-get install -y git binutils liblzma-dev libz-dev
> RUN apt-get install -y python-all
> RUN apt-get install -y cmake ninja-build
> RUN apt-get install -y subversion
>
> WORKDIR /root
> RUN wget -qO- https://github.com/google/protobuf/releases/download/v3.
> 3.0/protobuf-cpp-3.3.0.tar.gz | tar zxf -
> RUN cd protobuf-3.3.0 && ./autogen.sh && ./configure && make -j $(nproc)
> && make check -j $(nproc) && make install && ldconfig
> RUN apt-get install -y pkg-config
> RUN svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm
> RUN cd llvm/tools && svn co http://llvm.org/svn/llvm-project/cfe/trunk
> clang -r $(cd ../ && svn info | grep Revision | awk '{print $2}')
> RUN cd llvm/projects && svn co http://llvm.org/svn/llvm-
> project/compiler-rt/trunk clang -r $(cd ../ && svn info | grep Revision |
> awk '{print $2}')
> RUN mkdir build0 && cd build0 && cmake -GNinja -DCMAKE_BUILD_TYPE=Release
> ../llvm && ninja
> RUN mkdir build1 && cd build1 && cmake -GNinja -DCMAKE_BUILD_TYPE=Release
> ../llvm -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_C_COMPILER=`pwd`/../build0/bin/clang
> -DCMAKE_CXX_COMPILER=`pwd`/../build0/bin/clang++
> -DLLVM_USE_SANITIZE_COVERAGE=YES -DLLVM_USE_SANITIZER=Address
> -DCLANG_ENABLE_PROTO_FUZZER=ON
> RUN cd build1 && ninja clang-fuzzer
> RUN cd build1 && ninja clang-proto-fuzzer
> #RUN cd build1 && ninja clang-proto-to-cxx
>
>
>
>> Maybe this could live in clang-extra instead?
>>
>
> clang-extra?
>
clang-tools-extra, sorry.
> That's a separate repo, right?
>
Yes.
> It may require more cmake trickery, and we'll also have to share the
> clang-fuzzer-specific code between two repos.
>
We could move the whole thing. I'd imagine that at most 3% of people who
use clang will use this fuzzer, so having it elsewhere seems reasonable.
(I'd imagine many more people to use clang-tidy for example, and that's in
the other repro.) Also see the "Contributing Extensions to Clang" section
on http://clang.llvm.org/get_involved.html
> I do want the original clang-fuzzer to remain where it was, and both
> (clang-fuzzer and clang-proto-fuzzer) share the code.
>
>
>
>
>>
>> On Aug 8, 2017 4:15 PM, "Matt Morehouse via cfe-commits" <
>> cfe-commits at lists.llvm.org> wrote:
>>
>>> Author: morehouse
>>> Date: Tue Aug 8 13:15:04 2017
>>> New Revision: 310408
>>>
>>> URL: http://llvm.org/viewvc/llvm-project?rev=310408&view=rev
>>> Log:
>>> Integrate Kostya's clang-proto-fuzzer with LLVM.
>>>
>>> Summary:
>>> The clang-proto-fuzzer models a subset of C++ as a protobuf and
>>> uses libprotobuf-mutator to generate interesting mutations of C++
>>> programs. Clang-proto-fuzzer has already found several bugs in
>>> Clang (e.g., https://bugs.llvm.org/show_bug.cgi?id=33747,
>>> https://bugs.llvm.org/show_bug.cgi?id=33749).
>>>
>>> As with clang-fuzzer, clang-proto-fuzzer requires the following
>>> cmake flags:
>>> - CMAKE_C_COMPILER=clang
>>> - CMAKE_CXX_COMPILER=clang++
>>> - LLVM_USE_SANITIZE_COVERAGE=YES // needed for libFuzzer
>>> - LLVM_USE_SANITIZER=Address // needed for libFuzzer
>>>
>>> In addition, clang-proto-fuzzer requires:
>>> - CLANG_ENABLE_PROTO_FUZZER=ON
>>>
>>> clang-proto-fuzzer also requires the following dependencies:
>>> - binutils // needed for libprotobuf-mutator
>>> - liblzma-dev // needed for libprotobuf-mutator
>>> - libz-dev // needed for libprotobuf-mutator
>>> - docbook2x // needed for libprotobuf-mutator
>>> - Recent version of protobuf [3.3.0 is known to work]
>>>
>>> A working version of libprotobuf-mutator will automatically be
>>> downloaded and built as an external project.
>>>
>>> Implementation of clang-proto-fuzzer provided by Kostya
>>> Serebryany.
>>>
>>> https://bugs.llvm.org/show_bug.cgi?id=33829
>>>
>>> Reviewers: kcc, vitalybuka, bogner
>>>
>>> Reviewed By: kcc, vitalybuka
>>>
>>> Subscribers: thakis, mgorny, cfe-commits
>>>
>>> Differential Revision: https://reviews.llvm.org/D36324
>>>
>>> Added:
>>> cfe/trunk/cmake/modules/ProtobufMutator.cmake
>>> cfe/trunk/tools/clang-fuzzer/ExampleClangProtoFuzzer.cpp
>>> cfe/trunk/tools/clang-fuzzer/README.txt
>>> cfe/trunk/tools/clang-fuzzer/cxx_proto.proto
>>> cfe/trunk/tools/clang-fuzzer/handle-cxx/
>>> cfe/trunk/tools/clang-fuzzer/handle-cxx/CMakeLists.txt
>>> cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.cpp
>>> cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.h
>>> cfe/trunk/tools/clang-fuzzer/proto-to-cxx/
>>> cfe/trunk/tools/clang-fuzzer/proto-to-cxx/CMakeLists.txt
>>> cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.cpp
>>> cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.h
>>> cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx_main.cpp
>>> Modified:
>>> cfe/trunk/CMakeLists.txt
>>> cfe/trunk/tools/clang-fuzzer/CMakeLists.txt
>>> cfe/trunk/tools/clang-fuzzer/ClangFuzzer.cpp
>>>
>>> Modified: cfe/trunk/CMakeLists.txt
>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/CMakeLists.txt
>>> ?rev=310408&r1=310407&r2=310408&view=diff
>>> ============================================================
>>> ==================
>>> --- cfe/trunk/CMakeLists.txt (original)
>>> +++ cfe/trunk/CMakeLists.txt Tue Aug 8 13:15:04 2017
>>> @@ -377,6 +377,8 @@ option(CLANG_ENABLE_STATIC_ANALYZER "Bui
>>> option(CLANG_ANALYZER_BUILD_Z3
>>> "Build the static analyzer with the Z3 constraint manager." OFF)
>>>
>>> +option(CLANG_ENABLE_PROTO_FUZZER "Build Clang protobuf fuzzer." OFF)
>>> +
>>> if(NOT CLANG_ENABLE_STATIC_ANALYZER AND (CLANG_ENABLE_ARCMT OR
>>> CLANG_ANALYZER_BUILD_Z3))
>>> message(FATAL_ERROR "Cannot disable static analyzer while enabling
>>> ARCMT or Z3")
>>> endif()
>>>
>>> Added: cfe/trunk/cmake/modules/ProtobufMutator.cmake
>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/cmake/modules/
>>> ProtobufMutator.cmake?rev=310408&view=auto
>>> ============================================================
>>> ==================
>>> --- cfe/trunk/cmake/modules/ProtobufMutator.cmake (added)
>>> +++ cfe/trunk/cmake/modules/ProtobufMutator.cmake Tue Aug 8 13:15:04
>>> 2017
>>> @@ -0,0 +1,24 @@
>>> +set(PBM_PREFIX protobuf_mutator)
>>> +set(PBM_PATH ${CMAKE_CURRENT_BINARY_DIR}/${
>>> PBM_PREFIX}/src/${PBM_PREFIX})
>>> +set(PBM_LIB_PATH ${PBM_PATH}/src/libprotobuf-mutator.a)
>>> +set(PBM_FUZZ_LIB_PATH ${PBM_PATH}/src/libfuzzer/libp
>>> rotobuf-mutator-libfuzzer.a)
>>> +
>>> +ExternalProject_Add(${PBM_PREFIX}
>>> + PREFIX ${PBM_PREFIX}
>>> + GIT_REPOSITORY https://github.com/google/libprotobuf-mutator.git
>>> + GIT_TAG 34287f8
>>> + CONFIGURE_COMMAND ${CMAKE_COMMAND} -G${CMAKE_GENERATOR}
>>> + -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
>>> + -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
>>> + -DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}
>>> + BUILD_COMMAND ${CMAKE_MAKE_PROGRAM}
>>> + BUILD_BYPRODUCTS ${PBM_LIB_PATH} ${PBM_FUZZ_LIB_PATH}
>>> + BUILD_IN_SOURCE 1
>>> + INSTALL_COMMAND ""
>>> + LOG_DOWNLOAD 1
>>> + LOG_CONFIGURE 1
>>> + LOG_BUILD 1
>>> + )
>>> +
>>> +set(ProtobufMutator_INCLUDE_DIRS ${PBM_PATH})
>>> +set(ProtobufMutator_LIBRARIES ${PBM_FUZZ_LIB_PATH} ${PBM_LIB_PATH})
>>>
>>> Modified: cfe/trunk/tools/clang-fuzzer/CMakeLists.txt
>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu
>>> zzer/CMakeLists.txt?rev=310408&r1=310407&r2=310408&view=diff
>>> ============================================================
>>> ==================
>>> --- cfe/trunk/tools/clang-fuzzer/CMakeLists.txt (original)
>>> +++ cfe/trunk/tools/clang-fuzzer/CMakeLists.txt Tue Aug 8 13:15:04 2017
>>> @@ -1,21 +1,60 @@
>>> if( LLVM_USE_SANITIZE_COVERAGE )
>>> set(LLVM_LINK_COMPONENTS ${LLVM_TARGETS_TO_BUILD})
>>>
>>> + if(CLANG_ENABLE_PROTO_FUZZER)
>>> + # Create protobuf .h and .cc files, and put them in a library for
>>> use by
>>> + # clang-proto-fuzzer components.
>>> + find_package(Protobuf REQUIRED)
>>> + add_definitions(-DGOOGLE_PROTOBUF_NO_RTTI)
>>> + include_directories(${PROTOBUF_INCLUDE_DIRS})
>>> + include_directories(${CMAKE_CURRENT_BINARY_DIR})
>>> + protobuf_generate_cpp(PROTO_SRCS PROTO_HDRS cxx_proto.proto)
>>> + # Hack to bypass LLVM's cmake sources check and allow multiple
>>> libraries and
>>> + # executables from this directory.
>>> + set(LLVM_OPTIONAL_SOURCES
>>> + ClangFuzzer.cpp
>>> + ExampleClangProtoFuzzer.cpp
>>> + ${PROTO_SRCS}
>>> + )
>>> + add_clang_library(clangCXXProto
>>> + ${PROTO_SRCS}
>>> + ${PROTO_HDRS}
>>> +
>>> + LINK_LIBS
>>> + ${PROTOBUF_LIBRARIES}
>>> + )
>>> +
>>> + # Build and include libprotobuf-mutator
>>> + include(ProtobufMutator)
>>> + include_directories(${ProtobufMutator_INCLUDE_DIRS})
>>> +
>>> + # Build the protobuf->C++ translation library and driver.
>>> + add_clang_subdirectory(proto-to-cxx)
>>> +
>>> + # Build the protobuf fuzzer
>>> + add_clang_executable(clang-proto-fuzzer
>>> ExampleClangProtoFuzzer.cpp)
>>> + target_link_libraries(clang-proto-fuzzer
>>> + ${ProtobufMutator_LIBRARIES}
>>> + clangCXXProto
>>> + clangHandleCXX
>>> + clangProtoToCXX
>>> + LLVMFuzzer
>>> + )
>>> + else()
>>> + # Hack to bypass LLVM's cmake sources check and allow multiple
>>> libraries and
>>> + # executables from this directory.
>>> + set(LLVM_OPTIONAL_SOURCES ClangFuzzer.cpp
>>> ExampleClangProtoFuzzer.cpp)
>>> + endif()
>>> +
>>> + add_clang_subdirectory(handle-cxx)
>>> +
>>> add_clang_executable(clang-fuzzer
>>> EXCLUDE_FROM_ALL
>>> ClangFuzzer.cpp
>>> )
>>>
>>> target_link_libraries(clang-fuzzer
>>> - ${CLANG_FORMAT_LIB_DEPS}
>>> - clangAST
>>> - clangBasic
>>> - clangCodeGen
>>> - clangDriver
>>> - clangFrontend
>>> - clangRewriteFrontend
>>> - clangStaticAnalyzerFrontend
>>> - clangTooling
>>> + clangHandleCXX
>>> LLVMFuzzer
>>> )
>>> endif()
>>>
>>> Modified: cfe/trunk/tools/clang-fuzzer/ClangFuzzer.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu
>>> zzer/ClangFuzzer.cpp?rev=310408&r1=310407&r2=310408&view=diff
>>> ============================================================
>>> ==================
>>> --- cfe/trunk/tools/clang-fuzzer/ClangFuzzer.cpp (original)
>>> +++ cfe/trunk/tools/clang-fuzzer/ClangFuzzer.cpp Tue Aug 8 13:15:04
>>> 2017
>>> @@ -13,43 +13,12 @@
>>> ///
>>> //===------------------------------------------------------
>>> ----------------===//
>>>
>>> -#include "clang/Tooling/Tooling.h"
>>> -#include "clang/CodeGen/CodeGenAction.h"
>>> -#include "clang/Frontend/CompilerInstance.h"
>>> -#include "clang/Lex/PreprocessorOptions.h"
>>> -#include "llvm/Option/Option.h"
>>> -#include "llvm/Support/TargetSelect.h"
>>> +#include "handle-cxx/handle_cxx.h"
>>>
>>> -using namespace clang;
>>> +using namespace clang_fuzzer;
>>>
>>> extern "C" int LLVMFuzzerTestOneInput(uint8_t *data, size_t size) {
>>> std::string s((const char *)data, size);
>>> - llvm::InitializeAllTargets();
>>> - llvm::InitializeAllTargetMCs();
>>> - llvm::InitializeAllAsmPrinters();
>>> - llvm::InitializeAllAsmParsers();
>>> -
>>> - llvm::opt::ArgStringList CC1Args;
>>> - CC1Args.push_back("-cc1");
>>> - CC1Args.push_back("./test.cc");
>>> - CC1Args.push_back("-O2");
>>> - llvm::IntrusiveRefCntPtr<FileManager> Files(
>>> - new FileManager(FileSystemOptions()));
>>> - IgnoringDiagConsumer Diags;
>>> - IntrusiveRefCntPtr<DiagnosticOptions> DiagOpts = new
>>> DiagnosticOptions();
>>> - DiagnosticsEngine Diagnostics(
>>> - IntrusiveRefCntPtr<clang::DiagnosticIDs>(new DiagnosticIDs()),
>>> &*DiagOpts,
>>> - &Diags, false);
>>> - std::unique_ptr<clang::CompilerInvocation> Invocation(
>>> - tooling::newInvocation(&Diagnostics, CC1Args));
>>> - std::unique_ptr<llvm::MemoryBuffer> Input =
>>> - llvm::MemoryBuffer::getMemBuffer(s);
>>> - Invocation->getPreprocessorOpts().addRemappedFile("./test.cc",
>>> Input.release());
>>> - std::unique_ptr<tooling::ToolAction> action(
>>> - tooling::newFrontendActionFactory<clang::EmitObjAction>());
>>> - std::shared_ptr<PCHContainerOperations> PCHContainerOps =
>>> - std::make_shared<PCHContainerOperations>();
>>> - action->runInvocation(std::move(Invocation), Files.get(),
>>> PCHContainerOps,
>>> - &Diags);
>>> + HandleCXX(s, {"-O2"});
>>> return 0;
>>> }
>>>
>>> Added: cfe/trunk/tools/clang-fuzzer/ExampleClangProtoFuzzer.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu
>>> zzer/ExampleClangProtoFuzzer.cpp?rev=310408&view=auto
>>> ============================================================
>>> ==================
>>> --- cfe/trunk/tools/clang-fuzzer/ExampleClangProtoFuzzer.cpp (added)
>>> +++ cfe/trunk/tools/clang-fuzzer/ExampleClangProtoFuzzer.cpp Tue Aug 8
>>> 13:15:04 2017
>>> @@ -0,0 +1,28 @@
>>> +//===-- ExampleClangProtoFuzzer.cpp - Fuzz Clang
>>> --------------------------===//
>>> +//
>>> +// The LLVM Compiler Infrastructure
>>> +//
>>> +// This file is distributed under the University of Illinois Open Source
>>> +// License. See LICENSE.TXT for details.
>>> +//
>>> +//===------------------------------------------------------
>>> ----------------===//
>>> +///
>>> +/// \file
>>> +/// \brief This file implements a function that runs Clang on a single
>>> +/// input and uses libprotobuf-mutator to find new inputs. This
>>> function is
>>> +/// then linked into the Fuzzer library.
>>> +///
>>> +//===------------------------------------------------------
>>> ----------------===//
>>> +
>>> +#include "cxx_proto.pb.h"
>>> +#include "handle-cxx/handle_cxx.h"
>>> +#include "proto-to-cxx/proto_to_cxx.h"
>>> +
>>> +#include "src/libfuzzer/libfuzzer_macro.h"
>>> +
>>> +using namespace clang_fuzzer;
>>> +
>>> +DEFINE_BINARY_PROTO_FUZZER(const Function& input) {
>>> + auto S = FunctionToString(input);
>>> + HandleCXX(S, {"-O2"});
>>> +}
>>>
>>> Added: cfe/trunk/tools/clang-fuzzer/README.txt
>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu
>>> zzer/README.txt?rev=310408&view=auto
>>> ============================================================
>>> ==================
>>> --- cfe/trunk/tools/clang-fuzzer/README.txt (added)
>>> +++ cfe/trunk/tools/clang-fuzzer/README.txt Tue Aug 8 13:15:04 2017
>>> @@ -0,0 +1,73 @@
>>> +This directory contains two utilities for fuzzing Clang: clang-fuzzer
>>> and
>>> +clang-proto-fuzzer. Both use libFuzzer to generate inputs to clang via
>>> +coverage-guided mutation.
>>> +
>>> +The two utilities differ, however, in how they structure inputs to
>>> Clang.
>>> +clang-fuzzer makes no attempt to generate valid C++ programs and is
>>> therefore
>>> +primarily useful for stressing the surface layers of Clang (i.e. lexer,
>>> parser).
>>> +clang-proto-fuzzer uses a protobuf class to describe a subset of the C++
>>> +language and then uses libprotobuf-mutator to mutate instantiations of
>>> that
>>> +class, producing valid C++ programs in the process. As a result,
>>> +clang-proto-fuzzer is better at stressing deeper layers of Clang and
>>> LLVM.
>>> +
>>> +===================================
>>> + Building clang-fuzzer
>>> +===================================
>>> +Within your LLVM build directory, run CMake with the following variable
>>> +definitions:
>>> +- CMAKE_C_COMPILER=clang
>>> +- CMAKE_CXX_COMPILER=clang++
>>> +- LLVM_USE_SANITIZE_COVERAGE=YES
>>> +- LLVM_USE_SANITIZER=Address
>>> +
>>> +Then build the clang-fuzzer target.
>>> +
>>> +Example:
>>> + cd $LLVM_SOURCE_DIR
>>> + mkdir build && cd build
>>> + cmake .. -GNinja -DCMAKE_C_COMPILER=clang
>>> -DCMAKE_CXX_COMPILER=clang++ \
>>> + -DLLVM_USE_SANITIZE_COVERAGE=YES -DLLVM_USE_SANITIZER=Address
>>> + ninja clang-fuzzer
>>> +
>>> +
>>> +=======================================================
>>> + Building clang-proto-fuzzer (Linux-only instructions)
>>> +=======================================================
>>> +Install the necessary dependencies:
>>> +- binutils // needed for libprotobuf-mutator
>>> +- liblzma-dev // needed for libprotobuf-mutator
>>> +- libz-dev // needed for libprotobuf-mutator
>>> +- docbook2x // needed for libprotobuf-mutator
>>> +- Recent version of protobuf [3.3.0 is known to work]
>>> +
>>> +Within your LLVM build directory, run CMake with the following variable
>>> +definitions:
>>> +- CMAKE_C_COMPILER=clang
>>> +- CMAKE_CXX_COMPILER=clang++
>>> +- LLVM_USE_SANITIZE_COVERAGE=YES
>>> +- LLVM_USE_SANITIZER=Address
>>> +- CLANG_ENABLE_PROTO_FUZZER=ON
>>> +
>>> +Then build the clang-proto-fuzzer and clang-proto-to-cxx targets.
>>> Optionally,
>>> +you may also build clang-fuzzer with this setup.
>>> +
>>> +Example:
>>> + cd $LLVM_SOURCE_DIR
>>> + mkdir build && cd build
>>> + cmake .. -GNinja -DCMAKE_C_COMPILER=clang
>>> -DCMAKE_CXX_COMPILER=clang++ \
>>> + -DLLVM_USE_SANITIZE_COVERAGE=YES -DLLVM_USE_SANITIZER=Address \
>>> + -DCLANG_ENABLE_PROTO_FUZZER=ON
>>> + ninja clang-proto-fuzzer clang-proto-to-cxx
>>> +
>>> +
>>> +=====================
>>> + Running the fuzzers
>>> +=====================
>>> +clang-fuzzer:
>>> + bin/clang-fuzzer CORPUS_DIR
>>> +
>>> +clang-proto-fuzzer:
>>> + bin/clang-proto-fuzzer CORPUS_DIR
>>> +
>>> +Translating a clang-proto-fuzzer corpus output to C++:
>>> + bin/clang-proto-to-cxx CORPUS_OUTPUT_FILE
>>>
>>> Added: cfe/trunk/tools/clang-fuzzer/cxx_proto.proto
>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu
>>> zzer/cxx_proto.proto?rev=310408&view=auto
>>> ============================================================
>>> ==================
>>> --- cfe/trunk/tools/clang-fuzzer/cxx_proto.proto (added)
>>> +++ cfe/trunk/tools/clang-fuzzer/cxx_proto.proto Tue Aug 8 13:15:04
>>> 2017
>>> @@ -0,0 +1,93 @@
>>> +//===-- cxx_proto.proto - Protobuf description of C++
>>> ---------------------===//
>>> +//
>>> +// The LLVM Compiler Infrastructure
>>> +//
>>> +// This file is distributed under the University of Illinois Open Source
>>> +// License. See LICENSE.TXT for details.
>>> +//
>>> +//===------------------------------------------------------
>>> ----------------===//
>>> +///
>>> +/// \file
>>> +/// \brief This file describes a subset of C++ as a protobuf. It is
>>> used to
>>> +/// more easily find interesting inputs for fuzzing Clang.
>>> +///
>>> +//===------------------------------------------------------
>>> ----------------===//
>>> +
>>> +syntax = "proto2";
>>> +
>>> +message VarRef {
>>> + required int32 varnum = 1;
>>> +}
>>> +
>>> +message Lvalue {
>>> + required VarRef varref = 1;
>>> +}
>>> +
>>> +message Const {
>>> + required int32 val = 1;
>>> +}
>>> +
>>> +message BinaryOp {
>>> + enum Op {
>>> + PLUS = 0;
>>> + MINUS = 1;
>>> + MUL = 2;
>>> + DIV = 3;
>>> + MOD = 4;
>>> + XOR = 5;
>>> + AND = 6;
>>> + OR = 7;
>>> + EQ = 8;
>>> + NE = 9;
>>> + LE = 10;
>>> + GE = 11;
>>> + LT = 12;
>>> + GT = 13;
>>> + };
>>> + required Op op = 1;
>>> + required Rvalue left = 2;
>>> + required Rvalue right = 3;
>>> +}
>>> +
>>> +message Rvalue {
>>> + oneof rvalue_oneof {
>>> + VarRef varref = 1;
>>> + Const cons = 2;
>>> + BinaryOp binop = 3;
>>> + }
>>> +}
>>> +
>>> +message AssignmentStatement {
>>> + required Lvalue lvalue = 1;
>>> + required Rvalue rvalue = 2;
>>> +}
>>> +
>>> +
>>> +message IfElse {
>>> + required Rvalue cond = 1;
>>> + required StatementSeq if_body = 2;
>>> + required StatementSeq else_body = 3;
>>> +}
>>> +
>>> +message While {
>>> + required Rvalue cond = 1;
>>> + required StatementSeq body = 2;
>>> +}
>>> +
>>> +message Statement {
>>> + oneof stmt_oneof {
>>> + AssignmentStatement assignment = 1;
>>> + IfElse ifelse = 2;
>>> + While while_loop = 3;
>>> + }
>>> +}
>>> +
>>> +message StatementSeq {
>>> + repeated Statement statements = 1;
>>> +}
>>> +
>>> +message Function {
>>> + required StatementSeq statements = 1;
>>> +}
>>> +
>>> +package clang_fuzzer;
>>>
>>> Added: cfe/trunk/tools/clang-fuzzer/handle-cxx/CMakeLists.txt
>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu
>>> zzer/handle-cxx/CMakeLists.txt?rev=310408&view=auto
>>> ============================================================
>>> ==================
>>> --- cfe/trunk/tools/clang-fuzzer/handle-cxx/CMakeLists.txt (added)
>>> +++ cfe/trunk/tools/clang-fuzzer/handle-cxx/CMakeLists.txt Tue Aug 8
>>> 13:15:04 2017
>>> @@ -0,0 +1,11 @@
>>> +set(LLVM_LINK_COMPONENTS ${LLVM_TARGETS_TO_BUILD})
>>> +
>>> +add_clang_library(clangHandleCXX
>>> + handle_cxx.cpp
>>> +
>>> + LINK_LIBS
>>> + clangCodeGen
>>> + clangFrontend
>>> + clangLex
>>> + clangTooling
>>> + )
>>>
>>> Added: cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu
>>> zzer/handle-cxx/handle_cxx.cpp?rev=310408&view=auto
>>> ============================================================
>>> ==================
>>> --- cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.cpp (added)
>>> +++ cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.cpp Tue Aug 8
>>> 13:15:04 2017
>>> @@ -0,0 +1,58 @@
>>> +//==-- handle_cxx.cpp - Helper function for Clang fuzzers
>>> ------------------==//
>>> +//
>>> +// The LLVM Compiler Infrastructure
>>> +//
>>> +// This file is distributed under the University of Illinois Open Source
>>> +// License. See LICENSE.TXT for details.
>>> +//
>>> +//===------------------------------------------------------
>>> ----------------===//
>>> +//
>>> +// Implements HandleCXX for use by the Clang fuzzers.
>>> +//
>>> +//===------------------------------------------------------
>>> ----------------===//
>>> +
>>> +#include "handle_cxx.h"
>>> +
>>> +#include "clang/CodeGen/CodeGenAction.h"
>>> +#include "clang/Frontend/CompilerInstance.h"
>>> +#include "clang/Lex/PreprocessorOptions.h"
>>> +#include "clang/Tooling/Tooling.h"
>>> +#include "llvm/Option/Option.h"
>>> +#include "llvm/Support/TargetSelect.h"
>>> +
>>> +using namespace clang;
>>> +
>>> +void clang_fuzzer::HandleCXX(const std::string &S,
>>> + const std::vector<const char *>
>>> &ExtraArgs) {
>>> + llvm::InitializeAllTargets();
>>> + llvm::InitializeAllTargetMCs();
>>> + llvm::InitializeAllAsmPrinters();
>>> + llvm::InitializeAllAsmParsers();
>>> +
>>> + llvm::opt::ArgStringList CC1Args;
>>> + CC1Args.push_back("-cc1");
>>> + for (auto &A : ExtraArgs)
>>> + CC1Args.push_back(A);
>>> + CC1Args.push_back("./test.cc");
>>> +
>>> + llvm::IntrusiveRefCntPtr<FileManager> Files(
>>> + new FileManager(FileSystemOptions()));
>>> + IgnoringDiagConsumer Diags;
>>> + IntrusiveRefCntPtr<DiagnosticOptions> DiagOpts = new
>>> DiagnosticOptions();
>>> + DiagnosticsEngine Diagnostics(
>>> + IntrusiveRefCntPtr<clang::DiagnosticIDs>(new DiagnosticIDs()),
>>> &*DiagOpts,
>>> + &Diags, false);
>>> + std::unique_ptr<clang::CompilerInvocation> Invocation(
>>> + tooling::newInvocation(&Diagnostics, CC1Args));
>>> + std::unique_ptr<llvm::MemoryBuffer> Input =
>>> + llvm::MemoryBuffer::getMemBuffer(S);
>>> + Invocation->getPreprocessorOpts().addRemappedFile("./test.cc",
>>> + Input.release());
>>> + std::unique_ptr<tooling::ToolAction> action(
>>> + tooling::newFrontendActionFactory<clang::EmitObjAction>());
>>> + std::shared_ptr<PCHContainerOperations> PCHContainerOps =
>>> + std::make_shared<PCHContainerOperations>();
>>> + action->runInvocation(std::move(Invocation), Files.get(),
>>> PCHContainerOps,
>>> + &Diags);
>>> +}
>>> +
>>>
>>> Added: cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.h
>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu
>>> zzer/handle-cxx/handle_cxx.h?rev=310408&view=auto
>>> ============================================================
>>> ==================
>>> --- cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.h (added)
>>> +++ cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.h Tue Aug 8
>>> 13:15:04 2017
>>> @@ -0,0 +1,25 @@
>>> +//==-- handle_cxx.h - Helper function for Clang fuzzers
>>> --------------------==//
>>> +//
>>> +// The LLVM Compiler Infrastructure
>>> +//
>>> +// This file is distributed under the University of Illinois Open Source
>>> +// License. See LICENSE.TXT for details.
>>> +//
>>> +//===------------------------------------------------------
>>> ----------------===//
>>> +//
>>> +// Defines HandleCXX for use by the Clang fuzzers.
>>> +//
>>> +//===------------------------------------------------------
>>> ----------------===//
>>> +
>>> +#ifndef LLVM_CLANG_TOOLS_CLANG_FUZZER_HANDLE_CXX_HANDLECXX_H
>>> +#define LLVM_CLANG_TOOLS_CLANG_FUZZER_HANDLE_CXX_HANDLECXX_H
>>> +
>>> +#include <string>
>>> +#include <vector>
>>> +
>>> +namespace clang_fuzzer {
>>> +void HandleCXX(const std::string &S,
>>> + const std::vector<const char *> &ExtraArgs);
>>> +} // namespace clang_fuzzer
>>> +
>>> +#endif
>>>
>>> Added: cfe/trunk/tools/clang-fuzzer/proto-to-cxx/CMakeLists.txt
>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu
>>> zzer/proto-to-cxx/CMakeLists.txt?rev=310408&view=auto
>>> ============================================================
>>> ==================
>>> --- cfe/trunk/tools/clang-fuzzer/proto-to-cxx/CMakeLists.txt (added)
>>> +++ cfe/trunk/tools/clang-fuzzer/proto-to-cxx/CMakeLists.txt Tue Aug 8
>>> 13:15:04 2017
>>> @@ -0,0 +1,10 @@
>>> +set(LLVM_LINK_COMPONENTS ${LLVM_TARGETS_TO_BUILD})
>>> +
>>> +# Hack to bypass LLVM's CMake source checks so we can have both a
>>> library and
>>> +# an executable built from this directory.
>>> +set(LLVM_OPTIONAL_SOURCES proto_to_cxx.cpp proto_to_cxx_main.cpp)
>>> +
>>> +add_clang_library(clangProtoToCXX proto_to_cxx.cpp LINK_LIBS
>>> clangCXXProto)
>>> +
>>> +add_clang_executable(clang-proto-to-cxx proto_to_cxx_main.cpp)
>>> +target_link_libraries(clang-proto-to-cxx clangProtoToCXX)
>>>
>>> Added: cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu
>>> zzer/proto-to-cxx/proto_to_cxx.cpp?rev=310408&view=auto
>>> ============================================================
>>> ==================
>>> --- cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.cpp (added)
>>> +++ cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.cpp Tue Aug
>>> 8 13:15:04 2017
>>> @@ -0,0 +1,102 @@
>>> +//==-- proto_to_cxx.cpp - Protobuf-C++ conversion
>>> --------------------------==//
>>> +//
>>> +// The LLVM Compiler Infrastructure
>>> +//
>>> +// This file is distributed under the University of Illinois Open Source
>>> +// License. See LICENSE.TXT for details.
>>> +//
>>> +//===------------------------------------------------------
>>> ----------------===//
>>> +//
>>> +// Implements functions for converting between protobufs and C++.
>>> +//
>>> +//===------------------------------------------------------
>>> ----------------===//
>>> +
>>> +#include "proto_to_cxx.h"
>>> +#include "cxx_proto.pb.h"
>>> +
>>> +#include <ostream>
>>> +#include <sstream>
>>> +
>>> +namespace clang_fuzzer {
>>> +
>>> +// Forward decls.
>>> +std::ostream &operator<<(std::ostream &os, const BinaryOp &x);
>>> +std::ostream &operator<<(std::ostream &os, const StatementSeq &x);
>>> +
>>> +// Proto to C++.
>>> +std::ostream &operator<<(std::ostream &os, const Const &x) {
>>> + return os << "(" << x.val() << ")";
>>> +}
>>> +std::ostream &operator<<(std::ostream &os, const VarRef &x) {
>>> + return os << "a[" << (static_cast<uint32_t>(x.varnum()) % 100) <<
>>> "]";
>>> +}
>>> +std::ostream &operator<<(std::ostream &os, const Lvalue &x) {
>>> + return os << x.varref();
>>> +}
>>> +std::ostream &operator<<(std::ostream &os, const Rvalue &x) {
>>> + if (x.has_varref()) return os << x.varref();
>>> + if (x.has_cons()) return os << x.cons();
>>> + if (x.has_binop()) return os << x.binop();
>>> + return os << "1";
>>> +}
>>> +std::ostream &operator<<(std::ostream &os, const BinaryOp &x) {
>>> + os << "(" << x.left();
>>> + switch (x.op()) {
>>> + case BinaryOp::PLUS: os << "+"; break;
>>> + case BinaryOp::MINUS: os << "-"; break;
>>> + case BinaryOp::MUL: os << "*"; break;
>>> + case BinaryOp::DIV: os << "/"; break;
>>> + case BinaryOp::MOD: os << "%"; break;
>>> + case BinaryOp::XOR: os << "^"; break;
>>> + case BinaryOp::AND: os << "&"; break;
>>> + case BinaryOp::OR: os << "|"; break;
>>> + case BinaryOp::EQ: os << "=="; break;
>>> + case BinaryOp::NE: os << "!="; break;
>>> + case BinaryOp::LE: os << "<="; break;
>>> + case BinaryOp::GE: os << ">="; break;
>>> + case BinaryOp::LT: os << "<"; break;
>>> + case BinaryOp::GT: os << ">"; break;
>>> + }
>>> + return os << x.right() << ")";
>>> +}
>>> +std::ostream &operator<<(std::ostream &os, const AssignmentStatement
>>> &x) {
>>> + return os << x.lvalue() << "=" << x.rvalue() << ";\n";
>>> +}
>>> +std::ostream &operator<<(std::ostream &os, const IfElse &x) {
>>> + return os << "if (" << x.cond() << "){\n"
>>> + << x.if_body() << "} else { \n"
>>> + << x.else_body() << "}\n";
>>> +}
>>> +std::ostream &operator<<(std::ostream &os, const While &x) {
>>> + return os << "while (" << x.cond() << "){\n" << x.body() << "}\n";
>>> +}
>>> +std::ostream &operator<<(std::ostream &os, const Statement &x) {
>>> + if (x.has_assignment()) return os << x.assignment();
>>> + if (x.has_ifelse()) return os << x.ifelse();
>>> + if (x.has_while_loop()) return os << x.while_loop();
>>> + return os << "(void)0;\n";
>>> +}
>>> +std::ostream &operator<<(std::ostream &os, const StatementSeq &x) {
>>> + for (auto &st : x.statements()) os << st;
>>> + return os;
>>> +}
>>> +std::ostream &operator<<(std::ostream &os, const Function &x) {
>>> + return os << "void foo(int *a) {\n" << x.statements() << "}\n";
>>> +}
>>> +
>>> +// ---------------------------------
>>> +
>>> +std::string FunctionToString(const Function &input) {
>>> + std::ostringstream os;
>>> + os << input;
>>> + return os.str();
>>> +
>>> +}
>>> +std::string ProtoToCxx(const uint8_t *data, size_t size) {
>>> + Function message;
>>> + if (!message.ParseFromArray(data, size))
>>> + return "#error invalid proto\n";
>>> + return FunctionToString(message);
>>> +}
>>> +
>>> +} // namespace clang_fuzzer
>>>
>>> Added: cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.h
>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu
>>> zzer/proto-to-cxx/proto_to_cxx.h?rev=310408&view=auto
>>> ============================================================
>>> ==================
>>> --- cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.h (added)
>>> +++ cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.h Tue Aug 8
>>> 13:15:04 2017
>>> @@ -0,0 +1,22 @@
>>> +//==-- proto_to_cxx.h - Protobuf-C++ conversion
>>> ----------------------------==//
>>> +//
>>> +// The LLVM Compiler Infrastructure
>>> +//
>>> +// This file is distributed under the University of Illinois Open Source
>>> +// License. See LICENSE.TXT for details.
>>> +//
>>> +//===------------------------------------------------------
>>> ----------------===//
>>> +//
>>> +// Defines functions for converting between protobufs and C++.
>>> +//
>>> +//===------------------------------------------------------
>>> ----------------===//
>>> +
>>> +#include <cstdint>
>>> +#include <cstddef>
>>> +#include <string>
>>> +
>>> +namespace clang_fuzzer {
>>> +class Function;
>>> +std::string FunctionToString(const Function &input);
>>> +std::string ProtoToCxx(const uint8_t *data, size_t size);
>>> +}
>>>
>>> Added: cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx_main.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu
>>> zzer/proto-to-cxx/proto_to_cxx_main.cpp?rev=310408&view=auto
>>> ============================================================
>>> ==================
>>> --- cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx_main.cpp
>>> (added)
>>> +++ cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx_main.cpp Tue
>>> Aug 8 13:15:04 2017
>>> @@ -0,0 +1,30 @@
>>> +//==-- proto_to_cxx_main.cpp - Driver for protobuf-C++ conversion
>>> ----------==//
>>> +//
>>> +// The LLVM Compiler Infrastructure
>>> +//
>>> +// This file is distributed under the University of Illinois Open Source
>>> +// License. See LICENSE.TXT for details.
>>> +//
>>> +//===------------------------------------------------------
>>> ----------------===//
>>> +//
>>> +// Implements a simple driver to print a C++ program from a protobuf.
>>> +//
>>> +//===------------------------------------------------------
>>> ----------------===//
>>> +#include <fstream>
>>> +#include <iostream>
>>> +#include <streambuf>
>>> +#include <string>
>>> +
>>> +#include "proto_to_cxx.h"
>>> +
>>> +int main(int argc, char **argv) {
>>> + for (int i = 1; i < argc; i++) {
>>> + std::fstream in(argv[i]);
>>> + std::string str((std::istreambuf_iterator<char>(in)),
>>> + std::istreambuf_iterator<char>());
>>> + std::cout << "// " << argv[i] << std::endl;
>>> + std::cout << clang_fuzzer::ProtoToCxx(
>>> + reinterpret_cast<const uint8_t *>(str.data()), str.size());
>>> + }
>>> +}
>>> +
>>>
>>>
>>> _______________________________________________
>>> cfe-commits mailing list
>>> cfe-commits at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
>>>
>>
>> _______________________________________________
>> cfe-commits mailing list
>> cfe-commits at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20170810/503d3375/attachment-0001.html>
More information about the cfe-commits
mailing list