[libcxx] r276049 - [libcxx] Add support for benchmark tests using Google Benchmark.
Eric Fiselier via cfe-commits
cfe-commits at lists.llvm.org
Tue Aug 2 13:29:44 PDT 2016
Hi Ben,
I checked in a similar fix in r277512.
Please let me know if it helps. This "benchmark" stuff is still immature so
non-production worthy patches are always welcome.
/Eric
On Mon, Aug 1, 2016 at 1:45 PM, Craig, Ben <ben.craig at codeaurora.org> wrote:
> The current setup for the benchmark tests don't work well when the
> compiler being used isn't the system compiler. I'm on a system with a gcc
> 4.6.3 installation. I typically put clang in my path and set
> CMAKE_CXX_COMPILER and CMAKE_C_COMPILER. I don't know cmake well enough to
> recommend the "correct" solution, but these are some (not production
> worthy) changes that I made locally so that I could build the benchmarks.
>
> diff --git a/benchmarks/CMakeLists.txt b/benchmarks/CMakeLists.txt
> index e95bfa8..21bc0a8 100644
> --- a/benchmarks/CMakeLists.txt
> +++ b/benchmarks/CMakeLists.txt
> @@ -26,7 +26,9 @@ ExternalProject_Add(google-benchmark-libcxx
> PREFIX benchmark-libcxx
> SOURCE_DIR ${LIBCXX_SOURCE_DIR}/utils/google-benchmark
> INSTALL_DIR ${CMAKE_CURRENT_BINARY_DIR}/benchmark-libcxx
> - CMAKE_CACHE_DEFAULT_ARGS
> + CMAKE_CACHE_ARGS
> + -DCMAKE_CXX_COMPILER:STRING=clang++
> + -DCMAKE_C_COMPILER:STRING=clang
> -DCMAKE_BUILD_TYPE:STRING=RELEASE
> -DCMAKE_INSTALL_PREFIX:PATH=<INSTALL_DIR>
> -DCMAKE_CXX_FLAGS:STRING=${BENCHMARK_LIBCXX_COMPILE_FLAGS}
> @@ -60,6 +62,7 @@ set(BENCHMARK_NATIVE_INSTALL
> ${CMAKE_CURRENT_BINARY_DIR}/benchmark-native)
> set(BENCHMARK_TEST_COMPILE_FLAGS
> -std=c++14 -O2
> -I${BENCHMARK_LIBCXX_INSTALL}/include
> + -cxx-isystem ${LIBCXX_SOURCE_DIR}/include
> )
> set(BENCHMARK_TEST_LIBCXX_COMPILE_FLAGS
> -nostdinc++
>
>
> On 7/19/2016 6:07 PM, Eric Fiselier via cfe-commits wrote:
>
>> Author: ericwf
>> Date: Tue Jul 19 18:07:03 2016
>> New Revision: 276049
>>
>> URL: http://llvm.org/viewvc/llvm-project?rev=276049&view=rev
>> Log:
>> [libcxx] Add support for benchmark tests using Google Benchmark.
>>
>> Summary:
>> This patch does the following:
>>
>> 1. Checks in a copy of the Google Benchmark library into the libc++ repo
>> under `utils/google-benchmark`.
>> 2. Teaches libc++ how to build Google Benchmark against both (A) in-tree
>> libc++ and (B) the platforms native STL.
>> 3. Allows performance benchmarks to be built as part of the libc++ build.
>>
>> Building the benchmarks (and Google Benchmark) is off by default. It must
>> be enabled using the CMake option `-DLIBCXX_INCLUDE_BENCHMARKS=ON`. When
>> this option is enabled the tests under `libcxx/benchmarks` can be built
>> using the `libcxx-benchmarks` target.
>>
>> On Linux platforms where libstdc++ is the default STL the CMake option
>> `-DLIBCXX_BUILD_BENCHMARKS_NATIVE_STDLIB=ON` can be used to build each
>> benchmark test against libstdc++ as well. This is useful for comparing
>> performance between standard libraries.
>>
>> Support for benchmarks is currently very minimal. They must be manually
>> run by the user and there is no mechanism for detecting performance
>> regressions.
>>
>> Known Issues:
>>
>> * `-DLIBCXX_INCLUDE_BENCHMARKS=ON` is only supported for Clang, and not
>> GCC, since the `-stdlib=libc++` option is needed to build Google Benchmark.
>>
>>
>>
>>
>>
>>
>>
>>
>> Reviewers: danalbert, dberlin, chandlerc, mclow.lists, jroelofs
>>
>> Subscribers: chandlerc, dberlin, tberghammer, danalbert, srhines, hfinkel
>>
>> Differential Revision: https://reviews.llvm.org/D22240
>>
>> Added:
>> libcxx/trunk/benchmarks/CMakeLists.txt
>> libcxx/trunk/benchmarks/ContainerBenchmarks.hpp
>> libcxx/trunk/benchmarks/GenerateInput.hpp
>> libcxx/trunk/benchmarks/algorithms.bench.cpp
>> libcxx/trunk/utils/google-benchmark/
>> libcxx/trunk/utils/google-benchmark/.gitignore
>> libcxx/trunk/utils/google-benchmark/AUTHORS
>> libcxx/trunk/utils/google-benchmark/CMakeLists.txt
>> libcxx/trunk/utils/google-benchmark/CONTRIBUTING.md
>> libcxx/trunk/utils/google-benchmark/CONTRIBUTORS
>> libcxx/trunk/utils/google-benchmark/LICENSE
>> libcxx/trunk/utils/google-benchmark/README.LLVM
>> libcxx/trunk/utils/google-benchmark/README.md
>> libcxx/trunk/utils/google-benchmark/cmake/
>> libcxx/trunk/utils/google-benchmark/cmake/AddCXXCompilerFlag.cmake
>> libcxx/trunk/utils/google-benchmark/cmake/CXXFeatureCheck.cmake
>> libcxx/trunk/utils/google-benchmark/cmake/GetGitVersion.cmake
>> libcxx/trunk/utils/google-benchmark/cmake/gnu_posix_regex.cpp
>> libcxx/trunk/utils/google-benchmark/cmake/posix_regex.cpp
>> libcxx/trunk/utils/google-benchmark/cmake/std_regex.cpp
>> libcxx/trunk/utils/google-benchmark/cmake/steady_clock.cpp
>>
>> libcxx/trunk/utils/google-benchmark/cmake/thread_safety_attributes.cpp
>> libcxx/trunk/utils/google-benchmark/include/
>> libcxx/trunk/utils/google-benchmark/include/benchmark/
>> libcxx/trunk/utils/google-benchmark/include/benchmark/benchmark.h
>> libcxx/trunk/utils/google-benchmark/include/benchmark/benchmark_api.h
>> libcxx/trunk/utils/google-benchmark/include/benchmark/macros.h
>> libcxx/trunk/utils/google-benchmark/include/benchmark/reporter.h
>> libcxx/trunk/utils/google-benchmark/mingw.py
>> libcxx/trunk/utils/google-benchmark/src/
>> libcxx/trunk/utils/google-benchmark/src/CMakeLists.txt
>> libcxx/trunk/utils/google-benchmark/src/arraysize.h
>> libcxx/trunk/utils/google-benchmark/src/benchmark.cc
>> libcxx/trunk/utils/google-benchmark/src/check.h
>> libcxx/trunk/utils/google-benchmark/src/colorprint.cc
>> libcxx/trunk/utils/google-benchmark/src/colorprint.h
>> libcxx/trunk/utils/google-benchmark/src/commandlineflags.cc
>> libcxx/trunk/utils/google-benchmark/src/commandlineflags.h
>> libcxx/trunk/utils/google-benchmark/src/complexity.cc
>> libcxx/trunk/utils/google-benchmark/src/complexity.h
>> libcxx/trunk/utils/google-benchmark/src/console_reporter.cc
>> libcxx/trunk/utils/google-benchmark/src/csv_reporter.cc
>> libcxx/trunk/utils/google-benchmark/src/cycleclock.h
>> libcxx/trunk/utils/google-benchmark/src/internal_macros.h
>> libcxx/trunk/utils/google-benchmark/src/json_reporter.cc
>> libcxx/trunk/utils/google-benchmark/src/log.cc
>> libcxx/trunk/utils/google-benchmark/src/log.h
>> libcxx/trunk/utils/google-benchmark/src/mutex.h
>> libcxx/trunk/utils/google-benchmark/src/re.h
>> libcxx/trunk/utils/google-benchmark/src/re_posix.cc
>> libcxx/trunk/utils/google-benchmark/src/re_std.cc
>> libcxx/trunk/utils/google-benchmark/src/reporter.cc
>> libcxx/trunk/utils/google-benchmark/src/sleep.cc
>> libcxx/trunk/utils/google-benchmark/src/sleep.h
>> libcxx/trunk/utils/google-benchmark/src/stat.h
>> libcxx/trunk/utils/google-benchmark/src/string_util.cc
>> libcxx/trunk/utils/google-benchmark/src/string_util.h
>> libcxx/trunk/utils/google-benchmark/src/sysinfo.cc
>> libcxx/trunk/utils/google-benchmark/src/sysinfo.h
>> libcxx/trunk/utils/google-benchmark/src/walltime.cc
>> libcxx/trunk/utils/google-benchmark/src/walltime.h
>> libcxx/trunk/utils/google-benchmark/test/
>> libcxx/trunk/utils/google-benchmark/test/CMakeLists.txt
>> libcxx/trunk/utils/google-benchmark/test/basic_test.cc
>> libcxx/trunk/utils/google-benchmark/test/benchmark_test.cc
>> libcxx/trunk/utils/google-benchmark/test/complexity_test.cc
>> libcxx/trunk/utils/google-benchmark/test/cxx03_test.cc
>> libcxx/trunk/utils/google-benchmark/test/diagnostics_test.cc
>> libcxx/trunk/utils/google-benchmark/test/donotoptimize_test.cc
>> libcxx/trunk/utils/google-benchmark/test/filter_test.cc
>> libcxx/trunk/utils/google-benchmark/test/fixture_test.cc
>> libcxx/trunk/utils/google-benchmark/test/map_test.cc
>> libcxx/trunk/utils/google-benchmark/test/options_test.cc
>> libcxx/trunk/utils/google-benchmark/test/reporter_output_test.cc
>> libcxx/trunk/utils/google-benchmark/test/skip_with_error_test.cc
>> Modified:
>> libcxx/trunk/CMakeLists.txt
>> libcxx/trunk/benchmarks/unordered_set_operations.bench.cpp
>> libcxx/trunk/docs/BuildingLibcxx.rst
>> libcxx/trunk/docs/TestingLibcxx.rst
>>
>> Modified: libcxx/trunk/CMakeLists.txt
>> URL:
>> http://llvm.org/viewvc/llvm-project/libcxx/trunk/CMakeLists.txt?rev=276049&r1=276048&r2=276049&view=diff
>>
>> ==============================================================================
>> --- libcxx/trunk/CMakeLists.txt (original)
>> +++ libcxx/trunk/CMakeLists.txt Tue Jul 19 18:07:03 2016
>> @@ -56,6 +56,8 @@ option(LIBCXX_ENABLE_EXPERIMENTAL_LIBRAR
>> option(LIBCXX_ENABLE_FILESYSTEM
>> "Build filesystem as part of libc++experimental.a"
>> ${LIBCXX_ENABLE_EXPERIMENTAL_LIBRARY})
>> option(LIBCXX_INCLUDE_TESTS "Build the libc++ tests."
>> ${LLVM_INCLUDE_TESTS})
>> +option(LIBCXX_INCLUDE_BENCHMARKS "Build the libc++ benchmarks and their
>> dependancies" OFF)
>> +option(LIBCXX_BUILD_BENCHMARK_NATIVE_STDLIB "Build the benchmarks
>> against the native STL" OFF)
>> option(LIBCXX_INCLUDE_DOCS "Build the libc++ documentation."
>> ${LLVM_INCLUDE_DOCS})
>> set(LIBCXX_LIBDIR_SUFFIX "${LLVM_LIBDIR_SUFFIX}" CACHE STRING
>> "Define suffix of library directory name (32/64)")
>> @@ -426,6 +428,9 @@ include_directories(include)
>> add_subdirectory(include)
>> add_subdirectory(lib)
>> +if (LIBCXX_INCLUDE_BENCHMARKS)
>> + add_subdirectory(benchmarks)
>> +endif()
>> if (LIBCXX_INCLUDE_TESTS)
>> add_subdirectory(test)
>> endif()
>>
>> Added: libcxx/trunk/benchmarks/CMakeLists.txt
>> URL:
>> http://llvm.org/viewvc/llvm-project/libcxx/trunk/benchmarks/CMakeLists.txt?rev=276049&view=auto
>>
>> ==============================================================================
>> --- libcxx/trunk/benchmarks/CMakeLists.txt (added)
>> +++ libcxx/trunk/benchmarks/CMakeLists.txt Tue Jul 19 18:07:03 2016
>> @@ -0,0 +1,119 @@
>> +include(ExternalProject)
>> +include(CheckCXXCompilerFlag)
>> +
>>
>> +#==============================================================================
>> +# Build Google Benchmark for libc++
>>
>> +#==============================================================================
>> +check_cxx_compiler_flag(-stdlib=libc++ LIBCXX_HAS_NO_STDLIB_LIBCXX_FLAG)
>> +if (NOT LIBCXX_HAS_NO_STDLIB_LIBCXX_FLAG)
>> + message(FATAL "Benchmark requires support for the -stdlib=libc++ flag")
>> +endif()
>> +
>> +set(BENCHMARK_LIBCXX_COMPILE_FLAGS
>> + -Wno-unused-command-line-argument
>> + -nostdinc++
>> + -cxx-isystem ${LIBCXX_SOURCE_DIR}/include
>> + -stdlib=libc++)
>> +set(BENCHMARK_LIBCXX_LINK_FLAGS
>> + -L${LIBCXX_LIBRARY_DIR}
>> + -Wl,-rpath,${LIBCXX_LIBRARY_DIR})
>> +split_list(BENCHMARK_LIBCXX_COMPILE_FLAGS)
>> +split_list(BENCHMARK_LIBCXX_LINK_FLAGS)
>> +
>> +ExternalProject_Add(google-benchmark-libcxx
>> + EXCLUDE_FROM_ALL ON
>> + DEPENDS cxx
>> + PREFIX benchmark-libcxx
>> + SOURCE_DIR ${LIBCXX_SOURCE_DIR}/utils/google-benchmark
>> + INSTALL_DIR ${CMAKE_CURRENT_BINARY_DIR}/benchmark-libcxx
>> + CMAKE_CACHE_DEFAULT_ARGS
>> + -DCMAKE_BUILD_TYPE:STRING=RELEASE
>> + -DCMAKE_INSTALL_PREFIX:PATH=<INSTALL_DIR>
>> + -DCMAKE_CXX_FLAGS:STRING=${BENCHMARK_LIBCXX_COMPILE_FLAGS}
>> + -DCMAKE_SHARED_LINK_FLAGS:STRING=${BENCHMARK_LIBCXX_LINK_FLAGS}
>> + -DCMAKE_EXE_LINK_FLAGS:STRING=${BENCHMARK_LIBCXX_LINK_FLAGS}
>> + -DBENCHMARK_ENABLE_TESTING:BOOL=OFF)
>> +
>>
>> +#==============================================================================
>> +# Build Google Benchmark for the native stdlib
>>
>> +#==============================================================================
>> +if (LIBCXX_BUILD_BENCHMARK_NATIVE_STDLIB)
>> + ExternalProject_Add(google-benchmark-native
>> + EXCLUDE_FROM_ALL ON
>> + PREFIX benchmark-native
>> + SOURCE_DIR ${LIBCXX_SOURCE_DIR}/utils/google-benchmark
>> + INSTALL_DIR ${CMAKE_CURRENT_BINARY_DIR}/benchmark-native
>> + CMAKE_CACHE_ARGS
>> + -DBENCHMARK_ENABLE_TESTING:BOOL=OFF
>> + CMAKE_CACHE_DEFAULT_ARGS
>> + -DCMAKE_BUILD_TYPE:STRING=RELEASE
>> + -DCMAKE_INSTALL_PREFIX:PATH=<INSTALL_DIR>)
>> +endif()
>> +
>>
>> +#==============================================================================
>> +# Benchmark tests configuration
>>
>> +#==============================================================================
>> +add_custom_target(libcxx-benchmarks)
>> +
>> +set(BENCHMARK_LIBCXX_INSTALL
>> ${CMAKE_CURRENT_BINARY_DIR}/benchmark-libcxx)
>> +set(BENCHMARK_NATIVE_INSTALL
>> ${CMAKE_CURRENT_BINARY_DIR}/benchmark-native)
>> +set(BENCHMARK_TEST_COMPILE_FLAGS
>> + -std=c++14 -O2
>> + -I${BENCHMARK_LIBCXX_INSTALL}/include
>> +)
>> +set(BENCHMARK_TEST_LIBCXX_COMPILE_FLAGS
>> + -nostdinc++
>> + ${BENCHMARK_TEST_COMPILE_FLAGS}
>> + -Wno-user-defined-literals
>> +)
>> +set(BENCHMARK_TEST_LIBCXX_LINK_FLAGS
>> + -nodefaultlibs
>> + -L${BENCHMARK_LIBCXX_INSTALL}/lib/
>> +)
>> +set(BENCHMARK_TEST_NATIVE_LINK_FLAGS
>> + -L${BENCHMARK_NATIVE_INSTALL}/lib/
>> +)
>> +split_list(BENCHMARK_TEST_COMPILE_FLAGS)
>> +split_list(BENCHMARK_TEST_LIBCXX_COMPILE_FLAGS)
>> +split_list(BENCHMARK_TEST_LIBCXX_LINK_FLAGS)
>> +split_list(BENCHMARK_TEST_NATIVE_LINK_FLAGS)
>> +macro(add_benchmark_test name source_file)
>> + set(libcxx_target ${name}_libcxx)
>> + add_executable(${libcxx_target} EXCLUDE_FROM_ALL ${source_file})
>> + add_dependencies(${libcxx_target} cxx google-benchmark-libcxx)
>> + add_dependencies(libcxx-benchmarks ${libcxx_target})
>> + target_link_libraries(${libcxx_target} cxx -lbenchmark)
>> + set_target_properties(${libcxx_target}
>> + PROPERTIES
>> + OUTPUT_NAME "${name}.libcxx.out"
>> + COMPILE_FLAGS "${BENCHMARK_TEST_LIBCXX_COMPILE_FLAGS}"
>> + LINK_FLAGS "${BENCHMARK_TEST_LIBCXX_LINK_FLAGS}")
>> + if (LIBCXX_BUILD_BENCHMARK_NATIVE_STDLIB)
>> + set(native_target ${name}_native)
>> + add_executable(${native_target} EXCLUDE_FROM_ALL ${source_file})
>> + add_dependencies(${native_target} google-benchmark-native)
>> + target_link_libraries(${native_target} -lbenchmark)
>> + if (LIBCXX_HAS_PTHREAD_LIB)
>> + target_link_libraries(${native_target} -pthread)
>> + endif()
>> + add_dependencies(libcxx-benchmarks ${native_target})
>> + set_target_properties(${native_target}
>> + PROPERTIES
>> + OUTPUT_NAME "${name}.native.out"
>> + INCLUDE_DIRECTORIES ""
>> + COMPILE_FLAGS "${BENCHMARK_TEST_COMPILE_FLAGS}"
>> + LINK_FLAGS "${BENCHMARK_TEST_NATIVE_LINK_FLAGS}")
>> + endif()
>> +endmacro()
>> +
>> +
>>
>> +#==============================================================================
>> +# Register Benchmark tests
>>
>> +#==============================================================================
>> +file(GLOB BENCHMARK_TESTS "*.bench.cpp")
>> +foreach(test_path ${BENCHMARK_TESTS})
>> + get_filename_component(test_file "${test_path}" NAME)
>> + message(STATUS "TEST: ${test_file}")
>> + string(REPLACE ".bench.cpp" "" test_name "${test_file}")
>> + add_benchmark_test(${test_name} ${test_file})
>> +endforeach()
>>
>> Added: libcxx/trunk/benchmarks/ContainerBenchmarks.hpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/libcxx/trunk/benchmarks/ContainerBenchmarks.hpp?rev=276049&view=auto
>>
>> ==============================================================================
>> --- libcxx/trunk/benchmarks/ContainerBenchmarks.hpp (added)
>> +++ libcxx/trunk/benchmarks/ContainerBenchmarks.hpp Tue Jul 19 18:07:03
>> 2016
>> @@ -0,0 +1,69 @@
>> +#ifndef BENCHMARK_CONTAINER_BENCHMARKS_HPP
>> +#define BENCHMARK_CONTAINER_BENCHMARKS_HPP
>> +
>> +#include <cassert>
>> +
>> +#include "benchmark/benchmark_api.h"
>> +
>> +namespace ContainerBenchmarks {
>> +
>> +template <class Container, class GenInputs>
>> +void BM_InsertValue(benchmark::State& st, Container c, GenInputs gen) {
>> + auto in = gen(st.range_x());
>> + const auto end = in.end();
>> + while (st.KeepRunning()) {
>> + c.clear();
>> + for (auto it = in.begin(); it != end; ++it) {
>> + benchmark::DoNotOptimize(&(*c.insert(*it).first));
>> + }
>> + benchmark::ClobberMemory();
>> + }
>> +}
>> +
>> +template <class Container, class GenInputs>
>> +void BM_InsertValueRehash(benchmark::State& st, Container c, GenInputs
>> gen) {
>> + auto in = gen(st.range_x());
>> + const auto end = in.end();
>> + while (st.KeepRunning()) {
>> + c.clear();
>> + c.rehash(16);
>> + for (auto it = in.begin(); it != end; ++it) {
>> + benchmark::DoNotOptimize(&(*c.insert(*it).first));
>> + }
>> + benchmark::ClobberMemory();
>> + }
>> +}
>> +
>> +template <class Container, class GenInputs>
>> +static void BM_Find(benchmark::State& st, Container c, GenInputs gen) {
>> + auto in = gen(st.range_x());
>> + c.insert(in.begin(), in.end());
>> + benchmark::DoNotOptimize(&(*c.begin()));
>> + const auto end = in.data() + in.size();
>> + while (st.KeepRunning()) {
>> + for (auto it = in.data(); it != end; ++it) {
>> + benchmark::DoNotOptimize(&(*c.find(*it)));
>> + }
>> + benchmark::ClobberMemory();
>> + }
>> +}
>> +
>> +template <class Container, class GenInputs>
>> +static void BM_FindRehash(benchmark::State& st, Container c, GenInputs
>> gen) {
>> + c.rehash(8);
>> + auto in = gen(st.range_x());
>> + c.insert(in.begin(), in.end());
>> + benchmark::DoNotOptimize(&(*c.begin()));
>> + const auto end = in.data() + in.size();
>> + while (st.KeepRunning()) {
>> + for (auto it = in.data(); it != end; ++it) {
>> + benchmark::DoNotOptimize(&(*c.find(*it)));
>> + }
>> + benchmark::ClobberMemory();
>> + }
>> +
>> +}
>> +
>> +} // end namespace ContainerBenchmarks
>> +
>> +#endif // BENCHMARK_CONTAINER_BENCHMARKS_HPP
>>
>> Added: libcxx/trunk/benchmarks/GenerateInput.hpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/libcxx/trunk/benchmarks/GenerateInput.hpp?rev=276049&view=auto
>>
>> ==============================================================================
>> --- libcxx/trunk/benchmarks/GenerateInput.hpp (added)
>> +++ libcxx/trunk/benchmarks/GenerateInput.hpp Tue Jul 19 18:07:03 2016
>> @@ -0,0 +1,133 @@
>> +#ifndef BENCHMARK_GENERATE_INPUT_HPP
>> +#define BENCHMARK_GENERATE_INPUT_HPP
>> +
>> +#include <algorithm>
>> +#include <random>
>> +#include <vector>
>> +#include <string>
>> +#include <climits>
>> +#include <cstddef>
>> +
>> +static const char Letters[] = {
>> + '0','1','2','3','4',
>> + '5','6','7','8','9',
>> + 'A','B','C','D','E','F',
>> + 'G','H','I','J','K',
>> + 'L','M','N','O','P',
>> + 'Q','R','S','T','U',
>> + 'V','W','X','Y','Z',
>> + 'a','b','c','d','e','f',
>> + 'g','h','i','j','k',
>> + 'l','m','n','o','p',
>> + 'q','r','s','t','u',
>> + 'v','w','x','y','z'
>> +};
>> +static const std::size_t LettersSize = sizeof(Letters);
>> +
>> +inline std::default_random_engine& getRandomEngine() {
>> + static std::default_random_engine RandEngine(std::random_device{}());
>> + return RandEngine;
>> +}
>> +
>> +inline char getRandomChar() {
>> + std::uniform_int_distribution<> LettersDist(0, LettersSize-1);
>> + return Letters[LettersDist(getRandomEngine())];
>> +}
>> +
>> +template <class IntT>
>> +inline IntT getRandomInteger() {
>> + std::uniform_int_distribution<IntT> dist;
>> + return dist(getRandomEngine());
>> +}
>> +
>> +inline std::string getRandomString(std::size_t Len) {
>> + std::string str(Len, 0);
>> + std::generate_n(str.begin(), Len, &getRandomChar);
>> + return str;
>> +}
>> +
>> +template <class IntT>
>> +inline std::vector<IntT> getDuplicateIntegerInputs(size_t N) {
>> + std::vector<IntT> inputs(N, static_cast<IntT>(-1));
>> + return inputs;
>> +}
>> +
>> +template <class IntT>
>> +inline std::vector<IntT> getSortedIntegerInputs(size_t N) {
>> + std::vector<IntT> inputs;
>> + for (size_t i=0; i < N; i += 1)
>> + inputs.push_back(i);
>> + return inputs;
>> +}
>> +
>> +template <class IntT>
>> +std::vector<IntT> getSortedLargeIntegerInputs(size_t N) {
>> + std::vector<IntT> inputs;
>> + for (size_t i=0; i < N; ++i) {
>> + inputs.push_back(i + N);
>> + }
>> + return inputs;
>> +}
>> +
>> +template <class IntT>
>> +std::vector<IntT> getSortedTopBitsIntegerInputs(size_t N) {
>> + std::vector<IntT> inputs = getSortedIntegerInputs<IntT>(N);
>> + for (auto& E : inputs) E <<= ((sizeof(IntT) / 2) * CHAR_BIT);
>> + return inputs;
>> +}
>> +
>> +template <class IntT>
>> +inline std::vector<IntT> getReverseSortedIntegerInputs(size_t N) {
>> + std::vector<IntT> inputs;
>> + std::size_t i = N;
>> + while (i > 0) {
>> + --i;
>> + inputs.push_back(i);
>> + }
>> + return inputs;
>> +}
>> +
>> +template <class IntT>
>> +std::vector<IntT> getPipeOrganIntegerInputs(size_t N) {
>> + std::vector<IntT> v; v.reserve(N);
>> + for (size_t i = 0; i < N/2; ++i) v.push_back(i);
>> + for (size_t i = N/2; i < N; ++i) v.push_back(N - i);
>> + return v;
>> +}
>> +
>> +
>> +template <class IntT>
>> +std::vector<IntT> getRandomIntegerInputs(size_t N) {
>> + std::vector<IntT> inputs;
>> + for (size_t i=0; i < N; ++i) {
>> + inputs.push_back(getRandomInteger<IntT>());
>> + }
>> + return inputs;
>> +}
>> +
>> +inline std::vector<std::string> getDuplicateStringInputs(size_t N) {
>> + std::vector<std::string> inputs(N, getRandomString(1024));
>> + return inputs;
>> +}
>> +
>> +inline std::vector<std::string> getRandomStringInputs(size_t N) {
>> + std::vector<std::string> inputs;
>> + for (int i=0; i < N; ++i) {
>> + inputs.push_back(getRandomString(1024));
>> + }
>> + return inputs;
>> +}
>> +
>> +inline std::vector<std::string> getSortedStringInputs(size_t N) {
>> + std::vector<std::string> inputs = getRandomStringInputs(N);
>> + std::sort(inputs.begin(), inputs.end());
>> + return inputs;
>> +}
>> +
>> +inline std::vector<std::string> getReverseSortedStringInputs(size_t N) {
>> + std::vector<std::string> inputs = getSortedStringInputs(N);
>> + std::reverse(inputs.begin(), inputs.end());
>> + return inputs;
>> +}
>> +
>> +#endif // BENCHMARK_GENERATE_INPUT_HPP
>>
>> Added: libcxx/trunk/benchmarks/algorithms.bench.cpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/libcxx/trunk/benchmarks/algorithms.bench.cpp?rev=276049&view=auto
>>
>> ==============================================================================
>> --- libcxx/trunk/benchmarks/algorithms.bench.cpp (added)
>> +++ libcxx/trunk/benchmarks/algorithms.bench.cpp Tue Jul 19 18:07:03 2016
>> @@ -0,0 +1,62 @@
>> +#include <unordered_set>
>> +#include <vector>
>> +#include <cstdint>
>> +
>> +#include "benchmark/benchmark_api.h"
>> +#include "GenerateInput.hpp"
>> +
>> +constexpr std::size_t TestNumInputs = 1024;
>> +
>> +template <class GenInputs>
>> +void BM_Sort(benchmark::State& st, GenInputs gen) {
>> + using ValueType = typename decltype(gen(0))::value_type;
>> + const auto in = gen(st.range_x());
>> + std::vector<ValueType> inputs[5];
>> + auto reset_inputs = [&]() {
>> + for (auto& C : inputs) {
>> + C = in;
>> + benchmark::DoNotOptimize(C.data());
>> + }
>> + };
>> + reset_inputs();
>> + while (st.KeepRunning()) {
>> + for (auto& I : inputs) {
>> + std::sort(I.data(), I.data() + I.size());
>> + benchmark::DoNotOptimize(I.data());
>> + }
>> + st.PauseTiming();
>> + reset_inputs();
>> + benchmark::ClobberMemory();
>> + st.ResumeTiming();
>> + }
>> +}
>> +
>> +BENCHMARK_CAPTURE(BM_Sort, random_uint32,
>> + getRandomIntegerInputs<uint32_t>)->Arg(TestNumInputs);
>> +
>> +BENCHMARK_CAPTURE(BM_Sort, sorted_ascending_uint32,
>> + getSortedIntegerInputs<uint32_t>)->Arg(TestNumInputs);
>> +
>> +BENCHMARK_CAPTURE(BM_Sort, sorted_descending_uint32,
>> + getReverseSortedIntegerInputs<uint32_t>)->Arg(TestNumInputs);
>> +
>> +BENCHMARK_CAPTURE(BM_Sort, single_element_uint32,
>> + getDuplicateIntegerInputs<uint32_t>)->Arg(TestNumInputs);
>> +
>> +BENCHMARK_CAPTURE(BM_Sort, pipe_organ_uint32,
>> + getPipeOrganIntegerInputs<uint32_t>)->Arg(TestNumInputs);
>> +
>> +BENCHMARK_CAPTURE(BM_Sort, random_strings,
>> + getRandomStringInputs)->Arg(TestNumInputs);
>> +
>> +BENCHMARK_CAPTURE(BM_Sort, sorted_ascending_strings,
>> + getSortedStringInputs)->Arg(TestNumInputs);
>> +
>> +BENCHMARK_CAPTURE(BM_Sort, sorted_descending_strings,
>> + getReverseSortedStringInputs)->Arg(TestNumInputs);
>> +
>> +BENCHMARK_CAPTURE(BM_Sort, single_element_strings,
>> + getDuplicateStringInputs)->Arg(TestNumInputs);
>> +
>> +
>> +BENCHMARK_MAIN()
>>
>> Modified: libcxx/trunk/benchmarks/unordered_set_operations.bench.cpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/libcxx/trunk/benchmarks/unordered_set_operations.bench.cpp?rev=276049&r1=276048&r2=276049&view=diff
>>
>> ==============================================================================
>> --- libcxx/trunk/benchmarks/unordered_set_operations.bench.cpp (original)
>> +++ libcxx/trunk/benchmarks/unordered_set_operations.bench.cpp Tue Jul 19
>> 18:07:03 2016
>> @@ -1,44 +1,268 @@
>> #include <unordered_set>
>> #include <vector>
>> +#include <functional>
>> #include <cstdint>
>> +#include <cstdlib>
>> +#include <cstring>
>> #include "benchmark/benchmark_api.h"
>> -template <class IntT>
>> -std::vector<IntT> getInputs(size_t N) {
>> - std::vector<IntT> inputs;
>> - for (size_t i=0; i < N; ++i) {
>> - inputs.push_back(i);
>> - }
>> - return inputs;
>> +#include "ContainerBenchmarks.hpp"
>> +#include "GenerateInput.hpp"
>> +
>> +using namespace ContainerBenchmarks;
>> +
>> +constexpr std::size_t TestNumInputs = 1024;
>> +
>> +template <class _Size>
>> +inline __attribute__((__always_inline__))
>> +_Size loadword(const void* __p) {
>> + _Size __r;
>> + std::memcpy(&__r, __p, sizeof(__r));
>> + return __r;
>> }
>> -template <class Container, class Inputs>
>> -void BM_SetInsert(benchmark::State& st, Container c, Inputs const& in) {
>> - const auto end = in.end();
>> - while (st.KeepRunning()) {
>> - c.clear();
>> - for (auto it = in.begin(); it != end; ++it) {
>> - benchmark::DoNotOptimize(c.insert(*it));
>> - }
>> - benchmark::DoNotOptimize(c);
>> - }
>> +inline __attribute__((__always_inline__))
>> +std::size_t rotate_by_at_least_1(std::size_t __val, int __shift) {
>> + return (__val >> __shift) | (__val << (64 - __shift));
>> +}
>> +
>> +inline __attribute__((__always_inline__))
>> +std::size_t hash_len_16(std::size_t __u, std::size_t __v) {
>> + const std::size_t __mul = 0x9ddfea08eb382d69ULL;
>> + std::size_t __a = (__u ^ __v) * __mul;
>> + __a ^= (__a >> 47);
>> + std::size_t __b = (__v ^ __a) * __mul;
>> + __b ^= (__b >> 47);
>> + __b *= __mul;
>> + return __b;
>> +}
>> +
>> +
>> +template <std::size_t _Len>
>> +inline __attribute__((__always_inline__))
>> +std::size_t hash_len_0_to_8(const char* __s) {
>> + static_assert(_Len == 4 || _Len == 8, "");
>> + const uint64_t __a = loadword<uint32_t>(__s);
>> + const uint64_t __b = loadword<uint32_t>(__s + _Len - 4);
>> + return hash_len_16(_Len + (__a << 3), __b);
>> }
>> -BENCHMARK_CAPTURE(BM_SetInsert, uint32_insert,
>> - std::unordered_set<uint32_t>{}, getInputs<uint32_t>(1024));
>> -template <class Container, class Inputs>
>> -void BM_SetFind(benchmark::State& st, Container c, Inputs const& in) {
>> - c.insert(in.begin(), in.end());
>> - const auto end = in.end();
>> +struct UInt32Hash {
>> + UInt32Hash() = default;
>> + inline __attribute__((__always_inline__))
>> + std::size_t operator()(uint32_t data) const {
>> + return hash_len_0_to_8<4>(reinterpret_cast<const char*>(&data));
>> + }
>> +};
>> +
>> +struct UInt64Hash {
>> + UInt64Hash() = default;
>> + inline __attribute__((__always_inline__))
>> + std::size_t operator()(uint64_t data) const {
>> + return hash_len_0_to_8<8>(reinterpret_cast<const char*>(&data));
>> + }
>> +};
>> +
>> +struct UInt128Hash {
>> + UInt128Hash() = default;
>> + inline __attribute__((__always_inline__))
>> + std::size_t operator()(__uint128_t data) const {
>> + const __uint128_t __mask = static_cast<std::size_t>(-1);
>> + const std::size_t __a = (std::size_t)(data & __mask);
>> + const std::size_t __b = (std::size_t)((data & (__mask << 64)) >>
>> 64);
>> + return hash_len_16(__a, rotate_by_at_least_1(__b + 16, 16)) ^ __b;
>> + }
>> +};
>> +
>> +struct UInt32Hash2 {
>> + UInt32Hash2() = default;
>> + inline __attribute__((__always_inline__))
>> + std::size_t operator()(uint32_t data) const {
>> + const uint32_t __m = 0x5bd1e995;
>> + const uint32_t __r = 24;
>> + uint32_t __h = 4;
>> + uint32_t __k = data;
>> + __k *= __m;
>> + __k ^= __k >> __r;
>> + __k *= __m;
>> + __h *= __m;
>> + __h ^= __k;
>> + __h ^= __h >> 13;
>> + __h *= __m;
>> + __h ^= __h >> 15;
>> + return __h;
>> + }
>> +};
>> +
>> +struct UInt64Hash2 {
>> + UInt64Hash2() = default;
>> + inline __attribute__((__always_inline__))
>> + std::size_t operator()(uint64_t data) const {
>> + return hash_len_0_to_8<8>(reinterpret_cast<const char*>(&data));
>> + }
>> +};
>> +
>>
>> +//----------------------------------------------------------------------------//
>> +// BM_Hash
>> +//
>> ---------------------------------------------------------------------------//
>> +
>> +template <class HashFn, class GenInputs>
>> +void BM_Hash(benchmark::State& st, HashFn fn, GenInputs gen) {
>> + auto in = gen(st.range_x());
>> + const auto end = in.data() + in.size();
>> + std::size_t last_hash = 0;
>> + benchmark::DoNotOptimize(&last_hash);
>> while (st.KeepRunning()) {
>> - for (auto it = in.begin(); it != end; ++it) {
>> - benchmark::DoNotOptimize(c.find(*it));
>> + for (auto it = in.data(); it != end; ++it) {
>> + benchmark::DoNotOptimize(last_hash += fn(*it));
>> }
>> + benchmark::ClobberMemory();
>> }
>> }
>> -BENCHMARK_CAPTURE(BM_SetFind, uint32_lookup,
>> - std::unordered_set<uint32_t>{}, getInputs<uint32_t>(1024));
>> +BENCHMARK_CAPTURE(BM_Hash,
>> + uint32_random_std_hash,
>> + std::hash<uint32_t>{},
>> + getRandomIntegerInputs<uint32_t>) -> Arg(TestNumInputs);
>> +
>> +BENCHMARK_CAPTURE(BM_Hash,
>> + uint32_random_custom_hash,
>> + UInt32Hash{},
>> + getRandomIntegerInputs<uint32_t>) -> Arg(TestNumInputs);
>> +
>> +BENCHMARK_CAPTURE(BM_Hash,
>> + uint32_top_std_hash,
>> + std::hash<uint32_t>{},
>> + getSortedTopBitsIntegerInputs<uint32_t>) -> Arg(TestNumInputs);
>> +
>> +BENCHMARK_CAPTURE(BM_Hash,
>> + uint32_top_custom_hash,
>> + UInt32Hash{},
>> + getSortedTopBitsIntegerInputs<uint32_t>) -> Arg(TestNumInputs);
>> +
>> +
>>
>> +//----------------------------------------------------------------------------//
>> +// BM_InsertValue
>> +//
>> ---------------------------------------------------------------------------//
>> +
>> +
>> +// Sorted Assending //
>> +BENCHMARK_CAPTURE(BM_InsertValue,
>> + unordered_set_uint32,
>> + std::unordered_set<uint32_t>{},
>> + getRandomIntegerInputs<uint32_t>)->Arg(TestNumInputs);
>> +
>> +BENCHMARK_CAPTURE(BM_InsertValue,
>> + unordered_set_uint32_sorted,
>> + std::unordered_set<uint32_t>{},
>> + getSortedIntegerInputs<uint32_t>)->Arg(TestNumInputs);
>> +
>> +// Top Bytes //
>> +BENCHMARK_CAPTURE(BM_InsertValue,
>> + unordered_set_top_bits_uint32,
>> + std::unordered_set<uint32_t>{},
>> + getSortedTopBitsIntegerInputs<uint32_t>)->Arg(TestNumInputs);
>> +
>> +BENCHMARK_CAPTURE(BM_InsertValueRehash,
>> + unordered_set_top_bits_uint32,
>> + std::unordered_set<uint32_t, UInt32Hash>{},
>> + getSortedTopBitsIntegerInputs<uint32_t>)->Arg(TestNumInputs);
>> +
>> +// String //
>> +BENCHMARK_CAPTURE(BM_InsertValue,
>> + unordered_set_string,
>> + std::unordered_set<std::string>{},
>> + getRandomStringInputs)->Arg(TestNumInputs);
>> +
>> +BENCHMARK_CAPTURE(BM_InsertValueRehash,
>> + unordered_set_string,
>> + std::unordered_set<std::string>{},
>> + getRandomStringInputs)->Arg(TestNumInputs);
>> +
>>
>> +//----------------------------------------------------------------------------//
>> +// BM_Find
>> +//
>> ---------------------------------------------------------------------------//
>> +
>> +// Random //
>> +BENCHMARK_CAPTURE(BM_Find,
>> + unordered_set_random_uint64,
>> + std::unordered_set<uint64_t>{},
>> + getRandomIntegerInputs<uint64_t>)->Arg(TestNumInputs);
>> +
>> +BENCHMARK_CAPTURE(BM_FindRehash,
>> + unordered_set_random_uint64,
>> + std::unordered_set<uint64_t, UInt64Hash>{},
>> + getRandomIntegerInputs<uint64_t>)->Arg(TestNumInputs);
>> +
>> +// Sorted //
>> +BENCHMARK_CAPTURE(BM_Find,
>> + unordered_set_sorted_uint64,
>> + std::unordered_set<uint64_t>{},
>> + getSortedIntegerInputs<uint64_t>)->Arg(TestNumInputs);
>> +
>> +BENCHMARK_CAPTURE(BM_FindRehash,
>> + unordered_set_sorted_uint64,
>> + std::unordered_set<uint64_t, UInt64Hash>{},
>> + getSortedIntegerInputs<uint64_t>)->Arg(TestNumInputs);
>> +
>> +
>> +// Sorted //
>> +#if 1
>> +BENCHMARK_CAPTURE(BM_Find,
>> + unordered_set_sorted_uint128,
>> + std::unordered_set<__uint128_t, UInt128Hash>{},
>> + getSortedTopBitsIntegerInputs<__uint128_t>)->Arg(TestNumInputs);
>> +
>> +BENCHMARK_CAPTURE(BM_FindRehash,
>> + unordered_set_sorted_uint128,
>> + std::unordered_set<__uint128_t, UInt128Hash>{},
>> + getSortedTopBitsIntegerInputs<__uint128_t>)->Arg(TestNumInputs);
>> +#endif
>> +
>> +// Sorted //
>> +BENCHMARK_CAPTURE(BM_Find,
>> + unordered_set_sorted_uint32,
>> + std::unordered_set<uint32_t>{},
>> + getSortedIntegerInputs<uint32_t>)->Arg(TestNumInputs);
>> +
>> +BENCHMARK_CAPTURE(BM_FindRehash,
>> + unordered_set_sorted_uint32,
>> + std::unordered_set<uint32_t, UInt32Hash2>{},
>> + getSortedIntegerInputs<uint32_t>)->Arg(TestNumInputs);
>> +
>> +// Sorted Ascending //
>> +BENCHMARK_CAPTURE(BM_Find,
>> + unordered_set_sorted_large_uint64,
>> + std::unordered_set<uint64_t>{},
>> + getSortedLargeIntegerInputs<uint64_t>)->Arg(TestNumInputs);
>> +
>> +BENCHMARK_CAPTURE(BM_FindRehash,
>> + unordered_set_sorted_large_uint64,
>> + std::unordered_set<uint64_t, UInt64Hash>{},
>> + getSortedLargeIntegerInputs<uint64_t>)->Arg(TestNumInputs);
>> +
>> +
>> +// Top Bits //
>> +BENCHMARK_CAPTURE(BM_Find,
>> + unordered_set_top_bits_uint64,
>> + std::unordered_set<uint64_t>{},
>> + getSortedTopBitsIntegerInputs<uint64_t>)->Arg(TestNumInputs);
>> +
>> +BENCHMARK_CAPTURE(BM_FindRehash,
>> + unordered_set_top_bits_uint64,
>> + std::unordered_set<uint64_t, UInt64Hash>{},
>> + getSortedTopBitsIntegerInputs<uint64_t>)->Arg(TestNumInputs);
>> +
>> +// String //
>> +BENCHMARK_CAPTURE(BM_Find,
>> + unordered_set_string,
>> + std::unordered_set<std::string>{},
>> + getRandomStringInputs)->Arg(TestNumInputs);
>> +
>> +BENCHMARK_CAPTURE(BM_FindRehash,
>> + unordered_set_string,
>> + std::unordered_set<std::string>{},
>> + getRandomStringInputs)->Arg(TestNumInputs);
>> BENCHMARK_MAIN()
>>
>> Modified: libcxx/trunk/docs/BuildingLibcxx.rst
>> URL:
>> http://llvm.org/viewvc/llvm-project/libcxx/trunk/docs/BuildingLibcxx.rst?rev=276049&r1=276048&r2=276049&view=diff
>>
>> ==============================================================================
>> --- libcxx/trunk/docs/BuildingLibcxx.rst (original)
>> +++ libcxx/trunk/docs/BuildingLibcxx.rst Tue Jul 19 18:07:03 2016
>> @@ -227,7 +227,7 @@ ABI Library Specific Options
>> libc++abi is the C++ ABI library used.
>> -libc++ Feature options
>> +libc++ Feature Options
>> ----------------------
>> .. option:: LIBCXX_ENABLE_EXCEPTIONS:BOOL
>> @@ -242,9 +242,25 @@ libc++ Feature options
>> Build libc++ with run time type information.
>> +.. option:: LIBCXX_INCLUDE_BENCHMARKS:BOOL
>> -libc++ Feature options
>> -----------------------
>> + **Default**: ``OFF``
>> +
>> + Build the libc++ benchmark tests and the Google Benchmark library
>> needed
>> + to support them.
>> +
>> +.. option:: LIBCXX_BUILD_BENCHMARK_NATIVE_STDLIB:BOOL
>> +
>> + **Default**:: ``OFF``
>> +
>> + Build the libc++ benchmark tests and Google Benchmark library against
>> the
>> + native standard library on the platform. On linux this can be used to
>> compare
>> + libc++ to libstdc++ by building the benchmark tests against both
>> standard
>> + libraries.
>> +
>> +
>> +libc++ ABI Feature Options
>> +--------------------------
>> The following options allow building libc++ for a different ABI
>> version.
>>
>> Modified: libcxx/trunk/docs/TestingLibcxx.rst
>> URL:
>> http://llvm.org/viewvc/llvm-project/libcxx/trunk/docs/TestingLibcxx.rst?rev=276049&r1=276048&r2=276049&view=diff
>>
>> ==============================================================================
>> --- libcxx/trunk/docs/TestingLibcxx.rst (original)
>> +++ libcxx/trunk/docs/TestingLibcxx.rst Tue Jul 19 18:07:03 2016
>> @@ -198,3 +198,62 @@ Environment Variables
>> If ``LIBCXX_COLOR_DIAGNOSTICS`` is defined then the test suite will
>> attempt
>> to use color diagnostic outputs from the compiler.
>> Also see :option:`color_diagnostics`.
>> +
>> +Benchmarks
>> +==========
>> +
>> +Libc++ contains benchmark tests separately from the test of the test
>> suite.
>> +The benchmarks are written using the `Google Benchmark`_ library, a copy
>> of which
>> +is stored in the libc++ repository.
>> +
>> +For more information about using the Google Benchmark library see the
>> +`official documentation <https://github.com/google/benchmark>`_.
>> +
>> +.. _`Google Benchmark`: https://github.com/google/benchmark
>> +
>> +Building Benchmarks
>> +-------------------
>> +
>> +The benchmark tests are not enabled by default. To build the benchmarks
>> +libc++ must be configured using the CMake option
>> ``-DLIBCXX_INCLUDE_BENCHMARKS=ON``.
>> +Then the benchmarks can be built using the ``libcxx-benchmarks`` target.
>> +
>> +An example build would look like:
>> +
>> +.. code-block:: bash
>> +
>> + $ cd build
>> + $ cmake [options] -DLIBCXX_INCLUDE_BENCHMARKS=ON <path to libcxx
>> sources>
>> + $ make libcxx-benchmarks
>> +
>> +This will build all of the benchmarks under ``<libcxx-src>/benchmarks``
>> to be
>> +built against the just-built libc++. The compiled tests are output into
>> +``build/benchmarks``.
>> +
>> +The benchmarks can also be built against the platforms native standard
>> library
>> +using the ``-DLIBCXX_BUILD_BENCHMARKS_NATIVE_STDLIB=ON`` CMake option.
>> This
>> +is useful for comparing the performance of libc++ to other standard
>> libraries.
>> +The compiled benchmarks are named ``<test>.libcxx.out`` if they test
>> libc++ and
>> +``<test>.native.out`` otherwise.
>> +
>> +Also See:
>> +
>> + * :ref:`Building Libc++ <build instructions>`
>> + * :ref:`CMake Options`
>> +
>> +Running Benchmarks
>> +------------------
>> +
>> +The benchmarks must be run manually by the user. Currently there is no
>> way
>> +to run them as part of the build.
>> +
>> +For example:
>> +
>> +.. code-block:: bash
>> +
>> + $ cd build/benchmarks
>> + $ make libcxx-benchmarks
>> + $ ./algorithms.libcxx.out # Runs all the benchmarks
>> + $ ./algorithms.libcxx.out --benchmark_filter=BM_Sort.* # Only runs the
>> sort benchmarks
>> +
>> +For more information about running benchmarks see `Google Benchmark`_.
>>
>> Added: libcxx/trunk/utils/google-benchmark/.gitignore
>> URL:
>> http://llvm.org/viewvc/llvm-project/libcxx/trunk/utils/google-benchmark/.gitignore?rev=276049&view=auto
>>
>> ==============================================================================
>> --- libcxx/trunk/utils/google-benchmark/.gitignore (added)
>> +++ libcxx/trunk/utils/google-benchmark/.gitignore Tue Jul 19 18:07:03
>> 2016
>> @@ -0,0 +1,46 @@
>> +*.a
>> +*.so
>> +*.so.?*
>> +*.dll
>> +*.exe
>> +*.dylib
>> +*.cmake
>> +!/cmake/*.cmake
>> +*~
>> +*.pyc
>> +__pycache__
>> +
>> +# lcov
>> +*.lcov
>> +/lcov
>> +
>> +# cmake files.
>> +/Testing
>> +CMakeCache.txt
>> +CMakeFiles/
>> +cmake_install.cmake
>> +
>> +# makefiles.
>> +Makefile
>> +
>> +# in-source build.
>> +bin/
>> +lib/
>> +/test/*_test
>> +
>> +# exuberant ctags.
>> +tags
>> +
>> +# YouCompleteMe configuration.
>> +.ycm_extra_conf.pyc
>> +
>> +# ninja generated files.
>> +.ninja_deps
>> +.ninja_log
>> +build.ninja
>> +install_manifest.txt
>> +rules.ninja
>> +
>> +# out-of-source build top-level folders.
>> +build/
>> +_build/
>>
>> Added: libcxx/trunk/utils/google-benchmark/AUTHORS
>> URL:
>> http://llvm.org/viewvc/llvm-project/libcxx/trunk/utils/google-benchmark/AUTHORS?rev=276049&view=auto
>>
>> ==============================================================================
>> --- libcxx/trunk/utils/google-benchmark/AUTHORS (added)
>> +++ libcxx/trunk/utils/google-benchmark/AUTHORS Tue Jul 19 18:07:03 2016
>> @@ -0,0 +1,33 @@
>> +# This is the official list of benchmark authors for copyright purposes.
>> +# This file is distinct from the CONTRIBUTORS files.
>> +# See the latter for an explanation.
>> +#
>> +# Names should be added to this file as:
>> +# Name or Organization <email address>
>> +# The email address is not required for organizations.
>> +#
>> +# Please keep the list sorted.
>> +
>> +Albert Pretorius <pretoalb at gmail.com>
>> +Arne Beer <arne at twobeer.de>
>> +Christopher Seymour <chris.j.seymour at hotmail.com>
>> +David Coeurjolly <david.coeurjolly at liris.cnrs.fr>
>> +Dominic Hamon <dma at stripysock.com>
>> +Eugene Zhuk <eugene.zhuk at gmail.com>
>> +Evgeny Safronov <division494 at gmail.com>
>> +Felix Homann <linuxaudio at showlabor.de>
>> +Google Inc.
>> +Ismael Jimenez Martinez <ismael.jimenez.martinez at gmail.com>
>> +JianXiong Zhou <zhoujianxiong2 at gmail.com>
>> +Jussi Knuuttila <jussi.knuuttila at gmail.com>
>> +Kaito Udagawa <umireon at gmail.com>
>> +Lei Xu <eddyxu at gmail.com>
>> +Matt Clarkson <mattyclarkson at gmail.com>
>> +Oleksandr Sochka <sasha.sochka at gmail.com>
>> +Paul Redmond <paul.redmond at gmail.com>
>> +Radoslav Yovchev <radoslav.tm at gmail.com>
>> +Shuo Chen <chenshuo at chenshuo.com>
>> +Yusuke Suzuki <utatane.tea at gmail.com>
>> +Dirac Research
>> +Zbigniew Skowron <zbychs at gmail.com>
>> +Dominik Czarnota <dominik.b.czarnota at gmail.com>
>>
>> Added: libcxx/trunk/utils/google-benchmark/CMakeLists.txt
>> URL:
>> http://llvm.org/viewvc/llvm-project/libcxx/trunk/utils/google-benchmark/CMakeLists.txt?rev=276049&view=auto
>>
>> ==============================================================================
>> --- libcxx/trunk/utils/google-benchmark/CMakeLists.txt (added)
>> +++ libcxx/trunk/utils/google-benchmark/CMakeLists.txt Tue Jul 19
>> 18:07:03 2016
>> @@ -0,0 +1,147 @@
>> +cmake_minimum_required (VERSION 2.8.11)
>> +project (benchmark)
>> +
>> +foreach(p
>> + CMP0054 # CMake 3.1
>> + CMP0056 # export EXE_LINKER_FLAGS to try_run
>> + )
>> + if(POLICY ${p})
>> + cmake_policy(SET ${p} NEW)
>> + endif()
>> +endforeach()
>> +
>> +option(BENCHMARK_ENABLE_TESTING "Enable testing of the benchmark
>> library." ON)
>> +option(BENCHMARK_ENABLE_LTO "Enable link time optimisation of the
>> benchmark library." OFF)
>> +# Make sure we can import out CMake functions
>> +list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake")
>> +
>> +# Read the git tags to determine the project version
>> +include(GetGitVersion)
>> +get_git_version(GIT_VERSION)
>> +
>> +# Tell the user what versions we are using
>> +string(REGEX MATCH "[0-9]+\\.[0-9]+\\.[0-9]+" VERSION ${GIT_VERSION})
>> +message("-- Version: ${VERSION}")
>> +
>> +# The version of the libraries
>> +set(GENERIC_LIB_VERSION ${VERSION})
>> +string(SUBSTRING ${VERSION} 0 1 GENERIC_LIB_SOVERSION)
>> +
>> +# Import our CMake modules
>> +include(CheckCXXCompilerFlag)
>> +include(AddCXXCompilerFlag)
>> +include(CXXFeatureCheck)
>> +
>> +if ("${CMAKE_CXX_COMPILER_ID}" STREQUAL "MSVC")
>> + # Turn compiler warnings up to 11
>> + string(REGEX REPLACE "[-/]W[1-4]" "" CMAKE_CXX_FLAGS
>> "${CMAKE_CXX_FLAGS}")
>> + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /W4")
>> + add_definitions(-D_CRT_SECURE_NO_WARNINGS)
>> +
>> + # Link time optimisation
>> + if (BENCHMARK_ENABLE_LTO)
>> + set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /GL")
>> + set(CMAKE_STATIC_LINKER_FLAGS_RELEASE
>> "${CMAKE_STATIC_LINKER_FLAGS_RELEASE} /LTCG")
>> + set(CMAKE_SHARED_LINKER_FLAGS_RELEASE
>> "${CMAKE_SHARED_LINKER_FLAGS_RELEASE} /LTCG")
>> + set(CMAKE_EXE_LINKER_FLAGS_RELEASE
>> "${CMAKE_EXE_LINKER_FLAGS_RELEASE} /LTCG")
>> +
>> + set(CMAKE_CXX_FLAGS_RELWITHDEBINFO
>> "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} /GL")
>> + string(REGEX REPLACE "[-/]INCREMENTAL" "/INCREMENTAL:NO"
>> CMAKE_STATIC_LINKER_FLAGS_RELWITHDEBINFO
>> "${CMAKE_STATIC_LINKER_FLAGS_RELWITHDEBINFO}")
>> + set(CMAKE_STATIC_LINKER_FLAGS_RELWITHDEBINFO
>> "${CMAKE_STATIC_LINKER_FLAGS_RELWITHDEBINFO} /LTCG")
>> + string(REGEX REPLACE "[-/]INCREMENTAL" "/INCREMENTAL:NO"
>> CMAKE_SHARED_LINKER_FLAGS_RELWITHDEBINFO
>> "${CMAKE_SHARED_LINKER_FLAGS_RELWITHDEBINFO}")
>> + set(CMAKE_SHARED_LINKER_FLAGS_RELWITHDEBINFO
>> "${CMAKE_SHARED_LINKER_FLAGS_RELWITHDEBINFO} /LTCG")
>> + string(REGEX REPLACE "[-/]INCREMENTAL" "/INCREMENTAL:NO"
>> CMAKE_EXE_LINKER_FLAGS_RELWITHDEBINFO
>> "${CMAKE_EXE_LINKER_FLAGS_RELWITHDEBINFO}")
>> + set(CMAKE_EXE_LINKER_FLAGS_RELWITHDEBINFO
>> "${CMAKE_EXE_LINKER_FLAGS_RELWITHDEBINFO} /LTCG")
>> +
>> + set(CMAKE_CXX_FLAGS_MINSIZEREL "${CMAKE_CXX_FLAGS_MINSIZEREL} /GL")
>> + set(CMAKE_STATIC_LINKER_FLAGS_MINSIZEREL
>> "${CMAKE_STATIC_LINKER_FLAGS_MINSIZEREL} /LTCG")
>> + set(CMAKE_SHARED_LINKER_FLAGS_MINSIZEREL
>> "${CMAKE_SHARED_LINKER_FLAGS_MINSIZEREL} /LTCG")
>> + set(CMAKE_EXE_LINKER_FLAGS_MINSIZEREL
>> "${CMAKE_EXE_LINKER_FLAGS_MINSIZEREL} /LTCG")
>> + endif()
>> +else()
>> + # Try and enable C++11. Don't use C++14 because it doesn't work in some
>> + # configurations.
>> + add_cxx_compiler_flag(-std=c++11)
>> + if (NOT HAVE_CXX_FLAG_STD_CXX11)
>> + add_cxx_compiler_flag(-std=c++0x)
>> + endif()
>> +
>> + # Turn compiler warnings up to 11
>> + add_cxx_compiler_flag(-Wall)
>> +
>> + add_cxx_compiler_flag(-Wextra)
>> + add_cxx_compiler_flag(-Wshadow)
>> + add_cxx_compiler_flag(-Werror RELEASE)
>> + add_cxx_compiler_flag(-Werror RELWITHDEBINFO)
>> + add_cxx_compiler_flag(-Werror MINSIZEREL)
>> + add_cxx_compiler_flag(-pedantic)
>> + add_cxx_compiler_flag(-pedantic-errors)
>> + add_cxx_compiler_flag(-Wshorten-64-to-32)
>> + add_cxx_compiler_flag(-Wfloat-equal)
>> + add_cxx_compiler_flag(-Wzero-as-null-pointer-constant)
>> + add_cxx_compiler_flag(-fstrict-aliasing)
>> + if (HAVE_CXX_FLAG_FSTRICT_ALIASING)
>> + add_cxx_compiler_flag(-Wstrict-aliasing)
>> + endif()
>> + add_cxx_compiler_flag(-Wthread-safety)
>> + if (HAVE_WTHREAD_SAFETY)
>> + add_definitions(-DHAVE_WTHREAD_SAFETY)
>> + cxx_feature_check(THREAD_SAFETY_ATTRIBUTES)
>> + endif()
>> +
>> + # Link time optimisation
>> + if (BENCHMARK_ENABLE_LTO)
>> + add_cxx_compiler_flag(-flto)
>> + if ("${CMAKE_C_COMPILER_ID}" STREQUAL "GNU")
>> + find_program(GCC_AR gcc-ar)
>> + if (GCC_AR)
>> + set(CMAKE_AR ${GCC_AR})
>> + endif()
>> + find_program(GCC_RANLIB gcc-ranlib)
>> + if (GCC_RANLIB)
>> + set(CMAKE_RANLIB ${GCC_RANLIB})
>> + endif()
>> + endif()
>> + endif()
>> +
>> + # Coverage build type
>> + set(CMAKE_CXX_FLAGS_COVERAGE "${CMAKE_CXX_FLAGS_DEBUG}" CACHE STRING
>> + "Flags used by the C++ compiler during coverage builds."
>> + FORCE)
>> + set(CMAKE_EXE_LINKER_FLAGS_COVERAGE
>> + "${CMAKE_EXE_LINKER_FLAGS_DEBUG}" CACHE STRING
>> + "Flags used for linking binaries during coverage builds."
>> + FORCE)
>> + set(CMAKE_SHARED_LINKER_FLAGS_COVERAGE
>> + "${CMAKE_SHARED_LINKER_FLAGS_DEBUG}" CACHE STRING
>> + "Flags used by the shared libraries linker during coverage builds."
>> + FORCE)
>> + mark_as_advanced(
>> + CMAKE_CXX_FLAGS_COVERAGE
>> + CMAKE_EXE_LINKER_FLAGS_COVERAGE
>> + CMAKE_SHARED_LINKER_FLAGS_COVERAGE)
>> + set(CMAKE_BUILD_TYPE "${CMAKE_BUILD_TYPE}" CACHE STRING
>> + "Choose the type of build, options are: None Debug Release
>> RelWithDebInfo MinSizeRel Coverage."
>> + FORCE)
>> + add_cxx_compiler_flag(--coverage COVERAGE)
>> +endif()
>> +
>> +# C++ feature checks
>> +cxx_feature_check(STD_REGEX)
>> +cxx_feature_check(GNU_POSIX_REGEX)
>> +cxx_feature_check(POSIX_REGEX)
>> +cxx_feature_check(STEADY_CLOCK)
>> +
>> +# Ensure we have pthreads
>> +find_package(Threads REQUIRED)
>> +
>> +# Set up directories
>> +include_directories(${PROJECT_SOURCE_DIR}/include)
>> +
>> +# Build the targets
>> +add_subdirectory(src)
>> +
>> +if (BENCHMARK_ENABLE_TESTING)
>> + enable_testing()
>> + add_subdirectory(test)
>> +endif()
>>
>> Added: libcxx/trunk/utils/google-benchmark/CONTRIBUTING.md
>> URL:
>> http://llvm.org/viewvc/llvm-project/libcxx/trunk/utils/google-benchmark/CONTRIBUTING.md?rev=276049&view=auto
>>
>> ==============================================================================
>> --- libcxx/trunk/utils/google-benchmark/CONTRIBUTING.md (added)
>> +++ libcxx/trunk/utils/google-benchmark/CONTRIBUTING.md Tue Jul 19
>> 18:07:03 2016
>> @@ -0,0 +1,58 @@
>> +# How to contribute #
>> +
>> +We'd love to accept your patches and contributions to this project.
>> There are
>> +a just a few small guidelines you need to follow.
>> +
>> +
>> +## Contributor License Agreement ##
>> +
>> +Contributions to any Google project must be accompanied by a Contributor
>> +License Agreement. This is not a copyright **assignment**, it simply
>> gives
>> +Google permission to use and redistribute your contributions as part of
>> the
>> +project.
>> +
>> + * If you are an individual writing original source code and you're
>> sure you
>> + own the intellectual property, then you'll need to sign an
>> [individual
>> + CLA][].
>> +
>> + * If you work for a company that wants to allow you to contribute your
>> work,
>> + then you'll need to sign a [corporate CLA][].
>> +
>> +You generally only need to submit a CLA once, so if you've already
>> submitted
>> +one (even if it was for a different project), you probably don't need to
>> do it
>> +again.
>> +
>> +[individual CLA]:
>> https://developers.google.com/open-source/cla/individual
>> +[corporate CLA]: https://developers.google.com/open-source/cla/corporate
>> +
>> +Once your CLA is submitted (or if you already submitted one for
>> +another Google project), make a commit adding yourself to the
>> +[AUTHORS][] and [CONTRIBUTORS][] files. This commit can be part
>> +of your first [pull request][].
>> +
>> +[AUTHORS]: AUTHORS
>> +[CONTRIBUTORS]: CONTRIBUTORS
>> +
>> +
>> +## Submitting a patch ##
>> +
>> + 1. It's generally best to start by opening a new issue describing the
>> bug or
>> + feature you're intending to fix. Even if you think it's relatively
>> minor,
>> + it's helpful to know what people are working on. Mention in the
>> initial
>> + issue that you are planning to work on that bug or feature so that
>> it can
>> + be assigned to you.
>> +
>> + 1. Follow the normal process of [forking][] the project, and setup a
>> new
>> + branch to work in. It's important that each group of changes be
>> done in
>> + separate branches in order to ensure that a pull request only
>> includes the
>> + commits related to that bug or feature.
>> +
>> + 1. Do your best to have [well-formed commit messages][] for each
>> change.
>> + This provides consistency throughout the project, and ensures that
>> commit
>> + messages are able to be formatted properly by various git tools.
>> +
>> + 1. Finally, push the commits to your fork and submit a [pull
>> request][].
>> +
>> +[forking]: https://help.github.com/articles/fork-a-repo
>> +[well-formed commit messages]:
>> http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html
>> +[pull request]: https://help.github.com/articles/creating-a-pull-request
>>
>> Added: libcxx/trunk/utils/google-benchmark/CONTRIBUTORS
>> URL:
>> http://llvm.org/viewvc/llvm-project/libcxx/trunk/utils/google-benchmark/CONTRIBUTORS?rev=276049&view=auto
>>
>> ==============================================================================
>> --- libcxx/trunk/utils/google-benchmark/CONTRIBUTORS (added)
>> +++ libcxx/trunk/utils/google-benchmark/CONTRIBUTORS Tue Jul 19 18:07:03
>> 2016
>> @@ -0,0 +1,52 @@
>> +# People who have agreed to one of the CLAs and can contribute patches.
>> +# The AUTHORS file lists the copyright holders; this file
>> +# lists people. For example, Google employees are listed here
>> +# but not in AUTHORS, because Google holds the copyright.
>> +#
>> +# Names should be added to this file only after verifying that
>> +# the individual or the individual's organization has agreed to
>> +# the appropriate Contributor License Agreement, found here:
>> +#
>> +# https://developers.google.com/open-source/cla/individual
>> +# https://developers.google.com/open-source/cla/corporate
>> +#
>> +# The agreement for individuals can be filled out on the web.
>> +#
>> +# When adding J Random Contributor's name to this file,
>> +# either J's name or J's organization's name should be
>> +# added to the AUTHORS file, depending on whether the
>> +# individual or corporate CLA was used.
>> +#
>> +# Names should be added to this file as:
>> +# Name <email address>
>> +#
>> +# Please keep the list sorted.
>> +
>> +Albert Pretorius <pretoalb at gmail.com>
>> +Arne Beer <arne at twobeer.de>
>> +Billy Robert O'Neal III <billy.oneal at gmail.com> <bion at microsoft.com>
>> +Chris Kennelly <ckennelly at google.com> <ckennelly at ckennelly.com>
>> +Christopher Seymour <chris.j.seymour at hotmail.com>
>> +David Coeurjolly <david.coeurjolly at liris.cnrs.fr>
>> +Dominic Hamon <dma at stripysock.com>
>> +Eric Fiselier <eric at efcs.ca>
>> +Eugene Zhuk <eugene.zhuk at gmail.com>
>> +Evgeny Safronov <division494 at gmail.com>
>> +Felix Homann <linuxaudio at showlabor.de>
>> +Ismael Jimenez Martinez <ismael.jimenez.martinez at gmail.com>
>> +JianXiong Zhou <zhoujianxiong2 at gmail.com>
>> +Jussi Knuuttila <jussi.knuuttila at gmail.com>
>> +Kaito Udagawa <umireon at gmail.com>
>> +Kai Wolf <kai.wolf at gmail.com>
>> +Lei Xu <eddyxu at gmail.com>
>> +Matt Clarkson <mattyclarkson at gmail.com>
>> +Oleksandr Sochka <sasha.sochka at gmail.com>
>> +Pascal Leroy <phl at google.com>
>> +Paul Redmond <paul.redmond at gmail.com>
>> +Pierre Phaneuf <pphaneuf at google.com>
>> +Radoslav Yovchev <radoslav.tm at gmail.com>
>> +Shuo Chen <chenshuo at chenshuo.com>
>> +Yusuke Suzuki <utatane.tea at gmail.com>
>> +Tobias Ulvgård <tobias.ulvgard at dirac.se>
>> +Zbigniew Skowron <zbychs at gmail.com>
>> +Dominik Czarnota <dominik.b.czarnota at gmail.com>
>>
>> Added: libcxx/trunk/utils/google-benchmark/LICENSE
>> URL:
>> http://llvm.org/viewvc/llvm-project/libcxx/trunk/utils/google-benchmark/LICENSE?rev=276049&view=auto
>>
>> ==============================================================================
>> --- libcxx/trunk/utils/google-benchmark/LICENSE (added)
>> +++ libcxx/trunk/utils/google-benchmark/LICENSE Tue Jul 19 18:07:03 2016
>> @@ -0,0 +1,202 @@
>> +
>> + Apache License
>> + Version 2.0, January 2004
>> + http://www.apache.org/licenses/
>> +
>> + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
>> +
>> + 1. Definitions.
>> +
>> + "License" shall mean the terms and conditions for use,
>> reproduction,
>> + and distribution as defined by Sections 1 through 9 of this
>> document.
>> +
>> + "Licensor" shall mean the copyright owner or entity authorized by
>> + the copyright owner that is granting the License.
>> +
>> + "Legal Entity" shall mean the union of the acting entity and all
>> + other entities that control, are controlled by, or are under common
>> + control with that entity. For the purposes of this definition,
>> + "control" means (i) the power, direct or indirect, to cause the
>> + direction or management of such entity, whether by contract or
>> + otherwise, or (ii) ownership of fifty percent (50%) or more of the
>> + outstanding shares, or (iii) beneficial ownership of such entity.
>> +
>> + "You" (or "Your") shall mean an individual or Legal Entity
>> + exercising permissions granted by this License.
>> +
>> + "Source" form shall mean the preferred form for making
>> modifications,
>> + including but not limited to software source code, documentation
>> + source, and configuration files.
>> +
>> + "Object" form shall mean any form resulting from mechanical
>> + transformation or translation of a Source form, including but
>> + not limited to compiled object code, generated documentation,
>> + and conversions to other media types.
>> +
>> + "Work" shall mean the work of authorship, whether in Source or
>> + Object form, made available under the License, as indicated by a
>> + copyright notice that is included in or attached to the work
>> + (an example is provided in the Appendix below).
>> +
>> + "Derivative Works" shall mean any work, whether in Source or Object
>> + form, that is based on (or derived from) the Work and for which the
>> + editorial revisions, annotations, elaborations, or other
>> modifications
>> + represent, as a whole, an original work of authorship. For the
>> purposes
>> + of this License, Derivative Works shall not include works that
>> remain
>> + separable from, or merely link (or bind by name) to the interfaces
>> of,
>> + the Work and Derivative Works thereof.
>> +
>> + "Contribution" shall mean any work of authorship, including
>> + the original version of the Work and any modifications or additions
>> + to that Work or Derivative Works thereof, that is intentionally
>> + submitted to Licensor for inclusion in the Work by the copyright
>> owner
>> + or by an individual or Legal Entity authorized to submit on behalf
>> of
>> + the copyright owner. For the purposes of this definition,
>> "submitted"
>> + means any form of electronic, verbal, or written communication sent
>> + to the Licensor or its representatives, including but not limited
>> to
>> + communication on electronic mailing lists, source code control
>> systems,
>> + and issue tracking systems that are managed by, or on behalf of,
>> the
>> + Licensor for the purpose of discussing and improving the Work, but
>> + excluding communication that is conspicuously marked or otherwise
>> + designated in writing by the copyright owner as "Not a
>> Contribution."
>> +
>> + "Contributor" shall mean Licensor and any individual or Legal
>> Entity
>> + on behalf of whom a Contribution has been received by Licensor and
>> + subsequently incorporated within the Work.
>> +
>> + 2. Grant of Copyright License. Subject to the terms and conditions of
>> + this License, each Contributor hereby grants to You a perpetual,
>> + worldwide, non-exclusive, no-charge, royalty-free, irrevocable
>> + copyright license to reproduce, prepare Derivative Works of,
>> + publicly display, publicly perform, sublicense, and distribute the
>> + Work and such Derivative Works in Source or Object form.
>> +
>> + 3. Grant of Patent License. Subject to the terms and conditions of
>> + this License, each Contributor hereby grants to You a perpetual,
>> + worldwide, non-exclusive, no-charge, royalty-free, irrevocable
>> + (except as stated in this section) patent license to make, have
>> made,
>> + use, offer to sell, sell, import, and otherwise transfer the Work,
>> + where such license applies only to those patent claims licensable
>> + by such Contributor that are necessarily infringed by their
>> + Contribution(s) alone or by combination of their Contribution(s)
>> + with the Work to which such Contribution(s) was submitted. If You
>> + institute patent litigation against any entity (including a
>> + cross-claim or counterclaim in a lawsuit) alleging that the Work
>> + or a Contribution incorporated within the Work constitutes direct
>> + or contributory patent infringement, then any patent licenses
>> + granted to You under this License for that Work shall terminate
>> + as of the date such litigation is filed.
>> +
>> + 4. Redistribution. You may reproduce and distribute copies of the
>> + Work or Derivative Works thereof in any medium, with or without
>> + modifications, and in Source or Object form, provided that You
>> + meet the following conditions:
>> +
>> + (a) You must give any other recipients of the Work or
>> + Derivative Works a copy of this License; and
>> +
>> + (b) You must cause any modified files to carry prominent notices
>> + stating that You changed the files; and
>> +
>> + (c) You must retain, in the Source form of any Derivative Works
>> + that You distribute, all copyright, patent, trademark, and
>> + attribution notices from the Source form of the Work,
>> + excluding those notices that do not pertain to any part of
>> + the Derivative Works; and
>> +
>> + (d) If the Work includes a "NOTICE" text file as part of its
>> + distribution, then any Derivative Works that You distribute
>> must
>> + include a readable copy of the attribution notices contained
>> + within such NOTICE file, excluding those notices that do not
>> + pertain to any part of the Derivative Works, in at least one
>> + of the following places: within a NOTICE text file distributed
>> + as part of the Derivative Works; within the Source form or
>> + documentation, if provided along with the Derivative Works; or,
>> + within a display generated by the Derivative Works, if and
>> + wherever such third-party notices normally appear. The contents
>> + of the NOTICE file are for informational purposes only and
>> + do not modify the License. You may add Your own attribution
>> + notices within Derivative Works that You distribute, alongside
>> + or as an addendum to the NOTICE text from the Work, provided
>> + that such additional attribution notices cannot be construed
>> + as modifying the License.
>> +
>> + You may add Your own copyright statement to Your modifications and
>> + may provide additional or different license terms and conditions
>> + for use, reproduction, or distribution of Your modifications, or
>> + for any such Derivative Works as a whole, provided Your use,
>> + reproduction, and distribution of the Work otherwise complies with
>> + the conditions stated in this License.
>> +
>> + 5. Submission of Contributions. Unless You explicitly state otherwise,
>> + any Contribution intentionally submitted for inclusion in the Work
>> + by You to the Licensor shall be under the terms and conditions of
>> + this License, without any additional terms or conditions.
>> + Notwithstanding the above, nothing herein shall supersede or modify
>> + the terms of any separate license agreement you may have executed
>> + with Licensor regarding such Contributions.
>> +
>> + 6. Trademarks. This License does not grant permission to use the trade
>> + names, trademarks, service marks, or product names of the Licensor,
>> + except as required for reasonable and customary use in describing
>> the
>> + origin of the Work and reproducing the content of the NOTICE file.
>> +
>> + 7. Disclaimer of Warranty. Unless required by applicable law or
>> + agreed to in writing, Licensor provides the Work (and each
>> + Contributor provides its Contributions) on an "AS IS" BASIS,
>> + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
>> + implied, including, without limitation, any warranties or
>> conditions
>> + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
>> + PARTICULAR PURPOSE. You are solely responsible for determining the
>> + appropriateness of using or redistributing the Work and assume any
>> + risks associated with Your exercise of permissions under this
>> License.
>> +
>> + 8. Limitation of Liability. In no event and under no legal theory,
>> + whether in tort (including negligence), contract, or otherwise,
>> + unless required by applicable law (such as deliberate and grossly
>> + negligent acts) or agreed to in writing, shall any Contributor be
>> + liable to You for damages, including any direct, indirect, special,
>> + incidental, or consequential damages of any character arising as a
>> + result of this License or out of the use or inability to use the
>> + Work (including but not limited to damages for loss of goodwill,
>> + work stoppage, computer failure or malfunction, or any and all
>> + other commercial damages or losses), even if such Contributor
>> + has been advised of the possibility of such damages.
>> +
>> + 9. Accepting Warranty or Additional Liability. While redistributing
>> + the Work or Derivative Works thereof, You may choose to offer,
>> + and charge a fee for, acceptance of support, warranty, indemnity,
>> + or other liability obligations and/or rights consistent with this
>> + License. However, in accepting such obligations, You may act only
>> + on Your own behalf and on Your sole responsibility, not on behalf
>> + of any other Contributor, and only if You agree to indemnify,
>> + defend, and hold each Contributor harmless for any liability
>> + incurred by, or claims asserted against, such Contributor by reason
>> + of your accepting any such warranty or additional liability.
>> +
>> + END OF TERMS AND CONDITIONS
>> +
>> + APPENDIX: How to apply the Apache License to your work.
>> +
>> + To apply the Apache License to your work, attach the following
>> + boilerplate notice, with the fields enclosed by brackets "[]"
>> + replaced with your own identifying information. (Don't include
>> + the brackets!) The text should be enclosed in the appropriate
>> + comment syntax for the file format. We also recommend that a
>> + file or class name and description of purpose be included on the
>> + same "printed page" as the copyright notice for easier
>> + identification within third-party archives.
>> +
>> + Copyright [yyyy] [name of copyright owner]
>> +
>> + Licensed under the Apache License, Version 2.0 (the "License");
>> + you may not use this file except in compliance with the License.
>> + You may obtain a copy of the License at
>> +
>> + http://www.apache.org/licenses/LICENSE-2.0
>> +
>> + Unless required by applicable law or agreed to in writing, software
>> + distributed under the License is distributed on an "AS IS" BASIS,
>> + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
>> implied.
>> + See the License for the specific language governing permissions and
>> + limitations under the License.
>>
>> Added: libcxx/trunk/utils/google-benchmark/README.LLVM
>> URL:
>> http://llvm.org/viewvc/llvm-project/libcxx/trunk/utils/google-benchmark/README.LLVM?rev=276049&view=auto
>>
>> ==============================================================================
>> --- libcxx/trunk/utils/google-benchmark/README.LLVM (added)
>> +++ libcxx/trunk/utils/google-benchmark/README.LLVM Tue Jul 19 18:07:03
>> 2016
>> @@ -0,0 +1,6 @@
>> +LLVM notes
>> +----------
>> +
>> +This directory contains the Google Benchmark source code with some
>> unnecessary
>> +files removed. Note that this directory is under a different license than
>> +libc++.
>>
>> Added: libcxx/trunk/utils/google-benchmark/README.md
>> URL:
>> http://llvm.org/viewvc/llvm-project/libcxx/trunk/utils/google-benchmark/README.md?rev=276049&view=auto
>>
>> ==============================================================================
>> --- libcxx/trunk/utils/google-benchmark/README.md (added)
>> +++ libcxx/trunk/utils/google-benchmark/README.md Tue Jul 19 18:07:03 2016
>> @@ -0,0 +1,510 @@
>> +# benchmark
>> +[](https://travis-ci.org/google/benchmark
>> )
>> +[](https://ci.appveyor.com/project/google/benchmark/branch/master
>> )
>> +[](https://coveralls.io/r/google/benchmark
>> )
>> +
>> +A library to support the benchmarking of functions, similar to
>> unit-tests.
>> +
>> +Discussion group: https://groups.google.com/d/forum/benchmark-discuss
>> +
>> +IRC channel: https://freenode.net #googlebenchmark
>> +
>> +## Example usage
>> +### Basic usage
>> +Define a function that executes the code to be measured.
>> +
>> +```c++
>> +static void BM_StringCreation(benchmark::State& state) {
>> + while (state.KeepRunning())
>> + std::string empty_string;
>> +}
>> +// Register the function as a benchmark
>> +BENCHMARK(BM_StringCreation);
>> +
>> +// Define another benchmark
>> +static void BM_StringCopy(benchmark::State& state) {
>> + std::string x = "hello";
>> + while (state.KeepRunning())
>> + std::string copy(x);
>> +}
>> +BENCHMARK(BM_StringCopy);
>> +
>> +BENCHMARK_MAIN();
>> +```
>> +
>> +### Passing arguments
>> +Sometimes a family of benchmarks can be implemented with just one
>> routine that
>> +takes an extra argument to specify which one of the family of benchmarks
>> to
>> +run. For example, the following code defines a family of benchmarks for
>> +measuring the speed of `memcpy()` calls of different lengths:
>> +
>> +```c++
>> +static void BM_memcpy(benchmark::State& state) {
>> + char* src = new char[state.range_x()];
>> + char* dst = new char[state.range_x()];
>> + memset(src, 'x', state.range_x());
>> + while (state.KeepRunning())
>> + memcpy(dst, src, state.range_x());
>> + state.SetBytesProcessed(int64_t(state.iterations()) *
>> + int64_t(state.range_x()));
>> + delete[] src;
>> + delete[] dst;
>> +}
>> +BENCHMARK(BM_memcpy)->Arg(8)->Arg(64)->Arg(512)->Arg(1<<10)->Arg(8<<10);
>> +```
>> +
>> +The preceding code is quite repetitive, and can be replaced with the
>> following
>> +short-hand. The following invocation will pick a few appropriate
>> arguments in
>> +the specified range and will generate a benchmark for each such argument.
>> +
>> +```c++
>> +BENCHMARK(BM_memcpy)->Range(8, 8<<10);
>> +```
>> +
>> +By default the arguments in the range are generated in multiples of
>> eight and
>> +the command above selects [ 8, 64, 512, 4k, 8k ]. In the following code
>> the
>> +range multiplier is changed to multiples of two.
>> +
>> +```c++
>> +BENCHMARK(BM_memcpy)->RangeMultiplier(2)->Range(8, 8<<10);
>> +```
>> +Now arguments generated are [ 8, 16, 32, 64, 128, 256, 512, 1024, 2k,
>> 4k, 8k ].
>> +
>> +You might have a benchmark that depends on two inputs. For example, the
>> +following code defines a family of benchmarks for measuring the speed of
>> set
>> +insertion.
>> +
>> +```c++
>> +static void BM_SetInsert(benchmark::State& state) {
>> + while (state.KeepRunning()) {
>> + state.PauseTiming();
>> + std::set<int> data = ConstructRandomSet(state.range_x());
>> + state.ResumeTiming();
>> + for (int j = 0; j < state.range_y(); ++j)
>> + data.insert(RandomNumber());
>> + }
>> +}
>> +BENCHMARK(BM_SetInsert)
>> + ->ArgPair(1<<10, 1)
>> + ->ArgPair(1<<10, 8)
>> + ->ArgPair(1<<10, 64)
>> + ->ArgPair(1<<10, 512)
>> + ->ArgPair(8<<10, 1)
>> + ->ArgPair(8<<10, 8)
>> + ->ArgPair(8<<10, 64)
>> + ->ArgPair(8<<10, 512);
>> +```
>> +
>> +The preceding code is quite repetitive, and can be replaced with the
>> following
>> +short-hand. The following macro will pick a few appropriate arguments in
>> the
>> +product of the two specified ranges and will generate a benchmark for
>> each such
>> +pair.
>> +
>> +```c++
>> +BENCHMARK(BM_SetInsert)->RangePair(1<<10, 8<<10, 1, 512);
>> +```
>> +
>> +For more complex patterns of inputs, passing a custom function to
>> `Apply` allows
>> +programmatic specification of an arbitrary set of arguments on which to
>> run the
>> +benchmark. The following example enumerates a dense range on one
>> parameter,
>> +and a sparse range on the second.
>> +
>> +```c++
>> +static void CustomArguments(benchmark::internal::Benchmark* b) {
>> + for (int i = 0; i <= 10; ++i)
>> + for (int j = 32; j <= 1024*1024; j *= 8)
>> + b->ArgPair(i, j);
>> +}
>> +BENCHMARK(BM_SetInsert)->Apply(CustomArguments);
>> +```
>> +
>> +### Calculate asymptotic complexity (Big O)
>> +Asymptotic complexity might be calculated for a family of benchmarks. The
>> +following code will calculate the coefficient for the high-order term in
>> the
>> +running time and the normalized root-mean square error of string
>> comparison.
>> +
>> +```c++
>> +static void BM_StringCompare(benchmark::State& state) {
>> + std::string s1(state.range_x(), '-');
>> + std::string s2(state.range_x(), '-');
>> + while (state.KeepRunning()) {
>> + benchmark::DoNotOptimize(s1.compare(s2));
>> + }
>> + state.SetComplexityN(state.range_x());
>> +}
>> +BENCHMARK(BM_StringCompare)
>> + ->RangeMultiplier(2)->Range(1<<10, 1<<18)->Complexity(benchmark::oN);
>> +```
>> +
>> +As shown in the following invocation, asymptotic complexity might also be
>> +calculated automatically.
>> +
>> +```c++
>> +BENCHMARK(BM_StringCompare)
>> + ->RangeMultiplier(2)->Range(1<<10, 1<<18)->Complexity();
>> +```
>> +
>> +The following code will specify asymptotic complexity with a lambda
>> function,
>> +that might be used to customize high-order term calculation.
>> +
>> +```c++
>> +BENCHMARK(BM_StringCompare)->RangeMultiplier(2)
>> + ->Range(1<<10, 1<<18)->Complexity([](int n)->double{return n; });
>> +```
>> +
>> +### Templated benchmarks
>> +Templated benchmarks work the same way: This example produces and
>> consumes
>> +messages of size `sizeof(v)` `range_x` times. It also outputs throughput
>> in the
>> +absence of multiprogramming.
>> +
>> +```c++
>> +template <class Q> int BM_Sequential(benchmark::State& state) {
>> + Q q;
>> + typename Q::value_type v;
>> + while (state.KeepRunning()) {
>> + for (int i = state.range_x(); i--; )
>> + q.push(v);
>> + for (int e = state.range_x(); e--; )
>> + q.Wait(&v);
>> + }
>> + // actually messages, not bytes:
>> + state.SetBytesProcessed(
>> + static_cast<int64_t>(state.iterations())*state.range_x());
>> +}
>> +BENCHMARK_TEMPLATE(BM_Sequential, WaitQueue<int>)->Range(1<<0, 1<<10);
>> +```
>> +
>> +Three macros are provided for adding benchmark templates.
>> +
>> +```c++
>> +#if __cplusplus >= 201103L // C++11 and greater.
>> +#define BENCHMARK_TEMPLATE(func, ...) // Takes any number of parameters.
>> +#else // C++ < C++11
>> +#define BENCHMARK_TEMPLATE(func, arg1)
>> +#endif
>> +#define BENCHMARK_TEMPLATE1(func, arg1)
>> +#define BENCHMARK_TEMPLATE2(func, arg1, arg2)
>> +```
>> +
>> +## Passing arbitrary arguments to a benchmark
>> +In C++11 it is possible to define a benchmark that takes an arbitrary
>> number
>> +of extra arguments. The `BENCHMARK_CAPTURE(func, test_case_name,
>> ...args)`
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20160802/8e07aede/attachment-0001.html>
More information about the cfe-commits
mailing list