[PATCH] [libcxx] Checking LIT benchmark testing framework.

[Eric Fiselier]

Hi mclow.lists, chandlerc, danalbert,

Hi All,

This is the initial commit for the benchmark testing framework I plan to use in libc++. It functions similarly to the existing LIT setup.

The benchmarks use the Google benchmark library found here: http://github.com/google/benchmark.
To enable building the benchmark library use the cmake option `-DLIBCXX_ENABLE_BENCHMARK=ON`. This option will checkout the library from github and build it in the build/external directory.

Once the library is built the benchmarks can be run in one of two ways.

1. Standalone (without baseline comparison): This simply runs the benchmarks and generates the output. Use `-o /path/to/OUTPUT` to save the results of the benchmarks.
Example usage:
```
lit -v -o /path/to/baseline.txt /path/to/benchmarks 
```
2. Comparison against baseline: This runs the benchmarks and compares the results to a baseline file specified. If the current results are slower by more than the "allowed difference" the test fails and the results are reported.
Example usage:
```
lit -sv -o /path/to/current_results.txt --param=baseline=/path/to/baseline.txt --param=allowed_difference=2.5 /path/to/benchmarks
```
The `allowed_difference` parameter takes the percentage which the results are allowed to differ from the baseline. The default is 5%.

The benchmark tests are not run as part of the regular test suite. They are too time consuming and do not provide much value unless they are compared to a baseline.  They are instead an entirely separate test suite.

http://reviews.llvm.org/D8107

Files:
  CMakeLists.txt
  external/CMakeLists.txt
  external/Toolchain.cmake.in
  test/benchmark/lit.cfg
  test/benchmark/test.bench.cpp
  test/libcxx/test/benchmark.py
  test/libcxx/test/config.py
  test/libcxx/test/format.py

EMAIL PREFERENCES
  http://reviews.llvm.org/settings/panel/emailpreferences/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: D8107.21358.patch
Type: text/x-patch
Size: 21436 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20150306/3b079d09/attachment.bin>