[libcxx-commits] [libcxx] [libc++] Move the benchmarks under libcxx/test (PR #99371)

[Louis Dionne via libcxx-commits]

https://github.com/ldionne created https://github.com/llvm/llvm-project/pull/99371

This is an intermediate and fairly mechanical step towards unifying the benchmarks with the rest of the test suite. Moving this around requires a few changes, notably making sure we don't throw a wrench into the discovery process of the normal test suite. This won't be a problem anymore once benchmarks are taken into account by the test setup out of the box.

>From 87e5c9054736fc037b7bfc907337493630d68bb5 Mon Sep 17 00:00:00 2001
From: Louis Dionne <ldionne.2 at gmail.com>
Date: Sat, 11 May 2024 10:50:14 -0400
Subject: [PATCH] [libc++] Move the benchmarks under libcxx/test

This is an intermediate and fairly mechanical step towards unifying the
benchmarks with the rest of the test suite. Moving this around requires
a few changes, notably making sure we don't throw a wrench into the
discovery process of the normal test suite. This won't be a problem
anymore once benchmarks are taken into account by the test setup out
of the box.
---
 libcxx/CMakeLists.txt                                  |  4 ----
 libcxx/docs/TestingLibcxx.rst                          | 10 +++++-----
 libcxx/test/CMakeLists.txt                             |  4 ++++
 libcxx/{ => test}/benchmarks/CMakeLists.txt            |  6 ++++--
 libcxx/{ => test}/benchmarks/CartesianBenchmarks.h     |  0
 libcxx/{ => test}/benchmarks/ContainerBenchmarks.h     |  0
 libcxx/{ => test}/benchmarks/GenerateInput.h           |  0
 libcxx/{ => test}/benchmarks/Utilities.h               |  0
 libcxx/{ => test}/benchmarks/VariantBenchmarks.h       |  0
 .../benchmarks/algorithms.partition_point.bench.cpp    |  0
 libcxx/{ => test}/benchmarks/algorithms/common.h       |  0
 .../{ => test}/benchmarks/algorithms/count.bench.cpp   |  0
 .../{ => test}/benchmarks/algorithms/equal.bench.cpp   |  0
 libcxx/{ => test}/benchmarks/algorithms/fill.bench.cpp |  0
 libcxx/{ => test}/benchmarks/algorithms/find.bench.cpp |  0
 .../benchmarks/algorithms/for_each.bench.cpp           |  0
 .../benchmarks/algorithms/lower_bound.bench.cpp        |  0
 .../benchmarks/algorithms/make_heap.bench.cpp          |  0
 .../algorithms/make_heap_then_sort_heap.bench.cpp      |  0
 libcxx/{ => test}/benchmarks/algorithms/min.bench.cpp  |  0
 .../benchmarks/algorithms/min_max_element.bench.cpp    |  0
 .../{ => test}/benchmarks/algorithms/minmax.bench.cpp  |  0
 .../benchmarks/algorithms/mismatch.bench.cpp           |  0
 .../benchmarks/algorithms/pop_heap.bench.cpp           |  0
 .../benchmarks/algorithms/pstl.stable_sort.bench.cpp   |  0
 .../benchmarks/algorithms/push_heap.bench.cpp          |  0
 .../benchmarks/algorithms/ranges_contains.bench.cpp    |  0
 .../benchmarks/algorithms/ranges_ends_with.bench.cpp   |  0
 .../benchmarks/algorithms/ranges_make_heap.bench.cpp   |  0
 .../ranges_make_heap_then_sort_heap.bench.cpp          |  0
 .../benchmarks/algorithms/ranges_pop_heap.bench.cpp    |  0
 .../benchmarks/algorithms/ranges_push_heap.bench.cpp   |  0
 .../benchmarks/algorithms/ranges_sort.bench.cpp        |  0
 .../benchmarks/algorithms/ranges_sort_heap.bench.cpp   |  0
 .../benchmarks/algorithms/ranges_stable_sort.bench.cpp |  0
 libcxx/{ => test}/benchmarks/algorithms/sort.bench.cpp |  0
 .../benchmarks/algorithms/sort_heap.bench.cpp          |  0
 .../benchmarks/algorithms/stable_sort.bench.cpp        |  0
 libcxx/{ => test}/benchmarks/allocation.bench.cpp      |  0
 libcxx/{ => test}/benchmarks/atomic_wait.bench.cpp     |  0
 .../benchmarks/atomic_wait_vs_mutex_lock.bench.cpp     |  0
 libcxx/{ => test}/benchmarks/deque.bench.cpp           |  0
 libcxx/{ => test}/benchmarks/deque_iterator.bench.cpp  |  0
 libcxx/{ => test}/benchmarks/exception_ptr.bench.cpp   |  0
 libcxx/{ => test}/benchmarks/filesystem.bench.cpp      |  0
 libcxx/{ => test}/benchmarks/format.bench.cpp          |  0
 libcxx/{ => test}/benchmarks/format_to.bench.cpp       |  0
 libcxx/{ => test}/benchmarks/format_to_n.bench.cpp     |  0
 libcxx/{ => test}/benchmarks/formatted_size.bench.cpp  |  0
 libcxx/{ => test}/benchmarks/formatter_float.bench.cpp |  0
 libcxx/{ => test}/benchmarks/formatter_int.bench.cpp   |  0
 libcxx/{ => test}/benchmarks/function.bench.cpp        |  0
 libcxx/{ => test}/benchmarks/join_view.bench.cpp       |  0
 .../lexicographical_compare_three_way.bench.cpp        |  0
 .../benchmarks/libcxxabi/dynamic_cast.bench.cpp        |  0
 .../libcxxabi/dynamic_cast_old_stress.bench.cpp        |  0
 .../lit.cfg.py => test/benchmarks/lit.cfg.py.in}       |  6 +++---
 libcxx/{ => test}/benchmarks/lit.site.cfg.py.in        |  2 +-
 libcxx/{ => test}/benchmarks/map.bench.cpp             |  0
 .../{ => test}/benchmarks/monotonic_buffer.bench.cpp   |  0
 libcxx/{ => test}/benchmarks/numeric/gcd.bench.cpp     |  0
 libcxx/{ => test}/benchmarks/ordered_set.bench.cpp     |  0
 libcxx/{ => test}/benchmarks/random.bench.cpp          |  0
 .../benchmarks/shared_mutex_vs_mutex.bench.cpp         |  0
 .../std_format_spec_string_unicode.bench.cpp           |  0
 .../std_format_spec_string_unicode_escape.bench.cpp    |  0
 libcxx/{ => test}/benchmarks/stop_token.bench.cpp      |  0
 libcxx/{ => test}/benchmarks/string.bench.cpp          |  0
 libcxx/{ => test}/benchmarks/stringstream.bench.cpp    |  0
 libcxx/{ => test}/benchmarks/system_error.bench.cpp    |  0
 libcxx/{ => test}/benchmarks/to_chars.bench.cpp        |  0
 .../benchmarks/unordered_set_operations.bench.cpp      |  0
 libcxx/{ => test}/benchmarks/util_smartptr.bench.cpp   |  0
 libcxx/{ => test}/benchmarks/variant_visit_1.bench.cpp |  0
 libcxx/{ => test}/benchmarks/variant_visit_2.bench.cpp |  0
 libcxx/{ => test}/benchmarks/variant_visit_3.bench.cpp |  0
 .../{ => test}/benchmarks/vector_operations.bench.cpp  |  0
 77 files changed, 17 insertions(+), 15 deletions(-)
 rename libcxx/{ => test}/benchmarks/CMakeLists.txt (98%)
 rename libcxx/{ => test}/benchmarks/CartesianBenchmarks.h (100%)
 rename libcxx/{ => test}/benchmarks/ContainerBenchmarks.h (100%)
 rename libcxx/{ => test}/benchmarks/GenerateInput.h (100%)
 rename libcxx/{ => test}/benchmarks/Utilities.h (100%)
 rename libcxx/{ => test}/benchmarks/VariantBenchmarks.h (100%)
 rename libcxx/{ => test}/benchmarks/algorithms.partition_point.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/common.h (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/count.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/equal.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/fill.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/find.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/for_each.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/lower_bound.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/make_heap.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/make_heap_then_sort_heap.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/min.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/min_max_element.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/minmax.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/mismatch.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/pop_heap.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/pstl.stable_sort.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/push_heap.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/ranges_contains.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/ranges_ends_with.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/ranges_make_heap.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/ranges_make_heap_then_sort_heap.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/ranges_pop_heap.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/ranges_push_heap.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/ranges_sort.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/ranges_sort_heap.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/ranges_stable_sort.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/sort.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/sort_heap.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/algorithms/stable_sort.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/allocation.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/atomic_wait.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/atomic_wait_vs_mutex_lock.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/deque.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/deque_iterator.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/exception_ptr.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/filesystem.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/format.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/format_to.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/format_to_n.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/formatted_size.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/formatter_float.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/formatter_int.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/function.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/join_view.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/lexicographical_compare_three_way.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/libcxxabi/dynamic_cast.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/libcxxabi/dynamic_cast_old_stress.bench.cpp (100%)
 rename libcxx/{benchmarks/lit.cfg.py => test/benchmarks/lit.cfg.py.in} (73%)
 rename libcxx/{ => test}/benchmarks/lit.site.cfg.py.in (76%)
 rename libcxx/{ => test}/benchmarks/map.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/monotonic_buffer.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/numeric/gcd.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/ordered_set.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/random.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/shared_mutex_vs_mutex.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/std_format_spec_string_unicode.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/std_format_spec_string_unicode_escape.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/stop_token.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/string.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/stringstream.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/system_error.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/to_chars.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/unordered_set_operations.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/util_smartptr.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/variant_visit_1.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/variant_visit_2.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/variant_visit_3.bench.cpp (100%)
 rename libcxx/{ => test}/benchmarks/vector_operations.bench.cpp (100%)

diff --git a/libcxx/CMakeLists.txt b/libcxx/CMakeLists.txt
index 155f81a74a974..e46c65039821e 100644
--- a/libcxx/CMakeLists.txt
+++ b/libcxx/CMakeLists.txt
@@ -855,10 +855,6 @@ add_subdirectory(src)
 add_subdirectory(utils)
 add_subdirectory(modules)
 
-if (LIBCXX_INCLUDE_BENCHMARKS)
-  add_subdirectory(benchmarks)
-endif()
-
 if (LIBCXX_INCLUDE_TESTS)
   add_subdirectory(test)
   add_subdirectory(lib/abi)
diff --git a/libcxx/docs/TestingLibcxx.rst b/libcxx/docs/TestingLibcxx.rst
index 6d3417cabfd61..96543251ca1cb 100644
--- a/libcxx/docs/TestingLibcxx.rst
+++ b/libcxx/docs/TestingLibcxx.rst
@@ -449,9 +449,9 @@ An example build would look like:
 
   $ ninja -C build cxx-benchmarks
 
-This will build all of the benchmarks under ``<libcxx-src>/benchmarks`` to be
+This will build all of the benchmarks under ``<libcxx>/test/benchmarks`` to be
 built against the just-built libc++. The compiled tests are output into
-``build/projects/libcxx/benchmarks``.
+``build/libcxx/test/benchmarks``.
 
 Also See:
 
@@ -468,9 +468,9 @@ For example:
 
 .. code-block:: bash
 
-  $ cd build/projects/libcxx/benchmarks
-  $ ./algorithms.bench.out # Runs all the benchmarks
-  $ ./algorithms.bench.out --benchmark_filter=BM_Sort.* # Only runs the sort benchmarks
+  $ cd build/libcxx/test/benchmarks
+  $ ./find.bench.out # Runs all the benchmarks
+  $ ./find.bench.out --benchmark_filter="bm_ranges_find<std::vector<char>>" # Only runs that specific benchmark
 
 For more information about running benchmarks see `Google Benchmark`_.
 
diff --git a/libcxx/test/CMakeLists.txt b/libcxx/test/CMakeLists.txt
index cdd1c2d90fbcf..4ce15c79d005a 100644
--- a/libcxx/test/CMakeLists.txt
+++ b/libcxx/test/CMakeLists.txt
@@ -1,6 +1,10 @@
 include(HandleLitArguments)
 add_subdirectory(tools)
 
+if (LIBCXX_INCLUDE_BENCHMARKS)
+  add_subdirectory(benchmarks)
+endif()
+
 # By default, libcxx and libcxxabi share a library directory.
 if (NOT LIBCXX_CXX_ABI_LIBRARY_PATH)
   set(LIBCXX_CXX_ABI_LIBRARY_PATH "${LIBCXX_LIBRARY_DIR}" CACHE PATH
diff --git a/libcxx/benchmarks/CMakeLists.txt b/libcxx/test/benchmarks/CMakeLists.txt
similarity index 98%
rename from libcxx/benchmarks/CMakeLists.txt
rename to libcxx/test/benchmarks/CMakeLists.txt
index 110672600213a..d9cb7be0f091e 100644
--- a/libcxx/benchmarks/CMakeLists.txt
+++ b/libcxx/test/benchmarks/CMakeLists.txt
@@ -5,8 +5,6 @@ include(CheckCXXCompilerFlag)
 # Build Google Benchmark
 #==============================================================================
 
-set(CMAKE_FOLDER "${CMAKE_FOLDER}/Benchmarks")
-
 set(BENCHMARK_COMPILE_FLAGS
     -Wno-unused-command-line-argument
     -nostdinc++
@@ -190,6 +188,10 @@ endforeach()
 if (LIBCXX_INCLUDE_TESTS)
   include(AddLLVM)
 
+  configure_lit_site_cfg(
+          ${CMAKE_CURRENT_SOURCE_DIR}/lit.cfg.py.in
+          ${CMAKE_CURRENT_BINARY_DIR}/lit.cfg.py)
+
   configure_lit_site_cfg(
           ${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.py.in
           ${CMAKE_CURRENT_BINARY_DIR}/lit.site.cfg.py)
diff --git a/libcxx/benchmarks/CartesianBenchmarks.h b/libcxx/test/benchmarks/CartesianBenchmarks.h
similarity index 100%
rename from libcxx/benchmarks/CartesianBenchmarks.h
rename to libcxx/test/benchmarks/CartesianBenchmarks.h
diff --git a/libcxx/benchmarks/ContainerBenchmarks.h b/libcxx/test/benchmarks/ContainerBenchmarks.h
similarity index 100%
rename from libcxx/benchmarks/ContainerBenchmarks.h
rename to libcxx/test/benchmarks/ContainerBenchmarks.h
diff --git a/libcxx/benchmarks/GenerateInput.h b/libcxx/test/benchmarks/GenerateInput.h
similarity index 100%
rename from libcxx/benchmarks/GenerateInput.h
rename to libcxx/test/benchmarks/GenerateInput.h
diff --git a/libcxx/benchmarks/Utilities.h b/libcxx/test/benchmarks/Utilities.h
similarity index 100%
rename from libcxx/benchmarks/Utilities.h
rename to libcxx/test/benchmarks/Utilities.h
diff --git a/libcxx/benchmarks/VariantBenchmarks.h b/libcxx/test/benchmarks/VariantBenchmarks.h
similarity index 100%
rename from libcxx/benchmarks/VariantBenchmarks.h
rename to libcxx/test/benchmarks/VariantBenchmarks.h
diff --git a/libcxx/benchmarks/algorithms.partition_point.bench.cpp b/libcxx/test/benchmarks/algorithms.partition_point.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms.partition_point.bench.cpp
rename to libcxx/test/benchmarks/algorithms.partition_point.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/common.h b/libcxx/test/benchmarks/algorithms/common.h
similarity index 100%
rename from libcxx/benchmarks/algorithms/common.h
rename to libcxx/test/benchmarks/algorithms/common.h
diff --git a/libcxx/benchmarks/algorithms/count.bench.cpp b/libcxx/test/benchmarks/algorithms/count.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/count.bench.cpp
rename to libcxx/test/benchmarks/algorithms/count.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/equal.bench.cpp b/libcxx/test/benchmarks/algorithms/equal.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/equal.bench.cpp
rename to libcxx/test/benchmarks/algorithms/equal.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/fill.bench.cpp b/libcxx/test/benchmarks/algorithms/fill.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/fill.bench.cpp
rename to libcxx/test/benchmarks/algorithms/fill.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/find.bench.cpp b/libcxx/test/benchmarks/algorithms/find.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/find.bench.cpp
rename to libcxx/test/benchmarks/algorithms/find.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/for_each.bench.cpp b/libcxx/test/benchmarks/algorithms/for_each.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/for_each.bench.cpp
rename to libcxx/test/benchmarks/algorithms/for_each.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/lower_bound.bench.cpp b/libcxx/test/benchmarks/algorithms/lower_bound.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/lower_bound.bench.cpp
rename to libcxx/test/benchmarks/algorithms/lower_bound.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/make_heap.bench.cpp b/libcxx/test/benchmarks/algorithms/make_heap.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/make_heap.bench.cpp
rename to libcxx/test/benchmarks/algorithms/make_heap.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/make_heap_then_sort_heap.bench.cpp b/libcxx/test/benchmarks/algorithms/make_heap_then_sort_heap.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/make_heap_then_sort_heap.bench.cpp
rename to libcxx/test/benchmarks/algorithms/make_heap_then_sort_heap.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/min.bench.cpp b/libcxx/test/benchmarks/algorithms/min.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/min.bench.cpp
rename to libcxx/test/benchmarks/algorithms/min.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/min_max_element.bench.cpp b/libcxx/test/benchmarks/algorithms/min_max_element.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/min_max_element.bench.cpp
rename to libcxx/test/benchmarks/algorithms/min_max_element.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/minmax.bench.cpp b/libcxx/test/benchmarks/algorithms/minmax.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/minmax.bench.cpp
rename to libcxx/test/benchmarks/algorithms/minmax.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/mismatch.bench.cpp b/libcxx/test/benchmarks/algorithms/mismatch.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/mismatch.bench.cpp
rename to libcxx/test/benchmarks/algorithms/mismatch.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/pop_heap.bench.cpp b/libcxx/test/benchmarks/algorithms/pop_heap.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/pop_heap.bench.cpp
rename to libcxx/test/benchmarks/algorithms/pop_heap.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/pstl.stable_sort.bench.cpp b/libcxx/test/benchmarks/algorithms/pstl.stable_sort.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/pstl.stable_sort.bench.cpp
rename to libcxx/test/benchmarks/algorithms/pstl.stable_sort.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/push_heap.bench.cpp b/libcxx/test/benchmarks/algorithms/push_heap.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/push_heap.bench.cpp
rename to libcxx/test/benchmarks/algorithms/push_heap.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/ranges_contains.bench.cpp b/libcxx/test/benchmarks/algorithms/ranges_contains.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/ranges_contains.bench.cpp
rename to libcxx/test/benchmarks/algorithms/ranges_contains.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/ranges_ends_with.bench.cpp b/libcxx/test/benchmarks/algorithms/ranges_ends_with.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/ranges_ends_with.bench.cpp
rename to libcxx/test/benchmarks/algorithms/ranges_ends_with.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/ranges_make_heap.bench.cpp b/libcxx/test/benchmarks/algorithms/ranges_make_heap.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/ranges_make_heap.bench.cpp
rename to libcxx/test/benchmarks/algorithms/ranges_make_heap.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/ranges_make_heap_then_sort_heap.bench.cpp b/libcxx/test/benchmarks/algorithms/ranges_make_heap_then_sort_heap.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/ranges_make_heap_then_sort_heap.bench.cpp
rename to libcxx/test/benchmarks/algorithms/ranges_make_heap_then_sort_heap.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/ranges_pop_heap.bench.cpp b/libcxx/test/benchmarks/algorithms/ranges_pop_heap.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/ranges_pop_heap.bench.cpp
rename to libcxx/test/benchmarks/algorithms/ranges_pop_heap.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/ranges_push_heap.bench.cpp b/libcxx/test/benchmarks/algorithms/ranges_push_heap.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/ranges_push_heap.bench.cpp
rename to libcxx/test/benchmarks/algorithms/ranges_push_heap.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/ranges_sort.bench.cpp b/libcxx/test/benchmarks/algorithms/ranges_sort.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/ranges_sort.bench.cpp
rename to libcxx/test/benchmarks/algorithms/ranges_sort.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/ranges_sort_heap.bench.cpp b/libcxx/test/benchmarks/algorithms/ranges_sort_heap.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/ranges_sort_heap.bench.cpp
rename to libcxx/test/benchmarks/algorithms/ranges_sort_heap.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/ranges_stable_sort.bench.cpp b/libcxx/test/benchmarks/algorithms/ranges_stable_sort.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/ranges_stable_sort.bench.cpp
rename to libcxx/test/benchmarks/algorithms/ranges_stable_sort.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/sort.bench.cpp b/libcxx/test/benchmarks/algorithms/sort.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/sort.bench.cpp
rename to libcxx/test/benchmarks/algorithms/sort.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/sort_heap.bench.cpp b/libcxx/test/benchmarks/algorithms/sort_heap.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/sort_heap.bench.cpp
rename to libcxx/test/benchmarks/algorithms/sort_heap.bench.cpp
diff --git a/libcxx/benchmarks/algorithms/stable_sort.bench.cpp b/libcxx/test/benchmarks/algorithms/stable_sort.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/algorithms/stable_sort.bench.cpp
rename to libcxx/test/benchmarks/algorithms/stable_sort.bench.cpp
diff --git a/libcxx/benchmarks/allocation.bench.cpp b/libcxx/test/benchmarks/allocation.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/allocation.bench.cpp
rename to libcxx/test/benchmarks/allocation.bench.cpp
diff --git a/libcxx/benchmarks/atomic_wait.bench.cpp b/libcxx/test/benchmarks/atomic_wait.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/atomic_wait.bench.cpp
rename to libcxx/test/benchmarks/atomic_wait.bench.cpp
diff --git a/libcxx/benchmarks/atomic_wait_vs_mutex_lock.bench.cpp b/libcxx/test/benchmarks/atomic_wait_vs_mutex_lock.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/atomic_wait_vs_mutex_lock.bench.cpp
rename to libcxx/test/benchmarks/atomic_wait_vs_mutex_lock.bench.cpp
diff --git a/libcxx/benchmarks/deque.bench.cpp b/libcxx/test/benchmarks/deque.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/deque.bench.cpp
rename to libcxx/test/benchmarks/deque.bench.cpp
diff --git a/libcxx/benchmarks/deque_iterator.bench.cpp b/libcxx/test/benchmarks/deque_iterator.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/deque_iterator.bench.cpp
rename to libcxx/test/benchmarks/deque_iterator.bench.cpp
diff --git a/libcxx/benchmarks/exception_ptr.bench.cpp b/libcxx/test/benchmarks/exception_ptr.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/exception_ptr.bench.cpp
rename to libcxx/test/benchmarks/exception_ptr.bench.cpp
diff --git a/libcxx/benchmarks/filesystem.bench.cpp b/libcxx/test/benchmarks/filesystem.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/filesystem.bench.cpp
rename to libcxx/test/benchmarks/filesystem.bench.cpp
diff --git a/libcxx/benchmarks/format.bench.cpp b/libcxx/test/benchmarks/format.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/format.bench.cpp
rename to libcxx/test/benchmarks/format.bench.cpp
diff --git a/libcxx/benchmarks/format_to.bench.cpp b/libcxx/test/benchmarks/format_to.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/format_to.bench.cpp
rename to libcxx/test/benchmarks/format_to.bench.cpp
diff --git a/libcxx/benchmarks/format_to_n.bench.cpp b/libcxx/test/benchmarks/format_to_n.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/format_to_n.bench.cpp
rename to libcxx/test/benchmarks/format_to_n.bench.cpp
diff --git a/libcxx/benchmarks/formatted_size.bench.cpp b/libcxx/test/benchmarks/formatted_size.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/formatted_size.bench.cpp
rename to libcxx/test/benchmarks/formatted_size.bench.cpp
diff --git a/libcxx/benchmarks/formatter_float.bench.cpp b/libcxx/test/benchmarks/formatter_float.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/formatter_float.bench.cpp
rename to libcxx/test/benchmarks/formatter_float.bench.cpp
diff --git a/libcxx/benchmarks/formatter_int.bench.cpp b/libcxx/test/benchmarks/formatter_int.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/formatter_int.bench.cpp
rename to libcxx/test/benchmarks/formatter_int.bench.cpp
diff --git a/libcxx/benchmarks/function.bench.cpp b/libcxx/test/benchmarks/function.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/function.bench.cpp
rename to libcxx/test/benchmarks/function.bench.cpp
diff --git a/libcxx/benchmarks/join_view.bench.cpp b/libcxx/test/benchmarks/join_view.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/join_view.bench.cpp
rename to libcxx/test/benchmarks/join_view.bench.cpp
diff --git a/libcxx/benchmarks/lexicographical_compare_three_way.bench.cpp b/libcxx/test/benchmarks/lexicographical_compare_three_way.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/lexicographical_compare_three_way.bench.cpp
rename to libcxx/test/benchmarks/lexicographical_compare_three_way.bench.cpp
diff --git a/libcxx/benchmarks/libcxxabi/dynamic_cast.bench.cpp b/libcxx/test/benchmarks/libcxxabi/dynamic_cast.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/libcxxabi/dynamic_cast.bench.cpp
rename to libcxx/test/benchmarks/libcxxabi/dynamic_cast.bench.cpp
diff --git a/libcxx/benchmarks/libcxxabi/dynamic_cast_old_stress.bench.cpp b/libcxx/test/benchmarks/libcxxabi/dynamic_cast_old_stress.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/libcxxabi/dynamic_cast_old_stress.bench.cpp
rename to libcxx/test/benchmarks/libcxxabi/dynamic_cast_old_stress.bench.cpp
diff --git a/libcxx/benchmarks/lit.cfg.py b/libcxx/test/benchmarks/lit.cfg.py.in
similarity index 73%
rename from libcxx/benchmarks/lit.cfg.py
rename to libcxx/test/benchmarks/lit.cfg.py.in
index 0d08966c26cc1..a3ce2477b9a85 100644
--- a/libcxx/benchmarks/lit.cfg.py
+++ b/libcxx/test/benchmarks/lit.cfg.py.in
@@ -3,7 +3,7 @@
 import os
 import site
 
-site.addsitedir(os.path.join(os.path.dirname(os.path.dirname(__file__)), "utils"))
+site.addsitedir(os.path.join("@LIBCXX_SOURCE_DIR@", "utils"))
 from libcxx.test.googlebenchmark import GoogleBenchmark
 
 # Tell pylint that we know config and lit_config exist somewhere.
@@ -15,8 +15,8 @@
 config.name = "libc++ benchmarks"
 config.suffixes = []
 
-config.test_exec_root = os.path.join(config.libcxx_obj_root, "benchmarks")
-config.test_source_root = config.test_exec_root
+config.test_exec_root = "@CMAKE_CURRENT_BINARY_DIR@"
+config.test_source_root = "@CMAKE_CURRENT_BINARY_DIR@"
 
 config.test_format = GoogleBenchmark(
     test_sub_dirs=".", test_suffix=".bench.out", benchmark_args=config.benchmark_args
diff --git a/libcxx/benchmarks/lit.site.cfg.py.in b/libcxx/test/benchmarks/lit.site.cfg.py.in
similarity index 76%
rename from libcxx/benchmarks/lit.site.cfg.py.in
rename to libcxx/test/benchmarks/lit.site.cfg.py.in
index e3ce8b22263e1..6d4b0ca3dae0e 100644
--- a/libcxx/benchmarks/lit.site.cfg.py.in
+++ b/libcxx/test/benchmarks/lit.site.cfg.py.in
@@ -7,4 +7,4 @@ config.libcxx_obj_root = "@LIBCXX_BINARY_DIR@"
 config.benchmark_args = "@LIBCXX_BENCHMARK_TEST_ARGS@".split(';')
 
 # Let the main config do the real work.
-lit_config.load_config(config, "@LIBCXX_SOURCE_DIR@/benchmarks/lit.cfg.py")
\ No newline at end of file
+lit_config.load_config(config, "@CMAKE_CURRENT_BINARY_DIR@/lit.cfg.py")
\ No newline at end of file
diff --git a/libcxx/benchmarks/map.bench.cpp b/libcxx/test/benchmarks/map.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/map.bench.cpp
rename to libcxx/test/benchmarks/map.bench.cpp
diff --git a/libcxx/benchmarks/monotonic_buffer.bench.cpp b/libcxx/test/benchmarks/monotonic_buffer.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/monotonic_buffer.bench.cpp
rename to libcxx/test/benchmarks/monotonic_buffer.bench.cpp
diff --git a/libcxx/benchmarks/numeric/gcd.bench.cpp b/libcxx/test/benchmarks/numeric/gcd.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/numeric/gcd.bench.cpp
rename to libcxx/test/benchmarks/numeric/gcd.bench.cpp
diff --git a/libcxx/benchmarks/ordered_set.bench.cpp b/libcxx/test/benchmarks/ordered_set.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/ordered_set.bench.cpp
rename to libcxx/test/benchmarks/ordered_set.bench.cpp
diff --git a/libcxx/benchmarks/random.bench.cpp b/libcxx/test/benchmarks/random.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/random.bench.cpp
rename to libcxx/test/benchmarks/random.bench.cpp
diff --git a/libcxx/benchmarks/shared_mutex_vs_mutex.bench.cpp b/libcxx/test/benchmarks/shared_mutex_vs_mutex.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/shared_mutex_vs_mutex.bench.cpp
rename to libcxx/test/benchmarks/shared_mutex_vs_mutex.bench.cpp
diff --git a/libcxx/benchmarks/std_format_spec_string_unicode.bench.cpp b/libcxx/test/benchmarks/std_format_spec_string_unicode.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/std_format_spec_string_unicode.bench.cpp
rename to libcxx/test/benchmarks/std_format_spec_string_unicode.bench.cpp
diff --git a/libcxx/benchmarks/std_format_spec_string_unicode_escape.bench.cpp b/libcxx/test/benchmarks/std_format_spec_string_unicode_escape.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/std_format_spec_string_unicode_escape.bench.cpp
rename to libcxx/test/benchmarks/std_format_spec_string_unicode_escape.bench.cpp
diff --git a/libcxx/benchmarks/stop_token.bench.cpp b/libcxx/test/benchmarks/stop_token.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/stop_token.bench.cpp
rename to libcxx/test/benchmarks/stop_token.bench.cpp
diff --git a/libcxx/benchmarks/string.bench.cpp b/libcxx/test/benchmarks/string.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/string.bench.cpp
rename to libcxx/test/benchmarks/string.bench.cpp
diff --git a/libcxx/benchmarks/stringstream.bench.cpp b/libcxx/test/benchmarks/stringstream.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/stringstream.bench.cpp
rename to libcxx/test/benchmarks/stringstream.bench.cpp
diff --git a/libcxx/benchmarks/system_error.bench.cpp b/libcxx/test/benchmarks/system_error.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/system_error.bench.cpp
rename to libcxx/test/benchmarks/system_error.bench.cpp
diff --git a/libcxx/benchmarks/to_chars.bench.cpp b/libcxx/test/benchmarks/to_chars.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/to_chars.bench.cpp
rename to libcxx/test/benchmarks/to_chars.bench.cpp
diff --git a/libcxx/benchmarks/unordered_set_operations.bench.cpp b/libcxx/test/benchmarks/unordered_set_operations.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/unordered_set_operations.bench.cpp
rename to libcxx/test/benchmarks/unordered_set_operations.bench.cpp
diff --git a/libcxx/benchmarks/util_smartptr.bench.cpp b/libcxx/test/benchmarks/util_smartptr.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/util_smartptr.bench.cpp
rename to libcxx/test/benchmarks/util_smartptr.bench.cpp
diff --git a/libcxx/benchmarks/variant_visit_1.bench.cpp b/libcxx/test/benchmarks/variant_visit_1.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/variant_visit_1.bench.cpp
rename to libcxx/test/benchmarks/variant_visit_1.bench.cpp
diff --git a/libcxx/benchmarks/variant_visit_2.bench.cpp b/libcxx/test/benchmarks/variant_visit_2.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/variant_visit_2.bench.cpp
rename to libcxx/test/benchmarks/variant_visit_2.bench.cpp
diff --git a/libcxx/benchmarks/variant_visit_3.bench.cpp b/libcxx/test/benchmarks/variant_visit_3.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/variant_visit_3.bench.cpp
rename to libcxx/test/benchmarks/variant_visit_3.bench.cpp
diff --git a/libcxx/benchmarks/vector_operations.bench.cpp b/libcxx/test/benchmarks/vector_operations.bench.cpp
similarity index 100%
rename from libcxx/benchmarks/vector_operations.bench.cpp
rename to libcxx/test/benchmarks/vector_operations.bench.cpp