[libcxx-commits] [libcxx] [libc++] Add an optional integration of libc++'s test suite with SPEC (PR #156953)

[Louis Dionne via libcxx-commits]

================
@@ -0,0 +1,78 @@
+# ===----------------------------------------------------------------------===##
+#
+# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+# See https://llvm.org/LICENSE.txt for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+#
+# ===----------------------------------------------------------------------===##
+
+# REQUIRES: enable-spec-benchmarks
+
+# RUN: mkdir -p %T
+# RUN: echo "%{cxx}" > %T/cxx.subs
+# RUN: echo "%{compile_flags}" > %T/compile_flags.subs
+# RUN: echo "%{flags}" > %T/flags.subs
+# RUN: echo "%{link_flags}" > %T/link_flags.subs
+# RUN: echo "%{spec_dir}" > %T/spec_dir.subs
+# RUN: %{python} %s %T
+# END.
+
+import json
+import pathlib
+import sys
+
+test_dir = pathlib.Path(sys.argv[1])
+cxx = (test_dir / 'cxx.subs').open().read().strip()
+compile_flags = (test_dir / 'compile_flags.subs').open().read().strip()
+flags = (test_dir / 'flags.subs').open().read().strip()
+link_flags = (test_dir / 'link_flags.subs').open().read().strip()
+spec_dir = pathlib.Path((test_dir / 'spec_dir.subs').open().read().strip())
+
+# Setup the configuration file
+test_dir.mkdir(parents=True, exist_ok=True)
+spec_config = test_dir / 'spec-config.cfg'
+spec_config.write_text(f"""
+default:
+    ignore_errors        = 1
+    iterations           = 1
+    label                = spec-stdlib
+    log_line_width       = 4096
+    makeflags            = --jobs=8
+    mean_anyway          = 1
+    output_format        = csv
+    preenv               = 0
+    reportable           = 0
+    tune                 = base
+    copies               = 1
+    threads              = 1
+    CC                   = cc -O3
----------------
ldionne wrote:

I agree. I think it's difficult to achieve at the moment and it shouldn't make a huge different for what we care about, but the way we're configuring the C compiler as a whole is less than ideal. Ideally we'd also use `CMAKE_C_COMPILER` but that would require threading additional information throughout the test suite just for that purpose, so for now I'm tempted to keep it simple.

https://github.com/llvm/llvm-project/pull/156953