[libcxx-commits] [libcxx] [libc++] Fix SPEC benchmarks not producing a .lnt result file (PR #207450)

[via libcxx-commits]

llvmorg-github-actions[bot] wrote:


<!--LLVM PR SUMMARY COMMENT-->

@llvm/pr-subscribers-libcxx

Author: Louis Dionne (ldionne)

<details>
<summary>Changes</summary>

The refactoring in 471e8f7f94e7 removed the output of a .lnt file, which is necessary for interoperation with consolidate-benchmarks.

---
Full diff: https://github.com/llvm/llvm-project/pull/207450.diff


2 Files Affected:

- (modified) libcxx/test/benchmarks/spec.gen.py (+1-1) 
- (modified) libcxx/utils/run-spec-benchmark (+14-4) 


``````````diff
diff --git a/libcxx/test/benchmarks/spec.gen.py b/libcxx/test/benchmarks/spec.gen.py
index 13e374adf1587..0eeb2192d47f0 100644
--- a/libcxx/test/benchmarks/spec.gen.py
+++ b/libcxx/test/benchmarks/spec.gen.py
@@ -34,4 +34,4 @@
 
 for benchmark in spec_benchmarks:
     print(f'#--- {benchmark}.sh.test')
-    print(f'RUN: %{{python}} %{{libcxx-dir}}/utils/run-spec-benchmark --spec-dir %{{spec_dir}} --temp-dir %{{temp}} --benchmark {benchmark} --clean -- %{{cxx}} %{{compile_flags}} %{{flags}} %{{link_flags}}')
+    print(f'RUN: %{{python}} %{{libcxx-dir}}/utils/run-spec-benchmark --spec-dir %{{spec_dir}} --temp-dir %{{temp}} --benchmark {benchmark} --output %{{temp}}/results.lnt --clean -- %{{cxx}} %{{compile_flags}} %{{flags}} %{{link_flags}}')
diff --git a/libcxx/utils/run-spec-benchmark b/libcxx/utils/run-spec-benchmark
index 7a4fb565d34ef..c9b91598ce3c6 100755
--- a/libcxx/utils/run-spec-benchmark
+++ b/libcxx/utils/run-spec-benchmark
@@ -42,6 +42,12 @@ def main():
         action="store_true",
         help="Clean up build artifacts after running. Useful to save disk space when running multiple benchmarks.",
     )
+    parser.add_argument(
+        "--output",
+        type=str,
+        default=None,
+        help="Path to write the LNT-format results to. Defaults to stdout.",
+    )
     parser.add_argument(
         "compiler_and_flags",
         nargs="+",
@@ -122,16 +128,20 @@ default:
             print(f"SPEC benchmark {args.benchmark} had a compilation or runtime error.", file=sys.stderr)
             sys.exit(1)
 
-    # Parse results into LNT format and print to stdout.
+    # Parse results into LNT format. Write them to the requested output file (or stdout by default).
+    # We only get here after the build and run succeeded, so a failed benchmark produces no output file.
     benchmark_name = args.benchmark.replace(".", "_")
     spec_results = subprocess.run([str(utils_dir / "parse-spec-results"), "--output-format=lnt"] + csv_files,
                                    capture_output=True, text=True, check=True)
-    sys.stdout.write(spec_results.stdout)
-
     time_results = subprocess.run([str(utils_dir / "parse-time-output"), str(time_output), f"--benchmark={benchmark_name}",
                                     "--extract", "instructions", "max_rss", "cycles", "peak_memory"],
                                     capture_output=True, text=True, check=True)
-    sys.stdout.write(time_results.stdout)
+
+    lnt_results = spec_results.stdout + time_results.stdout
+    if args.output:
+        pathlib.Path(args.output).write_text(lnt_results)
+    else:
+        sys.stdout.write(lnt_results)
 
     # Clean up build artifacts since they can be very large.
     if args.clean:

``````````

</details>


https://github.com/llvm/llvm-project/pull/207450