[PATCH] D37421: [XRay] [test-suite] Add LNT support to retref-bench benchmarks.

[Hal Finkel via Phabricator via llvm-commits]

hfinkel added a comment.

Thanks for working on this! As a general note, I plan to add other microbenchmarks (and to transition at least MultiSource/Benchmarks/TSVC into this form). Can we take the code in  LNTBased/XRay/TestModule and move it into, say, LNTBased/lib/microbenchmarks.py. We can have LNTBased/Microbenchmarks/XRay, and the we can just iterate over all top-level directories in LNTBased/Microbenchmarks. The name 'XRay' is currently hard-coded in LNTBased/XRay/TestModule, but that could be a parameter taken from the directory name.



================
Comment at: LNTBased/XRay/TestModule:35
+            ['{}/{}'.format(options['OBJROOT'], 'retref-bench'),
+             '--benchmark_format=csv', '--benchmark_repetitions=10',
+             '--benchmark_report_aggregates_only=true'],
----------------
Please don't fix the number of iterations (i.e., --benchmark_repetitions=10). A major advantage of Google's benchmark library is its ability to dynamically vary the number of iterations in favor of determining accurately the per-iteration time. We should let it do that.


================
Comment at: LNTBased/XRay/TestModule:47
+        results = []
+        for line in csv_lines:
+          if line[0] == "name":
----------------
Please add a comment here with the columns so that it's clear what's happening:

  # name,iterations,real_time,cpu_time,time_unit,...



https://reviews.llvm.org/D37421