[PATCH] D43314: [lit] - Allow 1 test to report multiple micro-test results to provide support for microbenchmarks.

[Roman Lebedev via Phabricator via llvm-commits]

lebedev.ri marked an inline comment as done.
lebedev.ri added inline comments.


================
Comment at: llvm/trunk/utils/lit/lit/Test.py:163-164
+        Attach a micro-test result to the test result, with the given name and
+        result.  It is an error to attempt to attach a micro-test with the 
+        same name multiple times.
+
----------------
homerdin wrote:
> lebedev.ri wrote:
> > Was this restriction motivated?
> > Does LNT fail to import multiple results with the same name from a single `.json`?
> LNT accepts multiple results with the same name in a `.json` from multiple runs `--exec-multisample` as a combination of the individual lit outputs.  If we give two tests the same name in the test suite we would be combining results from two separate tests.  If you want to get multiple samples you can use `--exec-multisample`.  Also google benchmark is already reporting the mean of many runs.  
> 
> If you are wondering if you can add a micro-test result with the same name as a micro-test result that runs under a different executable that works because the names are extended with the parent's.
Thanks.
I was actually thinking about using `benchmark`'s repetitions instead of `--exec-multisample`,
but it appears that doesn't work, can't pass a list of measurements into `addMicroResult()`.


Repository:
  rL LLVM

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D43314/new/

https://reviews.llvm.org/D43314