[PATCH] D43314: [lit] - Allow 1 test to report multiple micro-test results to provide support for microbenchmarks.

Brian Homerding via Phabricator via llvm-commits llvm-commits at lists.llvm.org
Thu Feb 15 08:23:14 PST 2018


homerdin updated this revision to Diff 134434.
homerdin edited the summary of this revision.
homerdin added a comment.

I updated to a full context diff and have made some of the python changes requested.  I changed the way the name is formed to make it clearer.

@delcypher I had misread your request for tests that show what I am trying to do, which is why I opened the revision for LCALS.  I will add a test to lit's test suite for these changes.  I'll also look into the documentation and see what I can contribute there.

I still need to look into the Python 2.x and 3.x behavior regarding `.items()` and if we force Python 3 for LIT.


https://reviews.llvm.org/D43314

Files:
  utils/lit/lit/Test.py
  utils/lit/lit/main.py


Index: utils/lit/lit/main.py
===================================================================
--- utils/lit/lit/main.py
+++ utils/lit/lit/main.py
@@ -81,6 +81,17 @@
                 print('%s: %s ' % (metric_name, value.format()))
             print("*" * 10)
 
+        # Report micro-tests, if present
+        if test.result.microResults:
+            items = sorted(test.result.microResults.items())
+            for micro_test_name, micro_test in items:
+                print("%s MICRO-TEST: %s" %
+                         ('*'*3, micro_test_name))
+   
+                if micro_test.metrics:
+                    for metric_name, value in micro_test.metrics.items():
+                        print('    %s:  %s ' % (metric_name, value.format()))
+
         # Ensure the output is flushed.
         sys.stdout.flush()
 
@@ -113,6 +124,26 @@
             for key, value in test.result.metrics.items():
                 metrics_data[key] = value.todata()
 
+        # Report micro-tests separately, if present
+        if test.result.microResults:
+            for key, micro_test in test.result.microResults.items():
+                # Expand parent test name with micro test name
+                micro_test_name = '/' + key + '.test'
+                micro_full_name = test.getFullName().replace(".test",
+                                                             micro_test_name)
+
+                micro_test_data = {
+                    'name' : micro_full_name,
+                    'code' : micro_test.code.name,
+                    'output' : micro_test.output,
+                    'elapsed' : micro_test.elapsed }
+                if micro_test.metrics:
+                    micro_test_data['metrics'] = micro_metrics_data = {}
+                    for key, value in micro_test.metrics.items():
+                        micro_metrics_data[key] = value.todata()
+
+                tests_data.append(micro_test_data)
+
         tests_data.append(test_data)
 
     # Write the output.
Index: utils/lit/lit/Test.py
===================================================================
--- utils/lit/lit/Test.py
+++ utils/lit/lit/Test.py
@@ -135,6 +135,8 @@
         self.elapsed = elapsed
         # The metrics reported by this test.
         self.metrics = {}
+        # The micro-test results reported by this test.
+        self.microResults = {}
 
     def addMetric(self, name, value):
         """
@@ -153,6 +155,24 @@
             raise TypeError("unexpected metric value: %r" % (value,))
         self.metrics[name] = value
 
+    def addMicroResult(self, name, microResult):
+        """
+        addMicroResult(microResult)
+
+        Attach a micro-test result to the test result, with the given name and
+        result.  It is an error to attempt to attach a micro-test with the 
+        same name multiple times.
+
+        Each micro-test result must be an instance of the Result class.
+        """
+        if name in self.microResults:
+            raise ValueError("Result already includes microResult for %r" % (
+                   name,))
+        if not isinstance(microResult, Result):
+            raise TypeError("unexpected MicroResult value %r" % (microResult,))
+        self.microResults[name] = microResult
+
+
 # Test classes.
 
 class TestSuite:


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D43314.134434.patch
Type: text/x-patch
Size: 3300 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20180215/b6e30049/attachment.bin>


More information about the llvm-commits mailing list