[PATCH] D16871: lit: Allow arbitrary key/value data in a test result

[Daniel Dunbar via llvm-commits]

I would just use addMetric for this. I see a binary signature as just
another kind of metric data, sometimes people even use the term "measure"
to the process of computing a hash signature for a binary.

If you really dislike the term metric, I would probably rather just change
the one method name than have two methods that basically do the same job.

Also, +1 on getting test suite support for reporting binary hashes!

 - Daniel


On Wed, Feb 3, 2016 at 5:38 PM, Matthias Braun <matze at braunis.de> wrote:

> MatzeB created this revision.
> MatzeB added reviewers: ddunbar, cmatthews, EricWF.
> MatzeB added a subscriber: llvm-commits.
> MatzeB set the repository for this revision to rL LLVM.
> Herald added a subscriber: mcrosier.
>
> The test-suite will use this to attach additional infos like the md5sum
> of the executable to the test result. We should not use the existing
> addMetric() interface for this as an md5sum is not really a metric.
>
> Repository:
>   rL LLVM
>
> http://reviews.llvm.org/D16871
>
> Files:
>   utils/lit/lit/Test.py
>   utils/lit/lit/main.py
>
> Index: utils/lit/lit/main.py
> ===================================================================
> --- utils/lit/lit/main.py
> +++ utils/lit/lit/main.py
> @@ -94,19 +94,19 @@
>      # Encode the tests.
>      data['tests'] = tests_data = []
>      for test in run.tests:
> -        test_data = {
> -            'name' : test.getFullName(),
> -            'code' : test.result.code.name,
> -            'output' : test.result.output,
> -            'elapsed' : test.result.elapsed }
> +        result = test.result
> +        result.data['name'] = test.getFullName()
> +        result.data['code'] = result.code.name
> +        result.data['output'] = result.output
> +        result.data['elapsed'] = result.elapsed
>
>          # Add test metrics, if present.
> -        if test.result.metrics:
> -            test_data['metrics'] = metrics_data = {}
> -            for key, value in test.result.metrics.items():
> +        if result.metrics:
> +            result.data['metrics'] = metrics_data = {}
> +            for key, value in result.metrics.items():
>                  metrics_data[key] = value.todata()
>
> -        tests_data.append(test_data)
> +        tests_data.append(result.data)
>
>      # Write the output.
>      f = open(output_path, 'w')
> Index: utils/lit/lit/Test.py
> ===================================================================
> --- utils/lit/lit/Test.py
> +++ utils/lit/lit/Test.py
> @@ -126,6 +126,8 @@
>          self.elapsed = elapsed
>          # The metrics reported by this test.
>          self.metrics = {}
> +        # Additional data reported by this test.
> +        self.data = {}
>
>      def addMetric(self, name, value):
>          """
> @@ -144,6 +146,15 @@
>              raise TypeError("unexpected metric value: %r" % (value,))
>          self.metrics[name] = value
>
> +    def addData(self, name, value):
> +        """
> +        Attach additional data to the test result.
> +
> +        The data must be encodable as JSon (strings, numbers, lists,
> +        dictionaires)
> +        """
> +        self.data[name] = value
> +
>  # Test classes.
>
>  class TestSuite:
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160210/a98cbb18/attachment.html>