[LNT] r249026 - Add support for storing hash of test binaries.

[Tobias Grosser via llvm-commits]

On 10/01/2015 06:06 PM, Kristof Beyls via llvm-commits wrote:
> Author: kbeyls
> Date: Thu Oct  1 11:06:12 2015
> New Revision: 249026
>
> URL: http://llvm.org/viewvc/llvm-project?rev=249026&view=rev
> Log:
> Add support for storing hash of test binaries.
>
> This commit adds support for storing hashes of the test binaries as a new
> sample type. The idea is that when the hash function of the same test is the
> same as on a previous run, the analyses can assume that the binary didn't
> change in a meaningful way.
>
> None of the analyses in LNT have been adapted yet to take advantage of this
> extra information.
>
> The hash value is stored as a new sample type.  Since the hash value in many
> cases should not be handled the same as other sample types, like execution time
> or compilation time, a distinction needs to be made. This patch changes
> terminology in LNT and calls sample types that have a meaningful order, like
> execution time and compilation time, "metric fields". The hash function of a
> binary - as one hash value cannot be said to be "better" than another hash
> value - is not considered a "metric field".
>
> Care has been taken to make sure LNT can keep on working with test-suite
> versions that don't produce the hash values yet, which was introduced in
> r246035.

Hi Kristof,

I am unsure if this is the precise commit that caused the issue, but 
since the introduction of hash values my performance bots cannot submit
benchmark results any more:

http://lab.llvm.org:8011/builders/perf-x86_64-penryn-O3-polly/builds/3475/steps/lnt.nightly-test/logs/stdio

===============================================================================
Importing u'data-2015-10-12_01-23-39Hiswx5.plist'
Import Failed:
--
import failure: Traceback (most recent call last):
   File 
"/opt/venv/perf/lib/python2.7/site-packages/LNT-0.4.1dev-py2.7.egg/lnt/util/ImportData.py", 
line 74, in import_and_report
     success,run = db.importDataFromDict(data, config=db_config)
   File 
"/opt/venv/perf/lib/python2.7/site-packages/LNT-0.4.1dev-py2.7.egg/lnt/server/db/v4db.py", 
line 167, in importDataFromDict
     return db.importDataFromDict(data, config)
   File 
"/opt/venv/perf/lib/python2.7/site-packages/LNT-0.4.1dev-py2.7.egg/lnt/server/db/testsuitedb.py", 
line 715, in importDataFromDict
     self._importSampleValues(data['Tests'], run, tag)
   File 
"/opt/venv/perf/lib/python2.7/site-packages/LNT-0.4.1dev-py2.7.egg/lnt/server/db/testsuitedb.py", 
line 671, in _importSampleValues
     name)
ValueError: test u'SingleSource/Benchmarks/Shootout/fib2.hash' does not 
map to a sample field in the reported suite/
===============================================================================

Maybe you have an idea?

Best,
Tobias