[LNT] r249026 - Add support for storing hash of test binaries.

[Kristof Beyls via llvm-commits]

Hi Tobias,

The instance of LNT running at llvm.org/perf needs to be upgraded
to at least this commit, to be able to understand these new hash fields.
Chris is planning to do so.

Given that nobody noticed this failure for a couple of days, we
thought it means that currently, nobody is critically dependent on
llvm.org/perf. Therefore, we decided to move forward and get llvm.org/perf
upgraded, rather than trying to downgrade the clients again, which may be
non-trivial.

The upgrade will also pick up a number of stability fixes, so hopefully
llvm.org/perf will be a bit more stable after the upgrade too.

Thanks,

Kristof

> -----Original Message-----
> From: Tobias Grosser [mailto:tobias at grosser.es]
> Sent: 12 October 2015 05:55
> To: Kristof Beyls; llvm-commits at lists.llvm.org
> Subject: Re: [LNT] r249026 - Add support for storing hash of test
> binaries.
> 
> On 10/01/2015 06:06 PM, Kristof Beyls via llvm-commits wrote:
> > Author: kbeyls
> > Date: Thu Oct  1 11:06:12 2015
> > New Revision: 249026
> >
> > URL: http://llvm.org/viewvc/llvm-project?rev=249026&view=rev
> > Log:
> > Add support for storing hash of test binaries.
> >
> > This commit adds support for storing hashes of the test binaries as a
> > new sample type. The idea is that when the hash function of the same
> > test is the same as on a previous run, the analyses can assume that
> > the binary didn't change in a meaningful way.
> >
> > None of the analyses in LNT have been adapted yet to take advantage of
> > this extra information.
> >
> > The hash value is stored as a new sample type.  Since the hash value
> > in many cases should not be handled the same as other sample types,
> > like execution time or compilation time, a distinction needs to be
> > made. This patch changes terminology in LNT and calls sample types
> > that have a meaningful order, like execution time and compilation
> > time, "metric fields". The hash function of a binary - as one hash
> > value cannot be said to be "better" than another hash value - is not
> considered a "metric field".
> >
> > Care has been taken to make sure LNT can keep on working with
> > test-suite versions that don't produce the hash values yet, which was
> > introduced in r246035.
> 
> Hi Kristof,
> 
> I am unsure if this is the precise commit that caused the issue, but
> since the introduction of hash values my performance bots cannot submit
> benchmark results any more:
> 
> http://lab.llvm.org:8011/builders/perf-x86_64-penryn-O3-
> polly/builds/3475/steps/lnt.nightly-test/logs/stdio
> 
> ========================================================================
> =======
> Importing u'data-2015-10-12_01-23-39Hiswx5.plist'
> Import Failed:
> --
> import failure: Traceback (most recent call last):
>    File
> "/opt/venv/perf/lib/python2.7/site-packages/LNT-0.4.1dev-
> py2.7.egg/lnt/util/ImportData.py",
> line 74, in import_and_report
>      success,run = db.importDataFromDict(data, config=db_config)
>    File
> "/opt/venv/perf/lib/python2.7/site-packages/LNT-0.4.1dev-
> py2.7.egg/lnt/server/db/v4db.py",
> line 167, in importDataFromDict
>      return db.importDataFromDict(data, config)
>    File
> "/opt/venv/perf/lib/python2.7/site-packages/LNT-0.4.1dev-
> py2.7.egg/lnt/server/db/testsuitedb.py",
> line 715, in importDataFromDict
>      self._importSampleValues(data['Tests'], run, tag)
>    File
> "/opt/venv/perf/lib/python2.7/site-packages/LNT-0.4.1dev-
> py2.7.egg/lnt/server/db/testsuitedb.py",
> line 671, in _importSampleValues
>      name)
> ValueError: test u'SingleSource/Benchmarks/Shootout/fib2.hash' does not
> map to a sample field in the reported suite/
> ========================================================================
> =======
> 
> Maybe you have an idea?
> 
> Best,
> Tobias