[LNT] r249026 - Add support for storing hash of test binaries.

[Chris Matthews via llvm-commits]

One thing I do is have the builds store their report.json file, so when the submissions get mucked up I can manually fix and then do a "lnt submit". This has saved hundreds of results from being lost on Green Dragon. Better still, when I bring up a test instance, I can blast the last few months of results into it, so I have real data. 

I think there is a constant danger of problems like this when our bots all work on ToT LNT.  I don't know if doing something more sane is worth the effort though? Maybe upgrade llvm.org perf at the start of every month? At least you'd know when it will fail. 

> On Oct 12, 2015, at 12:51 AM, Tobias Grosser <tobias at grosser.es> wrote:
> 
>> On 10/12/2015 09:37 AM, Kristof Beyls wrote:
>> Hi Tobias,
>> 
>> The instance of LNT running at llvm.org/perf needs to be upgraded
>> to at least this commit, to be able to understand these new hash fields.
>> Chris is planning to do so.
>> 
>> Given that nobody noticed this failure for a couple of days, we
>> thought it means that currently, nobody is critically dependent on
>> llvm.org/perf. Therefore, we decided to move forward and get llvm.org/perf
>> upgraded, rather than trying to downgrade the clients again, which may be
>> non-trivial.
> 
> Yes, please update quickly.
> 
> I did not notice this change earlier as some new library dependences added by Chris broke my LNT builds (now fixed). Also for some reason the LNT builder does not turn red if the submission fails. So it took a while to notice this.
> 
> In general my current use case is not to look every day but to look at the full per-commit performance history for reference whenever I do something performance critical or when I track down performance bugs.
> Unfortunately, now a couple of days performance data is lost. :( It would be nice if this could be fixed quickly (or the offending LNT changes be disabled until the LNT builder is upgraded).
> 
>> The upgrade will also pick up a number of stability fixes, so hopefully
>> llvm.org/perf will be a bit more stable after the upgrade too.
> 
> Cool.
> 
> Best,
> Tobias
> 
> P.S: I know we did not have LNT access for now, but maybe in the future first updating the LNT server and then committing problematic client changes would be nice.