[LNT] r249026 - Add support for storing hash of test binaries.

[Tobias Grosser via llvm-commits]

On 10/13/2015 10:44 PM, Kristof Beyls wrote:
> Hi Tobias,
>
> I think the least risky way to get your submissions through
> is to add the
> --exclude-stat-from-submission=hash
> command line option to the lnt runtest nt command.
>
> If you're using a setup based on zorg, this probably means
> adding '--exclude-stat-from-submission=hash' to the list
> specified by the nt_flags parameter of the
> LNTBuilder.getLNTFactory that defines your runs.
>
> This should result in not including hashes in the jsons your
> bots create, and should get your submissions accepted by the
> current version of LNT running at llvm.org/perf.
 >
>
> It does require a change in your slave setup, but this
> probably is the quickest way to get your bot submissions through
> to the server before the it is upgraded.

I just committed this change and requested a restart of the buildmaster.

However, I must say I am unhappy about how this change went in. I 
appreciate both Chris' and your work on LNT and am aware that I do not
contribute as much to LNT. Nevertheless, I spent a significant amount of 
time on getting the performance bots work for me. Hence, I personally 
care that they are not broken.

I understand that some changes may by accident break performance bots,
but I would appreciate if breaking changes are reverted as soon as
the breakage has been detected and only reapplied after the issues have
been resolved. This seems common tradition in LLVM and I do not see a 
reason why we could not have the same policy for LNT.

Asking Galina for a restart adds unnecessary additional work to her,
breaks all current builds and has a significant delay.

Second, if future changes to LNT require additional packages to be 
installed, I would appreciate an email 24 hours before committing these
changes such that I can update the relevant package-caches.

I am clearly not in the position to demand things, as both Chris and you 
spend their valuable time on LNT and llvm.org maintenance, but
still wanted to raise the issues I have.

Best,
Tobias