[LNT] r249026 - Add support for storing hash of test binaries.

Kristof Beyls via llvm-commits llvm-commits at lists.llvm.org
Tue Oct 13 14:39:11 PDT 2015


You're absolutely right in being annoyed and I apologize for the
breakage. I would've reverted this, but the commit does include a
change to the database schema and I'm afraid that reverting that
change could cause people's database instances to get corrupted
in the worst case.

Thinking a bit more on how to get up-to-date clients to submit
again to out-of-date servers: we could make the
--exclude-stat-from-submission=hash
the default value for now, until the llvm.org/perf server is updated.
While this would help for working around the llvm.org/perf server issue,
it would regress the functionality for people who run their own LNT servers,
who will see that hash values will no longer be submitted to their servers
which can process them.

Putting everything in balance, I think it's a good idea in this
instance to make this change until llvm.org/perf is upgraded; which
would mean no clients/slaves would need to be changed and would avoid
the buildbot master restart.

All of this is obviously far from ideal, and there should be a few
lessons learned from this.

Thanks,

Kristof

> -----Original Message-----
> From: Tobias Grosser [mailto:tobias at grosser.es]
> Sent: 13 October 2015 22:17
> To: Kristof Beyls; Chris Matthews
> Cc: Chris Matthews; llvm-commits at lists.llvm.org
> Subject: Re: [LNT] r249026 - Add support for storing hash of test
> binaries.
> 
> On 10/13/2015 10:44 PM, Kristof Beyls wrote:
> > Hi Tobias,
> >
> > I think the least risky way to get your submissions through is to add
> > the --exclude-stat-from-submission=hash
> > command line option to the lnt runtest nt command.
> >
> > If you're using a setup based on zorg, this probably means adding
> > '--exclude-stat-from-submission=hash' to the list specified by the
> > nt_flags parameter of the LNTBuilder.getLNTFactory that defines your
> > runs.
> >
> > This should result in not including hashes in the jsons your bots
> > create, and should get your submissions accepted by the current
> > version of LNT running at llvm.org/perf.
>  >
> >
> > It does require a change in your slave setup, but this probably is the
> > quickest way to get your bot submissions through to the server before
> > the it is upgraded.
> 
> I just committed this change and requested a restart of the buildmaster.
> 
> However, I must say I am unhappy about how this change went in. I
> appreciate both Chris' and your work on LNT and am aware that I do not
> contribute as much to LNT. Nevertheless, I spent a significant amount of
> time on getting the performance bots work for me. Hence, I personally
> care that they are not broken.
> 
> I understand that some changes may by accident break performance bots,
> but I would appreciate if breaking changes are reverted as soon as the
> breakage has been detected and only reapplied after the issues have been
> resolved. This seems common tradition in LLVM and I do not see a reason
> why we could not have the same policy for LNT.
> 
> Asking Galina for a restart adds unnecessary additional work to her,
> breaks all current builds and has a significant delay.
> 
> Second, if future changes to LNT require additional packages to be
> installed, I would appreciate an email 24 hours before committing these
> changes such that I can update the relevant package-caches.
> 
> I am clearly not in the position to demand things, as both Chris and you
> spend their valuable time on LNT and llvm.org maintenance, but still
> wanted to raise the issues I have.
> 
> Best,
> Tobias





More information about the llvm-commits mailing list