[llvm-dev] Buildbot Noise

[Chris Matthews via llvm-dev]

> On Oct 7, 2015, at 4:23 PM, Renato Golin <renato.golin at linaro.org> wrote:
> 
> On 7 October 2015 at 23:58, Chris Matthews <chris.matthews at apple.com> wrote:
>> Unfortunately, Jenkins does not let me determine who to email based on the failure cause. That would be wonderful! The detected problem is right at the top of the email though, so at least you don’t have to click the link.
> 
> Yes, I've seen this, and I loved it.
> 
> 
>> For infrastructure problems sometimes we add helpful messages, for instance we had an issue with about 1 in 20 builds failing with a “killed -9” message, when that happened we could just say sorry an print a link to the bug.  Mostly is is just a nice fast link right to the test case failure or build failure.
> 
> Indeed.
> 
> 
>> I define flaky as a build that fails for a reason unrelated to the code on the blame list.
> 
> Me too, but latent *code* problems introduced before this build need
> to show up somewhere, and it's non-trivial to detect those kind of
> errors based on the blame list alone. An example of this is the C++
> ABI bugs.
> 
> They're rare enough that we can deal with them. But they look *a lot*
> like flaky bots, enough that could trigger people to disable bots
> without further consideration.
> 
> 
>> We should move towards getting this setup then.  There is some code that needs to be setup in buildbot, as well as an agreement on what gets attached to what.
> 
> Excellent! I have some spare hardware that I can use for experimental builders.
> 
> But, this being Jenkins, I suspect you'll need to push the jobs
> through, rather than me pulling them like buildbots. For that, I'll
> need some firewall configurations.

This can be done with buildbot or Jenkins. Both platforms support it.   It is a huge amount of work to port jobs to Jenkins, so that is not to be taken lightly.  This might be a good discussion to have at the conference, either way there is going to be some work to change configurations and link things up.

A simple buildbot specific fix is to just switch some of our bots to manual only launches. Then use curl commands at the end of other builds to tigger them.    That is even possible to do between Jenkins and buildbot.  That might be a quick and dirty way to phase some builds.

> 
> Just let me know when you're ready! :)
> 
> cheers,
> --renato