[llvm-dev] Clarification on expectations of buildbot email notifications

[Zachary Turner via llvm-dev]

Hi all,

Over the past year or so, all of us have broken the buildbots on many
occasions.  Usually we get notified on IRC, or via an buildbot email
notification sent to everyone on the blamelist.
If I happen to be on IRC I'll see the notification, but if not, the next
best thing is an email that was automatically sent to me (along with
everyone else on the blamelist) from the buildbot with information about
the failure.
And then finally, I'll occasionally get a response to my commit message
telling me that it's broken, and the patch may be reverted with information
in the commit message explaining which bot was broken and providing a link
to it.

However, we have some buildbots on the public waterfall which are
specifically configured not to send emails to people.  In some cases it's
because the bots are experimental, but there are a handful where the
reasoning I've been given is that it "wastes peoples time and contributes
to build blindness", but we are still expected to keep them green (usually
by people manually reaching out to us when they fail, or patches getting
reverted and us getting notified of the revert).

It is this last case that I'm concerned about, as it appears to be in
direct conflict with our own developer policy [
https://llvm.org/docs/DeveloperPolicy.html#id14], which states this
-----
We prefer for this to be handled before submission but understand that it
isn’t possible to test all of this for every submission. Our build bots and
nightly testing infrastructure normally finds these problems. A good rule
of thumb is to check the nightly testers for regressions the day after your
change. Build bots will directly email you if a group of commits that
included yours caused a failure. You are expected to check the build bot
messages to see if they are your fault and, if so, fix the breakage.

Commits that violate these quality standards (e.g. are very broken) may be
reverted. This is necessary when the change blocks other developers from
making progress. The developer is welcome to re-commit the change after the
problem has been fixed.

-----

I'm sending this email to get a sense of the community's views on this
matter.  If I'm correctly reading between the lines in the above passage,
buildbots which do not send emails should not be subject to the
revert-to-green policy.  To be honest, it's actually not even clear from
reading the above passage where the burden of fixing a "broken" patch on a
silent buildbot lies at all - with the patch author or with the bot
maintainer.


Would anyone care to weigh in with an unbiased opinion here?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190219/9a44ce36/attachment.html>