[llvm-dev] Buildbot roundtable writeup

[Nico Weber via llvm-dev]

Hi,

during LLVM conf we had a roundtable discussing the state of buildbot. Here
are the notes of what we discussed.

The summary is that there's lots of appetite for improving the state of
LLVM's infra, with lots of good shorter and longer term ideas (see below).

Several people were interested in starting an (open for all who are
interested) "LLVM infra team", with possibly a dedicated mailing list, and
with possibly the powers to make infra changes with just consensus from
people on that infra team.

(Sorry for the delayed email, I wrote this up right after the meeting but
forgot to hit "Send".)

The actual notes:

Problems with buildbot
- console view loads slowly
- many bots take a long time to cycle
- many bots are perma-red
- test output on some bots is huge due to the bots printing all tests, not
just failing ones, making it difficult to see failing tests
- it's sometimes difficult to reproduce failures on the bots locally

Possible improvements
- display better machine info on all builders (OS, host compiler with
detailed version, binutils version, cmake version, ninja version,
kernel/userspace bitness)
- require bots to use a cmake cache file, for easy local matching execution?
- remove perma-red bots
- remove slow bots
-- or put on faster hw, llvm foundation has funds
--- what about slow boards?
---- decouple build and test phases?
---- shard tests over multiple devices?
- make fast bots trigger slow bots, only when fast builds are successful,
for fewer emails
- have support tier lists?
-- e.g. tier 0 pledges bots that cycle in < 15 min, in return are on tier 0
waterfall and can revert breakages after ~ 15 min
-- tier 1 pledges bots that cyclle in < 1 day, can revert breakages after 1
day
- update buildbot to current version?
-- lots of api changes
- have pre-commit tests
-- kuhnel has prototype for this on linux, will send separate announcement,
positive reception
--- several requests to have the same for win
- move build off buildbot to github actions?
-- jyknight has prototype, works great, except that custom hardware isn't
(yet?) supported, so cycle times are prohibitively long
- have a dedicated llvm infra team
-- dedicated llvm-infra mailing list
-- and group of deciders with llvm foundation's blessing?
- have a buildbot view that shows only red bots?
- have an in-tree script for setting up a build + prereqs (eg. new-enough
host gcc, gnuwin tools on win, new enough cmake, etc)?
- current bots don't cover multi-config cmake generators (ie Xcode, msvc
before 2019)
-- explicitly say we don't support those? would allow some cleanups
--- msvc 2019 cmake support generates ninja builds for both debug and
release and calls ninja for the actual build
--- maybe do something similar for xcode?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20191107/5b3afc51/attachment.html>