[llvm-dev] Proposal: Make the VE target official

[Simon Moll via llvm-dev]

Hi,

we've addressed all points raised on the proposal to make VE an
official target:

* Buildbot worker times are down to 7-8mins. We use ccache (thx!).
* We're set up to enable a worker reporting to the main buildbot
instance.
* The `clang-ve-ninja` buildbot is green and its testing represents
what is supported for VE.

We're ready to switch to official status. We're planning to proceed as
follows once D113247 and D113389 are accepted.

1. Connect the VE worker to main buildbot instance.
2. Add `VE` to the LLVM target list, thereby making it official
(D113247).
3. Update llvm-zorg to build VE as an official target and enable
buildbot reporting for VE (D113389).
4. Attentively follow all builders until the expected aftermath has
settled.

Only after this, we will unfreeze VE feature development and spin up
the staging worker for work-in-progress features.

- Simon

On Thu, 2021-11-11 at 08:09 +0000, Simon Moll via llvm-dev wrote:
> On Wed, 2021-11-10 at 17:52 +0000, Renato Golin wrote:
> > On Wed, 10 Nov 2021 at 14:10, Simon Moll <Simon.Moll at emea.nec.com>
> > wrote:
> > > We should be able to bring that down. clang-ve-ninja currently
> > > builds
> > > everything from scratch (and it's all static - i'd love to have
> > > working
> > > shared component libraries for faster/incremental builds).
> > > 
> > > We are also considering a second, faster builder that only builds
> > > and
> > > tests LLVM+Clang. That would be the canary for any issues with
> > > the
> > > VE
> > > backend.
> > > 
> > 
> >  
> > IIUC, your builder is an x86_64 machine cross testing on a VE
> > target. 
> > 
> > If the canary builder doesn't run anything on VE hardware, then
> > after
> > VE is official, it won't be different than any other x86_64 builder
> > building/testing VE.
> > 
> > 
> > > Once D113093 is in, clang-ve-ninja is expected to be green.
> > > We can call that the stable state - everything that's tested is
> > > supposed to work and any red-ness implies breakage.
> > > 
> > 
> > 
> > Excellent! Make sure once green that the bot be moved to the
> > production server. Check with Galina to make sure you're on the
> > right
> > place.
> > 
> > 
> > > This may be a good moment to mention that the compiler-rt patch
> > > disables tests that will never work on VE - there is no fp
> > > denormal
> > > support, for example.
> > > 
> > 
> > 
> > Ah, right, so then permanently disabled is the right thing to do,
> > no
> > need for bugzilla entries.
> > 
> > 
> > > We add a new staging buildbot that builds everything clang-ve-
> > > ninja
> > > does plus the yet-unsupported features that we are currently
> > > working
> > > on.
> > > 
> In essense, we keep the current staging clang-ve-ninja as is and
> attach
> a clone to the main buildbot. The clone is then the official bot and
> the staging one is the one for work-in-progress features.
> 
> > 
> > 
> > If you do that, make sure you add it to the staging server (or a
> > private one). This bot cannot notify people of breakages (email,
> > IRC,
> > nothing).
> > 
> > 
> > > Initially, that bot will be 'red' while the official one has to
> > > be
> > > kept
> > > 'green'. Once we are confident about the feature/runtime - both
> > > bots
> > > are 'green' - we will make the official bot test that feature,
> > > thereby
> > > declaring the feature official. The staging bot will turn to new
> > > experimental features.
> > > 
> > 
> > 
> > Once it's official, and green, move the bot to the production
> > (noisy)
> > server and create a new one for new features in the staging one.
> > 
> > Once the target is out of experimental, every buildbot that doesn't
> > restrict the targets it builds (most bots) will build VE and run
> > its
> > LIT tests.
> > 
> > Because you're the VE code owner, if your target breaks other
> > people's bots, developers will (hopefully) notify you, too.
> > 
> > So your strategy would have at the very least three (classes of)
> > bots:
> > 
> > 1. Fast "canary" bot, building only Clang and LLVM and running LIT
> > tests (maybe you'll want to add some cross-VE testing here, too)
> > 
> > 2. Slow complete bot, building everything that is supposed to be
> > supported. This can be multiple bots with different configurations
> > or
> > one huge build.
> > 
> > 3. A staging (silent) build with new stuff that the team is working
> > on. This can also be different bots, totally up to the team, and
> > doesn't even need to be public.
> That's the plan. Given that other/fast bots will build the VE target
> and run its lit tests, we may not need the canary though.
> 
> > 
> > I did a similar division for Arm and we've been doing it since for
> > Arm 32 and 64. Even if we use the same server for multiple bots,
> > it's
> > easier to debug breakages when we build less stuff.
> libunwind, compiler-rt, libcxxabi, libcxx really dependend on each
> other. As long as build times are reasonable, we will keep it to full
> stack testing.
> 
> - Simon
> 
> > 
> > cheers,
> > --renato
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>