[Release-testers] [llvm-dev] 4.0.1 Release Schedule + Need feedback for improving stable releases

[Mehdi Amini via Release-testers]

> On Mar 27, 2017, at 8:58 AM, Tom Stellard via llvm-dev <llvm-dev at lists.llvm.org> wrote:
> 
> Hi,
> 
> I would like to start a discussion about improvements to the
> stable release process, but first, here is a proposed schedule
> for the 4.0.1 release.
> 
> May 1, 2017 -rc1
> May 22, 2017 Deadline for submitting merge request
> May 29, 2017 Deadline for merging changes.
> June 3, 2017 -rc2
> June 10, 2017 Final Release
> 
> This is slightly different from previous stable releases in that
> we are doing an early -rc1 release.  This matches what we do for
> major releases, and I think it will help us get more feedback from
> testers sooner.
> 
> There is now a script in utils/release, called merge-request.sh, which
> you can use to submit merge requests to bugzilla.  It's recommend
> that you use this script in order to give us consistent bugzilla entries.
> However, you can still file bugzilla requests manually if for some
> reason you are unable to use the script.

I like the fact that there is a script!
It’d be really nice to have an entry on the website describing how to use this. I always struggle to find the right email that tells me how to handle cherry-picks.


> If you file bugs manually, make sure you mark it as a blocker for
> the release metabug: 32061 and add a link the commit on phabricator
> in the url field: e.g http://reviews.llvm.org/rL12345
> 
> Description for the bug should be in the form of:
> 
> Merge r123456 into the 4.0 branch : <80 character commit message summary>
> 
> 
> * Stable Release Improvements *
> 
> I would like to start an informal discussion about stable release
> improvements.  My goals for this discussion are to find ways to
> get more community members/organizations using the official stable
> releases, and also to help simplify the release process for users,
> testers, and release managers.
> 
> * Ideas:
> 
> 1. Longer support for stable release. We currently have 2 major and 2
> stable releases per year.  Each major release is effectively supported
> for only 3 months, because after the x.y.1 release , we move on to the
> next major version.  Our current release schedule is approximately:
> 
> March: x.y.0
> June x.y.1
> Sept: (x+1).y.0 
> Dec: (x+1).y.1
> 
> One idea is to move to 6 releases per year and have an additional
> x.y.2 release after (x+1).y.0, but before (x+1).y.0.  This would
> mean that stable releases are supported for ~ 8 months. So something
> like:
> 
> Jan:   (x-1).y.1 
> March: x.y.0
> May:   (x-1).y.2
> July:  x.y.1
> Sept:  (x+1).y.0
> Nov:   x.y.2 
> 
> What do people think about this or other alternative release schedules?
> Would this kind of extended support make stable releases more
> attractive to users/organizations that use LLVM.
> 
> My main concern with adding an extra release is placing an extra
> burden on release testers.  For release testers is doing two
> more releases per year practical?
> 
> 
> 
> 2. Automated release testing.  This is connected to the previous
> idea.  Can we automate the release testing?  It would be great if
> we had some release tester bots that would test the release
> candidates and upload the binaries.  It seems like this should
> be doable within the current buildbot infrastructure.  Is the
> only thing that keeps us from doing this, lack of dedicated test
> hardware?


+1 for the automation! This is key to me.

I’m not sure we need *dedicated* hardware for this: the release testing does not consume a lot of cycles continuously, does it? I expect we need some hardware for a limited period of time when preparing a release.

What’s missing in my opinion is rather the software infrastructure: an easy way to run a script and validate the compiler more extensively than the LLVM test-suite.
Maybe we should lead an effort towards an “extended test-suite”: having the ability to clone a bunch of selected open-source project and build & run their unitests. Even if it is something that takes a full day to run, it seems possible to do this on each RCs.

> 3. Automated merge requests.  I would like to come up with a way
> that we can tag commit messages so that a post-commit hook would
> be able to automatically create merge requests in bugzilla.
> There are a few ways I can think of to tag the messages, but I'm
> just going to propose that adding the following line:
> 
> STABLE: 4.0
> 
> To commit messages would make it a candidate for an automated
> merge request.  Any thoughts?

I’m not a big fan of hijacking commit messages for this.
What does this by? Just saving running the script?
I rather have this as part of the scripts that will be available after we migrate to git.
For instance you can already commit from git with `git llvm push` (see http://llvm.org/docs/GettingStarted.html#for-developers-to-work-with-a-git-monorepo ), we can imagine extending this in the future to be able to send merge request easily from the command line.

— 
Mehdi