[cfe-dev] [Openmp-dev] RFC: Release process changes

[Hans Wennborg via cfe-dev]

On Thu, May 21, 2020 at 8:59 PM Tom Stellard via Openmp-dev
<openmp-dev at lists.llvm.org> wrote:
>
> Hi,
>
> I would like to propose a few changes to the LLVM release process.  The
> current process is documented here:  https://llvm.org/docs/HowToReleaseLLVM.html
>
> There are two parts to this proposal.  The first is a list of clarifications,
> which are things we are currently doing that aren't documented. The second
> is a list of changes which would actually modify how releases are currently
> managed.
>
>
>
> *** Proposed Clarifications ***
>
>
>
> **  Release manager is allowed to commit changes to the release branch without
>     code owner approval.  However, the release manager is encouraged to consult
>     with code owners or patch reviewers for non-trivial changes.
>
> It's not practical to get code owner approval every time.  Either because there
> is no code owner or because the number of backports is too high (e.g. pre-rc1 / pre-rc2).
> This proposed clarification matches how releases are currently managed.

+1

Maybe even stronger than "is allowed to commit", I think we should
really think about it as the release manager owning the branch, and
has full authority over what goes into it or not. Consulting code
owners often makes sense of course, but for many patches, consulting
the code owner (when there is one) is an unnecessary slowdown.

> ** There is no official release criteria.
>
> We have time-based releases and when the release is 'ready' has been
> up to the discretion of the release manager.  Changing the release
> criteria is out of the scope of this proposal, but I do think it would
> be good to have a discussion about this as a community, so I'm going to
> start a separate thread to discuss this.
>
>
>
> *** Proposed Changes ***
>
>
>
> ** Create a time-based bug-fix release schedule.  After each major release, make
>    a new bug-fix release every 2 weeks for 12 weeks (6 releases total).
>
> ** Eliminate release candidates for bug-fix releases.
>
> The current unofficial bug-fix release schedule is:
>
> X.Y.1-rc1 (6 weeks after major release)
> X.Y.1-rc2 (10 weeks after major release)
> X.Y.1-final (12 weeks after major release)
>
> I think this change will improve the overall test coverage of the release branch.
> I don't think the branch itself or even the release candidates get the same
> level of testing as the final releases.  If we are consistently snapshotting
> the release branch and putting out releases, I think this will make it easier
> and thus more likely that users will test out the release branch code.
>
> Additionally, with more frequent bug-fix release it removes the need to have
> release candidate releases. Every bug-fix release (up until the last one)
> would serve the same purpose as our current release candidates in that they
> are intended to give users an easier way to test the code before the final
> release.

My first thought is that doing all these releases sounds like a lot of
work. Would you be doing all of them, or would there be some other
arrangement? I suppose if we release this often, and also skip the
RCs, we might become more efficient at it :-)

Secondly, is having this many releases useful for downstream? One
concern might be that downstream consumers just wait for the .6 one,
and then there's no benefit and also no extra testing of the branch.
Is it mainly increasing test coverage of the branch that's the
motivation, or is it the demand for more bug-fix releases?

Not having at least one release candidate sounds a bit scary to be.
Without them we could get into a situation where everything works fine
on the release manager's machines, but is completely broken on other
platforms, and no way to fix until the next dot release. Maybe that
kind of breakage is less likely after the major release, but it still
seems it could make these dot releases less stable?

> ** Create clear rules for what kind of backports are accepted during each
>    release phase.
>
> * Before RC1:Patches should be limited to bug fixes, important optimization
>   improvements, or completion of features that were started before the branch
>   was created.  As with all phases, release managers and code owners can reject
>   patches that are deemed too invasive.
>
> * Before RC2: Patches should be limited to bug fixes or backend specific
>   improvements that are determined to be very safe.
>
> * Before RC3/Final: Major Release* Patches should be limited to critical
>   bugs or regressions.
>
> * Bug fix releases: Patches should be limited to bug fixes or very safe
>   and critical performance improvements.  Patches must maintain both API and
>   ABI compatibility with the previous major release.
>
> * Final bug fix release: Patches should be limited to critical bug fixes only.

These sound good to me.


Thanks,
Hans