[Release-testers] [Openmp-dev] RFC: Release process changes
Hans Wennborg via Release-testers
release-testers at lists.llvm.org
Wed May 27 02:11:59 PDT 2020
That all makes sense to me. The new process sgtm.
On Wed, May 27, 2020 at 5:20 AM Tom Stellard <tstellar at redhat.com> wrote:
> On 05/25/2020 05:48 AM, Hans Wennborg wrote:
> > On Thu, May 21, 2020 at 8:59 PM Tom Stellard via Openmp-dev
> > <openmp-dev at lists.llvm.org> wrote:
> >> Hi,
> >> I would like to propose a few changes to the LLVM release process. The
> >> current process is documented here: https://llvm.org/docs/HowToReleaseLLVM.html
> >> There are two parts to this proposal. The first is a list of clarifications,
> >> which are things we are currently doing that aren't documented. The second
> >> is a list of changes which would actually modify how releases are currently
> >> managed.
> >> *** Proposed Clarifications ***
> >> ** Release manager is allowed to commit changes to the release branch without
> >> code owner approval. However, the release manager is encouraged to consult
> >> with code owners or patch reviewers for non-trivial changes.
> >> It's not practical to get code owner approval every time. Either because there
> >> is no code owner or because the number of backports is too high (e.g. pre-rc1 / pre-rc2).
> >> This proposed clarification matches how releases are currently managed.
> > +1
> > Maybe even stronger than "is allowed to commit", I think we should
> > really think about it as the release manager owning the branch, and
> > has full authority over what goes into it or not. Consulting code
> > owners often makes sense of course, but for many patches, consulting
> > the code owner (when there is one) is an unnecessary slowdown.
> >> ** There is no official release criteria.
> >> We have time-based releases and when the release is 'ready' has been
> >> up to the discretion of the release manager. Changing the release
> >> criteria is out of the scope of this proposal, but I do think it would
> >> be good to have a discussion about this as a community, so I'm going to
> >> start a separate thread to discuss this.
> >> *** Proposed Changes ***
> >> ** Create a time-based bug-fix release schedule. After each major release, make
> >> a new bug-fix release every 2 weeks for 12 weeks (6 releases total).
> >> ** Eliminate release candidates for bug-fix releases.
> >> The current unofficial bug-fix release schedule is:
> >> X.Y.1-rc1 (6 weeks after major release)
> >> X.Y.1-rc2 (10 weeks after major release)
> >> X.Y.1-final (12 weeks after major release)
> >> I think this change will improve the overall test coverage of the release branch.
> >> I don't think the branch itself or even the release candidates get the same
> >> level of testing as the final releases. If we are consistently snapshotting
> >> the release branch and putting out releases, I think this will make it easier
> >> and thus more likely that users will test out the release branch code.
> >> Additionally, with more frequent bug-fix release it removes the need to have
> >> release candidate releases. Every bug-fix release (up until the last one)
> >> would serve the same purpose as our current release candidates in that they
> >> are intended to give users an easier way to test the code before the final
> >> release.
> > My first thought is that doing all these releases sounds like a lot of
> > work. Would you be doing all of them, or would there be some other
> > arrangement? I suppose if we release this often, and also skip the
> > RCs, we might become more efficient at it :-)
> Yes, I would plan to do all the releases. For 9.0.1, there were
> 3 RCs, so 4 releases in total. Doing 6 instead of 4 is not that much
> more work in my opinion. Also, we may end up skipping releases if
> there aren't any new changes in the branch. But doing extra
> releases would be good motivation to try to automates more parts of the
> release process.
> If we do feel like 6 is too many we could lengthen the interval to 3 weeks,
> which would give us just 4 releases.
> > Secondly, is having this many releases useful for downstream? One
> > concern might be that downstream consumers just wait for the .6 one,
> > and then there's no benefit and also no extra testing of the branch.
> > Is it mainly increasing test coverage of the branch that's the
> > motivation, or is it the demand for more bug-fix releases?
> From me as a distro package maintainer, I'm more likely to package
> a final release than a bug-fix release. Especially if I know there won't
> be another release candidate or final release coming very soon after.
> Besides increasing testing coverage, I think it helps other projects
> avoid having to do things like this: https://lkml.org/lkml/2020/5/5/1446
> In this case, the kernel release cycle is about 2 months, so they can't
> wait 3 months for a fix.
> > Not having at least one release candidate sounds a bit scary to be.
> > Without them we could get into a situation where everything works fine
> > on the release manager's machines, but is completely broken on other
> > platforms, and no way to fix until the next dot release. Maybe that
> > kind of breakage is less likely after the major release, but it still
> > seems it could make these dot releases less stable?
> I think in order to do this we need to have a certain level of CI testing.
> I've been experimenting with this in the last few release cycles. Right now,
> in the release branch we have tests running make-check for clang, llvm, lld,
> libclc, and running make for lldb. Each test runs on Linux, Mac, and Windows.
> I feel comfortable enough with this that it would catch most obvious issues
> with new bug-fix releases. Especially since the number of changes is low
> as compared to major releases.
> I think there would need to be some flexibility to do another bug-fix release
> ASAP if there is a major problem.
> >> ** Create clear rules for what kind of backports are accepted during each
> >> release phase.
> >> * Before RC1:Patches should be limited to bug fixes, important optimization
> >> improvements, or completion of features that were started before the branch
> >> was created. As with all phases, release managers and code owners can reject
> >> patches that are deemed too invasive.
> >> * Before RC2: Patches should be limited to bug fixes or backend specific
> >> improvements that are determined to be very safe.
> >> * Before RC3/Final: Major Release* Patches should be limited to critical
> >> bugs or regressions.
> >> * Bug fix releases: Patches should be limited to bug fixes or very safe
> >> and critical performance improvements. Patches must maintain both API and
> >> ABI compatibility with the previous major release.
> >> * Final bug fix release: Patches should be limited to critical bug fixes only.
> > These sound good to me.
> > Thanks,
> > Hans
More information about the Release-testers