[llvm-dev] RFC: New Automated Release Workflow (using Issues and Pull Requests)

[Tom Stellard via llvm-dev]

Hi,

Here is a proposal for a new automated workflow for managing parts of the release
process.  I've been experimenting with this over the past few releases and
now that we have migrated to GitHub issues, it would be possible for us to
implement this in the main repo.

The workflow is pretty straight forward, but it does use pull requests.  My
idea is to enable pull requests for only this automated workflow and not
for general development (i.e. We would still use Phabricator for code review).
Let me know what you think about this:


# Workflow

* On an existing issue or a newly created issue, a user who wants to backport
one or more commits to the release branch adds a comment:

/cherry-pick <commit_sha> <..>

* This starts a GitHub Action job that attempts to cherry-pick the commit(s)
to the current release branch.

* If the commit(s) can be cherry-picked cleanly, then the GitHub Action:
     * Pushes the result of the cherry-pick to a branch in the
       llvmbot/llvm-project repo called issue<n>, where n is the number of the
       GitHub Issue that launched the Action.

     * Adds this comment on the issue: /branch llvmbot/llvm-project/issue<n>

     * Creates a pull request from llvmbot/llvm-project/issue<n> to
       llvm/llvm-project/release/XX.x

     * Adds a comment on the issue: /pull-request #<n>
       where n is the number of the pull request.

* If the commit(s) can't be cherry-picked cleanly, then the GitHub Action job adds
the release:cherry-pick-failed label to the issue and adds a comment:
"Failed to cherry-pick <commit_sha> <..>" along with a link to the failing
Action.

* If a user has manually cherry-picked the fixes, resolved the conflicts, and
pushed the result to a branch on github, they can automatically create a pull
request by adding this comment to an issue: /branch <user>/<repo>/<branch>

* Once a pull request has been created, this launches more GitHub Actions
to run pre-commit tests.

* Once the tests complete successfully and the changes have been approved
by the release manager, the pull request can me merged into the release branch.

* After the pull request is merged, a GitHub Action automatically closes the
associated issue.

Some Examples:

Cherry-pick success: https://github.com/tstellar/llvm-project/issues/729
Cherry-pick failure: https://github.com/tstellar/llvm-project/issues/730
Manual Branch comment: https://github.com/tstellar/llvm-project/issues/710


# Motivation

Why do this?  The goal is to make the release process more efficient and transparent.
With this new workflow, users can get automatic and immediate feedback when a commit
they want backported doesn't apply cleanly or introduces some test failures.  With
the current process, these kinds of issues are communicated by the release manager,
and it can be days or even weeks before a problem is discovered and communicated back
to the users.

Another advantage of this workflow is it introduces pre-commit CI to the release branch,
which is important for the stability of the branch and the releases, but also gives
the project an opportunity to experiment with new CI workflows in a way that
does not disrupt development on the main branch.

# Implementation

If this proposal is accepted, I would plan to implement this for the LLVM 14 release cycle based
on the following proof of concept that I have been testing for the last few releases:

https://github.com/tstellar/llvm-project/blob/release-automation/.github/workflows/release-workflow.yml
https://github.com/tstellar/llvm-project/blob/release-automation/.github/workflows/release-workflow-create-pr.yml
https://github.com/tstellar/llvm-project/blob/release-automation/.github/workflows/release-merge-pr.yml

Thanks,
Tom