[cfe-dev] RFC: Using GitHub Actions for CI testing on the release/* branches

[Tom Stellard via cfe-dev]

On 11/11/2019 04:32 PM, Tom Stellard wrote:
> Hi,
> 
> I would like to start using GitHub Actions[1] for CI testing on the release/*
> branches.  As far as I know we don't have any buildbots listening to the
> release branches, and I think GitHub Actions are a good way for us to quickly
> bring-up some CI jobs there.
> 
> My proposal is to start by adding two post-commit CI jobs to the release/9.x branch.
> One for building and testing (ninja checka-all) llvm/clang/lld on Linux,
> Windows, and Mac, and another for detecting ABI changes since the 9.0.0 release.
> 
> I have already implemented these two CI jobs in my llvm-project fork on GitHub[2][3],
> but in order to get these running in the main repository, I would need to:
> 
> 1. Create a new repository in the LLVM organization called 'actions' for storing some custom
> builds steps for our CI jobs (see [4]).
> 2. Commit yaml CI definitions to the .github/workflows directory in the release/9.x
> branch.
> 
> In the future, I would also like to add buil and tests jobs for other sub-projects
> once I am able to get those working.
> 
> In addition to being used for post-commit testing, having these CI definitions in the
> main tree will make it easier for me (or anyone) to do pre-commit testing for the
> release branch in a personal fork.  It will also allow me to experiment with some new
> workflows to help make managing the releases much easier.
> 
> I think this will be a good way to test Actions in a low traffic environment to
> see if they are something we would want to use for CI on the master branch.
> 
> Given that we are close to the end of the 9.0.1 cycle, unless there are any
> strong objections, I would like to get this enabled by Mon Nov 18, to maximize its
> usefulness.  Let me know what you think.
> 

The CI tests are running now on the stable branch.  Here is the
first run [1].  Passing commits[2] now have a green check mark in
the web UI that you can click on to view the results.

These tests have already caught 1 bug that did not appear in my local
builds, so they are proving to be very useful.

Now that this is in place, I'm going to start looking into some more automation
around backporting fixes to the stable branch.  I think we could do something like
filing backport requests as a GitHub issues and then have that issue automatically
trigger a pull request and do some pre-commit testing.  I'm still trying to learn
what is technically possible with GitHub and if anyone has any ideas or suggestions,
let me know.  I'm hoping to have something in place for the 10.0.1 release cycle.

Thanks,
Tom

[1] https://github.com/llvm/llvm-project/commit/6c86aa62d502ac2df58e207d4792544cc96d79bd/checks?check_suite_id=316648311
[2] https://github.com/llvm/llvm-project/commit/6c86aa62d502ac2df58e207d4792544cc96d79bd

> Thanks,
> Tom
> 
> [1] https://github.com/features/actions
> [2] https://github.com/tstellar/llvm-project/commit/952d80e8509ecc95797b2ddbf1af40abad2dcf4e/checks?check_suite_id=305765621
> [3] https://github.com/tstellar/llvm-project/commit/6d74f1b81632ef081dffa1e0c0434f47d4954423/checks?check_suite_id=303074176
> [4] https://github.com/tstellar/actions
>