[flang-dev] [Classic flang] RFC: Github Actions for automatic CI post-push actions

Wed Oct 21 23:47:37 PDT 2020

Hello,

Richard Barton asked me to check if we could use github actions to get
automatic builds and tests for flang on push actions: PR creation or update
and merge to master.

We would like to request your comments on the following proposal of how
this could be implemented:

1) On every push action to master branch in flang-compiler/flang repository
Github action will:
  * Download the pre-built llvm and flang-driver artifacts for installation
and install them.
  * Build and install OpenMP (about 30 seconds)
  * Build libpgmath and flang project.
  * Run make check-all to test the build.

  The action takes about 7-8 minutes and gives a clear pass/fail result in
the Pull Request, including links to the build logs.

  Example PRs to demonstrate how this works:
  * Breaking the build: https://github.com/michalpasztamobica/flang/pull/1
  * Breaking the tests: https://github.com/michalpasztamobica/flang/pull/3
  * Harmless change resulting in a successful build:
https://github.com/michalpasztamobica/flang/pull/2

  The script itself:
https://github.com/michalpasztamobica/flang/blob/master/.github/workflows/build_flang.yml

2) On every push action to release_90 in flang-compiler/flang-driver
repository Github actions will:
  * Download the pre-built llvm artifacts for installation and install them.
  * Build flang-driver project according to official instructions.
  * Upload the resulting build directory as an artifact to be used in step
(1).

  The script is here:
https://github.com/michalpasztamobica/flang-driver/blob/release_90/.github/workflows/build_flang-driver.yml

3) On every push action to release_90 in flang-compiler/llvm repository
Github actions will:
  * Build llvm project according to official instructions.
  * Upload the resulting build directory as an artifact to be used in step
(1 and 2).

  The script is here:
https://github.com/michalpasztamobica/llvm/blob/release_90/.github/workflows/build_llvm.yml

Here's a few points giving the rationale behind the described solutions

1) We want to keep the whole process on github. Hence - we rely on
artifacts built and uploaded by github actions. Github holds artifacts for
90 days, so sometimes a manual trigger of some jobs may be necessary. I
haven't found a way to automate this so it works as if cron was setup to
trigger it every 90 days. We could consider doing binary releases instead,
especially that this topic has been discussed already during Wednesday
calls, but we suggest we start small and then expand a working solution.

2) We focus on flang repository commits - we tried to keep them as quick as
possible, hence the builds rely on other jobs to build the artifacts and we
don't run tests for llvm or flang-driver.

3) We build X86, Aarch64 and PowerPC for llvm and flang-driver. Github only
provides X86 machines with different operating systems, so for building and
testing flang itself we are bound to X86 only. We could set up "self-hosted
runners" to be triggerred with github actions if we had a machine available
somewhere.

Next steps:

Once we get the comments we will try to implement them and submit PRs for
merging the presented yml scripts to all three repositories.

Many thanks to Andrzej Warzynski for their insightful review remarks to the
scripts and to Richard Barton for driving this idea.

I am looking forward to hearing your comments! You can also reach me on the
flang-compiler Slack channel.

Best regards,

Michał Paszta
Senior Consultant / Senior Group Manager

Tel: +48 664 964 158
RingCentral: 2524
Address: Skierniewicka 10a, 01-230 Warszawa

-- 
Mobica is a global software services company, delivering and enabling 
technologies that transform business outcomes for the leading brands in 
Automotive, Silicon, FinTech, Media and Telecoms. Headquartered in Wilmslow 
UK with offices across Europe and the US, our established technical and 
delivery excellence in high quality software engineering drives success for 
our multinational customers on every continent, every day.
Find out more at 
Mobica.com <https://www.mobica.com>

Mobica Limited is a limited 
company registered in England and Wales with registered number 05169596 and 
VAT registered number 223837508. Our registered office is at Crown House, 
Manchester Road, Wilmslow, Cheshire, SK9 1BH, UK.
This message is intended 
solely for the addressee(s) and may contain confidential information.
If 
you have received this message in error, please send it back to us, and 
immediately and permanently delete it.
Do not use, copy or disclose the 
information contained in this message or in any attachment.
Mobica complies 
with all requirements of GDPR and other relevant data protection law. You 
can view our Privacy Policy at https://mobica.com/privacy-policy/ 
<https://mobica.com/privacy-policy/>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/flang-dev/attachments/20201022/2079edd7/attachment.html>