[llvm] [github] pre-cache llvm-project in Linux CI Docker images (PR #133137)

[via llvm-commits]

llvmbot wrote:


<!--LLVM PR SUMMARY COMMENT-->

@llvm/pr-subscribers-github-workflow

Author: George Burgess IV (gburgessiv)

<details>
<summary>Changes</summary>

**Meta**: I'm not familiar with the actual deployment process here, but I'd _ideally_ like to be able to test this PR prior to submitting it. Not sure whether that'll be worth the effort compared to landing it (or splitting it, waiting for the new Docker image to appear, then messing with the `premerge.yaml` part of this) though?

Any guidance on how to test any part of this out-of-production is appreciated. :) 

=====

Linux CI bots take a bit over a minute to fully set their git repos up, always starting from scratch. This can end up being a significant percentage of their overall runtime.

If llvm-project is built into the images & used as a cache, we could potentially speed that up quite a bit. Try it out and see.

If this is successful, should be easy to port to Windows CI, too. I'm preferring to start small though.

---
Full diff: https://github.com/llvm/llvm-project/pull/133137.diff


2 Files Affected:

- (modified) .github/workflows/containers/github-action-ci/Dockerfile (+11) 
- (modified) .github/workflows/premerge.yaml (+6-1) 


``````````diff
diff --git a/.github/workflows/containers/github-action-ci/Dockerfile b/.github/workflows/containers/github-action-ci/Dockerfile
index bd3720017b7f7..8774c3f8b384f 100644
--- a/.github/workflows/containers/github-action-ci/Dockerfile
+++ b/.github/workflows/containers/github-action-ci/Dockerfile
@@ -103,3 +103,14 @@ RUN mkdir actions-runner && \
     tar xzf ./actions-runner-linux-x64-$GITHUB_RUNNER_VERSION.tar.gz && \
     rm ./actions-runner-linux-x64-$GITHUB_RUNNER_VERSION.tar.gz
 
+# Pre-cache llvm-project in these images; this works with the premerge action
+# to speed up runs; cloning LLVM fresh takes >1min otherwise.
+#
+# A local experiment showed an explicit `git gc --aggressive` reduced this
+# layer's size by 20% (~800MB). Disable GC'ing afterward, so we don't waste time
+# trying to do that in CI.
+RUN \
+  git clone https://github.com/llvm/llvm-project --tags && \
+  cd llvm-project && \
+  git gc --aggressive && \
+  git config gc.auto 0
diff --git a/.github/workflows/premerge.yaml b/.github/workflows/premerge.yaml
index c488421d37450..951c3273d3f4f 100644
--- a/.github/workflows/premerge.yaml
+++ b/.github/workflows/premerge.yaml
@@ -26,6 +26,11 @@ concurrency:
 jobs:
   premerge-checks-linux:
     name: Linux Premerge Checks (Test Only - Please Ignore Results)
+    defaults:
+      run:
+        # The premerge Linux docker instance keeps a pristine clone of LLVM in
+        # /home/gha/llvm-project; that should be used as our main workdir.
+        working-directory: /home/gha/llvm-project
     if: >-
         github.repository_owner == 'llvm' &&
         (github.event_name != 'pull_request' || github.event.action != 'closed')
@@ -34,7 +39,7 @@ jobs:
       - name: Checkout LLVM
         uses: actions/checkout at 11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
         with:
-          fetch-depth: 2
+          path: /home/gha/llvm-project
       - name: Setup ccache
         uses: hendrikmuhs/ccache-action at a1209f81afb8c005c13b4296c32e363431bffea5 # v1.2.17
         with:

``````````

</details>


https://github.com/llvm/llvm-project/pull/133137