[llvm] [Github][RFC] Add workflow to diff codegen on llvm-test-suite (PR #190010)

[Petr Hosek via llvm-commits]

================
@@ -0,0 +1,130 @@
+# When /test-suite is commented on a PR, checks out the PR, builds clang and
+# then the test-suite in several configurations. It then checks out the base of
+# the PR, builds clang and the test-suite again, and then uploads the diff of
+# the codegen.
+
+name: Diff test-suite codegen
+
+on:
+  issue_comment:
+    types:
+      - created
+
+jobs:
+  test-suite:
+    name: Build and diff
+    runs-on: ubuntu-24.04
+    permissions:
+      pull-requests: write
+    if: >-
+      !startswith(github.event.comment.body, '<!--IGNORE-->') &&
+      github.event.issue.pull_request && contains(github.event.comment.body, '/test-suite')
+    steps:
+      - name: Get pull request
+        id: get-pr
+        uses: actions/github-script at ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
+        with:
+          script: |
+            const { data: pr } = await github.rest.pulls.get({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              pull_number: context.payload.issue.number
+            })
+            if (!pr.mergeable)
+              await github.rest.issues.createComment({
+                owner: context.repo.owner,
+                repo: context.repo.repo,
+                body: "Can't diff PR, PR isn't mergeable"
+              })
+            return pr
+      - name: Check pull request is mergeable
+        if: ${{ !fromJSON(steps.get-pr.outputs.result).mergeable }}
+        run: exit 1
+      - name: Thumbs up comment
+        uses: actions/github-script at ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
+        with:
+          script: |
+            github.rest.reactions.createForIssueComment({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              comment_id: context.payload.comment.id,
+              content: '+1'
+            })
+      - name: Checkout pull request
+        uses: actions/checkout at 8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
+        with:
+          ref: ${{ fromJSON(steps.get-pr.outputs.result).merge_commit_sha }}
+          repository: ${{ fromJSON(steps.get-pr.outputs.result).head.repo.full_name }}
+          fetch-depth: 2
+      # Check out the test-suite scripts and CMake files from the default branch
+      # on the upstream repository. Use these instead of the scripts in the pull
+      # request's branch since it's untrusted.
+      - name: Checkout scripts and CMake files
+        uses: actions/checkout at 8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
+        with:
+          path: test-suite
+          sparse-checkout: .github/workflows/test-suite
+      - name: Checkout llvm/llvm-test-suite
+        uses: actions/checkout at 8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
+        with:
+          repository: llvm/llvm-test-suite
+          path: llvm-test-suite
+      - name: Setup environment variables
+        run: |
+          echo "HEAD_SHA=$(git rev-parse HEAD)" >> $GITHUB_ENV
+          echo "BASE_SHA=$(git rev-parse HEAD^)" >> $GITHUB_ENV
+          SCRIPTS_DIR="$GITHUB_WORKSPACE/test-suite/.github/workflows/test-suite"
+          echo "SCRIPTS_DIR=$SCRIPTS_DIR" >> $GITHUB_ENV
+          echo "$SCRIPTS_DIR" >> $GITHUB_PATH
+      - name: Install system dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y cmake ninja-build libc6-dev-{arm64,riscv64}-cross libgcc-14-dev-{arm64,riscv64}-cross libstdc++-14-dev-{arm64,riscv64}-cross
+      - name: Configure Clang
+        run: cmake -B build -DCMAKE_BUILD_TYPE=Release -DLLVM_TARGETS_TO_BUILD='AArch64;X86;RISCV' -DLLVM_ENABLE_PROJECTS='clang;lld' -DLLVM_APPEND_VC_REV=OFF llvm -GNinja
+      - name: Build Clang @ head
+        run: ninja -C build
+      - name: Configure and build test-suite @ head
+        run: |
+          configure-and-build.sh rva23u64-O3-b $SCRIPTS_DIR/riscv64.cmake
+          configure-and-build.sh armv9-a-O3-b $SCRIPTS_DIR/aarch64.cmake
+          configure-and-build.sh x86_64-O3-b $SCRIPTS_DIR/x86_64.cmake
+        working-directory: llvm-test-suite
+      - name: Build test-suite @ base
+        run: git checkout $BASE_SHA && ninja -C build
+      - name: Configure and build test-suite @ base
+        run: |
+          configure-and-build.sh rva23u64-O3-a $SCRIPTS_DIR/riscv64.cmake
+          configure-and-build.sh armv9-a-O3-a $SCRIPTS_DIR/aarch64.cmake
+          configure-and-build.sh x86_64-O3-a $SCRIPTS_DIR/x86_64.cmake
----------------
petrhosek wrote:

I'd use `base` as a suffix to make it clearer which one is which.
```suggestion
          configure-and-build.sh rva23u64-O3-base $SCRIPTS_DIR/riscv64.cmake
          configure-and-build.sh armv9-a-O3-base $SCRIPTS_DIR/aarch64.cmake
          configure-and-build.sh x86_64-O3-base $SCRIPTS_DIR/x86_64.cmake
```

https://github.com/llvm/llvm-project/pull/190010