[llvm] [Flang] Github action to check for integration tests (PR #221766)
Kiran Chandramohan via llvm-commits
llvm-commits at lists.llvm.org
Tue Sep 22 12:46:27 PDT 2026
https://github.com/kiranchandramohan updated https://github.com/llvm/llvm-project/pull/221766
>From d3c8c390758f3c7dc97ea56cf9d8354ac8bd3a31 Mon Sep 17 00:00:00 2001
From: Kiran Chandramohan <kiran.chandramohan at arm.com>
Date: Fri, 21 Aug 2026 22:23:56 +0200
Subject: [PATCH 1/5] [Flang] Github action to check for integration tests
---
.../flang-integration-test-check.yml | 104 ++++++++++++++++++
1 file changed, 104 insertions(+)
create mode 100644 .github/workflows/flang-integration-test-check.yml
diff --git a/.github/workflows/flang-integration-test-check.yml b/.github/workflows/flang-integration-test-check.yml
new file mode 100644
index 0000000000000..452b650db0beb
--- /dev/null
+++ b/.github/workflows/flang-integration-test-check.yml
@@ -0,0 +1,104 @@
+name: Flang integration test check
+
+on:
+ pull_request_target:
+ types: [opened, synchronize, reopened]
+ paths:
+ - "flang/test/**"
+
+# This workflow only inspects file patches through the GitHub API. In
+# particular, it must not check out or execute code from the pull request.
+permissions:
+ contents: read
+ pull-requests: write
+
+jobs:
+ check-for-full-pipeline-tests:
+ runs-on: ubuntu-24.04
+ steps:
+ - name: Check for new full-pipeline test invocations
+ uses: actions/github-script at 3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+ with:
+ script: |
+ const marker = "<!-- flang-integration-test-check -->";
+ const pr = context.payload.pull_request;
+
+ const files = await github.paginate(
+ github.rest.pulls.listFiles,
+ {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ pull_number: pr.number,
+ per_page: 100,
+ },
+ );
+
+ const findings = [];
+ for (const file of files) {
+ if (!file.filename.startsWith("flang/test/") || !file.patch)
+ continue;
+
+ // Only report newly added uses. Existing full-pipeline tests do
+ // not need to be justified again when an unrelated line changes.
+ const addsEmitLLVM = file.patch.split("\n").some(
+ line => /^\+(?!\+\+)/.test(line) && /\bRUN:/.test(line) &&
+ /(^|\s)-emit-llvm(?=\s|=|$)/.test(line),
+ );
+ if (addsEmitLLVM)
+ findings.push(`- \`${file.filename}\``);
+ }
+
+ const comments = await github.paginate(
+ github.rest.issues.listComments,
+ {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ issue_number: pr.number,
+ per_page: 100,
+ },
+ );
+ const previous = comments.find(
+ comment => comment.user.type === "Bot" && comment.body.includes(marker),
+ );
+
+ if (findings.length === 0) {
+ if (previous) {
+ await github.rest.issues.deleteComment({
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ comment_id: previous.id,
+ });
+ }
+ return;
+ }
+
+ const body = [
+ marker,
+ "This pull request adds a Flang test invocation that exercises the compiler " +
+ "pipeline through LLVM IR generation:",
+ "",
+ findings.join("\n"),
+ "",
+ "Please consider whether the same behavior can be covered by a focused test " +
+ "that stops at the relevant compiler stage. If LLVM IR generation is " +
+ "necessary, add a nearby, test-specific comment explaining why. See the " +
+ "[Flang integration test guidance](https://github.com/llvm/llvm-project/blob/main/flang/test/Integration/README.md).",
+ "",
+ "_This is an informational reminder; it does not fail the workflow._",
+ ].join("\n");
+
+ if (previous) {
+ await github.rest.issues.updateComment({
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ comment_id: previous.id,
+ body,
+ });
+ } else {
+ await github.rest.issues.createComment({
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ issue_number: pr.number,
+ body,
+ });
+ }
>From 11e14ee13fdebc29bd0a4f8f4c1d9320f60b91bb Mon Sep 17 00:00:00 2001
From: Kiran Chandramohan <kiran.chandramohan at arm.com>
Date: Tue, 22 Sep 2026 17:46:38 +0200
Subject: [PATCH 2/5] Address security warning and limit the action to the
llvm-project repo
---
.github/workflows/flang-integration-test-check.yml | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/.github/workflows/flang-integration-test-check.yml b/.github/workflows/flang-integration-test-check.yml
index 452b650db0beb..b03af46ee2caa 100644
--- a/.github/workflows/flang-integration-test-check.yml
+++ b/.github/workflows/flang-integration-test-check.yml
@@ -1,19 +1,23 @@
name: Flang integration test check
on:
- pull_request_target:
+ # This workflow needs pull_request_target so it can comment on pull requests
+ # from forks. It only examines patches using the GitHub API and never checks
+ # out or executes code from the pull request branch.
+ pull_request_target: # zizmor: ignore[dangerous-triggers]
types: [opened, synchronize, reopened]
paths:
- "flang/test/**"
-# This workflow only inspects file patches through the GitHub API. In
-# particular, it must not check out or execute code from the pull request.
permissions:
contents: read
- pull-requests: write
jobs:
check-for-full-pipeline-tests:
+ if: github.repository == 'llvm/llvm-project'
+ permissions:
+ contents: read
+ pull-requests: write
runs-on: ubuntu-24.04
steps:
- name: Check for new full-pipeline test invocations
>From 9d75cb2270bd72ad8e640f837e063812c19d179b Mon Sep 17 00:00:00 2001
From: Kiran Chandramohan <kiran.chandramohan at arm.com>
Date: Tue, 22 Sep 2026 19:25:46 +0200
Subject: [PATCH 3/5] [Flang] Reuse the shared PR comment workflow
Run the integration test check with read-only pull_request permissions
and upload comments for issue-write.yml. Update an existing reminder
when its findings disappear, and remove the pull_request_target suppression.
---
.../flang-integration-test-check.yml | 54 +++++++++----------
.github/workflows/issue-write.yml | 1 +
2 files changed, 25 insertions(+), 30 deletions(-)
diff --git a/.github/workflows/flang-integration-test-check.yml b/.github/workflows/flang-integration-test-check.yml
index b03af46ee2caa..bda5b2eaaa8e5 100644
--- a/.github/workflows/flang-integration-test-check.yml
+++ b/.github/workflows/flang-integration-test-check.yml
@@ -1,29 +1,26 @@
name: Flang integration test check
on:
- # This workflow needs pull_request_target so it can comment on pull requests
- # from forks. It only examines patches using the GitHub API and never checks
- # out or executes code from the pull request branch.
- pull_request_target: # zizmor: ignore[dangerous-triggers]
+ pull_request:
types: [opened, synchronize, reopened]
paths:
- "flang/test/**"
+ - ".github/workflows/flang-integration-test-check.yml"
permissions:
contents: read
+ pull-requests: read
jobs:
check-for-full-pipeline-tests:
if: github.repository == 'llvm/llvm-project'
- permissions:
- contents: read
- pull-requests: write
runs-on: ubuntu-24.04
steps:
- name: Check for new full-pipeline test invocations
uses: actions/github-script at 3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
+ const fs = require("fs");
const marker = "<!-- flang-integration-test-check -->";
const pr = context.payload.pull_request;
@@ -62,17 +59,19 @@ jobs:
},
);
const previous = comments.find(
- comment => comment.user.type === "Bot" && comment.body.includes(marker),
+ comment => comment.user.login === "github-actions[bot]" &&
+ comment.body.includes(marker),
);
+ // issue-write.yml creates or updates comments from this artifact.
+ // Resolve an existing reminder when its findings disappear.
if (findings.length === 0) {
- if (previous) {
- await github.rest.issues.deleteComment({
- owner: context.repo.owner,
- repo: context.repo.repo,
- comment_id: previous.id,
- });
- }
+ const resolved = previous ? [{
+ id: previous.id,
+ body: marker + "\n\nNo newly added `-emit-llvm` RUN lines remain " +
+ "under `flang/test/`.",
+ }] : [];
+ fs.writeFileSync("comments", JSON.stringify(resolved));
return;
}
@@ -91,18 +90,13 @@ jobs:
"_This is an informational reminder; it does not fail the workflow._",
].join("\n");
- if (previous) {
- await github.rest.issues.updateComment({
- owner: context.repo.owner,
- repo: context.repo.repo,
- comment_id: previous.id,
- body,
- });
- } else {
- await github.rest.issues.createComment({
- owner: context.repo.owner,
- repo: context.repo.repo,
- issue_number: pr.number,
- body,
- });
- }
+ const comment = { body };
+ if (previous)
+ comment.id = previous.id;
+ fs.writeFileSync("comments", JSON.stringify([comment]));
+
+ - name: Upload comment for issue-write.yml
+ uses: actions/upload-artifact at 043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+ with:
+ name: workflow-args
+ path: comments
diff --git a/.github/workflows/issue-write.yml b/.github/workflows/issue-write.yml
index 289ec207386e6..c7e1f2e4f13f1 100644
--- a/.github/workflows/issue-write.yml
+++ b/.github/workflows/issue-write.yml
@@ -11,6 +11,7 @@ on:
- "Test Issue Write"
- "Check LLVM ABI annotations"
- "Diff test-suite codegen"
+ - "Flang integration test check"
types:
- completed
>From a9d9a134f761dd0c83c80a915a5d0eb5a5b87241 Mon Sep 17 00:00:00 2001
From: Kiran Chandramohan <kiran.chandramohan at arm.com>
Date: Tue, 22 Sep 2026 20:09:15 +0200
Subject: [PATCH 4/5] [Flang] Run the integration test reminder for all Flang
changes
This is to avoid the situation where a subsequent change removes
the integration test from flang/test but the workflow would not
run because there are no changes in flang/test.
---
.github/workflows/flang-integration-test-check.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.github/workflows/flang-integration-test-check.yml b/.github/workflows/flang-integration-test-check.yml
index bda5b2eaaa8e5..6ca84fbf4d12a 100644
--- a/.github/workflows/flang-integration-test-check.yml
+++ b/.github/workflows/flang-integration-test-check.yml
@@ -4,7 +4,7 @@ on:
pull_request:
types: [opened, synchronize, reopened]
paths:
- - "flang/test/**"
+ - "flang/**"
- ".github/workflows/flang-integration-test-check.yml"
permissions:
>From 9fc932da920ee25ce6faf42830dc545981c2fe37 Mon Sep 17 00:00:00 2001
From: Kiran Chandramohan <kiran.chandramohan at arm.com>
Date: Tue, 22 Sep 2026 21:44:55 +0200
Subject: [PATCH 5/5] Remove registration of Flang integration test comment
workflow
This is done separately in #225500
---
.github/workflows/issue-write.yml | 1 -
1 file changed, 1 deletion(-)
diff --git a/.github/workflows/issue-write.yml b/.github/workflows/issue-write.yml
index c7e1f2e4f13f1..289ec207386e6 100644
--- a/.github/workflows/issue-write.yml
+++ b/.github/workflows/issue-write.yml
@@ -11,7 +11,6 @@ on:
- "Test Issue Write"
- "Check LLVM ABI annotations"
- "Diff test-suite codegen"
- - "Flang integration test check"
types:
- completed
More information about the llvm-commits
mailing list