[libcxx-commits] [libcxx] [llvm] Try adding Github Problem Matchers to libc++ workflow. (PR #146768)

[via libcxx-commits]

https://github.com/EricWF updated https://github.com/llvm/llvm-project/pull/146768

>From 92ccb58088a315e8363750752d5a92bcfe08acb5 Mon Sep 17 00:00:00 2001
From: Eric Fiselier <eric at efcs.ca>
Date: Wed, 2 Jul 2025 16:08:37 -0400
Subject: [PATCH 1/2] Try adding Github Problem Matchers to libc++ workflow.

The problem matchers are simple regex's that will be applied
to the output of particular steps, and when a match is found,
an annotation is applied to the relevent github workflow.

The idea is to more easily surface relevent failure information,
which may otherwise be hidden among cancelled tasks.

The Problem Matcher specification/format is rather limited in its
descriptive abliity, meaning we lose most context about the error.
Further, because many of the source files are re-arranged before use,
it prevents us from actually pointing to the file:line in the github UI;
Boo :-(.

A more complete solution would likely hook into LIT, remap source file
paths as needed, and use the "echo ::error::" toolkit command to create
the annotations.
---
 .github/libcxx-problem-matchers.json         | 27 +++++++++++++++++++
 .github/workflows/libcxx-build-and-test.yaml | 28 +++++++++++++++++---
 2 files changed, 52 insertions(+), 3 deletions(-)
 create mode 100644 .github/libcxx-problem-matchers.json

diff --git a/.github/libcxx-problem-matchers.json b/.github/libcxx-problem-matchers.json
new file mode 100644
index 0000000000000..d681b83f069a1
--- /dev/null
+++ b/.github/libcxx-problem-matchers.json
@@ -0,0 +1,27 @@
+{
+    "problemMatcher": [
+        {
+            "owner": "clang",
+            "pattern": [
+                {
+                    "regexp": "^(.+):(\\d+):(\\d+):\\s+(error|warning):\\s+(.+)$",
+                    "file": 1,
+                    "line": 2,
+                    "column": 3,
+                    "severity": 4,
+                    "message": 5
+                }
+            ]
+        },
+        {
+              "owner": "lit",
+              "severity": "error",
+              "pattern": [
+                  {
+                      "regexp": "^\\s*(FAIL|XPASS):\\s+([^\\s]+)\\s+::\\s+(.+?)\\s+\\(\\d+\\s+of\\s+\\d+\\)$",
+                      "message": 2
+                  }
+              ]
+          }
+    ]
+}
diff --git a/.github/workflows/libcxx-build-and-test.yaml b/.github/workflows/libcxx-build-and-test.yaml
index f0bdf6c0b5899..0145fe27d7563 100644
--- a/.github/workflows/libcxx-build-and-test.yaml
+++ b/.github/workflows/libcxx-build-and-test.yaml
@@ -57,10 +57,17 @@ jobs:
     steps:
       - uses: actions/checkout at 11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
       - name: ${{ matrix.config }}.${{ matrix.cxx }}
-        run: libcxx/utils/ci/run-buildbot ${{ matrix.config }}
+        run: |
+          echo "::add-matcher::.github/libcxx-problem-matchers.json"
+          libcxx/utils/ci/run-buildbot ${{ matrix.config }}
         env:
           CC: ${{ matrix.cc }}
           CXX: ${{ matrix.cxx }}
+      - if: always()
+        name: Disable  Problem Matchers
+        run: |
+          echo "::remove-matcher owner=lit::"
+          echo "::remove-matcher owner=clang::"
       - uses: actions/upload-artifact at 26f96dfa697d77e81fd5907df203aa23a56210a8 # v4.3.0
         if: always()
         with:
@@ -103,10 +110,17 @@ jobs:
     steps:
       - uses: actions/checkout at 11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
       - name: ${{ matrix.config }}
-        run: libcxx/utils/ci/run-buildbot ${{ matrix.config }}
+        run: |
+          echo "::add-matcher::.github/libcxx-problem-matchers.json"
+          libcxx/utils/ci/run-buildbot ${{ matrix.config }}
         env:
           CC: ${{ matrix.cc }}
           CXX: ${{ matrix.cxx }}
+      - if: always()
+        name: Disable Problem Matchers
+        run: |
+          echo "::remove-matcher owner=lit::"
+          echo "::remove-matcher owner=clang::"
       - uses: actions/upload-artifact at 26f96dfa697d77e81fd5907df203aa23a56210a8 # v4.3.0
         if: always()  # Upload artifacts even if the build or test suite fails
         with:
@@ -167,10 +181,17 @@ jobs:
     steps:
       - uses: actions/checkout at 11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
       - name: ${{ matrix.config }}
-        run: libcxx/utils/ci/run-buildbot ${{ matrix.config }}
+        run: |
+          echo "::add-matcher::.github/libcxx-problem-matchers.json"
+          libcxx/utils/ci/run-buildbot ${{ matrix.config }}
         env:
           CC: clang-21
           CXX: clang++-21
+      - if: always()
+        name: Disable Problem Matchers
+        run: |
+          echo "::remove-matcher owner=lit::"
+          echo "::remove-matcher owner=clang::"
       - uses: actions/upload-artifact at 26f96dfa697d77e81fd5907df203aa23a56210a8 # v4.3.0
         if: always()
         with:
@@ -221,6 +242,7 @@ jobs:
       - uses: seanmiddleditch/gha-setup-ninja at 3b1f8f94a2f8254bd26914c4ab9474d4f0015f67 # v6
       - name: Build and test
         run: |
+          echo "::add-matcher::.github/libcxx-problem-matchers.json"
           python3 -m venv .venv
           source .venv/bin/activate
           python -m pip install psutil

>From 180f46c7234dd2097dacc188a95e017db69c3eb1 Mon Sep 17 00:00:00 2001
From: Eric Fiselier <eric at efcs.ca>
Date: Wed, 2 Jul 2025 16:15:30 -0400
Subject: [PATCH 2/2] Temporarily add a test failure to demonstrate the
 matchers in action.

---
 libcxx/test/broken-on-purpose-1.pass.cpp | 4 ++++
 1 file changed, 4 insertions(+)
 create mode 100644 libcxx/test/broken-on-purpose-1.pass.cpp

diff --git a/libcxx/test/broken-on-purpose-1.pass.cpp b/libcxx/test/broken-on-purpose-1.pass.cpp
new file mode 100644
index 0000000000000..ff8100c806520
--- /dev/null
+++ b/libcxx/test/broken-on-purpose-1.pass.cpp
@@ -0,0 +1,4 @@
+
+DoesNotExist bar() {
+
+}