[llvm] [libc++][CI] TEMP: investigate macOS LNT install failure (DO NOT MERGE) (PR #211783)

Louis Dionne via llvm-commits llvm-commits at lists.llvm.org
Fri Jul 24 05:47:21 PDT 2026


https://github.com/ldionne updated https://github.com/llvm/llvm-project/pull/211783

>From ffc1b3d441875041fc74cd06952bcb6fe159288b Mon Sep 17 00:00:00 2001
From: Louis Dionne <ldionne.2 at gmail.com>
Date: Fri, 24 Jul 2026 08:37:32 -0400
Subject: [PATCH 1/2] [libc++][CI] TEMP: investigate macOS LNT install failure

Reproduce and diagnose the 'pip install llvm-lnt' failure on the macOS
self-hosted benchmark runners. DO NOT MERGE.
---
 .github/workflows/libcxx-investigate-lnt.yml | 80 ++++++++++++++++++++
 1 file changed, 80 insertions(+)
 create mode 100644 .github/workflows/libcxx-investigate-lnt.yml

diff --git a/.github/workflows/libcxx-investigate-lnt.yml b/.github/workflows/libcxx-investigate-lnt.yml
new file mode 100644
index 0000000000000..e95224a49f18d
--- /dev/null
+++ b/.github/workflows/libcxx-investigate-lnt.yml
@@ -0,0 +1,80 @@
+# TEMPORARY — DO NOT MERGE.
+#
+# This workflow exists only to reproduce and investigate the LNT installation
+# failure seen on the macOS self-hosted benchmark runners:
+#
+#   https://github.com/llvm/llvm-project/actions/runs/30092892111/job/89479960018
+#
+# The benchmark job installs `python at 3.14` via Homebrew and prepends
+# `$(brew --prefix python at 3.14)/libexec/bin` to the PATH. That directory only
+# provides *unversioned* `python`/`pip` symlinks -- it has no `python3`. So a
+# `python3` invocation (used by run-benchmarks to create the venv) still
+# resolves to Apple's `/usr/bin/python3` (3.9.6), and `pip install llvm-lnt`
+# then fails because every llvm-lnt release requires Python >= 3.10.
+#
+# This job mirrors the benchmark setup, dumps the relevant diagnostics, then
+# reproduces the failure and validates a candidate fix on the same runners.
+
+name: Investigate LNT install failure (macOS)
+
+permissions:
+  contents: read
+
+on:
+  workflow_dispatch:
+  pull_request:
+    paths:
+      - '.github/workflows/libcxx-investigate-lnt.yml'
+
+jobs:
+  investigate:
+    runs-on: ["self-hosted", "macOS", "ARM64", "26", "26.5"]
+    env:
+      DEVELOPER_DIR: /Applications/Xcode_26.5.app/Contents/Developer
+    steps:
+      - name: Install python at 3.14 via Homebrew (same as the benchmark job)
+        run: |
+          brew update
+          brew install python at 3.14
+          echo "$(brew --prefix python at 3.14)/libexec/bin" >> "$GITHUB_PATH"
+
+      - name: Diagnose which python3 / pip3 are actually on PATH
+        run: |
+          echo "PATH=${PATH}"
+          echo "--- which -a python3 python pip3 pip ---"
+          which -a python3 python pip3 pip || true
+          echo "--- versions ---"
+          echo "python3 -> $(command -v python3): $(python3 --version 2>&1)"
+          echo "python  -> $(command -v python)  : $(python  --version 2>&1)"
+          prefix="$(brew --prefix python at 3.14)"
+          echo "--- brew --prefix python at 3.14 = ${prefix} ---"
+          echo "libexec/bin (this is what we add to PATH -- note: no python3):"
+          ls -l "${prefix}/libexec/bin"
+          echo "bin (this one DOES contain python3/pip3):"
+          ls -l "${prefix}/bin"
+
+      - name: Reproduce the failure -- venv via 'python3' + pip install llvm-lnt
+        run: |
+          set -x
+          venv="$(mktemp -d)/venv"
+          python3 -m venv "${venv}"
+          "${venv}/bin/python" --version
+          "${venv}/bin/pip" --version
+          # This is the exact command that fails in the benchmark job. We expect
+          # it to fail here too; capture the status instead of aborting the job
+          # so the candidate-fix step below still runs.
+          if "${venv}/bin/pip" install llvm-lnt; then
+            echo "::warning::UNEXPECTED: 'pip install llvm-lnt' succeeded with python3"
+          else
+            echo "::notice::REPRODUCED: 'pip install llvm-lnt' failed with python3 (Apple 3.9.6), as in CI"
+          fi
+
+      - name: Candidate fix -- venv via 'python3.14' + pip install llvm-lnt
+        run: |
+          set -x
+          venv="$(mktemp -d)/venv"
+          python3.14 -m venv "${venv}"
+          "${venv}/bin/python" --version
+          "${venv}/bin/pip" install llvm-lnt
+          "${venv}/bin/lnt" --help | head -n 5
+          echo "::notice::FIX CONFIRMED: 'pip install llvm-lnt' works when the venv uses python3.14"

>From 6a0217eaa3dd96aaf4d9d7941569ba816227e368 Mon Sep 17 00:00:00 2001
From: Louis Dionne <ldionne.2 at gmail.com>
Date: Fri, 24 Jul 2026 08:47:01 -0400
Subject: [PATCH 2/2] [libc++][CI] Mirror the real benchmark flow end-to-end

Run the actual run-benchmarks script with the hash.bench.cpp filter, using
the PATH fix under test (brew python at 3.14 'bin' dir, which has a real
python3). DO NOT MERGE.
---
 .github/workflows/libcxx-investigate-lnt.yml | 94 ++++++++++----------
 1 file changed, 46 insertions(+), 48 deletions(-)

diff --git a/.github/workflows/libcxx-investigate-lnt.yml b/.github/workflows/libcxx-investigate-lnt.yml
index e95224a49f18d..883dea347fbae 100644
--- a/.github/workflows/libcxx-investigate-lnt.yml
+++ b/.github/workflows/libcxx-investigate-lnt.yml
@@ -1,19 +1,20 @@
 # TEMPORARY — DO NOT MERGE.
 #
-# This workflow exists only to reproduce and investigate the LNT installation
-# failure seen on the macOS self-hosted benchmark runners:
+# This workflow reproduces the macOS benchmark job end-to-end in order to
+# investigate (and validate the fix for) the LNT installation failure seen at:
 #
 #   https://github.com/llvm/llvm-project/actions/runs/30092892111/job/89479960018
 #
-# The benchmark job installs `python at 3.14` via Homebrew and prepends
-# `$(brew --prefix python at 3.14)/libexec/bin` to the PATH. That directory only
-# provides *unversioned* `python`/`pip` symlinks -- it has no `python3`. So a
-# `python3` invocation (used by run-benchmarks to create the venv) still
-# resolves to Apple's `/usr/bin/python3` (3.9.6), and `pip install llvm-lnt`
-# then fails because every llvm-lnt release requires Python >= 3.10.
+# It mirrors the macOS matrix entry of libcxx-benchmark-commit.yml as closely as
+# possible -- same runner, Xcode, Homebrew deps, and the same `run-benchmarks`
+# invocation -- but scoped to a single quick benchmark (hash.bench.cpp) so the
+# whole flow runs fast.
 #
-# This job mirrors the benchmark setup, dumps the relevant diagnostics, then
-# reproduces the failure and validates a candidate fix on the same runners.
+# The one intentional difference is the fix under test: we add
+# `$(brew --prefix python at 3.14)/bin` to PATH (it contains a real `python3`)
+# instead of `.../libexec/bin` (which only has unversioned `python`/`pip`, so a
+# bare `python3` fell through to Apple's 3.9.6 and llvm-lnt -- which needs
+# Python >= 3.10 -- could not be installed).
 
 name: Investigate LNT install failure (macOS)
 
@@ -27,54 +28,51 @@ on:
       - '.github/workflows/libcxx-investigate-lnt.yml'
 
 jobs:
-  investigate:
+  run-benchmarks:
     runs-on: ["self-hosted", "macOS", "ARM64", "26", "26.5"]
     env:
+      COMPILER: clang++
       DEVELOPER_DIR: /Applications/Xcode_26.5.app/Contents/Developer
+      # Pinned to the same commit the failing job benchmarked.
+      COMMIT: 62b809a37aac6cf2f64e99e9efba89dd8731920d
+      BENCHMARK_SUITE_VERSION: 62b809a37aac6cf2f64e99e9efba89dd8731920d
+      FILTER: hash.bench.cpp
     steps:
-      - name: Install python at 3.14 via Homebrew (same as the benchmark job)
+      - name: Checkout the LLVM monorepo
+        uses: actions/checkout at df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
+        with:
+          persist-credentials: false
+          # We benchmark arbitrary historical commits, which requires the full history to be available.
+          fetch-depth: 0
+          fetch-tags: true
+
+      - name: Install dependencies via Homebrew
         run: |
           brew update
-          brew install python at 3.14
-          echo "$(brew --prefix python at 3.14)/libexec/bin" >> "$GITHUB_PATH"
+          brew install ninja cmake python at 3.14
+          # FIX under test: use the dir that provides a real `python3`/`pip3`,
+          # not libexec/bin (which only has unversioned python/pip).
+          echo "$(brew --prefix python at 3.14)/bin" >> "$GITHUB_PATH"
 
-      - name: Diagnose which python3 / pip3 are actually on PATH
+      - name: Diagnose tools in use
         run: |
-          echo "PATH=${PATH}"
-          echo "--- which -a python3 python pip3 pip ---"
-          which -a python3 python pip3 pip || true
-          echo "--- versions ---"
+          cmake --version
+          ninja --version
+          "${COMPILER}" --version
           echo "python3 -> $(command -v python3): $(python3 --version 2>&1)"
-          echo "python  -> $(command -v python)  : $(python  --version 2>&1)"
-          prefix="$(brew --prefix python at 3.14)"
-          echo "--- brew --prefix python at 3.14 = ${prefix} ---"
-          echo "libexec/bin (this is what we add to PATH -- note: no python3):"
-          ls -l "${prefix}/libexec/bin"
-          echo "bin (this one DOES contain python3/pip3):"
-          ls -l "${prefix}/bin"
 
-      - name: Reproduce the failure -- venv via 'python3' + pip install llvm-lnt
+      - name: Run the benchmarks
         run: |
-          set -x
-          venv="$(mktemp -d)/venv"
-          python3 -m venv "${venv}"
-          "${venv}/bin/python" --version
-          "${venv}/bin/pip" --version
-          # This is the exact command that fails in the benchmark job. We expect
-          # it to fail here too; capture the status instead of aborting the job
-          # so the candidate-fix step below still runs.
-          if "${venv}/bin/pip" install llvm-lnt; then
-            echo "::warning::UNEXPECTED: 'pip install llvm-lnt' succeeded with python3"
-          else
-            echo "::notice::REPRODUCED: 'pip install llvm-lnt' failed with python3 (Apple 3.9.6), as in CI"
+          filter_arg=()
+          if [ -n "${FILTER}" ]; then
+            filter_arg=(--filter "${FILTER}")
           fi
+          libcxx/utils/ci/lnt/run-benchmarks                            \
+            --test-suite-commit "${BENCHMARK_SUITE_VERSION}"            \
+            --machine macos-26.5-arm64                                  \
+            --compiler "${COMPILER}"                                    \
+            --benchmark-commit "${COMMIT}"                              \
+            "${filter_arg[@]}"                                          \
+            --output "${COMMIT}.json"
 
-      - name: Candidate fix -- venv via 'python3.14' + pip install llvm-lnt
-        run: |
-          set -x
-          venv="$(mktemp -d)/venv"
-          python3.14 -m venv "${venv}"
-          "${venv}/bin/python" --version
-          "${venv}/bin/pip" install llvm-lnt
-          "${venv}/bin/lnt" --help | head -n 5
-          echo "::notice::FIX CONFIRMED: 'pip install llvm-lnt' works when the venv uses python3.14"
+          cat "${COMMIT}.json"



More information about the llvm-commits mailing list