[llvm] [libc++][CI] TEMP: investigate macOS LNT install failure (DO NOT MERGE) (PR #211783)
Louis Dionne via llvm-commits
llvm-commits at lists.llvm.org
Fri Jul 24 05:47:21 PDT 2026
https://github.com/ldionne updated https://github.com/llvm/llvm-project/pull/211783
>From ffc1b3d441875041fc74cd06952bcb6fe159288b Mon Sep 17 00:00:00 2001
From: Louis Dionne <ldionne.2 at gmail.com>
Date: Fri, 24 Jul 2026 08:37:32 -0400
Subject: [PATCH 1/2] [libc++][CI] TEMP: investigate macOS LNT install failure
Reproduce and diagnose the 'pip install llvm-lnt' failure on the macOS
self-hosted benchmark runners. DO NOT MERGE.
---
.github/workflows/libcxx-investigate-lnt.yml | 80 ++++++++++++++++++++
1 file changed, 80 insertions(+)
create mode 100644 .github/workflows/libcxx-investigate-lnt.yml
diff --git a/.github/workflows/libcxx-investigate-lnt.yml b/.github/workflows/libcxx-investigate-lnt.yml
new file mode 100644
index 0000000000000..e95224a49f18d
--- /dev/null
+++ b/.github/workflows/libcxx-investigate-lnt.yml
@@ -0,0 +1,80 @@
+# TEMPORARY — DO NOT MERGE.
+#
+# This workflow exists only to reproduce and investigate the LNT installation
+# failure seen on the macOS self-hosted benchmark runners:
+#
+# https://github.com/llvm/llvm-project/actions/runs/30092892111/job/89479960018
+#
+# The benchmark job installs `python at 3.14` via Homebrew and prepends
+# `$(brew --prefix python at 3.14)/libexec/bin` to the PATH. That directory only
+# provides *unversioned* `python`/`pip` symlinks -- it has no `python3`. So a
+# `python3` invocation (used by run-benchmarks to create the venv) still
+# resolves to Apple's `/usr/bin/python3` (3.9.6), and `pip install llvm-lnt`
+# then fails because every llvm-lnt release requires Python >= 3.10.
+#
+# This job mirrors the benchmark setup, dumps the relevant diagnostics, then
+# reproduces the failure and validates a candidate fix on the same runners.
+
+name: Investigate LNT install failure (macOS)
+
+permissions:
+ contents: read
+
+on:
+ workflow_dispatch:
+ pull_request:
+ paths:
+ - '.github/workflows/libcxx-investigate-lnt.yml'
+
+jobs:
+ investigate:
+ runs-on: ["self-hosted", "macOS", "ARM64", "26", "26.5"]
+ env:
+ DEVELOPER_DIR: /Applications/Xcode_26.5.app/Contents/Developer
+ steps:
+ - name: Install python at 3.14 via Homebrew (same as the benchmark job)
+ run: |
+ brew update
+ brew install python at 3.14
+ echo "$(brew --prefix python at 3.14)/libexec/bin" >> "$GITHUB_PATH"
+
+ - name: Diagnose which python3 / pip3 are actually on PATH
+ run: |
+ echo "PATH=${PATH}"
+ echo "--- which -a python3 python pip3 pip ---"
+ which -a python3 python pip3 pip || true
+ echo "--- versions ---"
+ echo "python3 -> $(command -v python3): $(python3 --version 2>&1)"
+ echo "python -> $(command -v python) : $(python --version 2>&1)"
+ prefix="$(brew --prefix python at 3.14)"
+ echo "--- brew --prefix python at 3.14 = ${prefix} ---"
+ echo "libexec/bin (this is what we add to PATH -- note: no python3):"
+ ls -l "${prefix}/libexec/bin"
+ echo "bin (this one DOES contain python3/pip3):"
+ ls -l "${prefix}/bin"
+
+ - name: Reproduce the failure -- venv via 'python3' + pip install llvm-lnt
+ run: |
+ set -x
+ venv="$(mktemp -d)/venv"
+ python3 -m venv "${venv}"
+ "${venv}/bin/python" --version
+ "${venv}/bin/pip" --version
+ # This is the exact command that fails in the benchmark job. We expect
+ # it to fail here too; capture the status instead of aborting the job
+ # so the candidate-fix step below still runs.
+ if "${venv}/bin/pip" install llvm-lnt; then
+ echo "::warning::UNEXPECTED: 'pip install llvm-lnt' succeeded with python3"
+ else
+ echo "::notice::REPRODUCED: 'pip install llvm-lnt' failed with python3 (Apple 3.9.6), as in CI"
+ fi
+
+ - name: Candidate fix -- venv via 'python3.14' + pip install llvm-lnt
+ run: |
+ set -x
+ venv="$(mktemp -d)/venv"
+ python3.14 -m venv "${venv}"
+ "${venv}/bin/python" --version
+ "${venv}/bin/pip" install llvm-lnt
+ "${venv}/bin/lnt" --help | head -n 5
+ echo "::notice::FIX CONFIRMED: 'pip install llvm-lnt' works when the venv uses python3.14"
>From 6a0217eaa3dd96aaf4d9d7941569ba816227e368 Mon Sep 17 00:00:00 2001
From: Louis Dionne <ldionne.2 at gmail.com>
Date: Fri, 24 Jul 2026 08:47:01 -0400
Subject: [PATCH 2/2] [libc++][CI] Mirror the real benchmark flow end-to-end
Run the actual run-benchmarks script with the hash.bench.cpp filter, using
the PATH fix under test (brew python at 3.14 'bin' dir, which has a real
python3). DO NOT MERGE.
---
.github/workflows/libcxx-investigate-lnt.yml | 94 ++++++++++----------
1 file changed, 46 insertions(+), 48 deletions(-)
diff --git a/.github/workflows/libcxx-investigate-lnt.yml b/.github/workflows/libcxx-investigate-lnt.yml
index e95224a49f18d..883dea347fbae 100644
--- a/.github/workflows/libcxx-investigate-lnt.yml
+++ b/.github/workflows/libcxx-investigate-lnt.yml
@@ -1,19 +1,20 @@
# TEMPORARY — DO NOT MERGE.
#
-# This workflow exists only to reproduce and investigate the LNT installation
-# failure seen on the macOS self-hosted benchmark runners:
+# This workflow reproduces the macOS benchmark job end-to-end in order to
+# investigate (and validate the fix for) the LNT installation failure seen at:
#
# https://github.com/llvm/llvm-project/actions/runs/30092892111/job/89479960018
#
-# The benchmark job installs `python at 3.14` via Homebrew and prepends
-# `$(brew --prefix python at 3.14)/libexec/bin` to the PATH. That directory only
-# provides *unversioned* `python`/`pip` symlinks -- it has no `python3`. So a
-# `python3` invocation (used by run-benchmarks to create the venv) still
-# resolves to Apple's `/usr/bin/python3` (3.9.6), and `pip install llvm-lnt`
-# then fails because every llvm-lnt release requires Python >= 3.10.
+# It mirrors the macOS matrix entry of libcxx-benchmark-commit.yml as closely as
+# possible -- same runner, Xcode, Homebrew deps, and the same `run-benchmarks`
+# invocation -- but scoped to a single quick benchmark (hash.bench.cpp) so the
+# whole flow runs fast.
#
-# This job mirrors the benchmark setup, dumps the relevant diagnostics, then
-# reproduces the failure and validates a candidate fix on the same runners.
+# The one intentional difference is the fix under test: we add
+# `$(brew --prefix python at 3.14)/bin` to PATH (it contains a real `python3`)
+# instead of `.../libexec/bin` (which only has unversioned `python`/`pip`, so a
+# bare `python3` fell through to Apple's 3.9.6 and llvm-lnt -- which needs
+# Python >= 3.10 -- could not be installed).
name: Investigate LNT install failure (macOS)
@@ -27,54 +28,51 @@ on:
- '.github/workflows/libcxx-investigate-lnt.yml'
jobs:
- investigate:
+ run-benchmarks:
runs-on: ["self-hosted", "macOS", "ARM64", "26", "26.5"]
env:
+ COMPILER: clang++
DEVELOPER_DIR: /Applications/Xcode_26.5.app/Contents/Developer
+ # Pinned to the same commit the failing job benchmarked.
+ COMMIT: 62b809a37aac6cf2f64e99e9efba89dd8731920d
+ BENCHMARK_SUITE_VERSION: 62b809a37aac6cf2f64e99e9efba89dd8731920d
+ FILTER: hash.bench.cpp
steps:
- - name: Install python at 3.14 via Homebrew (same as the benchmark job)
+ - name: Checkout the LLVM monorepo
+ uses: actions/checkout at df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
+ with:
+ persist-credentials: false
+ # We benchmark arbitrary historical commits, which requires the full history to be available.
+ fetch-depth: 0
+ fetch-tags: true
+
+ - name: Install dependencies via Homebrew
run: |
brew update
- brew install python at 3.14
- echo "$(brew --prefix python at 3.14)/libexec/bin" >> "$GITHUB_PATH"
+ brew install ninja cmake python at 3.14
+ # FIX under test: use the dir that provides a real `python3`/`pip3`,
+ # not libexec/bin (which only has unversioned python/pip).
+ echo "$(brew --prefix python at 3.14)/bin" >> "$GITHUB_PATH"
- - name: Diagnose which python3 / pip3 are actually on PATH
+ - name: Diagnose tools in use
run: |
- echo "PATH=${PATH}"
- echo "--- which -a python3 python pip3 pip ---"
- which -a python3 python pip3 pip || true
- echo "--- versions ---"
+ cmake --version
+ ninja --version
+ "${COMPILER}" --version
echo "python3 -> $(command -v python3): $(python3 --version 2>&1)"
- echo "python -> $(command -v python) : $(python --version 2>&1)"
- prefix="$(brew --prefix python at 3.14)"
- echo "--- brew --prefix python at 3.14 = ${prefix} ---"
- echo "libexec/bin (this is what we add to PATH -- note: no python3):"
- ls -l "${prefix}/libexec/bin"
- echo "bin (this one DOES contain python3/pip3):"
- ls -l "${prefix}/bin"
- - name: Reproduce the failure -- venv via 'python3' + pip install llvm-lnt
+ - name: Run the benchmarks
run: |
- set -x
- venv="$(mktemp -d)/venv"
- python3 -m venv "${venv}"
- "${venv}/bin/python" --version
- "${venv}/bin/pip" --version
- # This is the exact command that fails in the benchmark job. We expect
- # it to fail here too; capture the status instead of aborting the job
- # so the candidate-fix step below still runs.
- if "${venv}/bin/pip" install llvm-lnt; then
- echo "::warning::UNEXPECTED: 'pip install llvm-lnt' succeeded with python3"
- else
- echo "::notice::REPRODUCED: 'pip install llvm-lnt' failed with python3 (Apple 3.9.6), as in CI"
+ filter_arg=()
+ if [ -n "${FILTER}" ]; then
+ filter_arg=(--filter "${FILTER}")
fi
+ libcxx/utils/ci/lnt/run-benchmarks \
+ --test-suite-commit "${BENCHMARK_SUITE_VERSION}" \
+ --machine macos-26.5-arm64 \
+ --compiler "${COMPILER}" \
+ --benchmark-commit "${COMMIT}" \
+ "${filter_arg[@]}" \
+ --output "${COMMIT}.json"
- - name: Candidate fix -- venv via 'python3.14' + pip install llvm-lnt
- run: |
- set -x
- venv="$(mktemp -d)/venv"
- python3.14 -m venv "${venv}"
- "${venv}/bin/python" --version
- "${venv}/bin/pip" install llvm-lnt
- "${venv}/bin/lnt" --help | head -n 5
- echo "::notice::FIX CONFIRMED: 'pip install llvm-lnt' works when the venv uses python3.14"
+ cat "${COMMIT}.json"
More information about the llvm-commits
mailing list