[libcxx-commits] [libcxx] [llvm] [libc++][WIP] Move to Github hosted builders (PR #109720)
Louis Dionne via libcxx-commits
libcxx-commits at lists.llvm.org
Tue Sep 24 06:08:26 PDT 2024
https://github.com/ldionne updated https://github.com/llvm/llvm-project/pull/109720
>From 22b80131356cfc4252f9dc8bb4eeb15e85af921a Mon Sep 17 00:00:00 2001
From: Louis Dionne <ldionne.2 at gmail.com>
Date: Mon, 23 Sep 2024 17:23:35 -0400
Subject: [PATCH 1/8] [libc++][WIP] Move to Github hosted builders
Libc++ experiences periodic outages of our Github actions builders,
which are quite disruptive. Unfortunately, the Github self-hosted
runner setup is very opaque. It works extremely well when it works,
but most contributors are unable to troubleshoot issues related to
the setup when they happen.
This patch attempts to move the libc++ builders to normal Github
hosted runners. This is primarily to evaluate the feasibility of
such a change, which would simplify our CI setup but may not scale
sufficiently for our needs.
---
.github/workflows/libcxx-build-and-test.yaml | 32 ++++++++++----------
1 file changed, 16 insertions(+), 16 deletions(-)
diff --git a/.github/workflows/libcxx-build-and-test.yaml b/.github/workflows/libcxx-build-and-test.yaml
index b5e60781e00064..bdf4c1b74fdbaa 100644
--- a/.github/workflows/libcxx-build-and-test.yaml
+++ b/.github/workflows/libcxx-build-and-test.yaml
@@ -49,7 +49,13 @@ env:
jobs:
stage1:
if: github.repository_owner == 'llvm'
- runs-on: libcxx-runners-8-set
+ runs: &docker-image
+ using: 'docker'
+ image: 'libcxx/utils/ci/Dockerfile'
+ args:
+ - BASE_IMAGE: ghcr.io/actions/actions-runner:2.317.0
+ - GCC_LATEST_VERSION: 14
+ - LLVM_HEAD_VERSION: 20
continue-on-error: false
strategy:
fail-fast: false
@@ -84,7 +90,8 @@ jobs:
**/crash_diagnostics/*
stage2:
if: github.repository_owner == 'llvm'
- runs-on: libcxx-runners-8-set
+ runs:
+ <<: *docker-image
needs: [ stage1 ]
continue-on-error: false
strategy:
@@ -136,12 +143,15 @@ jobs:
matrix:
config: [
'generic-abi-unstable',
+ 'generic-asan',
+ 'generic-cxx26',
'generic-hardening-mode-debug',
'generic-hardening-mode-extensive',
'generic-hardening-mode-fast',
'generic-hardening-mode-fast-with-abi-breaks',
'generic-merged',
'generic-modules-lsv',
+ 'generic-msan',
'generic-no-exceptions',
'generic-no-experimental',
'generic-no-filesystem',
@@ -155,25 +165,15 @@ jobs:
'generic-no-rtti',
'generic-optimized-speed',
'generic-static',
+ 'generic-tsan',
+ 'generic-ubsan',
# TODO Find a better place for the benchmark and bootstrapping builds to live. They're either very expensive
# or don't provide much value since the benchmark run results are too noise on the bots.
'benchmarks',
'bootstrapping-build'
]
- machine: [ 'libcxx-runners-8-set' ]
- include:
- - config: 'generic-cxx26'
- machine: libcxx-runners-8-set
- - config: 'generic-asan'
- machine: libcxx-runners-8-set
- - config: 'generic-tsan'
- machine: libcxx-runners-8-set
- - config: 'generic-ubsan'
- machine: libcxx-runners-8-set
- # Use a larger machine for MSAN to avoid timeout and memory allocation issues.
- - config: 'generic-msan'
- machine: libcxx-runners-8-set
- runs-on: ${{ matrix.machine }}
+ runs:
+ <<: *docker-image
steps:
- uses: actions/checkout at v4
- name: ${{ matrix.config }}
>From 2fdbb387fd0109cd3d9b223c1e5b88b3f3b75e9b Mon Sep 17 00:00:00 2001
From: Louis Dionne <ldionne.2 at gmail.com>
Date: Mon, 23 Sep 2024 17:36:33 -0400
Subject: [PATCH 2/8] Touch libc++ file to trigger CI
---
libcxx/foo | 0
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 libcxx/foo
diff --git a/libcxx/foo b/libcxx/foo
new file mode 100644
index 00000000000000..e69de29bb2d1d6
>From a494ba9c3ea9d5e20154c978f5d917e1b6877da2 Mon Sep 17 00:00:00 2001
From: Louis Dionne <ldionne.2 at gmail.com>
Date: Mon, 23 Sep 2024 17:39:05 -0400
Subject: [PATCH 3/8] WIP: Try on ubuntu-latest
---
.github/workflows/libcxx-build-and-test.yaml | 21 ++++++++++----------
1 file changed, 10 insertions(+), 11 deletions(-)
diff --git a/.github/workflows/libcxx-build-and-test.yaml b/.github/workflows/libcxx-build-and-test.yaml
index bdf4c1b74fdbaa..55e8f7ba16ff56 100644
--- a/.github/workflows/libcxx-build-and-test.yaml
+++ b/.github/workflows/libcxx-build-and-test.yaml
@@ -49,13 +49,14 @@ env:
jobs:
stage1:
if: github.repository_owner == 'llvm'
- runs: &docker-image
- using: 'docker'
- image: 'libcxx/utils/ci/Dockerfile'
- args:
- - BASE_IMAGE: ghcr.io/actions/actions-runner:2.317.0
- - GCC_LATEST_VERSION: 14
- - LLVM_HEAD_VERSION: 20
+ runs-on: ubuntu-latest
+ # runs: &docker-image
+ # using: 'docker'
+ # image: 'libcxx/utils/ci/Dockerfile'
+ # args:
+ # - BASE_IMAGE: ghcr.io/actions/actions-runner:2.317.0
+ # - GCC_LATEST_VERSION: 14
+ # - LLVM_HEAD_VERSION: 20
continue-on-error: false
strategy:
fail-fast: false
@@ -90,8 +91,7 @@ jobs:
**/crash_diagnostics/*
stage2:
if: github.repository_owner == 'llvm'
- runs:
- <<: *docker-image
+ runs-on: ubuntu-latest
needs: [ stage1 ]
continue-on-error: false
strategy:
@@ -172,8 +172,7 @@ jobs:
'benchmarks',
'bootstrapping-build'
]
- runs:
- <<: *docker-image
+ runs-on: ubuntu-latest
steps:
- uses: actions/checkout at v4
- name: ${{ matrix.config }}
>From 24e87959c4e573ef1c1147ea26652ced64f83bcd Mon Sep 17 00:00:00 2001
From: Louis Dionne <ldionne.2 at gmail.com>
Date: Mon, 23 Sep 2024 17:56:36 -0400
Subject: [PATCH 4/8] Try with fozzie-live Docker image
---
.github/workflows/libcxx-build-and-test.yaml | 2 ++
1 file changed, 2 insertions(+)
diff --git a/.github/workflows/libcxx-build-and-test.yaml b/.github/workflows/libcxx-build-and-test.yaml
index 55e8f7ba16ff56..226ac92939696e 100644
--- a/.github/workflows/libcxx-build-and-test.yaml
+++ b/.github/workflows/libcxx-build-and-test.yaml
@@ -50,6 +50,8 @@ jobs:
stage1:
if: github.repository_owner == 'llvm'
runs-on: ubuntu-latest
+ container:
+ image: ghcr.io/libcxx/actions-builder:fozzie-live
# runs: &docker-image
# using: 'docker'
# image: 'libcxx/utils/ci/Dockerfile'
>From 0456f4c77eb6001e6035d9d6a63d888ed1bfb502 Mon Sep 17 00:00:00 2001
From: Louis Dionne <ldionne.2 at gmail.com>
Date: Mon, 23 Sep 2024 18:03:28 -0400
Subject: [PATCH 5/8] Run more verbose
---
libcxx/utils/ci/run-buildbot | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/libcxx/utils/ci/run-buildbot b/libcxx/utils/ci/run-buildbot
index b0533cb9a49c93..18229333a34a7f 100755
--- a/libcxx/utils/ci/run-buildbot
+++ b/libcxx/utils/ci/run-buildbot
@@ -119,7 +119,7 @@ function generate-cmake-base() {
-DLIBCXX_ENABLE_WERROR=YES \
-DLIBCXXABI_ENABLE_WERROR=YES \
-DLIBUNWIND_ENABLE_WERROR=YES \
- -DLLVM_LIT_ARGS="-sv --xunit-xml-output test-results.xml --timeout=1500 --time-tests" \
+ -DLLVM_LIT_ARGS="--xunit-xml-output test-results.xml --timeout=1500 --time-tests" \
"${@}"
}
>From cc626cf49f4fe01a78f0310a03cf154b70edadbe Mon Sep 17 00:00:00 2001
From: Louis Dionne <ldionne.2 at gmail.com>
Date: Mon, 23 Sep 2024 18:19:00 -0400
Subject: [PATCH 6/8] Use fozzie-live image everywhere
---
.github/workflows/libcxx-build-and-test.yaml | 11 ++++-------
1 file changed, 4 insertions(+), 7 deletions(-)
diff --git a/.github/workflows/libcxx-build-and-test.yaml b/.github/workflows/libcxx-build-and-test.yaml
index 226ac92939696e..2e8cdbda304a4e 100644
--- a/.github/workflows/libcxx-build-and-test.yaml
+++ b/.github/workflows/libcxx-build-and-test.yaml
@@ -52,13 +52,6 @@ jobs:
runs-on: ubuntu-latest
container:
image: ghcr.io/libcxx/actions-builder:fozzie-live
- # runs: &docker-image
- # using: 'docker'
- # image: 'libcxx/utils/ci/Dockerfile'
- # args:
- # - BASE_IMAGE: ghcr.io/actions/actions-runner:2.317.0
- # - GCC_LATEST_VERSION: 14
- # - LLVM_HEAD_VERSION: 20
continue-on-error: false
strategy:
fail-fast: false
@@ -94,6 +87,8 @@ jobs:
stage2:
if: github.repository_owner == 'llvm'
runs-on: ubuntu-latest
+ container:
+ image: ghcr.io/libcxx/actions-builder:fozzie-live
needs: [ stage1 ]
continue-on-error: false
strategy:
@@ -175,6 +170,8 @@ jobs:
'bootstrapping-build'
]
runs-on: ubuntu-latest
+ container:
+ image: ghcr.io/libcxx/actions-builder:fozzie-live
steps:
- uses: actions/checkout at v4
- name: ${{ matrix.config }}
>From 88177cb7f200f34cdfbbd0c4e683bc85c3576182 Mon Sep 17 00:00:00 2001
From: Louis Dionne <ldionne.2 at gmail.com>
Date: Mon, 23 Sep 2024 19:14:23 -0400
Subject: [PATCH 7/8] Run lit verbose
---
libcxx/utils/ci/run-buildbot | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/libcxx/utils/ci/run-buildbot b/libcxx/utils/ci/run-buildbot
index 18229333a34a7f..315242781c4bf9 100755
--- a/libcxx/utils/ci/run-buildbot
+++ b/libcxx/utils/ci/run-buildbot
@@ -119,7 +119,7 @@ function generate-cmake-base() {
-DLIBCXX_ENABLE_WERROR=YES \
-DLIBCXXABI_ENABLE_WERROR=YES \
-DLIBUNWIND_ENABLE_WERROR=YES \
- -DLLVM_LIT_ARGS="--xunit-xml-output test-results.xml --timeout=1500 --time-tests" \
+ -DLLVM_LIT_ARGS="-v --xunit-xml-output test-results.xml --timeout=1500 --time-tests" \
"${@}"
}
>From 4fa1a176a09e30185d258a430e3bf3dd69f7cc29 Mon Sep 17 00:00:00 2001
From: Louis Dionne <ldionne.2 at gmail.com>
Date: Tue, 24 Sep 2024 09:08:06 -0400
Subject: [PATCH 8/8] Try tweaking awk command
---
libcxx/test/libcxx/transitive_includes.gen.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/libcxx/test/libcxx/transitive_includes.gen.py b/libcxx/test/libcxx/transitive_includes.gen.py
index 22075364bf1b79..97d36c2b2a5f77 100644
--- a/libcxx/test/libcxx/transitive_includes.gen.py
+++ b/libcxx/test/libcxx/transitive_includes.gen.py
@@ -93,7 +93,7 @@
// RUN: mkdir %t
// RUN: %{{cxx}} %s %{{flags}} %{{compile_flags}} --trace-includes -fshow-skipped-includes --preprocess > /dev/null 2> %t/trace-includes.txt
// RUN: %{{python}} %{{libcxx-dir}}/test/libcxx/transitive_includes_to_csv.py %t/trace-includes.txt > %t/actual_transitive_includes.csv
-// RUN: cat %{{libcxx-dir}}/test/libcxx/transitive_includes/%{{cxx_std}}.csv | awk '/^{escaped_header} / {{ print }}' > %t/expected_transitive_includes.csv
+// RUN: cat %{{libcxx-dir}}/test/libcxx/transitive_includes/%{{cxx_std}}.csv | awk '/^{escaped_header}[[:space:]]/ {{ print }}' > %t/expected_transitive_includes.csv
// RUN: diff -w %t/expected_transitive_includes.csv %t/actual_transitive_includes.csv
#include <{header}>
"""
More information about the libcxx-commits
mailing list