[llvm] 5136c6d - [Tools][Docker] Update Dockerfiles and Docker guide (#123841)
via llvm-commits
llvm-commits at lists.llvm.org
Wed Jan 22 04:53:28 PST 2025
Author: Marc Auberer
Date: 2025-01-22T13:53:23+01:00
New Revision: 5136c6d9d207b72135c92567e58e5cdbb86efc09
URL: https://github.com/llvm/llvm-project/commit/5136c6d9d207b72135c92567e58e5cdbb86efc09
DIFF: https://github.com/llvm/llvm-project/commit/5136c6d9d207b72135c92567e58e5cdbb86efc09.diff
LOG: [Tools][Docker] Update Dockerfiles and Docker guide (#123841)
Fixes #123669
- Update Dockerfiles to work with the LLVM trunk
- Adapt Documentation accordingly
- Fix duplicate `-c` flag
Added:
llvm/utils/docker/debian12/Dockerfile
Modified:
llvm/docs/Docker.rst
llvm/utils/docker/build_docker_image.sh
llvm/utils/docker/example/Dockerfile
llvm/utils/docker/nvidia-cuda/Dockerfile
llvm/utils/docker/scripts/checkout.sh
Removed:
llvm/utils/docker/debian10/Dockerfile
################################################################################
diff --git a/llvm/docs/Docker.rst b/llvm/docs/Docker.rst
index 6b3c80da684b42..5d976eddb3130f 100644
--- a/llvm/docs/Docker.rst
+++ b/llvm/docs/Docker.rst
@@ -9,7 +9,7 @@ You can find a number of sources to build docker images with LLVM components in
images for their own use, or as a starting point for someone who wants to write
their own Dockerfiles.
-We currently provide Dockerfiles with ``debian10`` and ``nvidia-cuda`` base images.
+We currently provide Dockerfiles with ``debian12`` and ``nvidia-cuda`` base images.
We also provide an ``example`` image, which contains placeholders that one would need
to fill out in order to produce Dockerfiles for a new docker image.
@@ -72,13 +72,13 @@ checkout from git and provide a list of CMake arguments to use during when
building LLVM inside docker container.
Here's a very simple example of getting a docker image with clang binary,
-compiled by the system compiler in the debian10 image:
+compiled by the system compiler in the debian12 image:
.. code-block:: bash
./llvm/utils/docker/build_docker_image.sh \
- --source debian10 \
- --docker-repository clang-debian10 --docker-tag "staging" \
+ --source debian12 \
+ --docker-repository clang-debian12 --docker-tag "staging" \
-p clang -i install-clang -i install-clang-resource-headers \
-- \
-DCMAKE_BUILD_TYPE=Release
@@ -93,51 +93,45 @@ this command will do that:
# LLVM_TARGETS_TO_BUILD=Native is to reduce stage1 compile time.
# Options, starting with BOOTSTRAP_* are passed to stage2 cmake invocation.
./build_docker_image.sh \
- --source debian10 \
- --docker-repository clang-debian10 --docker-tag "staging" \
+ --source debian12 \
+ --docker-repository clang-debian12 --docker-tag "staging" \
-p clang -i stage2-install-clang -i stage2-install-clang-resource-headers \
-- \
-DLLVM_TARGETS_TO_BUILD=Native -DCMAKE_BUILD_TYPE=Release \
-DBOOTSTRAP_CMAKE_BUILD_TYPE=Release \
-DCLANG_ENABLE_BOOTSTRAP=ON -DCLANG_BOOTSTRAP_TARGETS="install-clang;install-clang-resource-headers"
-This will produce a new image ``clang-debian10:staging`` from the latest
+This will produce a new image ``clang-debian12:staging`` from the latest
upstream revision.
After the image is built you can run bash inside a container based on your image
like this:
.. code-block:: bash
- docker run -ti clang-debian10:staging bash
+ docker run -ti clang-debian12:staging bash
Now you can run bash commands as you normally would:
.. code-block:: bash
root at 80f351b51825:/# clang -v
- clang version 5.0.0 (trunk 305064)
+ clang version 19.1.7 (trunk 524462)
+ Target: x86_64-unknown-linux-gnu
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /bin
- Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.8
- Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.8.4
- Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.9
- Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.9.2
- Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.9
- Candidate multilib: .;@m64
- Selected multilib: .;@m64
Which image should I choose?
============================
-We currently provide two images: Debian10-based and nvidia-cuda-based. They
+We currently provide two images: Debian12-based and nvidia-cuda-based. They
diff er in the base image that they use, i.e. they have a
diff erent set of
preinstalled binaries. Debian8 is very minimal, nvidia-cuda is larger, but has
preinstalled CUDA libraries and allows to access a GPU, installed on your
machine.
If you need a minimal linux distribution with only clang and libstdc++ included,
-you should try Debian10-based image.
+you should try Debian12-based image.
If you want to use CUDA libraries and have access to a GPU on your machine,
you should choose nvidia-cuda-based image and use `nvidia-docker
@@ -150,7 +144,7 @@ If you have a
diff erent use-case, you could create your own image based on
``example/`` folder.
Any docker image can be built and run using only the docker binary, i.e. you can
-run debian10 build on Fedora or any other Linux distribution. You don't need to
+run debian12 build on Fedora or any other Linux distribution. You don't need to
install CMake, compilers or any other clang dependencies. It is all handled
during the build process inside Docker's isolated environment.
@@ -158,12 +152,12 @@ Stable build
============
If you want a somewhat recent and somewhat stable build, use the
``branches/google/stable`` branch, i.e. the following command will produce a
-Debian10-based image using the latest ``google/stable`` sources for you:
+Debian12-based image using the latest ``google/stable`` sources for you:
.. code-block:: bash
./llvm/utils/docker/build_docker_image.sh \
- -s debian10 --d clang-debian10 -t "staging" \
+ -s debian12 --d clang-debian12 -t "staging" \
--branch branches/google/stable \
-p clang -i install-clang -i install-clang-resource-headers \
-- \
diff --git a/llvm/utils/docker/build_docker_image.sh b/llvm/utils/docker/build_docker_image.sh
index f2d43934f4ccac..68de54709c2950 100755
--- a/llvm/utils/docker/build_docker_image.sh
+++ b/llvm/utils/docker/build_docker_image.sh
@@ -23,7 +23,7 @@ Available options:
General:
-h|--help show this help message
Docker-specific:
- -s|--source image source dir (i.e. debian10, nvidia-cuda, etc)
+ -s|--source image source dir (i.e. debian12, nvidia-cuda, etc)
-d|--docker-repository docker repository for the image
-t|--docker-tag docker tag for the image
Checkout arguments:
@@ -38,7 +38,7 @@ Available options:
-p|--llvm-project Add the project to a list LLVM_ENABLE_PROJECTS, passed to
CMake.
Can be specified multiple times.
- -c|--checksums name of a file, containing checksums of llvm checkout.
+ --checksums name of a file, containing checksums of llvm checkout.
Script will fail if checksums of the checkout do not
match.
Build-specific:
@@ -51,18 +51,18 @@ Required options: --source and --docker-repository, at least one
All options after '--' are passed to CMake invocation.
For example, running:
-$ build_docker_image.sh -s debian10 -d mydocker/debian10-clang -t latest \
+$ build_docker_image.sh -s debian12 -d mydocker/debian12-clang -t latest \
-p clang -i install-clang -i install-clang-resource-headers
will produce two docker images:
- mydocker/debian10-clang-build:latest - an intermediate image used to compile
+ mydocker/debian12-clang-build:latest - an intermediate image used to compile
clang.
- mydocker/clang-debian10:latest - a small image with preinstalled clang.
+ mydocker/clang-debian12:latest - a small image with preinstalled clang.
Please note that this example produces a not very useful installation, since it
doesn't override CMake defaults, which produces a Debug and non-boostrapped
version of clang.
To get a 2-stage clang build, you could use this command:
-$ ./build_docker_image.sh -s debian10 -d mydocker/clang-debian10 -t "latest" \
+$ ./build_docker_image.sh -s debian12 -d mydocker/clang-debian12 -t "latest" \
-p clang -i stage2-install-clang -i stage2-install-clang-resource-headers \
-- \
-DLLVM_TARGETS_TO_BUILD=Native -DCMAKE_BUILD_TYPE=Release \
@@ -110,7 +110,7 @@ while [[ $# -gt 0 ]]; do
CMAKE_ENABLED_PROJECTS="$CMAKE_ENABLED_PROJECTS;$PROJ"
shift 2
;;
- -c|--checksums)
+ --checksums)
shift
CHECKSUMS_FILE="$1"
shift
@@ -160,7 +160,7 @@ if [ $SEEN_INSTALL_TARGET -eq 0 ]; then
exit 1
fi
-SOURCE_DIR=$(dirname $0)
+SOURCE_DIR=$(dirname "$0")
if [ ! -d "$SOURCE_DIR/$IMAGE_SOURCE" ]; then
echo "No sources for '$IMAGE_SOURCE' were found in $SOURCE_DIR"
exit 1
diff --git a/llvm/utils/docker/debian10/Dockerfile b/llvm/utils/docker/debian12/Dockerfile
similarity index 79%
rename from llvm/utils/docker/debian10/Dockerfile
rename to llvm/utils/docker/debian12/Dockerfile
index b898c935b9726f..84123c439aced1 100644
--- a/llvm/utils/docker/debian10/Dockerfile
+++ b/llvm/utils/docker/debian12/Dockerfile
@@ -1,4 +1,4 @@
-#===- llvm/utils/docker/debian10/build/Dockerfile -------------------------===//
+#===- llvm/utils/docker/debian12/build/Dockerfile -------------------------===//
#
# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
# See https://llvm.org/LICENSE.txt for license information.
@@ -6,8 +6,8 @@
#
#===----------------------------------------------------------------------===//
# Stage 1. Check out LLVM source code and run the build.
-FROM launcher.gcr.io/google/debian10:latest as builder
-LABEL maintainer "LLVM Developers"
+FROM launcher.gcr.io/google/debian12:latest AS builder
+LABEL maintainer="LLVM Developers"
# Install build dependencies of llvm.
# First, Update the apt's source list and include the sources of the packages.
RUN grep deb /etc/apt/sources.list | \
@@ -19,8 +19,8 @@ RUN apt-get update && \
rm -rf /var/lib/apt/lists/*
# Install a newer ninja release. It seems the older version in the debian repos
# randomly crashes when compiling llvm.
-RUN wget "https://github.com/ninja-build/ninja/releases/download/v1.8.2/ninja-linux.zip" && \
- echo "d2fea9ff33b3ef353161ed906f260d565ca55b8ca0568fa07b1d2cab90a84a07 ninja-linux.zip" \
+RUN wget "https://github.com/ninja-build/ninja/releases/download/v1.12.1/ninja-linux.zip" && \
+ echo "6f98805688d19672bd699fbbfa2c2cf0fc054ac3df1f0e6a47664d963d530255 ninja-linux.zip" \
| sha256sum -c && \
unzip ninja-linux.zip -d /usr/local/bin && \
rm ninja-linux.zip
@@ -37,11 +37,11 @@ RUN /tmp/scripts/build_install_llvm.sh --to /tmp/clang-install ${buildscript_arg
# Stage 2. Produce a minimal release image with build results.
-FROM launcher.gcr.io/google/debian10:latest
-LABEL maintainer "LLVM Developers"
+FROM launcher.gcr.io/google/debian12:latest
+LABEL maintainer="LLVM Developers"
# Install packages for minimal useful image.
RUN apt-get update && \
- apt-get install -y --no-install-recommends libstdc++-7-dev binutils && \
+ apt-get install -y --no-install-recommends binutils && \
rm -rf /var/lib/apt/lists/*
# Copy build results of stage 1 to /usr/local.
COPY --from=builder /tmp/clang-install/ /usr/local/
diff --git a/llvm/utils/docker/example/Dockerfile b/llvm/utils/docker/example/Dockerfile
index ebfb0e49c82c86..197716fa7bdff9 100644
--- a/llvm/utils/docker/example/Dockerfile
+++ b/llvm/utils/docker/example/Dockerfile
@@ -10,9 +10,9 @@
# Stage 1. Check out LLVM source code and run the build.
# FIXME: Replace 'ubuntu' with your base image
-FROM ubuntu as builder
+FROM ubuntu AS builder
# FIXME: Change maintainer name
-LABEL maintainer "Maintainer <maintainer at email>"
+LABEL maintainer="Maintainer <maintainer at email>"
# FIXME: Install llvm/clang build dependencies here. Including compiler to
# build stage1, cmake, subversion, ninja, etc.
@@ -31,7 +31,7 @@ RUN /tmp/scripts/build_install_llvm.sh --to /tmp/clang-install ${buildscript_arg
# FIXME: Replace 'ubuntu' with your base image.
FROM ubuntu
# FIXME: Change maintainer name.
-LABEL maintainer "Maintainer <maintainer at email>"
+LABEL maintainer="Maintainer <maintainer at email>"
# FIXME: Install all packages you want to have in your release container.
# A minimal useful installation should include at least libstdc++ and binutils.
diff --git a/llvm/utils/docker/nvidia-cuda/Dockerfile b/llvm/utils/docker/nvidia-cuda/Dockerfile
index 91ad53c57b1655..035c5825df0fda 100644
--- a/llvm/utils/docker/nvidia-cuda/Dockerfile
+++ b/llvm/utils/docker/nvidia-cuda/Dockerfile
@@ -6,11 +6,11 @@
#
#===----------------------------------------------------------------------===//
# Stage 1. Check out LLVM source code and run the build.
-FROM nvidia/cuda:8.0-devel as builder
-LABEL maintainer "LLVM Developers"
+FROM nvidia/cuda:12.6.3-devel-ubuntu24.04 AS builder
+LABEL maintainer="LLVM Developers"
# Install llvm build dependencies.
RUN apt-get update && \
- apt-get install -y --no-install-recommends ca-certificates cmake python \
+ apt-get install -y --no-install-recommends ca-certificates cmake 2to3 python-is-python3 \
subversion ninja-build git && \
rm -rf /var/lib/apt/lists/*
@@ -26,8 +26,8 @@ RUN /tmp/scripts/build_install_llvm.sh --to /tmp/clang-install ${buildscript_arg
# Stage 2. Produce a minimal release image with build results.
-FROM nvidia/cuda:8.0-devel
-LABEL maintainer "LLVM Developers"
+FROM nvidia/cuda:12.6.3-devel-ubuntu24.04
+LABEL maintainer="LLVM Developers"
# Copy clang installation into this container.
COPY --from=builder /tmp/clang-install/ /usr/local/
# C++ standard library and binutils are already included in the base package.
diff --git a/llvm/utils/docker/scripts/checkout.sh b/llvm/utils/docker/scripts/checkout.sh
index 8fe7c6141d86b2..677fa61e6ae514 100755
--- a/llvm/utils/docker/scripts/checkout.sh
+++ b/llvm/utils/docker/scripts/checkout.sh
@@ -18,7 +18,7 @@ Checkout git sources into /tmp/clang-build/src. Used inside a docker container.
Available options:
-h|--help show this help message
-b|--branch git branch to checkout, i.e. 'main',
- 'release/10.x'
+ 'release/19.x'
(default: 'main')
-r|--revision git revision to checkout
-c|--cherrypick revision to cherry-pick. Can be specified multiple times.
@@ -83,7 +83,7 @@ function apply_cherrypicks() {
# This function is always called on a sorted list of cherrypicks.
for CHERRY_REV in $CHERRYPICKS; do
echo "Cherry-picking $CHERRY_REV into $CHECKOUT_DIR"
- EMAIL="someone at somewhere.net" git cherry-pick $CHERRY_REV
+ EMAIL="someone at somewhere.net" git cherry-pick "$CHERRY_REV"
done
popd
@@ -97,7 +97,7 @@ mkdir -p "$CLANG_BUILD_DIR/src"
CHECKOUT_DIR="$CLANG_BUILD_DIR/src"
echo "Checking out https://github.com/llvm/llvm-project.git to $CHECKOUT_DIR"
-git clone -b $LLVM_BRANCH --single-branch \
+git clone -b "$LLVM_BRANCH" --single-branch \
"https://github.com/llvm/llvm-project.git" \
"$CHECKOUT_DIR"
@@ -114,7 +114,7 @@ CHECKSUMS_FILE="/tmp/checksums/checksums.txt"
if [ -f "$CHECKSUMS_FILE" ]; then
echo "Validating checksums for LLVM checkout..."
- python "$(dirname $0)/llvm_checksum/llvm_checksum.py" -c "$CHECKSUMS_FILE" \
+ python "$(dirname "$0")/llvm_checksum/llvm_checksum.py" -c "$CHECKSUMS_FILE" \
--partial --multi_dir "$CLANG_BUILD_DIR/src"
else
echo "Skipping checksumming checks..."
More information about the llvm-commits
mailing list