[polly] c55c2ab - [Buildbot][Polly] Move polly-x86_64-linux-test-suite build instructions into main repository (#166809)
via llvm-commits
llvm-commits at lists.llvm.org
Wed Jan 7 01:02:09 PST 2026
Author: Michael Kruse
Date: 2026-01-07T10:02:04+01:00
New Revision: c55c2ab8067eef649a8b04fc8173b8e164cb56b3
URL: https://github.com/llvm/llvm-project/commit/c55c2ab8067eef649a8b04fc8173b8e164cb56b3
DIFF: https://github.com/llvm/llvm-project/commit/c55c2ab8067eef649a8b04fc8173b8e164cb56b3.diff
LOG: [Buildbot][Polly] Move polly-x86_64-linux-test-suite build instructions into main repository (#166809)
Allow the main llvm-project repository to contain the buildbot builder
instructions, instead of storing them in llvm-zorg. The corresponding
llvm-zorg PR is https://github.com/llvm/llvm-zorg/pull/648.
Using polly-x86_64-linux-test-suite as a proof-of-concept because that
builder is currently offline, I am its maintainer, and is easier to
build than an configuration supporting offload. Once the design has been
decided, more builders can follow.
Advantages are:
* It is easier to make changes in the llvm-project repository. There are
more reviewers than for the llvm-zorg repository.
* Buildbot changes can be made in the same PR with changes that require
updating the buildbot, e.g. changing the name of a CMake option.
* Configuration changes take effect immeditately when landing; no
buildbot master restart needed.
* Some builders store a CMake cache file in the llvm-project repository
for the reasons above. However, the number of changes that can be made
with a CMake cache file alone are limited.
Compared to AnnotatedBuilder, advantages are:
* Reproducing a buildbot configuration locally made easy: just execute
the script in-place. No llvm-zorg, local buildbot worker, or buildbot
master needed.
* Same for testing a change of a builder before landing it in llvm-zorg.
Doing so with an AnnotatedBuilder requires two llvm-zorg checkouts:
One for making the change of the builder script itself, which then is
pushed to a private llvm-zorg branch on GitHub, and a second that is
modified to fetch that branch instead of
https://github.com/llvm/llvm-zorg/tree/main.
* The AnnotatedBuilder scripts are located in the llvm-zorg repository
and the buildbot-workers always checkout is always the top-of-trunk.
This means that a buildbot configuration is split over three checkouts:
* The checkout of llvm-project to be tested
* The checkout of llvm-zorg by the buildbot-worker fetches; always the
top-of-trunk, i.e may not match the revision of llvm-project that is
executed (such as the CMake cache files located there), especially when
using the "Force build" feature.
* The checkout of llvm-zorg that the buildbot-master is running, which
is updated only when the master is manually restarted.
* The "Force Build" feature also allows for test-building any
llvm-project PR. This is correctly handled by zorg's
`addGetSourcecodeSteps`, but does not work with AnnotatedBuilders that
checkout the llvm-project source on their own.
The goal is to move as much as possible into the llvm-project repository
such that there cannot be a mismatch between checkouts of different
repositories. Ideally, the buildbot-master only needs to be
updated+restarted for adding/removing workers, not for build
configuration changes.
---------
Co-authored-by: Jan Patrick Lehr <jp.lehr at gmail.com>
Added:
.ci/buildbot/README.md
.ci/buildbot/worker.py
polly/ci/.gitignore
polly/ci/polly-x86_64-linux-test-suite.cmake
polly/ci/polly-x86_64-linux-test-suite.py
Modified:
Removed:
################################################################################
diff --git a/.ci/buildbot/README.md b/.ci/buildbot/README.md
new file mode 100644
index 0000000000000..8daf081bdf11a
--- /dev/null
+++ b/.ci/buildbot/README.md
@@ -0,0 +1,49 @@
+# ScriptedBuilder Buildbot Workers
+
+This directory contains code shared by LLVM Buildbot workers. The typical
+pipeline of a ScriptedBuilder-based builder is as follows.
+
+1. A commit is pushed to [main](https://github.com/llvm/llvm-project/tree/main)
+
+2. The [Buildbot master](https://lab.llvm.org/) polls the repository and finds
+ new commits. It schedules build requests on every relevant worker.
+ Alternatively, a build request of a specific llvm-project commit can be
+ created using the "Force Build" or "Rebuild" buttons.
+
+3. When a worker is ready, the master sends the build steps determined by
+ [ScriptedBuilder](https://github.com/llvm/llvm-zorg/blob/main/zorg/buildbot/builders/ScriptedBuilder.py)
+ to the worker.
+
+4. The checkout step checks out llvm-project commit into a directory named
+ `llvm.src` on the worker.
+
+5. The annotate step executes a predefined Python script from the llvm-project
+ source tree on the worker. Its working directory is an initially empty
+ sibling directory named `build`. The argument `--workdir=.` is passed to
+ override the default build directory (which is
diff erent to avoid
+ accidentally spilling clutter into the cwd)
+
+6. The script is expected to use the utilities from
+ [`worker.py`](https://github.com/llvm/llvm-project/blob/main/.ci/buildbot/worker.py)
+ to build LLVM. The `with w.step("stepname"):` pattern is used to visually
+ separate additional steps in the [Buildbot GUI](https://lab.llvm.org/).
+
+
+## Reproducing Builds
+
+Users can execute the worker script directly to reproduce a build problem with
+a worker using the llvm-project source tree in which it is located. By default
+it will use a new directory with `.workdir` suffix (so it can be
+`.gitignore`-ignored) next to the script as build directory.
+
+The ScriptedBuilder system tries to keep all worker/build settings within the
+script, but some parameters can be overridden using command line parameters.
+For instance, `--jobs` overrides the Ninja and llvm-lit `-j` argument that the
+worker would use. Script should be written to honor these overrides where they
+apply, and may also add additional ones.
+
+See
+[`worker.py`](https://github.com/llvm/llvm-project/blob/main/.ci/buildbot/worker.py)
+or a
+[reference script](https://github.com/llvm/llvm-project/blob/main/polly/polly-x86_64-linux-test-suite)
+for further details.
diff --git a/.ci/buildbot/worker.py b/.ci/buildbot/worker.py
new file mode 100644
index 0000000000000..99de7faab10db
--- /dev/null
+++ b/.ci/buildbot/worker.py
@@ -0,0 +1,577 @@
+# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+# See https://llvm.org/LICENSE.txt for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+"""Utilities for ScriptedBuilder Buildbot worker scripts"""
+
+import argparse
+import filecmp
+import os
+import stat
+import pathlib
+import re
+import shlex
+import shutil
+import subprocess
+import sys
+import traceback
+import platform
+import multiprocessing
+from contextlib import contextmanager
+
+_SHQUOTE_WINDOWS_ESCAPEDCHARS = re.compile(r'(["\\])')
+_SHQUOTE_WINDOWS_QUOTEDCHARS = re.compile("[ \t\n]")
+
+
+def _shquote_windows(txt):
+ """shlex.quote for Windows cmd.exe"""
+ txt = txt.replace("%", "%%")
+ quoted = re.sub(_SHQUOTE_WINDOWS_ESCAPEDCHARS, r"\\\1", txt)
+ if len(quoted) == len(txt) and not _SHQUOTE_WINDOWS_QUOTEDCHARS.search(txt):
+ return txt
+ else:
+ return '"' + quoted + '"'
+
+
+def shjoin(args):
+ """Convert a list of shell arguments to an appropriately quoted string."""
+ if os.name in set(("nt", "os2", "ce")):
+ return " ".join(map(_shquote_windows, args))
+ else:
+ return shlex.join(args)
+
+
+def report(msg):
+ """
+ Emit a message to the build log. Appears in red font. Lines surrounded
+ by @@@ may be interpreted as meta-instructions.
+ """
+ print(msg, file=sys.stderr, flush=True)
+
+
+def report_prog_version(name, cmd):
+ try:
+ p = subprocess.run(cmd, check=True, capture_output=True, text=True)
+ outlines = p.stdout.strip().splitlines()
+ report_list(name, outlines[0])
+ except BaseException:
+ pass
+
+
+def report_list(category, *items):
+ items = list(items)
+ filtered = []
+
+ while items:
+ item = items.pop()
+ match item:
+ case tuple() | list():
+ items += item
+ continue
+ case None:
+ continue
+ case _:
+ item = str(item).strip()
+ if not item:
+ continue
+ if item in filtered:
+ continue
+ filtered.append(item)
+ category += ":"
+ report(f"{category:<9}{', '.join(reversed( filtered))}")
+
+
+def report_platform():
+ report_list(
+ "CPU",
+ platform.machine(),
+ platform.architecture()[0],
+ platform.processor(),
+ f"{multiprocessing.cpu_count()} native threads",
+ )
+ try:
+ releaseinfo = platform.freedesktop_os_release()
+ except BaseException:
+ releaseinfo = dict()
+ report_list(
+ "OS",
+ platform.system(),
+ platform.architecture()[1],
+ platform.platform(),
+ releaseinfo.get("PRETTY_NAME"),
+ )
+ report_list("Python", platform.python_implementation(), platform.python_version())
+
+ report_prog_version("CMake", ["cmake", "--version"])
+ report_prog_version("Ninja", ["ninja", "--version"])
+ report_prog_version("Sphinx", ["sphinx-build", "--version"])
+ report_prog_version("Doxygen", ["doxygen", "--version"])
+
+ report_prog_version("gcc", ["gcc", "--version"])
+ report_prog_version("ld", ["ld", "--version"])
+
+ report_prog_version("LLVM", ["llvm-config", "--version"])
+ report_prog_version("Clang", ["clang", "--version"])
+ report_prog_version("LLD", ["ld.lld", "--version"])
+
+
+def run_command(cmd, shell=False, **kwargs):
+ """
+ Report which command is being run, then execute it using
+ subprocess.check_call.
+ """
+ report(f"Running: {cmd if shell else shjoin(cmd)}")
+ sys.stderr.flush()
+ subprocess.check_call(cmd, shell=shell, **kwargs)
+
+
+def _remove_readonly(func, path, _):
+ """Clear the readonly bit and reattempt the removal."""
+ try:
+ os.chmod(path, stat.S_IWRITE)
+ except Exception:
+ pass
+ func(path)
+
+
+def rmtree(path):
+ """Remove directory path and all its subdirectories. Includes a workaround
+ for Windows where shutil.rmtree errors on read-only files.
+
+ Taken from official Python docs
+ https://docs.python.org/3/library/shutil.html#rmtree-example
+ """
+ shutil.rmtree(path, onexc=_remove_readonly)
+
+
+def try_delete(path):
+ """
+ Delete the file or directory;
+ if not successful, print a warning but continue
+ """
+ try:
+ os.unlink(path)
+ except Exception:
+ try:
+ _remove_readonly(os.unlink, path, _)
+ except Exception:
+ try:
+ rmtree(path)
+ except Exception as e:
+ print(f"Warning: Could not delete {path}: {e}")
+
+
+def checkout(giturl, sourcepath):
+ """
+ Use git to checkout the remote repository giturl at local directory
+ sourcepath.
+
+ If the repository already exists, clear all local changes and check out the
+ latest main branch.
+ """
+ if not os.path.exists(sourcepath):
+ run_command(["git", "clone", giturl, sourcepath])
+
+ # Reset repository state no matter what there was before
+ run_command(["git", "-C", sourcepath, "stash", "--all"])
+ run_command(["git", "-C", sourcepath, "stash", "clear"])
+
+ # Fetch and checkout the newest
+ run_command(["git", "-C", sourcepath, "fetch", "origin"])
+ run_command(["git", "-C", sourcepath, "checkout", "origin/main", "--detach"])
+
+
+ at contextmanager
+def step(step_name, halt_on_fail=False):
+ """Report a new build step being started.
+
+ Use like this::
+ with step("greet-step"):
+ report("Hello World!")
+ """
+ # Barrier to separate stdio output for the the previous step
+ sys.stderr.flush()
+ sys.stdout.flush()
+
+ report(f"@@@BUILD_STEP {step_name}@@@")
+ if halt_on_fail:
+ report("@@@HALT_ON_FAILURE@@@")
+ try:
+ yield
+ except Exception as e:
+ if isinstance(e, subprocess.CalledProcessError):
+ report(f"{shjoin(e.cmd)} exited with return code {e.returncode}.")
+ report("@@@STEP_FAILURE@@@")
+ else:
+ traceback.print_exc()
+ report("@@@STEP_EXCEPTION@@@")
+ if halt_on_fail:
+ # Do not continue with the next steps, but allow except/finally
+ # blocks to execute
+ raise e
+
+
+class Worker:
+ """Helper class to keep context in a worker.run() environment"""
+
+ def __init__(self, args, clean, clobber, workdir, jobs, cachefile, llvmsrcroot):
+ self.args = args
+ self.clean = clean
+ self.clobber = clobber
+ self.workdir = workdir
+ self.jobs = jobs
+ self.cachefile = cachefile
+ self.llvmsrcroot = llvmsrcroot
+
+ def in_llvmsrc(self, path):
+ """
+ Convert a path in the llvm-project source checkout to an absolute path
+ """
+ return os.path.join(self.llvmsrcroot, path)
+
+ def in_workdir(self, path):
+ """Convert a path in the workdir to an absolute path"""
+ return os.path.join(self.workdir, path)
+
+ def run_ninja(
+ self, targets: list = [], *, builddir, ccache_stats: bool = False, **kwargs
+ ):
+ """
+ Run ninja in builddir. If self.jobs is set, automatically adds a
+ -j option to set the number of parallel jobs.
+
+ Parameters
+ ----------
+ targets : list
+ List of build targets; build the default target 'all' if list is
+ empty
+ builddir
+ Directory of the build.ninja file
+ ccache_stats : bool
+ If true, also emit ccache statistics when finishing the build
+ """
+ cmd = ["ninja"]
+ if builddir is not None:
+ cmd += ["-C", builddir]
+ cmd += targets
+ if self.jobs:
+ cmd.append(f"-j{self.jobs}")
+ if ccache_stats:
+ run_command(["ccache", "-z"])
+ try:
+ run_command(cmd, **kwargs)
+ finally:
+ # TODO: Pipe to stderr to separate from build log itself
+ run_command(["ccache", "-sv"])
+ else:
+ run_command(cmd, **kwargs)
+
+ @contextmanager
+ def step(self, step_name, halt_on_fail=False):
+ """Convenience wrapper for step()"""
+ with step(step_name, halt_on_fail=halt_on_fail) as s:
+ yield s
+
+ def report(self, msg):
+ """Convenience wrapper for report()"""
+ report(msg)
+
+ def run_command(self, *args, **kwargs):
+ """Convenience wrapper for run_command()"""
+ return run_command(*args, **kwargs)
+
+ def rmtree(self, *args, **kwargs):
+ """Convenience wrapper for rmtree()"""
+ return rmtree(*args, *kwargs)
+
+ def checkout(self, giturl, sourcepath):
+ """Convenience wrapper for checkout()"""
+ return checkout(giturl, sourcepath)
+
+
+def convert_bool(v):
+ """Convert input to bool type
+
+ Use to convert the value of bool environment variables. Specifically, the
+ buildbot master sets 'false' to build properties, which by default Python
+ would interpret as true-ish.
+ """
+ match v:
+ case None:
+ return False
+ case bool(b):
+ return b
+ case str(s):
+ return not s.strip().upper() in ["", "0", "N", "NO", "FALSE", "OFF"]
+ case _:
+ return bool(v)
+
+
+def relative_if_possible(path, relative_to):
+ """
+ Like os.path.relpath, but does not fail if path is not a parent of
+ relative_to; keeps the original path in that case
+ """
+ path = os.path.normpath(path)
+ if not os.path.isabs(path):
+ # Path is already relative (assumed to relative_to)
+ return path
+ try:
+ result = os.path.relpath(path, start=relative_to)
+ return result if result else path
+ except ValueError:
+ return path
+
+
+ at contextmanager
+def run(
+ scriptpath,
+ llvmsrcroot,
+ parser=None,
+ cachefile=None,
+ clobberpaths=[],
+ workerjobs=None,
+ incremental=None,
+):
+ """
+ Runs the boilerplate for a ScriptedBuilder buildbot. It is not necessary to
+ use this function (one can also call run_command() etc. directly), but
+ allows for some more flexibility and safety checks. Arguments passed to this
+ function represent the worker configuration.
+
+ We use the term 'clean' for resetting the worker to an empty state. This
+ involves deleting ${prefix}/llvm.src as well as ${prefix}/build.
+ The term 'clobber' means deleting build artifacts, but not already
+ downloaded git repositories. Build artifacts include build- and
+ install-directories. Changes in the llvm.src directory will
+ either be force-reset by the buildbot's 'checkout' step anyway,
+ or -- in case of local invocation -- represents the source the user wants
+ to reproduce without being tied to a specific commit. In either case the
+ source directories should not be touched. We consider 'clean' to comprise
+ 'clobber'. llvm-zorg also uses the term 'clean_obj' instead of 'clobber'.
+ By default, we will always clobber to get the same starting point at every
+ build. If incremental=True or the --incremental command line option is used,
+ the starting point is the previous build.
+
+ A buildbot worker will invoke this script using this directory structure,
+ where ${prefix} is a dedicated directory for this builder:
+ ${prefix}/llvm.src # Checkout location for the llvm-source
+ ${prefix}/build # cwd when launching the build script
+
+ The build script is called with --workdir=. parameter, i.e. the build
+ artifacts are written into ${prefix}/build. When cleaning, the worker (NOT
+ the build script) will delete ${prefix}/llvm.src; Deleting any contents of
+ ${prefix}/build is to be done by the builder script, e.g. by this function.
+ The builder script can choose to not delete the complete workdir, e.g.
+ additional source checkouts such as the llvm-test-suite.
+
+ The buildbot master will set the 'clean' build property and the environment
+ variable BUILDBOT_CLEAN when in the GUI the option "Clean source code and
+ build directory" is checked by the user. The 'clean_obj' build property and
+ the BUILDBOT_CLEAN_OBJ environment variable will be set when either the
+ "Clean build directory" GUI option is set, or the master detects a change
+ to a CMakeLists.txt or *.cmake file.
+
+ Parameters
+ ----------
+ scriptpath
+ Pass __file__ from the main builder script.
+ llvmsrcroot
+ Absolute path to the llvm-project source checkout. Since the builder
+ script is supposed to be a part of llvm-project itself, the builder
+ script can compute it from __file__.
+ parser
+ Use this argparse.ArgumentParser instead of creating a new one. Allows
+ adding additional command line switches in addition to the pre-defined
+ ones. Build scripts are encouraged to apply the pre-defined switches.
+ cachefile
+ Path (relative to llvmsrcroot) of the CMake cache file to
+ use. `None` indicates that the script does not use a cache file. Can be
+ overridden using --cachefile.
+ clobberpaths
+ Directories relative to workdir that need to be deleted if the build
+ configuration changes (due to changes of CMakeLists.txt or changes of
+ configuration parameters). Typically, only source checkouts are not
+ deleted.
+ workerjobs
+ Default number of build and test jobs; If set, expected to be the number
+ of jobs of the actual buildbot worker that executes this script. Can be
+ overridden using the --jobs parameter so in case someone needs to
+ reproduce this build, they can adjust the number of jobs for the
+ reproducer platform. Alternatively, the worker can set the
+ BUILDBOT_JOBS environment variable or keep ninja/llvm-lit defaults.
+ incremental
+ Only clobber the build artifacts when the build configuration changes.
+ Can be overridden using --incremental.
+ """
+
+ scriptpath = os.path.abspath(scriptpath)
+ llvmsrcroot = os.path.abspath(llvmsrcroot)
+ stem = pathlib.Path(scriptpath).stem
+ workdir_default = f"{stem}.workdir"
+
+ jobs_default = None
+ if jobs_env := os.environ.get("BUILDBOT_JOBS"):
+ jobs_default = int(jobs_env)
+ if not jobs_default:
+ jobs_default = workerjobs
+ if not jobs_default:
+ jobs_default = None
+
+ incremental_default = None if incremental else False
+
+ parser = parser or argparse.ArgumentParser(
+ allow_abbrev=True,
+ description="When executed without arguments, builds the worker's "
+ f"LLVM build configuration in {os.path.abspath(workdir_default)}. "
+ "Some build configuration parameters can be altered using the "
+ "following switches:",
+ )
+ parser.add_argument(
+ "--workdir",
+ default=workdir_default,
+ help="Use this dir (relative to cwd) as workdir to write the build "
+ "artifacts into; --workdir=. uses the current directory.\nWarning: The "
+ "content of this directory may be deleted",
+ )
+ if cachefile is not None:
+ parser.add_argument(
+ "--cachefile",
+ default=relative_if_possible(cachefile, llvmsrcroot),
+ help="File containing the initial values for the CMakeCache.txt "
+ "for the llvm build.",
+ )
+ parser.add_argument(
+ "--clean",
+ action=argparse.BooleanOptionalAction,
+ default=convert_bool(os.environ.get("BUILDBOT_CLEAN")),
+ help="Delete the entire workdir before starting the build, including "
+ "source directories",
+ )
+ parser.add_argument(
+ "--incremental",
+ action=argparse.BooleanOptionalAction,
+ default=incremental_default,
+ help="Keep previous build artifacts when starting the build",
+ )
+ parser.add_argument(
+ "--jobs",
+ "-j",
+ type=int,
+ default=jobs_default,
+ help="Number of build- and test-jobs",
+ )
+ args = parser.parse_args()
+
+ workdir = os.path.abspath(args.workdir)
+ incremental = args.incremental
+ clean = args.clean
+ if cachefile is not None:
+ cachefile = os.path.join(llvmsrcroot, args.cachefile)
+ if not os.path.isfile(cachefile):
+ raise Exception(f"--cachefile={cachefile} does not exist")
+
+ prevcachepath = os.path.join(workdir, "prevcache.cmake")
+ prevscriptpath = os.path.join(workdir, "prevscript.py")
+
+ if clean:
+ # Clean implies clobber
+ clobber = False
+ elif incremental is None:
+ # Automatically determine whether to clobber
+ def has_config_change():
+ # Has the master scheduler determined a CMakeLists.txt has changed?
+ if convert_bool(os.environ.get("BUILDBOT_CLOBBER")):
+ return True
+ if convert_bool(os.environ.get("BUILDBOT_CLEAN_OBJ")):
+ return True
+
+ # Has the build script changed?
+ if not os.path.isfile(prevscriptpath):
+ return True
+ if not filecmp.cmp(scriptpath, prevscriptpath, shallow=False):
+ return True
+
+ # Has the cache file (if any) changed?
+ if cachefile:
+ if not os.path.isfile(prevcachepath):
+ return True
+ if not os.path.isfile(cachefile):
+ return True
+ if not filecmp.cmp(cachefile, prevcachepath, shallow=False):
+ return True
+
+ return False
+
+ clobber = has_config_change()
+ else:
+ # Adhere to explicitly set incremental option
+ clobber = not incremental
+
+ # Safety check
+ parentdir = os.path.dirname(scriptpath)
+ while True:
+ if os.path.exists(workdir) and os.path.samefile(parentdir, workdir):
+ raise Exception(
+ f"Cannot use {args.workdir} as workdir; it contains the source "
+ "itself in '{parentdir}'"
+ )
+ newparentdir = os.path.dirname(parentdir)
+ if newparentdir == parentdir:
+ break
+ parentdir = newparentdir
+
+ w = Worker(
+ args,
+ clean=clean,
+ clobber=clobber,
+ workdir=workdir,
+ jobs=args.jobs,
+ cachefile=cachefile,
+ llvmsrcroot=llvmsrcroot,
+ )
+
+ with step("platform-info"):
+ report_platform()
+
+ # Ensure that the cwd is not the directory we are going to delete. This
+ # would not work e.g. under Windows. We will chdir to workdir in the next
+ # step anyway.
+ os.chdir("/")
+
+ if clean:
+ if os.path.exists(workdir):
+ print("Deleting previous build state including sources", file=sys.stderr)
+
+ with w.step(f"clean"):
+ if os.path.exists(workdir):
+ # Do not delete the directory itself, just the contents; it might be
+ # a symlink to somewhere else
+ for d in os.listdir(workdir):
+ try_delete(os.path.join(workdir, d))
+ elif clobber:
+ # Warn user if deleting anything
+ for p in clobberpaths:
+ if os.path.exists(os.path.join(workdir, p)):
+ print(
+ "Deleting previous build artifacts; use --incremental to keep",
+ file=sys.stderr,
+ )
+ break
+
+ with w.step(f"clobber"):
+ for d in clobberpaths:
+ try_delete(os.path.join(workdir, d))
+ try_delete(prevscriptpath)
+ try_delete(prevcachepath)
+
+ os.makedirs(workdir, exist_ok=True)
+ os.chdir(workdir)
+
+ # Remember used script and cachefile to detect changes
+ shutil.copy(scriptpath, prevscriptpath)
+ if cachefile:
+ shutil.copy(cachefile, prevcachepath)
+
+ os.environ["NINJA_STATUS"] = "[%p/%es :: %u->%r->%f (of %t)] "
+
+ yield w
diff --git a/polly/ci/.gitignore b/polly/ci/.gitignore
new file mode 100644
index 0000000000000..f2198f0f4ef8e
--- /dev/null
+++ b/polly/ci/.gitignore
@@ -0,0 +1 @@
+/*.workdir
diff --git a/polly/ci/polly-x86_64-linux-test-suite.cmake b/polly/ci/polly-x86_64-linux-test-suite.cmake
new file mode 100644
index 0000000000000..f5699804f2109
--- /dev/null
+++ b/polly/ci/polly-x86_64-linux-test-suite.cmake
@@ -0,0 +1,11 @@
+# General settings
+set(CMAKE_BUILD_TYPE "Release" CACHE STRING "")
+set(CMAKE_C_COMPILER_LAUNCHER "ccache" CACHE STRING "")
+set(CMAKE_CXX_COMPILER_LAUNCHER "ccache" CACHE STRING "")
+set(LLVM_ENABLE_LLD ON CACHE BOOL "")
+
+set(LLVM_ENABLE_ASSERTIONS ON CACHE BOOL "")
+set(LLVM_ENABLE_PROJECTS "clang;polly" CACHE STRING "")
+set(LLVM_TARGETS_TO_BUILD "X86" CACHE STRING "")
+
+set(LLVM_POLLY_LINK_INTO_TOOLS ON CACHE BOOL "")
diff --git a/polly/ci/polly-x86_64-linux-test-suite.py b/polly/ci/polly-x86_64-linux-test-suite.py
new file mode 100644
index 0000000000000..2a91f479d222a
--- /dev/null
+++ b/polly/ci/polly-x86_64-linux-test-suite.py
@@ -0,0 +1,76 @@
+#! /usr/bin/env python3
+# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+# See https://llvm.org/LICENSE.txt for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+"""Check Polly optimizations on llvm-test-suite"""
+
+import os
+import sys
+
+# Adapt to location in source tree
+llvmsrcroot = os.path.normpath(f"{__file__}/../../..")
+
+sys.path.insert(0, os.path.join(llvmsrcroot, ".ci/buildbot"))
+import worker
+
+llvmbuilddir = "llvm.build"
+llvminstalldir = "llvm.install"
+testsuitesrcdir = "testsuite.src"
+testsuitebuilddir = "testsuite.build"
+
+with worker.run(
+ __file__,
+ llvmsrcroot,
+ cachefile="polly/ci/polly-x86_64-linux-test-suite.cmake",
+ clobberpaths=[llvmbuilddir, testsuitebuilddir, llvminstalldir],
+ incremental=True,
+) as w:
+ with w.step("configure-llvm", halt_on_fail=True):
+ cmakecmd = [
+ "cmake",
+ f"-S{w.in_llvmsrc('llvm')}",
+ f"-B{llvmbuilddir}",
+ "-GNinja",
+ f"-C{w.in_llvmsrc(w.cachefile)}",
+ f"-DCMAKE_INSTALL_PREFIX={llvminstalldir}",
+ ]
+ if w.jobs:
+ cmakecmd.append(f"-DLLVM_LIT_ARGS=-sv;-j{w.jobs}")
+ w.run_command(cmakecmd)
+
+ with w.step("build-llvm", halt_on_fail=True):
+ w.run_ninja(builddir=llvmbuilddir, ccache_stats=True)
+
+ with w.step("check-polly"):
+ w.run_ninja(["check-polly"], builddir=llvmbuilddir)
+
+ with w.step("install-llvm", halt_on_fail=True):
+ w.run_ninja(["install"], builddir=llvmbuilddir)
+
+ with w.step("checkout-testsuite", halt_on_fail=True):
+ w.checkout("https://github.com/llvm/llvm-test-suite", testsuitesrcdir)
+
+ with w.step("configure-testsuite", halt_on_fail=True):
+ jobsarg = f";-j{w.jobs}" if w.jobs else ""
+ w.run_command(
+ [
+ "cmake",
+ f"-S{testsuitesrcdir}",
+ f"-B{testsuitebuilddir}",
+ "-GNinja",
+ "-DCMAKE_BUILD_TYPE=Release",
+ f"-DCMAKE_C_COMPILER={os.path.abspath(llvminstalldir)}/bin/clang",
+ f"-DCMAKE_CXX_COMPILER={os.path.abspath(llvminstalldir)}/bin/clang++",
+ f"-DTEST_SUITE_LIT={os.path.abspath(llvmbuilddir)}/bin/llvm-lit",
+ f"-DTEST_SUITE_LLVM_SIZE={os.path.abspath(llvmbuilddir)}/bin/llvm-size",
+ "-DTEST_SUITE_EXTRA_C_FLAGS=-Wno-unused-command-line-argument -mllvm -polly",
+ "-DTEST_SUITE_EXTRA_CXX_FLAGS=-Wno-unused-command-line-argument -mllvm -polly",
+ f"-DLLVM_LIT_ARGS=-sv{jobsarg};-o;report.json",
+ ]
+ )
+
+ with w.step("build-testsuite", halt_on_fail=True):
+ w.run_ninja(builddir=testsuitebuilddir)
+
+ with w.step("check-testsuite"):
+ w.run_ninja(["check"], builddir=testsuitebuilddir)
More information about the llvm-commits
mailing list