[llvm] r276264 - [docs] Update release docs
Renato Golin via llvm-commits
llvm-commits at lists.llvm.org
Thu Jul 21 05:00:51 PDT 2016
Author: rengolin
Date: Thu Jul 21 07:00:50 2016
New Revision: 276264
URL: http://llvm.org/viewvc/llvm-project?rev=276264&view=rev
Log:
[docs] Update release docs
Modified:
llvm/trunk/docs/HowToReleaseLLVM.rst
Modified: llvm/trunk/docs/HowToReleaseLLVM.rst
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/HowToReleaseLLVM.rst?rev=276264&r1=276263&r2=276264&view=diff
==============================================================================
--- llvm/trunk/docs/HowToReleaseLLVM.rst (original)
+++ llvm/trunk/docs/HowToReleaseLLVM.rst Thu Jul 21 07:00:50 2016
@@ -2,17 +2,13 @@
How To Release LLVM To The Public
=================================
-.. contents::
- :local:
- :depth: 1
-
Introduction
============
This document contains information about successfully releasing LLVM ---
-including subprojects: e.g., ``clang`` and ``dragonegg`` --- to the public. It
-is the Release Manager's responsibility to ensure that a high quality build of
-LLVM is released.
+including sub-projects: e.g., ``clang`` and ``compiler-rt`` --- to the public.
+It is the Release Manager's responsibility to ensure that a high quality build
+of LLVM is released.
If you're looking for the document on how to test the release candidates and
create the binary packages, please refer to the :doc:`ReleaseProcess` instead.
@@ -46,7 +42,7 @@ The release process is roughly as follow
the end of the first round of testing will be removed or disabled for the
release.
-* Generate and send out the second release candidate sources. Only *critial*
+* Generate and send out the second release candidate sources. Only *critical*
bugs found during this testing phase will be fixed. Any bugs introduced by
merged patches will be fixed. If so a third round of testing is needed.
@@ -89,24 +85,10 @@ Branch the Subversion trunk using the fo
#. Verify that the current Subversion trunk is in decent shape by
examining nightly tester and buildbot results.
-#. Create the release branch for ``llvm``, ``clang``, the ``test-suite``, and
- ``dragonegg`` from the last known good revision. The branch's name is
+#. Create the release branch for ``llvm``, ``clang``, and other sub-projects,
+ from the last known good revision. The branch's name is
``release_XY``, where ``X`` is the major and ``Y`` the minor release
- numbers. The branches should be created using the following commands:
-
- ::
-
- $ svn copy https://llvm.org/svn/llvm-project/llvm/trunk \
- https://llvm.org/svn/llvm-project/llvm/branches/release_XY
-
- $ svn copy https://llvm.org/svn/llvm-project/cfe/trunk \
- https://llvm.org/svn/llvm-project/cfe/branches/release_XY
-
- $ svn copy https://llvm.org/svn/llvm-project/dragonegg/trunk \
- https://llvm.org/svn/llvm-project/dragonegg/branches/release_XY
-
- $ svn copy https://llvm.org/svn/llvm-project/test-suite/trunk \
- https://llvm.org/svn/llvm-project/test-suite/branches/release_XY
+ numbers. Use ``utils/release/tag.sh`` to tag the release.
#. Advise developers that they may now check their patches into the Subversion
tree again.
@@ -121,8 +103,6 @@ Branch the Subversion trunk using the fo
$ svn co https://llvm.org/svn/llvm-project/cfe/branches/release_XY clang-X.Y
- $ svn co https://llvm.org/svn/llvm-project/dragonegg/branches/release_XY dragonegg-X.Y
-
$ svn co https://llvm.org/svn/llvm-project/test-suite/branches/release_XY test-suite-X.Y
Update LLVM Version
@@ -155,71 +135,19 @@ be done with the export.sh script in uti
This will generate source tarballs for each LLVM project being validated, which
can be uploaded to the website for further testing.
-Building the Release
---------------------
-
-The builds of ``llvm``, ``clang``, and ``dragonegg`` *must* be free of
-errors and warnings in Debug, Release+Asserts, and Release builds. If all
-builds are clean, then the release passes Build Qualification.
-
-The ``make`` options for building the different modes:
-
-+-----------------+---------------------------------------------+
-| Mode | Options |
-+=================+=============================================+
-| Debug | ``ENABLE_OPTIMIZED=0`` |
-+-----------------+---------------------------------------------+
-| Release+Asserts | ``ENABLE_OPTIMIZED=1`` |
-+-----------------+---------------------------------------------+
-| Release | ``ENABLE_OPTIMIZED=1 DISABLE_ASSERTIONS=1`` |
-+-----------------+---------------------------------------------+
-
-Build LLVM
-^^^^^^^^^^
-
-Build ``Debug``, ``Release+Asserts``, and ``Release`` versions
-of ``llvm`` on all supported platforms. Directions to build ``llvm``
-are :doc:`here <GettingStarted>`.
-
Build Clang Binary Distribution
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-Creating the ``clang`` binary distribution (Debug/Release+Asserts/Release)
-requires performing the following steps for each supported platform:
-
-#. Build clang according to the directions `here
- <http://clang.llvm.org/get_started.html>`__.
+Creating the ``clang`` binary distribution requires following the instructions
+:doc:`here <ReleaseProcess>`.
-#. Build both a Debug and Release version of clang. The binary will be the
- Release build.
+That process will perform both Release+Asserts and Release builds but only
+pack the Release build for upload. You should use the Release+Asserts sysroot,
+normally under ``final/Phase3/Release+Asserts/llvmCore-3.8.1-RCn.install/``,
+for test-suite and run-time benchmarks, to make sure nothing serious has
+passed through the net. For compile-time benchmarks, use the Release version.
-#. Package ``clang`` (details to follow).
-
-Target Specific Build Details
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-The table below specifies which compilers are used for each Arch/OS combination
-when qualifying the build of ``llvm``, ``clang``, and ``dragonegg``.
-
-+--------------+---------------+----------------------+
-| Architecture | OS | compiler |
-+==============+===============+======================+
-| x86-32 | Mac OS 10.5 | gcc 4.0.1 |
-+--------------+---------------+----------------------+
-| x86-32 | Linux | gcc 4.2.X, gcc 4.3.X |
-+--------------+---------------+----------------------+
-| x86-32 | FreeBSD | gcc 4.2.X |
-+--------------+---------------+----------------------+
-| x86-32 | mingw | gcc 3.4.5 |
-+--------------+---------------+----------------------+
-| x86-64 | Mac OS 10.5 | gcc 4.0.1 |
-+--------------+---------------+----------------------+
-| x86-64 | Linux | gcc 4.2.X, gcc 4.3.X |
-+--------------+---------------+----------------------+
-| x86-64 | FreeBSD | gcc 4.2.X |
-+--------------+---------------+----------------------+
-| ARMv7 | Linux | gcc 4.6.X, gcc 4.7.X |
-+--------------+---------------+----------------------+
+The minimum required version of the tools you'll need are :doc:`here <GettingStarted>`
Release Qualification Criteria
------------------------------
@@ -229,68 +157,50 @@ baseline). Regressions are related to c
(We may tolerate some minor performance regressions if they are deemed
necessary for the general quality of the compiler.)
-**Regressions are new failures in the set of tests that are used to qualify
+More specifically, Clang/LLVM is qualified when it has a clean test with all
+supported sub-projects included (``make check-all``), per target, and it has no
+regressions with the ``test-suite`` in relation to the previous release.
+
+Regressions are new failures in the set of tests that are used to qualify
each product and only include things on the list. Every release will have
some bugs in it. It is the reality of developing a complex piece of
software. We need a very concrete and definitive release criteria that
ensures we have monotonically improving quality on some metric. The metric we
use is described below. This doesn't mean that we don't care about other
criteria, but these are the criteria which we found to be most important and
-which must be satisfied before a release can go out.**
+which must be satisfied before a release can go out.
+
+Official Testing
+----------------
+
+A few developers in the community have dedicated time to validate the release
+candidates and volunteered to be the official release testers for each
+architecture.
+
+These will be the ones testing, generating and uploading the official binaries
+to the server, and will be the minimum tests *necessary* for the release to
+proceed.
-Qualify LLVM
-^^^^^^^^^^^^
+This will obviously not cover all OSs and distributions, so additional community
+validation is important. However, if community input is not reached before the
+release is out, all bugs reported will have to go on the next stable release.
-LLVM is qualified when it has a clean test run without a front-end. And it has
-no regressions when using either ``clang`` or ``dragonegg`` with the
-``test-suite`` from the previous release.
-
-Qualify Clang
-^^^^^^^^^^^^^
-
-``Clang`` is qualified when front-end specific tests in the ``llvm`` regression
-test suite all pass, clang's own test suite passes cleanly, and there are no
-regressions in the ``test-suite``.
-
-Specific Target Qualification Details
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-+--------------+-------------+----------------+-----------------------------+
-| Architecture | OS | clang baseline | tests |
-+==============+=============+================+=============================+
-| x86-32 | Linux | last release | llvm regression tests, |
-| | | | clang regression tests, |
-| | | | test-suite (including spec) |
-+--------------+-------------+----------------+-----------------------------+
-| x86-32 | FreeBSD | last release | llvm regression tests, |
-| | | | clang regression tests, |
-| | | | test-suite |
-+--------------+-------------+----------------+-----------------------------+
-| x86-32 | mingw | none | QT |
-+--------------+-------------+----------------+-----------------------------+
-| x86-64 | Mac OS 10.X | last release | llvm regression tests, |
-| | | | clang regression tests, |
-| | | | test-suite (including spec) |
-+--------------+-------------+----------------+-----------------------------+
-| x86-64 | Linux | last release | llvm regression tests, |
-| | | | clang regression tests, |
-| | | | test-suite (including spec) |
-+--------------+-------------+----------------+-----------------------------+
-| x86-64 | FreeBSD | last release | llvm regression tests, |
-| | | | clang regression tests, |
-| | | | test-suite |
-+--------------+-------------+----------------+-----------------------------+
-| ARMv7A | Linux | last release | llvm regression tests, |
-| | | | clang regression tests, |
-| | | | test-suite |
-+--------------+-------------+----------------+-----------------------------+
+The official release managers are:
+
+* Major releases (X.0): Hans Wennborg
+* Stable releases (X.n): Tom Stellard
+
+The official release testers are volunteered from the community and have
+consistently validated and released binaries for their targets/OSs. To contact
+them, you should email the ``release-testers at lists.llvm.org`` mailing list.
Community Testing
-----------------
Once all testing has been completed and appropriate bugs filed, the release
candidate tarballs are put on the website and the LLVM community is notified.
-Ask that all LLVM developers test the release in 2 ways:
+
+We ask that all LLVM developers test the release in any the following ways:
#. Download ``llvm-X.Y``, ``llvm-test-X.Y``, and the appropriate ``clang``
binary. Build LLVM. Run ``make check`` and the full LLVM test suite (``make
@@ -300,28 +210,57 @@ Ask that all LLVM developers test the re
everything. Run ``make check`` and the full LLVM test suite (``make
TEST=nightly report``).
-Ask LLVM developers to submit the test suite report and ``make check`` results
-to the list. Verify that there are no regressions from the previous release.
-The results are not used to qualify a release, but to spot other potential
-problems. For unsupported targets, verify that ``make check`` is at least
-clean.
+#. Download ``llvm-X.Y``, ``llvm-test-X.Y``, and the appropriate ``clang``
+ binary. Build whole programs with it (ex. Chromium, Firefox, Apache) for
+ your platform.
+
+#. Download ``llvm-X.Y``, ``llvm-test-X.Y``, and the appropriate ``clang``
+ binary. Build *your* programs with it and check for conformance and
+ performance regressions.
+
+#. Run the :doc:`release process <ReleaseProcess>`, if your platform is
+ *different* than that which is officially supported, and report back errors
+ only if they were not reported by the official release tester for that
+ architecture.
+
+We also ask that the OS distribution release managers test their packages with
+the first candidate of every release, and report any *new* errors in Bugzilla.
+If the bug can be reproduced with an unpatched upstream version of the release
+candidate (as opposed to the distribution's own build), the priority should be
+release blocker.
During the first round of testing, all regressions must be fixed before the
second release candidate is tagged.
-If this is the second round of testing, the testing is only to ensure that bug
+In the subsequent stages, the testing is only to ensure that bug
fixes previously merged in have not created new major problems. *This is not
the time to solve additional and unrelated bugs!* If no patches are merged in,
the release is determined to be ready and the release manager may move onto the
next stage.
+Reporting Regressions
+---------------------
+
+Every regression that is found during the tests (as per the criteria above),
+should be filled in a bug in Bugzilla with the priority *release blocker* and
+blocking a specific release.
+
+To help manage all the bugs reported and which ones are blockers or not, a new
+"[meta]" bug should be created and all regressions *blocking* that Meta. Once
+all blockers are done, the Meta can be closed.
+
+If a bug can't be reproduced, or stops being a blocker, it should be removed
+from the Meta and its priority decreased to *normal*. Debugging can continue,
+but on trunk.
+
Release Patch Rules
-------------------
Below are the rules regarding patching the release branch:
#. Patches applied to the release branch may only be applied by the release
- manager.
+ manager, the official release testers or the code owners with approval from
+ the release manager.
#. During the first round of testing, patches that fix regressions or that are
small and relatively risk free (verified by the appropriate code owner) are
@@ -333,7 +272,7 @@ Below are the rules regarding patching t
regressions may be applied.
#. For dot releases all patches must maintain both API and ABI compatibility with
- the previous major release. Only bugfixes will be accepted.
+ the previous major release. Only bug-fixes will be accepted.
Merging Patches
^^^^^^^^^^^^^^^
@@ -394,10 +333,10 @@ is what to do:
#. Check out the ``www`` module from Subversion.
-#. Create a new subdirectory ``X.Y`` in the releases directory.
+#. Create a new sub-directory ``X.Y`` in the releases directory.
-#. Commit the ``llvm``, ``test-suite``, ``clang`` source, ``clang binaries``,
- ``dragonegg`` source, and ``dragonegg`` binaries in this new directory.
+#. Commit the ``llvm``, ``test-suite``, ``clang`` source and binaries in this
+ new directory.
#. Copy and commit the ``llvm/docs`` and ``LICENSE.txt`` files into this new
directory. The docs should be built with ``BUILD_FOR_WEBSITE=1``.
@@ -417,5 +356,6 @@ is what to do:
Announce the Release
^^^^^^^^^^^^^^^^^^^^
-Have Chris send out the release announcement when everything is finished.
+Send an email to the list announcing the release, pointing people to all the
+relevant documentation, download pages and bugs fixed.
More information about the llvm-commits
mailing list