[LLVMdev] make dist?
Reid Spencer
reid at x10sys.com
Tue Apr 11 14:11:11 PDT 2006
Hi Tanya,
The first thing you need to understand is that there are multiple make
targets to support this feature. I'll briefly describe each here so you
have an overview and then delve into the details later.
* distdir - builds the distribution directory from which the
distribution will be packaged
* dist - builds each of the distribution tarballs (tar.gz,
tar.bzip2, .zip). These can be built individually as well, with
separate targets.
* dist-check - this is identical to "dist" but includes a check on
the distribution that ensures the tarball can: be unpacked,
compiles correctly, passes 'make check', passes 'make clean'.
* dist-clean - this just does a normal clean but also cleans up
the stuff generated by the other three "dist" targets (above).
Okay, that's the basic functionality. When making a release, we want to
ensure that the tree you build the distribution from passes "dist-
check". Beyond fixing the usual bugs, there is generally one impediment
to making the release in this fashion: missing files. The "dist-check"
process guards against that possibility. It will either fail and that
failure will indicate what's missing, or it will succeed meaning that it
has proved that the tarballs can actually succeed in building LLVM
correctly and that it passes "make check".
distdir
This target builds the distribution directory which is the directory
from which the tarballs are generated. The distribution directory has
the same name as the release, e.g. LLVM-1.7). This target goes through
he following process:
1. First, if there was an old distribution directory (for the
current release), it is removed in its entirety and you see
"Removing old LLVM-1.7"
2. Second, it issues a "make all ENABLE_OPTIMIZED=1" to ensure that
the everything in your tree can be built in release mode. Often
times there are discrepancies in building between debug and
release modes so it enforces release mode first. If that fails,
the distdir target fails too. This is preceded by the message
"Making 'all' to verify build".
3. Next, it traverses your source tree and copies it to a new
directory that has the name of the release (LLVM-1.7 in our
current case). This is the directory that will get tar'd. It
contains all the software that needs to be in the distribution.
During the copying process, it omits generated files, CVS
directories, and any other "cruft" that's in your build tree.
This is done to eliminate the possibility of huge distribution
tarballs that include useless or irrelevant stuff in them. This
is the trickiest part of making the distribution. Done manually
you will either include stuff that shouldn't be in the
distribution or exclude stuff that should. This step is preceded
by the message "Building Distribution Directory LLVM-1.7"
4. The distribution directory is then traversed and all CVS or .svn
directories are removed. You see: "Eliminating CVS/.svn
directories from distribution"
5. The recursive dist-hook target is executed. This gives each
directory a chance to modify the distribution in some way (more
on this below).
6. The distribution directory is traversed and the correct file
permissions and modes are set based on the type of file.
To control the process of making the distribution directory correctly,
each Makefile can utilize two features:
1. EXTRA_DIST - this make variable specifies which files it should
distribute. By default, all source files are automatically
included for distribution as well as certain "well known" files
(see DistAlways variable in Makefile.rules for details). Each
Makefile specifies, via the EXTRA_DIST variable, which
additional files need to be distributed. Only those files that
are needed to build LLVM should be added to EXTRA_DIST.
EXTRA_DIST contains a list of file or directory names that
should be distributed. For example, the top level Makefile
contains "EXTRA_DIST := test llvm.spec include". This means that
in addition to regular things that are distributed at the top
level (CREDITS.txt, LICENSE.txt, etc.) the distribution should
contain the entire test and include directories as well as the
llvm.spec file.
2. dist-hook - this make target can be used to alter the content of
the distribution directory. For example, in the top level
Makefile there is some logic to eliminate files in the include
subtree that are generated by the configure script. These should
not be distributed. Similarly, any dist-hook target found in any
directory can add or remove or modify things just before it gets
packaged. Any transformation is permitted. Generally, not much
is needed.
You will see various messages if things go wrong:
1. During the copying process, any files that are missing will be
flagged with: "===== WARNING: Distribution Source 'dir/file' Not
Found!" These must be corrected by either adding the file or
removing it from EXTRA_DIST.
2. If you build the distribution with VERBOSE=1, then you might
also see: "Skipping non-existent 'dir/file'" in certain cases
where its okay to skip the file.
3. The target can fail if any of the things it does fail .. error
messages should indicate what went wrong.
dist
This target does exactly what distdir target does, but also includes
assembling the tarballs. There are actually four related targets here:
* dist-gzip: package the gzipped distribution tar file. The
distribution directory is packaged into a single file ending
in .tar.gz which is gzip compressed.
* dist-bzip2: package the bzip2 distribution tar file. The
distribution directory is packaged into a single file ending
in .tar.bzip2 which is bzip2 compressed.
* dist-zip: package the zip distribution file. The distribution
directory is packaged into a single file ending in .zip which is
zip compressed.
* dist: does all three, dist-gzip, dist-bzip2, dist-zip
dist-check
This target checks the distribution. The basic idea is that it unpacks
the distribution tarball and ensures that it can build. It takes the
following actions:
1. It depends on the dist-gzip target which, if it hasn't already
been built, builds the gzip tar bundle (see dist and distdir
above)
2. removes any pre-existing _distcheckdir at the top level.
3. creates a new _distcheckdir directory at the top level
4. creates a build subdirectory and an install subdirectory under
_distcheckdir
5. unzips and untars the release tarball into _distcheckdir ,
creating LLVM-1.7 directory (from the tarball)
6. in the build subdirectory, it configures with appropriate
options to build from the unpacked source tarball into the build
directory with installation in the install directory
7. runs make all
8. runs make check
9. runs make install
10. runs make uninstall
11. runs make dist
12. runs make clean
13. runs make dist-clean
If it can pass all that, the distribution will be deemed distribution
worthy and you will see:
===== LLVM-1.7.tar.gz Ready For Distribution =====
This means the tarball should then be tested on other platforms and have
the nightly test run against it. If those all pass, THEN its ready for
distribution.
A note about disk space: using "dist-check" will easily triple the
amount of disk space your build tree is using. Might want to check
available space before you being :)
dist-clean
In addition to doing a normal "clean", this target will clean up the
files and directories created by the distribution targets. In particular
the distribution directory (LLVM-X.X), check directory (_distcheckdir),
and the various tarballs will be removed. You do this after the release
has shipped and you no longer need this stuff in your build tree.
I hope this brief tutorial has helped. Feel free to ask questions.
Further details can be found in Makefile.rules.
Reid.
On Tue, 2006-04-11 at 13:07 -0500, Tanya Lattner wrote:
> Reid,
>
> Could you explain in detail what make dist does? :) I'd like to see how it
> can be integrated into the release process.
>
> Thanks,
> Tanya
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20060411/ff7e44b0/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20060411/ff7e44b0/attachment.sig>
More information about the llvm-dev
mailing list