[LLVMdev] Re: [llvm-commits] CVS: */Makefile.am

[John Criswell]

Reid Spencer wrote:
> On Sun, 2004-10-10, Misha Brukman asked "Why can't we use wildcards
> instead of listing all the sources" and then wrote in response to my
> reply:
> 
>>On Sun, Oct 10, 2004 at 04:40:48PM -0700, Reid Spencer wrote:
>>
>>>http://www.gnu.org/software/automake/manual/html_mono/automake.html#wildcards
>>>
>>>is your answer.
>>
>>I think their "answer" is lame; we have already sacrificed portability
>>because we are using enough of gmake-only functionality because we
>>cannot depend on the system "make" being anything useful, yet gmake is
>>portable, and hence, the "standard".
> 
> 
> I agree completely.
> 
> 
>>The thing about "distribute only what you list", well... maybe, but the
>>way "WE" do distribution is (I believe) checkout, build, test, make
>>clean, make dist.  That way, you *know* that what you're distributing
>>actually works.  But I'm not the distro person, that's John. :)

Typically, I do the following:

Checkout LLVM with release tag.
Make Distro.
Unpackage distro.
Build Distro.
Test Distro.

Yes, I'm paranoid.
:)

> 
> 
> Well, automake changes all that. Right now it takes John 1-2 weeks to do
> a "distro". automake can produce a VALIDATED distribution automatically
> with only CPU time involved. The "distcheck" target builds a
> distribution candidate, produces a tar.gz of it, unpacks it, rebuilds
> it, and if everything is okay, the tar.gz is good to go. This will find
> ALL KINDS of subtle errors and a big part of getting the distribution
> right is setting up the _SOURCES variables to list things correctly.

FWIW, I don't believe that the build system for LLVM proper is the 
problem.  I think the problems stem from the following:

o) The llvm-gcc binary distributions take awhile due to the following 
complications:
	o After building it, I put the build into a cfrontend directory and add 
copyright files and fixheader scripts.
	o For systems in which I don't want to deal with header file licensing 
issues (i.e. Solaris and MacOS X), I remove all of the system headers 
that GCC fixed when I built it.
	o Building llvm-gcc and LLVM is sort of a chicken and egg problem.  I 
have to build LLVM, then llvm-gcc, and then build llvm/runtime and 
install it into the llvm-gcc binary tree.  And if something gets fixed 
in LLVM, it might change the libraries created by llvm/runtime, so 
keeping everything in sync is kind of a pain.

o) Problems which don't affect day to day work typically don't get fixed 
until release time.  Despite many people's efforts to fix bugs 
immediently when they show up in the nightly tester, problems slip 
through because our day to day priorities don't allow us to fix every 
problem.

Part of that two weeks is one week worth of hurriedly fixing bugs that 
we've neglected (because for releases, we want everything that worked in 
the previous release to work in this current release).

Chris has recommended that we schedule some "cleaning up" time more 
regularly to relieve this burden.

o) We support 3 platforms in-house (one of which gets neglected), and 
each new platform that I can build increases the problems caused by the 
two things above.

I believe the two best things that we can do to shorten releases is the 
following:

1. Re-arrange the LLVM and LLVM-GCC builds to build in one fell swoop. 
I checkout out, I type make, and it all builds.  This requires moving 
parts of llvm/runtime into llvm-gcc, or making llvm-gcc an LLVM project, 
or some other approach.

2. Schedule some time at least a month before the release to start 
polishing LLVM and fix regressions that haven't already been tackled, 
per Chris's suggestion.

I don't believe automake will make releases go any faster because it 
doesn't solve the 2 problems listed above.  The LLVM core works pretty 
well; it's the interaction with llvm-gcc that is problematic.

That said, I'm not against automake.  I just want to point out what I 
think the release bottlenecks are.

-- John T.

-- 
*********************************************************************
* John T. Criswell                         Email: criswell at uiuc.edu *
* Research Programmer                                               *
* University of Illinois at Urbana-Champaign                        *
*                                                                   *
* "It's today!" said Piglet. "My favorite day," said Pooh.          *
*********************************************************************