[LLVMdev] UPDATE: Automake Difficulties (Long)

[Alkis Evlogimenos]

On Thursday 21 October 2004 01:54, Vladimir Prus wrote:
> On Wednesday 20 October 2004 12:01, Reid Spencer wrote:
> > I'm re-thinking my penchant for automake. automake is great for many
> > standard applications that just need to get portable makefiles up and
> > running quickly.  However, it turns out that LLVM is "different enough"
> > from a standard application that automake might not be the best choice.
>
> I might just here to suggest that Boost.Build
> (http://boost.org/boost-build2) might be good.
>
> > Here's some of the problems I've run into:
> >
> > 1. There's no way to tell automake to build bytecode. Without a lot of
> > customization of automake itself, it just can't grok the fact that there
> > might be multiple ways to compile a c/c++ program producing different
> > results and requiring different tools.
>
> Boost.Build supports this very good. In particular, on my project on work I
> can run:
>
>    bjam toolset=gcc
>    bjam toolset=nm
>    bjam toolset=nmm
>
> and the first will do a regular build, the third will compile for some
> embedded processor, and the third will pass the sources via some annotation
> tool and produce an annotated x86 binary.
>
> Besides, "toolset=gcc" can become "toolset=msvc" on windows, and it will
> have high chances of working.

I don't think Reid was referring to the use of different compilers on 
(possibly) different platforms. If you look at runtime/GC/SemiSpace: those 
files are not compiled using the compiler used to compile the rest of the 
project (llc, lli and so on). They use the llvm-gcc-frontend to compile and 
link those libraries into llvm bytecode. The resulting files end up in 
runtime/GC/SemiSpace/BytecodeObj

> > 2. The entire llvm-test project would have to either be completely
> > rethought or stay the way it is, it just can't be easily automake'd
> > because it doesn't follow the automake pattern.
> >
> > 3. The llvm/test directory would require significant rework to get it to
> > automake correctly (via dejagnu).
>
> No comments on the two points above yet.

Any ideas if the boost testing framework can apply to these two points?

> > 4. There's no way to avoid listing all the sources for every library
> > (I've tried several alternatives with no good results).
>
> Easy, I've already have a half-working setup (only the targets I use are
> converted), and, for example,
>
>    lib/Analysis/DataStructure/Jamfile
>
> has nothing but:
>
>    llvm-lib datastructure ;
>
> > 5. There's no way to install bytecode libraries somewhere separately
> > from other libraries.
>
> Should be possible.
>
> > 6. Creating a distribution tarball would require additional Makefile.am
> > files to be inserted in all the directories under llvm/include, the
> > traversal of which would cause additional overhead for every non-dist
> > target. That's 17 gratuitous "make" processes per build.  There doesn't
> > seem to be a way around this.
>
> Why distribution tarball has anything to do with build system?

This is done in automake. Because the distribution consists of files processed 
by automake, the "distcheck" target makes sure that those files are in the 
distribution and the distribution tarball can build properly without needing 
automake/autoconf.

> > 7. I can't get automake to stop doing a double configure. That is, when
> > you configure all the Makefiles are updated. You then go do a build and
> > it thinks it needs to reconfigure again so it does it twice. This is
> > just a waste of time (40 seconds on my machine).
>
> Ehm.... I'd say that configure should be orthogonal to building. No?

I totally agree with this as well.

> > 8. automake's notion of buiding a library is very fixed. It basically
> > supports two things: libtool built shared libraries (linking) and ar
> > built libraries that are also run through ranlib.  While its possible to
> > override AR and LINK, there isn't a way to override RANLIB on a
> > per-target basis so you can't build both a regular library (requiring
> > RANLIB=ranlib) and a bytecode library (requiring something like
> > RANLIB=true) in the same directory. We have in LLVM at least 4 ways to
> > build libraries: regular .a, pre-linked .o (combination of .o files),
> > shared-library, and bytecode-library. While I figured out a hack to do
> > pre-linked .o, it basically uses GNU Make, not automake to make it work
> > and therefore breaks automake's "make" portability.
>
> I think given Boost.Build's "properties" (like "toolset" above), this
> should be doable.
>
> > 1. Get dependency generation in a single pass like automake. This will
> > give us about a 30% speed up on builds .. and one of the main reasons
> > for moving to automake is taken away.  Estimate: 1-2 hours.
>
> Well, Boost.Build is single pass already. Need to compare performance on
> LLVM, though.
>
> Basically, I can try to finish my attempt to add Boost.Build files to LLVM.
> But I wonder if that makes sense? Is it already decided to just improve the
> current system?

I don't think it is set on stone to keep the current system. We are always 
open to new ideas and if they improve things we will most likely adopt them. 
I would say you should proceed on finishing what you have and we can test it 
and see how it compares with the current system.

Just for clarification: the only requirement for Boost.Build is bjam itself, 
right?

> Is anything with "Boost" in name will be rejected right away?

Absolutely not! We were using one of the boost libraries before and we 
replaced it because it was easy to do and it removed one external dependency 
for us, not because it had the "Boost" in name :-)

-- 

Alkis