[cfe-dev] [LLVMdev] RFC: Upcoming Build System Changes

[Óscar Fuentes]

greened at obbligato.org (David A. Greene) writes:

> Ok, here are some hard numbers for empty builds:
>
> LLVM empty build:
> /usr/bin/time make -j16
> 4.32user 2.47system 0:03.21elapsed 211%CPU (0avgtext+0avgdata 13376maxresident)k
> 0inputs+0outputs (0major+671804minor)pagefaults 0swaps

So your 16-way machine takes 2.5 times more than my 4-way cheap desktop...

> Cray empty build:
> /usr/bin/time make dynamic-developer -j16
> 2.88user 1.06system 0:04.94elapsed 79%CPU (0avgtext+0avgdata 64208maxresident)k
> 0inputs+0outputs (0major+184605minor)pagefaults 0swaps
>
> The Cray empty build includes some really horrendous shell script
> trickery known as "modules."  You can read the gory details here:
>
> http://modules.sourceforge.net/
>
> It's possibly the dumbest thing ever invented.  But that's what I've got
> to work with.
>
> The point is, the Cray build includes some major shell overhead that the
> LLVM build doesn't have to deal with.  And the Cray build has more
> targets to build.  And the Cray build has more dependencies to examine.
> And the Cray build still blows the socks off the LLVM build.

... and you hypothesize that the shell work adds substantial time to the
Cray build, thus making even more dramatic the difference on the
LLVM-Cray build systems.

> Here is actual data comparing an empty LLVM build done recursively (the
> LLVM build) and non-recursively (the Cray build).
>
> See this?
>
> 0inputs+0outputs (0major+671804minor)pagefaults 0swaps
>
> vs. this?
>
> 0inputs+0outputs (0major+184605minor)pagefaults 0swaps
>
> That's I/O.

Let me guess: you are starting from a cold cache. Other than a
difference caused by the OS, that's the only thing that could explain
it.

The format used by your `time' is not familiar to me, so excuse me if I
make a question related to your measures:

LLVM: 4.32user 2.47system 0:03.21elapsed 211%CPU
Cray: 2.88user 1.06system 0:04.94elapsed 79%CPU

Looks like the Cray build is using less CPU but takes 1.7 seconds more
than LLVM of wall clock time. But then you note that the LLVM build uses
a lot more I/O. How can this be? It looks as if the LLVM build were much
more parallel. Is this explained because the shell scripts you mention
above introduce serialization points?

And the decisive question: is 3.2 seconds considered an annoying amount
of time for you deciding to support a substantial change on the LLVM
build, but 4.94 seconds is not annoying enough to get rid of those shell
scripts? :-)