[cfe-dev] RFC: Upcoming Build System Changes

[Óscar Fuentes]

greened at obbligato.org (David A. Greene) writes:

> Óscar Fuentes <ofv at wanadoo.es> writes:
>
>> Chris Lattner <clattner at apple.com> writes:
>>
>>> There are different overheads in different scenarios.  The makefiles
>>> get really poor utilization out of a 8 or 16-way machine because of
>>> implicit synchronization between different sublibraries.
>>
>> BTW, adding explicit library dependencies will make the parallel builds
>> worse, 
>
> No they won't, for exactly the reason Chris stated.

That's makes no sense. The automatic system we had generated an optimal
dependency graph: no missing or unnecesary edges. You can't do better
than than optimal, can you?. Furthermore, while building static
libraries, those didn't block one another, because there was no
dependency declarations among them. I don't know how the current system
works, but the clang build, which always used explicit dependencies,
created unnecessary block points among libraries because of that.

>> because you know when a dependency is missing (the build fails)
>> but you don't know when a dependency is superfluous.
>
> That's true but is not a show-stopper IME.  It's easier to remove
> superfluous dependencies than try to track down a missing one in a
> recursive make build.

It's quite the opposite. You'll not notice superfluous dependencies
unless you look for them. BTW, long time ago I scanned the clang library
dependencies (which used an explicit scheme all along, as mentioned
above) by comparing them with the output of GenLibDeps.pl (once tweaked
for managing the clang libs.) There was a few unnecesary/redundant
dependencies, on a set of libraries that is very small compared to LLVM
(keep in mind that the number of edges on the dependency graph grows
much faster than linear.)

To think that no unnecesary dependencies will pop up on the build is
delusional, unless you introduce some sort of automatic check. And once
you have the automatic check, why not delegate to it the handling of the
dependencies?

As mentioned elsewhere, what we want is to know when a source code
change creates a modification on the dependency graph, to decide if we
are okay with that. Manually keeping up to date the dependency files is
just an unnecessary bureaucratic burden.