[LLVMdev] RFC: Upcoming Build System Changes

[David A. Greene]

Óscar Fuentes <ofv at wanadoo.es> writes:

>> Chris is absolutely on-target as to why the current build is slow.  It's
>> slow because recursive make hides the parallelism.  It hides the
>> parallelism because it hides the dependencies.  There is no way to get
>> around that problem with a recursive make build system.
>
> You keep repeating that and I say that it is wrong. Can you mention a
> serialization point on the LLVM build caused by recursive make?
> (GenLibDeps is not such example, as described on a previous message.)

The fact that the LLVM has to run through all of the directories, read
Makefiles, check dependencies in each Makefile, etc.  In essence, a
recursive make adds implicit dependencies on all of the sub-Makefiles.
Those Makefiles have to be processed before any real work can begin.
That includes shell overhead, which can be significant.

That's just one example.

Another is the artificial barriers to work stealing that recursive make
imposes.  I believe that once a group of threads is assigned to do a
sub-make, that group of threads is tied up until the sub-make is
finished, which means there are always a few idle threads at the end of
each sub-make.  In a non-recursive build, idle threads can go work on
something else immediately.  This could probably be fixed via GNU make's
-j pipe communication with sub-makes but I'm not sure it has been.

                               -Dave