[LLVMdev] [Proposal] Parallelize post-IPO stage.

[Diego Novillo]

On Fri, Jul 12, 2013 at 3:49 PM, Shuxin Yang <shuxin.llvm at gmail.com> wrote:

> 3. How to parallelize post-IPO stage
> ====================================
>
>   From 5k' high, the concept is very simple, just to
>    step 1).divide the merged IR into small pieces,
>    step 2).and compile each of this pieces independendly.
>    step 3) the objects of each piece are fed back to linker to are linked
>            into an executable, or a dynamic lib.

You seem to be describing GCC's strategy for whole program
optimization (http://gcc.gnu.org/wiki/LinkTimeOptimization).

What is a bit confusing in your description is that you seem to be
wanting to do more work after IPO is *done*.  If the optimizations are
done, then there is no need to parallelize anything.

In GCC, we have two parallel stages: the generation of bytecode (which
is naturally parallelized by your build system) and the final
optimization phase, which is parallelized by partitioning the
callgraph into disjoint subsets.  These subsets are then parallelized
by spawning the compiler on each of the partitions.

The only sequential part is the actual analysis (what we call WPA or
whole program analysis).  That is a single threaded phase that makes
all the optimization decisions, annotates the summaries on each
callgraph node, partitions the callgraph and then spawns all the
optimizers to work on each section of the callgraph.


Diego.