[PATCH] D27247: Parallelize ICF to make LLD's ICF really fast.

[Rui Ueyama via Phabricator via llvm-commits]

ruiu added a comment.

> Is the large (23) number of iterations due to slowness of propagating identicalness across references (one level of references per iteration currently) or due to ICF<ELFT>::segregate only being able to split into two at each iteration? (or a combination of both?). Here are two ideas for reducing the number of iterations:

Very good question. Single-threaded iteration with a single GroupId for each InputSection took 11 iterations, so this patch slows down single threaded execution. The main loop gets faster as we have less number of input sections to mutate, but still executing an empty iteration takes ~100ms. I made a change to code to optimize for that case. (I could optimize it even more if I added more code for single thread, but I think in practice this should be enough.)

> do some sort of topological sorting (even approximate) and then do partial iterations which only sort only part of the array. (more generally, avoid revisiting sections that are unlikely to change this iteration). This can speed up the convergence since we avoid wasting work on nodes that won't change.

Interesting idea. I could imagine that we might be able to compute strongly connected component and toposort them to do some sort of optimization. I wouldn't do that in this patch, but that's probably worth trying.

> make the "equal" comparison actually be a "less". That will allow ICF<ELFT>::segregate to sort instead of partition, which allows it to generate multiple ranges at a time.

This is very interesting idea too. I'll try to do that after submit this patch.



================
Comment at: ELF/InputSection.h:292
   // Used by ICF.
-  uint64_t GroupId = 0;
+  uint64_t GroupId[2] = {0, 0};
 
----------------
silvas wrote:
> Would
> 
> ```
> struct {
>   uint64_t Current;
>   uint64_t Next;
> } GroupId;
> ```
> 
> be a bit better?
I'm using GroupId[0] and GroupId[1] alternately, so I can't make that change.


https://reviews.llvm.org/D27247