[PATCH] D38266: Parallelize string merging.

[Rui Ueyama via llvm-commits]

Actually I just came up with a different and better idea. I'll upload a
patch shortly.

On Thu, Sep 28, 2017 at 7:02 PM, Rafael Espíndola <
rafael.espindola at gmail.com> wrote:

> The results are attached.
>
> scylla is probably the most interesting. Without threads it gets 1.11x
> slower. With 8 cores it is 1.45x faster.
>
> So this is probably fine, but what do you think of the idea of sorting
> by hash to have a reproducible output with any number of shards?
>
> Cheers,
> Rafael
>
> On 28 September 2017 at 15:42, Rafael Avila de Espindola
> <rafael.espindola at gmail.com> wrote:
> > Rafael Avila de Espindola <rafael.espindola at gmail.com> writes:
> >
> >>> +  // Parallelism. Changing this number causes benign changes in the
> >>> +  // order of output section pieces. For build reproducibility, we
> >>> +  // always use the same number.
> >>> +  static constexpr size_t NumShards = 8;
> >>
> >> One idea that might allow using a variable number of shards:
> >>
> >> Sort the strings once they are know to be unique. The sort order can be
> >> based on the hash and look at the string itself only if two hashes are
> >> identical, so it shouldn't be too slow.
> >
> > In fact, we could use the N most significant bits of the hash to do the
> > sharding. That way we only need to sort inside each shard.
> >
> > Cheers,
> > Rafael
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20170928/7bb93212/attachment-0001.html>