[PATCH] D22438: [llvm-profdata] Speed up merging by using a thread pool

[Xinliang David Li via llvm-commits]

On Sun, Jul 17, 2016 at 1:46 PM, Vedant Kumar <vsk at apple.com> wrote:

> vsk added a comment.
>
> In https://reviews.llvm.org/D22438#486320, @silvas wrote:
>
> > The way you arrived at sqrt(N) seems sketchy to me.
>
>
> I didn't consider the option of splitting the input by twos. You're right
> that it's possible to do better than T - 1 + (N/T) serial merges.
>
> > Assuming infinite parallelism, the first step performs N/2 merges and
> then you recurse on the N/2 merged profiles [...]. Which must work out to
> something like log(N) serial merges.
>
>
> In practice (dropping the assumption of infinite parallelism), I don't
> think we can get to log(N) serial merges. You have to at least visit each
> input once (using any one of T threads), so N/T serial operations looks
> like the lower bound.
>
> That said, your suggested approach seems better and I'll take a shot at it.
>
> In https://reviews.llvm.org/D22438#486320, @davidxl wrote:
>
> > There isn't any reduction of the number of merges compared with serial
> case. Say there are T threads, the total number of merges is: T * f(N/T) +
> T - 1, so f(N) is still N-1.
>
>
> No disagreement here. We will count the same (or higher) number of merges
> in total when using T threads. It's strictly meant as a speed improvement.
>
> > I suspect a much larger speed up can be obtained by unifying symbol
> table creation across profiles (that are generated from the same binary --
> as their symtab is identical). I was planning doing that but did not find
> time. If you can help with that, that will be a great win.
>
>
> That sounds like a great idea. Did you have in mind some sort of cache of
> InstrProfSymtabs which can optionally be passed to the raw profile reader?
> I can investigate that after this patch is done.
>
>
Yes, the idea is that after a raw profile reader is created, its symtab can
be saved (for sharing later). It can be retrieved later (using a key that
is specific to the binary that produces the profile), and passed to other
reader's creator method.

David


>
> https://reviews.llvm.org/D22438
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160717/285b5981/attachment.html>