[lld] r287042 - Use one task per iteration in parallel_for_loop.

[Rui Ueyama via llvm-commits]

I think it is not beautiful to take care of granularity of tasks on caller
side, and it can be resolved with this. It works for me. What do you think?

template <class Iterator, class Func>
void parallel_for_each(Iterator begin, Iterator end, Func func) {
  ptrdiff_t taskSize = std::distance(begin, end) / 1024;
  if (taskSize == 0)
    taskSize = 1;

  TaskGroup tg;
  while (taskSize <= std::distance(begin, end)) {
    tg.spawn([=, &func] { std::for_each(begin, begin + taskSize, func); });
    begin += taskSize;
  }
  std::for_each(begin, end, func);
}

On Wed, Nov 16, 2016 at 5:47 AM, Rafael Espíndola <
rafael.espindola at gmail.com> wrote:

> > If you don't want to revert it, how about this.
> >
> > The problem in the original code is the task size is fixed to 1024. We
> can
> > make it adaptive to the size of the input, so that we will always have
> > reasonable number of tasks.
>
> So, there is quite a bit of work to be done if we get serious about
> threads. We have to investigate why the pool executor has such a high
> overhead and figure out the right way to split work so that each
> thread is not stepping over each other. We should very likely also
> create one thread per core, not one per SMT.
>
> For example of possible improvement, when outputting the file it is
> probably profitable to make write do nothing but write and partition
> the input sections over all the file so that each thread can allocate
> local memory, relocate there and then write its work to the correct
> output offset with a single contiguous write call.
>
> So in general finding the correct granularity is something that I
> think should be explicitly done in the caller.
>
> Given that we still have a lot of work before threading becomes a
> priority, how about the attached compromise. It just writes each
> output thread in parallel. In my testcase it brings the linker back to
> the previous performance when not using --block-id.
>
> Cheers,
> Rafael
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20161116/1cae7fa7/attachment.html>