[lldb-dev] Parallelizing loading of shared libraries

[Scott Smith via lldb-dev]

After a dealing with a bunch of microoptimizations, I'm back to
parallelizing loading of shared modules.  My naive approach was to just
create a new thread per shared library.  I have a feeling some users may
not like that; I think I read an email from someone who has thousands of
shared libraries.  That's a lot of threads :-)

The problem is loading a shared library can cause downstream
parallelization through TaskPool.  I can't then also have the loading of a
shared library itself go through TaskPool, as that could cause a deadlock -
if all the worker threads are waiting on work that TaskPool needs to run on
a worker thread.... then nothing will happen.

Three possible solutions:

1. Remove the notion of a single global TaskPool, but instead have a static
pool at each callsite that wants it.  That way multiple paths into the same
code would share the same pool, but different places in the code would have
their own pool.

2. Change the wait code for TaskRunner to note whether it is already on a
TaskPool thread, and if so, spawn another one.  However, I don't think that
fully solves the issue of having too many threads loading shared libraries,
as there is no guarantee the new worker would work on the "deepest" work.
I suppose each task would be annotated with depth, and the work could be
sorted in TaskPool though...

3. Leave a separate thread per shared library.

Thoughts?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20170426/551b67d0/attachment.html>