[PATCH] Add preload method.

[Rui Ueyama]

On Fri, Sep 12, 2014 at 4:22 PM, kledzik at apple.com <kledzik at apple.com>
wrote:

> I assume that preload() return immediately, and that it is expected to
> spin off some thread to parse an archive member?  If so, we have no overall
> throttle on how many threads will be started (a hundred undefines could
> spin up 100 threads).  Also, how is the archive reader to coordinate if the
> Resolver gets to the point it really wants an object file to fulfill and
> undefine but some other thread is busy parsing that member?
>

We should throttle on how many thread will be started using a tasks
scheduler. The global scheduler should restrict the number of tasks less
than some number that varies depending on the number of available cores,
and that should also reuse (kernel) threads.

We also should make a rendezvous point at where the main thread wait for
the speculative load task if they are trying to parse the same file. The
easiest way to do that is probably representing a file begin parsed as a
future. A pre-parse task first create a future and sets the value to it
when it's done. The main thread is blocked if the result is not yet
available.

Don't we just have a producer/consumer problem where the archive reader is
> the producer and the resolver is the consumer.  The consumer is single
> threaded and currently queries (pulls from) the producer on the consumer
> thread.  Can the driver start up some producer task for archive reading to
> pre-parse archives?  Rather than blindly parsing all members, your idea of
> passing undefined symbols names to the producer is a good idea.
>

The driver could trigger pre-parsing archives, but I'm afraid it's
inefficient, as you might have implied. The driver does not know anything
about symbols, so it has no idea which files in an archive file is going to
be used. If most of the files in an archive file are going to be used, we
could blindly parse all of them, but I think that's not the case. Only the
resolver can make a good guess.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20140912/bbb8c079/attachment.html>