[PATCH] D17609: [lit] Enqueue tests on a separate thread to not hit limits on parallel queues

[Sean Silva via llvm-commits]

On Fri, Mar 11, 2016 at 7:53 AM, Filipe Cabecinhas <
filcab+llvm.phabricator at gmail.com> wrote:

> filcab added a subscriber: filcab.
> filcab added a comment.
>
> No bug, and there's no specific limit in the docs (well, you can see
> how it's implemented and check the limits of the Queue's innards).
>
> multiprocessing.Queue is allowed to behave like this, though:
>
> https://docs.python.org/2/library/multiprocessing.html#multiprocessing.Queue.put
>
> "put(obj[, block[, timeout]])
> Put obj into the queue. If the optional argument block is True (the
> default) and timeout is None (the default), block if necessary until a
> free slot is available."
>
> Do you want me to add this to the commit message? I don't think it's
> the most useful thing to have in a comment there, reminding people to
> look at the docs. :-)
>

I think it makes sense to have it in a comment, because it is a non-obvious
aspect of the API which is affecting how the code needs to be structured.
Probably worth pointing out in the commit message why this seemed to work
until lately: the actual limit at which it blocks that we have seen in
practice in multiprocessing.Queue.put is on mac due to the 2^15 semaphore
up limit.

-- Sean Silva


>
> Thank you,
>
>   Filipe
>
>
> http://reviews.llvm.org/D17609
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160311/02860f78/attachment.html>