[PATCH] [Lit] Use multiprocessing instead of threading

[Daniel Dunbar]

On Thu, Oct 24, 2013 at 4:50 PM, Sean Silva <silvas at purdue.edu> wrote:

>
>
>
> On Wed, Oct 23, 2013 at 3:44 PM, Rafael Espíndola <
> rafael.espindola at gmail.com> wrote:
>
>> On 23 October 2013 00:26, Daniel Dunbar <daniel at zuster.org> wrote:
>> > I wanted to have a little bit of time to check out how things were on
>> > Windows, but I haven't found it.
>> >
>> > Off the top of my head, I think that was the main thing, other than also
>> > just giving it a little bake time.
>> >
>> > We could go ahead and switch the default for non-Windows and see what
>> > happens if you are interested...
>>
>> Probably a good idea. With check-all the time goes from  1m10.644s to
>> 0m50.506s in my machine :-)
>>
>
> Wow, that means that 20s of the check-all time were just lit overhead,
> which is insane.
>

No, it doesn't, it means that 20s of check-all time could be saved by
increased parallelism.

The lit overhead is fairly small, you can check it by running with
--no-execute. On my machine for just LLVM, lit can "run" all the tests in
<3s if it doesn't actually execute them -- this includes the time for
discovery, and reading and parsing every single test script. It takes < .5s
to handle just discovery and processing of all the tests without reading
and parsing the test scripts, so you can see that most of the time is in
parsing the "ShTest" format, but even that is a small percentage of the
overall testing time.

 - Daniel

Can you estimate the total lit overhead after switching to multiprocessing?
> E.g., can we gain another 20s of check-all speed by reducing lit overhead?
>
> -- Sean Silva
>
>
>>
>> Cheers,
>> Rafael
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20131024/4b5db6fa/attachment.html>