[lldb-dev] possible test runner speedup

[Todd Fiala via lldb-dev]

Hi all,

I've started looking into test runner speeds based on the number of
threads.  Since we first introduced the parallel test runner, we've
defaulted to using a number of work queues equal to the number of logical
processors reported by the machine.  Thus on a 24 core Linux box (12
hyperthreaded cores), you would get 24 work queues if you didn't specify
anything differently.  Similarly, on a typical MacBook Pro on OS X, you'll
get 8 worker queues.

There was no magic to say that this was a good default.  It was just a
reasonable place to start.  And when I first played with it, I had a 24
core machine as my daily setup, which took something like 2.5 minutes to
run all the tests.

But, on the lower end of the CPU spectrum, I end up taking more like 10-ish
minutes on an 8 core MBP.  Hence some measurements:

|-----------+--------------+-----------------------+------------------------|
| # workers | time         | % slower than fastest |
 |
|-----------+--------------+-----------------------+------------------------|
|         8 | 10m 7s 580ms |                 26.5% | ** this is the default
|
|        13 | 8m 29s 590ms |                  6.1% |
 |
|        16 | 8m 0s 160ms  |                  0.0% |
 |
|        19 | 8m 31s 780ms |                  6.6% |
 |
|-----------+--------------+-----------------------+------------------------|

The net of it is that roughly 2x the current number of work queues on the
MBPs yielded a fastest wall clock time to getting the tests done.  The
limiting factor at this point is the timeouts, which we may want to trim.

I played with it some on a 24 logical core Linux box.  This also did well
with 48 worker queues.  Again the limiting factor seems to be the timeouts,
and made it harder to see how much faster it went (the timeout really got
in the way here).  But it certainly wasn't slower.

Just wanted to pass that along.  Moral of the story is that settings
--threads 2xnum_logical_cores may get you a win.

I may up the default in the nearish future if I don't hear reports of pain
and agony on trials with that.

Thanks!
-- 
-Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20150927/963a510d/attachment.html>