[lldb-dev] changing default test runner from multiprocessing-based to threading-based

[Zachary Turner via lldb-dev]

After our last discussion, I thought about it some more and there are at
least some problems with this.  The biggest problem is that with only a
single process, you are doing all tests from effectively a single instance
of LLDB.  There's a TestMultipleDebuggers.py for example, and whether or
not that test passes is equivalent to whether or not the test suite can
even work without dying horribly.  In other words, you are inherently
relying on multiple debuggers working to even run the test suite.

I don't know if that's a problem, but at the very least, it's kind of
unfortunate.  And of course the problem grows to other areas.  What other
things fail horribly when a single instance of LLDB is debugging 100
processes at the same time?

It's worth adding this as an alternate run mode, but I don't think we
should make it default until it's more battle-tested.

On Mon, Sep 21, 2015 at 12:49 PM Todd Fiala via lldb-dev <
lldb-dev at lists.llvm.org> wrote:

> Hi all,
>
> I'm considering changing the default lldb test runner from
> multiprocessing-based to threading-based.  Long ago I switched it from
> threading to multiprocessing.  The only reason I did this was because OS X
> was failing to allow more than one exec at a time in the worker threads -
> way down in the Python Global Interpreter Lock (GIL).  And, at the time, I
> didn't have the time to break out the test runner strategies.
>
> We have verified the threading-based issue is no longer manifesting on OS
> X 10.10 and 10.11 beta.  That being the case, I'd like to convert us back
> to being threading-based by default.  Specifically, this will have the same
> effect as doing the following:
> (non-Windows): --test-runner-name threading
> (Windows): --test-runner-name threading-pool
>
> There are a couple benefits here:
> 1. We'll remove a fork for creating the worker queues.  Each of those are
> just threads when using threading, rather than being forked processes.
> Depending on the underlying OS, a thread is typically cheaper.  Also, some
> of the inter-worker communication now becomes cheap intra-process
> communication instead of heavier multiprocessing constructs.
> 2. Debugging is a bit easier.  The worker queues make a lot of noise in
> 'ps aux'-style greps, and are a pain to debug relatively speaking vs. the
> threaded version.
>
> I'm not yet looking to remove the multiprocessing support.  It is likely
> I'll check the OS X version and default to the multiprocessing test runner
> if it wasn't explicitly specified and the OS X version is < 10.10 as I'm
> pretty sure I hit the issue on 10.9's python.
>
> Thoughts?
> --
> -Todd
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20150921/819fc140/attachment.html>