[LNT] r303244 - async_ops.py: only set up worker processes once

[James Molloy via llvm-commits]

Author: jamesm
Date: Wed May 17 04:38:55 2017
New Revision: 303244

URL: http://llvm.org/viewvc/llvm-project?rev=303244&view=rev
Log:
async_ops.py: only set up worker processes once

This was obviously the intended behaviour but since WORKERS never got set to anything we ended up creating new processes for every async process. This is all well and good until one async process outlasts a newer one, in which case the log buffer object it writes to has been destroyed in the parent and multiprocessing madness ensues.

Fixes PR33067.


Modified:
    lnt/trunk/lnt/util/async_ops.py

Modified: lnt/trunk/lnt/util/async_ops.py
URL: http://llvm.org/viewvc/llvm-project/lnt/trunk/lnt/util/async_ops.py?rev=303244&r1=303243&r2=303244&view=diff
==============================================================================
--- lnt/trunk/lnt/util/async_ops.py (original)
+++ lnt/trunk/lnt/util/async_ops.py Wed May 17 04:38:55 2017
@@ -26,6 +26,7 @@ from threading import Lock
 from lnt.testing.util.commands import note, warning, timed, error
 NUM_WORKERS = 4  # The number of subprocesses to spawn per LNT process.
 WORKERS = None  # The worker pool.
+WORKERS_LOCK = Lock()
 
 JOBS = []
 
@@ -33,18 +34,23 @@ JOBS = []
 def launch_workers():
     """Make sure we have a worker pool ready to queue."""
     global WORKERS
-    if not WORKERS:
-        note("Starting workers")
-        manager = Manager()
-        try:
-            current_app.config['mem_logger'].buffer = \
-                manager.list(current_app.config['mem_logger'].buffer)
-        except RuntimeError:
-            #  It might be the case that we are not running in the app.
-            #  In this case, don't bother memory logging, stdout should
-            #  sufficient for console mode.
-            pass
-
+    global WORKERS_LOCK
+    WORKERS_LOCK.acquire()
+    try:
+        if not WORKERS:
+            note("Starting workers")
+            manager = Manager()
+            WORKERS = True
+            try:
+                current_app.config['mem_logger'].buffer = \
+                    manager.list(current_app.config['mem_logger'].buffer)
+            except RuntimeError:
+                #  It might be the case that we are not running in the app.
+                #  In this case, don't bother memory logging, stdout should
+                #  sufficient for console mode.
+                pass
+    finally:
+        WORKERS_LOCK.release()
 
 def sig_handler(signo, frame):
     cleanup()