[PATCH] D24185: [lit] Fix discovery of GTest unit tests

[Diana Picus via llvm-commits]

rovka created this revision.
rovka added a reviewer: ddunbar.
rovka added a subscriber: llvm-commits.

During test discovery, we invoke each test executable with --gtest_list_tests.
For this we use subprocess.Popen, which according to the Python docs only raises
exceptions when called with invalid arguments, or when the child process raises
an exception before the new program starts to execute. If the invocation returns
a non-zero code, we silently assume that it has succeeded and there are no tests
to be discovered there.

This happens when building with BUILD_SHARED_LIBS - due to a mishap in CMake, we
set the wrong rpath for tests that live 2 directories away from unittest/. When
invoking those executables with --gtest_list_tests, they fail to find their
libraries and error out. Lit just assumes everything went well and we end up
unknowingly running fewer tests in this configuration (~1700 instead of ~1800).

We can fix this by replacing the call to subprocess.Popen with
subprocess.check_output, so we get an exception when the child process returns
with a non-zero return code.


https://reviews.llvm.org/D24185

Files:
  utils/lit/lit/formats/googletest.py
  utils/lit/lit/util.py

Index: utils/lit/lit/util.py
===================================================================
--- utils/lit/lit/util.py
+++ utils/lit/lit/util.py
@@ -66,9 +66,7 @@
 def capture(args, env=None):
     """capture(command) - Run the given command (or argv list) in a shell and
     return the standard output."""
-    p = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
-                         env=env)
-    out,_ = p.communicate()
+    out = subprocess.check_output(args, env=env, stderr=subprocess.STDOUT)
     return convert_string(out)
 
 def which(command, paths = None):
Index: utils/lit/lit/formats/googletest.py
===================================================================
--- utils/lit/lit/formats/googletest.py
+++ utils/lit/lit/formats/googletest.py
@@ -1,6 +1,7 @@
 from __future__ import absolute_import
 import os
 import sys
+from subprocess import CalledProcessError
 
 import lit.Test
 import lit.TestRunner
@@ -34,6 +35,9 @@
             if kIsWindows:
               lines = lines.replace('\r', '')
             lines = lines.split('\n')
+        except CalledProcessError as exception:
+            litConfig.error(exception.output)
+            raise StopIteration
         except:
             litConfig.error("unable to discover google-tests in %r" % path)
             raise StopIteration


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D24185.70148.patch
Type: text/x-patch
Size: 1344 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160902/e0faca13/attachment.bin>