[llvm] r265034 - [lit][googletest] Handle upstream gtest output

[Daniel Dunbar via llvm-commits]

Author: ddunbar
Date: Thu Mar 31 13:22:55 2016
New Revision: 265034

URL: http://llvm.org/viewvc/llvm-project?rev=265034&view=rev
Log:
[lit][googletest] Handle upstream gtest output

Summary:
Upstream googletest prints "Running main() from gtest_main.cc" to stdout prior
to running tests. LLVM removed that print statement in r61540. If a user were
to use lit to run tests that use upstream googletest, however, lit
reports "Running main()" as an invalid test name.

To avoid such a failure, add an extra conditional to `formats/googletest.py`.
Also add tests to demonstrate the modified behavior.

Reviewers: abdulras, ddunbar

Subscribers: ddunbar, llvm-commits, kastiglione

Differential Revision: http://reviews.llvm.org/D18606

Added:
    llvm/trunk/utils/lit/tests/Inputs/googletest-upstream-format/
    llvm/trunk/utils/lit/tests/Inputs/googletest-upstream-format/DummySubDir/
    llvm/trunk/utils/lit/tests/Inputs/googletest-upstream-format/DummySubDir/OneTest   (with props)
    llvm/trunk/utils/lit/tests/Inputs/googletest-upstream-format/lit.cfg
    llvm/trunk/utils/lit/tests/googletest-upstream-format.py
Modified:
    llvm/trunk/utils/lit/lit/formats/googletest.py

Modified: llvm/trunk/utils/lit/lit/formats/googletest.py
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/lit/formats/googletest.py?rev=265034&r1=265033&r2=265034&view=diff
==============================================================================
--- llvm/trunk/utils/lit/lit/formats/googletest.py (original)
+++ llvm/trunk/utils/lit/lit/formats/googletest.py Thu Mar 31 13:22:55 2016
@@ -43,6 +43,12 @@ class GoogleTest(TestFormat):
             if not ln.strip():
                 continue
 
+            if 'Running main() from gtest_main.cc' in ln:
+                # Upstream googletest prints this to stdout prior to running
+                # tests. LLVM removed that print statement in r61540, but we
+                # handle it here in case upstream googletest is being used.
+                continue
+
             prefix = ''
             index = 0
             while ln[index*2:index*2+2] == '  ':

Added: llvm/trunk/utils/lit/tests/Inputs/googletest-upstream-format/DummySubDir/OneTest
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/tests/Inputs/googletest-upstream-format/DummySubDir/OneTest?rev=265034&view=auto
==============================================================================
--- llvm/trunk/utils/lit/tests/Inputs/googletest-upstream-format/DummySubDir/OneTest (added)
+++ llvm/trunk/utils/lit/tests/Inputs/googletest-upstream-format/DummySubDir/OneTest Thu Mar 31 13:22:55 2016
@@ -0,0 +1,38 @@
+#!/usr/bin/env python
+
+import sys
+
+if len(sys.argv) != 2:
+    raise ValueError("unexpected number of args")
+
+if sys.argv[1] == "--gtest_list_tests":
+    print("""\
+Running main() from gtest_main.cc
+FirstTest.
+  subTestA
+  subTestB
+ParameterizedTest/0.
+  subTest
+ParameterizedTest/1.
+  subTest""")
+    sys.exit(0)
+elif not sys.argv[1].startswith("--gtest_filter="):
+    raise ValueError("unexpected argument: %r" % (sys.argv[1]))
+
+test_name = sys.argv[1].split('=',1)[1]
+print('Running main() from gtest_main.cc')
+if test_name == 'FirstTest.subTestA':
+    print('I am subTest A, I PASS')
+    print('[  PASSED  ] 1 test.')
+    sys.exit(0)
+elif test_name == 'FirstTest.subTestB':
+    print('I am subTest B, I FAIL')
+    print('And I have two lines of output')
+    sys.exit(1)
+elif test_name in ('ParameterizedTest/0.subTest',
+                   'ParameterizedTest/1.subTest'):
+    print('I am a parameterized test, I also PASS')
+    print('[  PASSED  ] 1 test.')
+    sys.exit(0)
+else:
+    raise SystemExit("error: invalid test name: %r" % (test_name,))

Propchange: llvm/trunk/utils/lit/tests/Inputs/googletest-upstream-format/DummySubDir/OneTest
------------------------------------------------------------------------------
    svn:executable = *

Added: llvm/trunk/utils/lit/tests/Inputs/googletest-upstream-format/lit.cfg
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/tests/Inputs/googletest-upstream-format/lit.cfg?rev=265034&view=auto
==============================================================================
--- llvm/trunk/utils/lit/tests/Inputs/googletest-upstream-format/lit.cfg (added)
+++ llvm/trunk/utils/lit/tests/Inputs/googletest-upstream-format/lit.cfg Thu Mar 31 13:22:55 2016
@@ -0,0 +1,3 @@
+import lit.formats
+config.name = 'googletest-upstream-format'
+config.test_format = lit.formats.GoogleTest('DummySubDir', 'Test')

Added: llvm/trunk/utils/lit/tests/googletest-upstream-format.py
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/tests/googletest-upstream-format.py?rev=265034&view=auto
==============================================================================
--- llvm/trunk/utils/lit/tests/googletest-upstream-format.py (added)
+++ llvm/trunk/utils/lit/tests/googletest-upstream-format.py Thu Mar 31 13:22:55 2016
@@ -0,0 +1,20 @@
+# Check the various features of the GoogleTest format.
+#
+# RUN: not %{lit} -j 1 -v %{inputs}/googletest-upstream-format > %t.out
+# RUN: FileCheck < %t.out %s
+#
+# END.
+
+# CHECK: -- Testing:
+# CHECK: PASS: googletest-upstream-format :: DummySubDir/OneTest/FirstTest.subTestA
+# CHECK: FAIL: googletest-upstream-format :: DummySubDir/OneTest/FirstTest.subTestB
+# CHECK-NEXT: *** TEST 'googletest-upstream-format :: DummySubDir/OneTest/FirstTest.subTestB' FAILED ***
+# CHECK-NEXT: Running main() from gtest_main.cc
+# CHECK-NEXT: I am subTest B, I FAIL
+# CHECK-NEXT: And I have two lines of output
+# CHECK: ***
+# CHECK: PASS: googletest-upstream-format :: DummySubDir/OneTest/ParameterizedTest/0.subTest
+# CHECK: PASS: googletest-upstream-format :: DummySubDir/OneTest/ParameterizedTest/1.subTest
+# CHECK: Failing Tests (1)
+# CHECK: Expected Passes    : 3
+# CHECK: Unexpected Failures: 1