[PATCH] [zorg] Fix LitTestCommand unexpected test result when TIMEOUT is returned.

[Rick Foos]

Support for LIT tests returning TIMEOUT was added a while back.

If a LIT test returns TIMEOUT, LitTestCommand reports an Unexpected test result error.

lldb.tests test lldb 28 expected failures Unexpected test result output TIMEOUT 28 expected passes
    stdio
    TIMEOUT: LLDB::15-breakpoint-by-symbol-disable-no-hit

with this fix, A count of timeouts and timeout reason are displayed in the title.

lldb.tests test lldb 6 unexpected failures 28 expected failures 2 timeout waiting for results 21 expected passes

http://reviews.llvm.org/D9022

Files:
  zorg/buildbot/commands/LitTestCommand.py

Index: zorg/buildbot/commands/LitTestCommand.py
===================================================================
--- zorg/buildbot/commands/LitTestCommand.py
+++ zorg/buildbot/commands/LitTestCommand.py
@@ -142,7 +142,8 @@
                  'UNTESTED':'untested testcases',
                  'REGRESSED':'runtime performance regression',
                  'IMPROVED':'runtime performance improvement',
-                 'UNSUPPORTED':'unsupported tests'}
+                 'UNSUPPORTED':'unsupported tests',
+                 'TIMEOUT':'timeout waiting for results'}
 
   def __init__(self, ignore=[], flaky=[], max_logs=20, parseSummaryOnly=False,
                *args, **kwargs):
@@ -198,13 +199,14 @@
 
   def test_basic(self):
     obs = self.parse_log("""
-PASS: test-one (1 of 3)
-FAIL: test-two (2 of 3)
-PASS: test-three (3 of 3)
+PASS: test-one (1 of 4)
+FAIL: test-two (2 of 4)
+PASS: test-three (3 of 4)
+TIMEOUT: test-four (4 of 4)
 """)
 
-    self.assertEqual(obs.resultCounts, { 'FAIL' : 1, 'PASS' : 2 })
-    self.assertEqual(obs.step.logs, [('FAIL: test-two', 'FAIL: test-two')])
+    self.assertEqual(obs.resultCounts, { 'FAIL' : 1, 'TIMEOUT' : 1, 'PASS' : 2 })
+    self.assertEqual(obs.step.logs, [('FAIL: test-two', 'FAIL: test-two'), ('TIMEOUT: test-four', 'TIMEOUT: test-four')])
 
   def test_verbose_logs(self):
     obs = self.parse_log("""
@@ -259,7 +261,7 @@
 
     # If there were failing tests, the status should be an error (even if the
     # test command didn't report as such).
-    for failing_code in ('FAIL', 'XPASS', 'KPASS', 'UNRESOLVED'):
+    for failing_code in set(['FAIL', 'XPASS', 'KPASS', 'UNRESOLVED', 'TIMEOUT']):
       cmd = self.parse_log("""%s: test-one (1 of 1)""" % (failing_code,))
       self.assertEqual(cmd.evaluateCommand(RemoteCommandProxy(0)), FAILURE)

EMAIL PREFERENCES
  http://reviews.llvm.org/settings/panel/emailpreferences/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: D9022.23744.patch
Type: text/x-patch
Size: 1820 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20150414/99430553/attachment.bin>