[Lldb-commits] [PATCH] D73384: [lldb/Lit] Change the lldbtest format to behave more like shell test.

[Jonas Devlieghere via Phabricator via lldb-commits]

JDevlieghere created this revision.
JDevlieghere added reviewers: LLDB, labath.
Herald added a subscriber: abidh.
Herald added a project: LLDB.

The current lldbtest format has a number of shortcomings, all related to how we omit information based on why the test fails. For example, a successful test would print nothing, even when `-a` is passed to lit. It's not up to the test format to decide whether to print something or not, that's handled by lit itself. For other test results we would sometimes print stdout and stderr, but sometimes we would forget, such as when a timeout was reached or we couldn't parse the dotest output.

This patch changes the lldbtest format and makes it behave more like lit. We now always print the dotest invocation, the exit code, the output to stdout & stderr. If you're used to dealing with ShTests in lit, this will feel all very familiar.


Repository:
  rLLDB LLDB

https://reviews.llvm.org/D73384

Files:
  lldb/test/API/lldbtest.py


Index: lldb/test/API/lldbtest.py
===================================================================
--- lldb/test/API/lldbtest.py
+++ lldb/test/API/lldbtest.py
@@ -86,33 +86,46 @@
                 shutil.copy(python, copied_python)
             cmd[0] = copied_python
 
+        timeoutInfo = None
         try:
             out, err, exitCode = lit.util.executeCommand(
                 cmd,
                 env=test.config.environment,
                 timeout=litConfig.maxIndividualTestTime)
         except lit.util.ExecuteCommandTimeoutException:
-            return (lit.Test.TIMEOUT, 'Reached timeout of {} seconds'.format(
-                litConfig.maxIndividualTestTime))
+            timeoutInfo = 'Reached timeout of {} seconds'.format(
+                litConfig.maxIndividualTestTime)
+
+        output = """Script:\n--\n%s\n--\nExit Code: %d\n""" % (
+            ' '.join(cmd), exitCode)
+        if timeoutInfo is not None:
+            output += """Timeout: %s\n""" % (timeoutInfo,)
+        output += "\n"
+
+        if out:
+            output += """Command Output (stdout):\n--\n%s\n--\n""" % (out,)
+        if err:
+            output += """Command Output (stderr):\n--\n%s\n--\n""" % (err,)
+
+        if timeoutInfo:
+            return lit.Test.TIMEOUT, output
 
         if exitCode:
             # Match FAIL but not XFAIL.
             for line in out.splitlines() + err.splitlines():
                 if line.startswith('FAIL:'):
-                    return lit.Test.FAIL, out + err
+                    return lit.Test.FAIL, output
 
             if 'XPASS:' in out or 'XPASS:' in err:
-                return lit.Test.XPASS, out + err
+                return lit.Test.XPASS, output
 
         has_unsupported_tests = 'UNSUPPORTED:' in out or 'UNSUPPORTED:' in err
         has_passing_tests = 'PASS:' in out or 'PASS:' in err
         if has_unsupported_tests and not has_passing_tests:
-            return lit.Test.UNSUPPORTED, out + err
+            return lit.Test.UNSUPPORTED, output
 
         passing_test_line = 'RESULT: PASSED'
         if passing_test_line not in out and passing_test_line not in err:
-            msg = ('Unable to find %r in dotest output (exit code %d):\n\n%s%s'
-                   % (passing_test_line, exitCode, out, err))
-            return lit.Test.UNRESOLVED, msg
+            return lit.Test.UNRESOLVED, output
 
-        return lit.Test.PASS, ''
+        return lit.Test.PASS, output


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D73384.240300.patch
Type: text/x-patch
Size: 2457 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/lldb-commits/attachments/20200124/db7e5886/attachment.bin>