[Lldb-commits] [lldb] 55d4b0d - [lldb] Fix that a crashing test is marked as unsupported when it prints UNSUPPORTED before crashing

[Raphael Isemann via lldb-commits]

Author: Raphael Isemann
Date: 2020-02-25T08:46:37+01:00
New Revision: 55d4b0d7dd70d5ecc9e641d93fbf234bb7104d1a

URL: https://github.com/llvm/llvm-project/commit/55d4b0d7dd70d5ecc9e641d93fbf234bb7104d1a
DIFF: https://github.com/llvm/llvm-project/commit/55d4b0d7dd70d5ecc9e641d93fbf234bb7104d1a.diff

LOG: [lldb] Fix that a crashing test is marked as unsupported when it prints UNSUPPORTED before crashing

Summary:
I added an `abort()` call to some code and noticed that the test suite was still passing and it just marked my test as "UNSUPPORTED".

It seems the reason for that is that we expect failing tests to print "FAIL:" which doesn't happen when we crash. If we then also
have an unsupported because we skipped some debug information in the output, we just mark the test passing because it is unsupported
on the current platform.

This patch marks any test that has a non-zero exit code as failing even if it doesn't print "FAIL:" (e.g., because it crashed).

Reviewers: labath, JDevlieghere

Reviewed By: labath, JDevlieghere

Subscribers: aprantl, lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D75031

Added: 
    

Modified: 
    lldb/test/API/lldbtest.py

Removed: 
    


################################################################################
diff  --git a/lldb/test/API/lldbtest.py b/lldb/test/API/lldbtest.py
index 65aafab5c617..94c0588fd9e6 100644
--- a/lldb/test/API/lldbtest.py
+++ b/lldb/test/API/lldbtest.py
@@ -114,14 +114,12 @@ def execute(self, test, litConfig):
             return lit.Test.TIMEOUT, output
 
         if exitCode:
-            # Match FAIL but not XFAIL.
-            for line in out.splitlines() + err.splitlines():
-                if line.startswith('FAIL:'):
-                    return lit.Test.FAIL, output
-
             if 'XPASS:' in out or 'XPASS:' in err:
                 return lit.Test.XPASS, output
 
+            # Otherwise this is just a failure.
+            return lit.Test.FAIL, output
+
         has_unsupported_tests = 'UNSUPPORTED:' in out or 'UNSUPPORTED:' in err
         has_passing_tests = 'PASS:' in out or 'PASS:' in err
         if has_unsupported_tests and not has_passing_tests: