[Lldb-commits] [PATCH] D127258: [lldb] Mark API tests as XFAIL if they have expected failures and no passing tests

[Jonas Devlieghere via Phabricator via lldb-commits]

JDevlieghere created this revision.
JDevlieghere added reviewers: aprantl, mib.
Herald added a project: All.
JDevlieghere requested review of this revision.

Currently, when an API test has no passing tests but at least one unsupported test, we report the test as unsupported. This patch slightly changes that behavior for tests that do not contain any passing tests, but contain both unsupported tests and expected failures. In that case, I think it's more useful to report the test as an expected failure rather than being unsupported.

For example, the output below now result in an "expected failure" rather than "unsupported":

  RESULT: PASSED (0 passes, 0 failures, 0 errors, 1 skipped, 3 expected failures, 0 unexpected successes)


https://reviews.llvm.org/D127258

Files:
  lldb/test/API/lldbtest.py


Index: lldb/test/API/lldbtest.py
===================================================================
--- lldb/test/API/lldbtest.py
+++ lldb/test/API/lldbtest.py
@@ -95,8 +95,12 @@
             # Otherwise this is just a failure.
             return lit.Test.FAIL, output
 
-        has_unsupported_tests = 'UNSUPPORTED:' in out or 'UNSUPPORTED:' in err
         has_passing_tests = 'PASS:' in out or 'PASS:' in err
+        has_expected_failures = 'XFAIL:' in out or 'XFAIL:' in err
+        if has_expected_failures and not has_passing_tests:
+            return lit.Test.XFAIL, output
+
+        has_unsupported_tests = 'UNSUPPORTED:' in out or 'UNSUPPORTED:' in err
         if has_unsupported_tests and not has_passing_tests:
             return lit.Test.UNSUPPORTED, output
 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D127258.434967.patch
Type: text/x-patch
Size: 781 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/lldb-commits/attachments/20220607/5e04aaa2/attachment.bin>