[llvm] 9f89d31 - [lldb][test] Mark gtest cases as XFAIL if the test suite is XFAIL (#102986)

[via llvm-commits]

Author: Kendal Harland
Date: 2024-08-16T10:54:25-07:00
New Revision: 9f89d31d5185015f8eea9c0f3d35e7ba9d353e67

URL: https://github.com/llvm/llvm-project/commit/9f89d31d5185015f8eea9c0f3d35e7ba9d353e67
DIFF: https://github.com/llvm/llvm-project/commit/9f89d31d5185015f8eea9c0f3d35e7ba9d353e67.diff

LOG: [lldb][test] Mark gtest cases as XFAIL if the test suite is XFAIL (#102986)

When a test case inside of a gtest suite fails, we report a failure
which causes the entire `ninja check-lldb` invocation to fail, even if
the outer test case is marked as XFAIL - each test case result is
reported as its own lit test run. This PR updates lit so it checks
whether each test case's parent test suite is XFAIL before setting the
status to FAIL.

This is especially problematic because the failing tests can't manually
be marked as XFAIL, due to
https://github.com/llvm/llvm-project/issues/102264.

Fixes https://github.com/llvm/llvm-project/issues/102265

### Repro instructions

1. Modify any gtest test case to generate a failure.
2. Mark the outer lit test with XFAIL using either `--xfail-tests` flag
or `LIT_XFAIL` env var.
3. Run the tests
4. Observe the lit test is XFAIL as expected, but the failed child test
cases show up as separate failures.

Co-authored-by: kendal <kendal at thebrowser.company>

Added: 
    

Modified: 
    llvm/utils/lit/lit/formats/googletest.py

Removed: 
    


################################################################################
diff  --git a/llvm/utils/lit/lit/formats/googletest.py b/llvm/utils/lit/lit/formats/googletest.py
index 8037094a910678..6c29cca891581f 100644
--- a/llvm/utils/lit/lit/formats/googletest.py
+++ b/llvm/utils/lit/lit/formats/googletest.py
@@ -334,7 +334,11 @@ def remove_gtest(tests):
                             returnCode = lit.Test.SKIPPED
                         elif "failures" in testinfo:
                             has_failure_in_shard = True
-                            returnCode = lit.Test.FAIL
+                            returnCode = (
+                                lit.Test.XFAIL
+                                if test.isExpectedToFail()
+                                else lit.Test.FAIL
+                            )
                             output = header
                             for fail in testinfo["failures"]:
                                 output += fail["failure"] + "\n"