[llvm-branch-commits] [CI] Setup generate_report to describe ninja failures (PR #152621)

[David Spickett via llvm-branch-commits]

================
@@ -246,6 +288,58 @@ def test_no_failures_build_failed(self):
             ),
         )
 
+    def test_no_failures_build_failed_ninja_log(self):
+        self.assertEqual(
+            generate_test_report_lib.generate_report(
+                "Foo",
+                1,
+                [
+                    junit_from_xml(
+                        dedent(
+                            """\
+          <?xml version="1.0" encoding="UTF-8"?>
+          <testsuites time="0.00">
+          <testsuite name="Passed" tests="1" failures="0" skipped="0" time="0.00">
+          <testcase classname="Bar/test_1" name="test_1" time="0.00"/>
+          </testsuite>
+          </testsuites>"""
+                        )
+                    )
+                ],
+                [
+                    [
+                        "[1/5] test/1.stamp",
+                        "[2/5] test/2.stamp",
+                        "[3/5] test/3.stamp",
+                        "[4/5] test/4.stamp",
+                        "FAILED: test/4.stamp",
+                        "touch test/4.stamp",
+                        "Wow! Close To You!",
+                        "[5/5] test/5.stamp",
+                    ]
+                ],
+            ),
+            (
+                dedent(
+                    """\
+                    # Foo
+
+                    * 1 test passed
----------------
DavidSpickett wrote:

I may also be mixing meanings of "build failure". As there are "build build failures" where it actually failed to run some non-test command, and "build failure" which includes anything ninja tried to run but failed, which includes tests.

Ultimately if the user doesn't have to dig in the log that's already an improvement even if a few things end up labelled incorrectly. So you've already got a good improvement here.

https://github.com/llvm/llvm-project/pull/152621