[llvm] [CI] Detect step failures in metrics job (PR #122564)

[Aiden Grossman via llvm-commits]

https://github.com/boomanaiden154 updated https://github.com/llvm/llvm-project/pull/122564

>From 09a8144a00f3b12302a2667b1ee8a1f80c0ed2d4 Mon Sep 17 00:00:00 2001
From: Aiden Grossman <aidengrossman at google.com>
Date: Sat, 11 Jan 2025 02:19:02 +0000
Subject: [PATCH] [CI] Detect step failures in metrics job

This patch makes the metrics job also detect failures in individual steps.
This is necessary now that we are setting continue-on-error in the premerge
jobs to prevent sending out unnecessary email to detect what jobs actually
fail.
---
 .ci/metrics/metrics.py | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/.ci/metrics/metrics.py b/.ci/metrics/metrics.py
index 55025e50d1081f..cbff478b9ba153 100644
--- a/.ci/metrics/metrics.py
+++ b/.ci/metrics/metrics.py
@@ -80,6 +80,18 @@ def get_metrics(github_repo: github.Repository, workflows_to_track: dict[str, in
         completed_at = workflow_jobs[0].completed_at
 
         job_result = int(workflow_jobs[0].conclusion == "success")
+        if job_result:
+            # We still might want to mark the job as a failure if one of the steps
+            # failed. This is required due to use setting continue-on-error in
+            # the premerge pipeline to prevent sending emails while we are
+            # testing the infrastructure.
+            # TODO(boomanaiden154): Remove this once the premerge pipeline is no
+            # longer in a testing state and we can directly assert the workflow
+            # result.
+            for step in workflow_jobs[0].steps:
+                if step.conclusion != "success":
+                    job_result = 0
+                    break
 
         queue_time = started_at - created_at
         run_time = completed_at - started_at