[llvm] [CI] Make cache_lit_timing_files.py Script Gracefully Fail (PR #162316)
Aiden Grossman via llvm-commits
llvm-commits at lists.llvm.org
Tue Oct 7 09:18:34 PDT 2025
https://github.com/boomanaiden154 created https://github.com/llvm/llvm-project/pull/162316
This is a performance optimization and does not impact test fidelity. There have been some flakes where this script will fail to download files, exit with code 1, causing the job to fail before it even starts running tests. This is undesirable as the tests will only run 10-15% slower without this, so catch the exceptions and emit a warning we can track later in the rare case we cannot download the timing files.
>From 19b40efec006fdb270852150a836c0ef48c59493 Mon Sep 17 00:00:00 2001
From: Aiden Grossman <aidengrossman at google.com>
Date: Tue, 7 Oct 2025 16:13:23 +0000
Subject: [PATCH] [CI] Make cache_lit_timing_files.py Script Gracefully Fail
This is a performance optimization and does not impact test fidelity. There
have been some flakes where this script will fail to download files, exit
with code 1, causing the job to fail before it even starts running tests.
This is undesirable as the tests will only run 10-15% slower without this,
so catch the exceptions and emit a warning we can track later in the rare
case we cannot download the timing files.
---
.ci/cache_lit_timing_files.py | 18 ++++++++++++++++--
1 file changed, 16 insertions(+), 2 deletions(-)
diff --git a/.ci/cache_lit_timing_files.py b/.ci/cache_lit_timing_files.py
index 2f43e46fc0e56..27a5cf6b0fda3 100644
--- a/.ci/cache_lit_timing_files.py
+++ b/.ci/cache_lit_timing_files.py
@@ -17,6 +17,7 @@
import glob
from google.cloud import storage
+from google.api_core import exceptions
GCS_PARALLELISM = 100
@@ -50,7 +51,14 @@ def _maybe_download_timing_file(blob):
def download_timing_files(storage_client, bucket_name: str):
bucket = storage_client.bucket(bucket_name)
- blobs = bucket.list_blobs(prefix="lit_timing")
+ try:
+ blobs = bucket.list_blobs(prefix="lit_timing")
+ except exceptions.ClientError as client_error:
+ print(
+ "::warning file=cache_lit_timing_files.py::Failed to list blobs "
+ "in bucket."
+ )
+ sys.exit(0)
with multiprocessing.pool.ThreadPool(GCS_PARALLELISM) as thread_pool:
futures = []
for timing_file_blob in blobs:
@@ -60,7 +68,13 @@ def download_timing_files(storage_client, bucket_name: str):
)
)
for future in futures:
- future.get()
+ future.wait()
+ if not future.successful():
+ print(
+ "::warning file=cache_lit_timing_files.py::Failed to "
+ "download lit timing file."
+ )
+ continue
print("Done downloading")
More information about the llvm-commits
mailing list