[libcxx-commits] [libcxx] [libc++][lit] Atomically update the persistent cache (PR #66538)

[Alexander Richardson via libcxx-commits]

================
@@ -69,8 +69,12 @@ def f(config, *args, **kwargs):
             if cacheKey not in cache:
                 cache[cacheKey] = function(config, *args, **kwargs)
                 # Update the persistent cache so it knows about the new key
-                with open(persistentCache, "wb") as cacheFile:
+                # We write to a temporary file and rename the result to ensure
+                # that the cache is not corrupted when running the test suite
+                # with multiple shards.
+                with open(persistentCache + ".tmp", "wb") as cacheFile:
                     pickle.dump(cache, cacheFile)
+                os.replace(persistentCache + ".tmp", persistentCache)
             return cache[cacheKey]
----------------
arichardson wrote:

yes this is lit's built-in support to run N subsets of the testsuite across N workers with the `--num-shards=` and `--run-shard=` flag. Quite useful if you are trying to distribute it for multiple slow emulated workers. See also the description of the patch I just uploaded: https://github.com/llvm/llvm-project/pull/66545

https://github.com/llvm/llvm-project/pull/66538