[libcxx-commits] [libcxx] [libc++][lit] Atomically update the persistent cache (PR #66538)

[via libcxx-commits]

================
@@ -69,8 +69,12 @@ def f(config, *args, **kwargs):
             if cacheKey not in cache:
                 cache[cacheKey] = function(config, *args, **kwargs)
                 # Update the persistent cache so it knows about the new key
-                with open(persistentCache, "wb") as cacheFile:
+                # We write to a temporary file and rename the result to ensure
+                # that the cache is not corrupted when running the test suite
+                # with multiple shards.
+                with open(persistentCache + ".tmp", "wb") as cacheFile:
                     pickle.dump(cache, cacheFile)
+                os.replace(persistentCache + ".tmp", persistentCache)
             return cache[cacheKey]
----------------
EricWF wrote:

This whole setup concerns me. It seems like a way to get very hard to diagnose test failures/behavior.

The desire to shard makes sense. But the cache existing and not being already full by the time the shards have started seems unfortunate. 

https://github.com/llvm/llvm-project/pull/66538