[llvm] [CAS] Add ObjectStore::getStandaloneMemoryBuffer() (PR #215378)
Steven Wu via llvm-commits
llvm-commits at lists.llvm.org
Mon Aug 10 13:01:52 PDT 2026
https://github.com/cachemeifyoucan created https://github.com/llvm/llvm-project/pull/215378
getMemoryBuffer() is documented as returning a buffer "whose lifetime is
independent of the CAS (it can live longer)", but it is implemented with
MemoryBuffer::getMemBuffer() over whatever getData() returns. For an
on-disk CAS that is a pointer into the store's own mapping, so the buffer
dies with the store. Fix the comment to say what it does, and add
getStandaloneMemoryBuffer() for callers that need the documented behavior.
The default implementation copies, which always satisfies the lifetime
requirement, so ObjectStore implementations are correct without changes.
An implementation that can hand out storage outliving itself overrides
getStandaloneMemoryBufferImpl() to avoid the copy.
OnDiskGraphDB does that for objects big enough to have a file of their own,
reading that file again rather than sharing the mapping it already has, so
the result stays valid even once the CAS directory is pruned and its pages
can be shared and reclaimed rather than charged to the process. Everything
else is copied, including the case where a nul terminator is required but
the file has no byte to spare for it.
>From 3e9b21918844e0ee2f52139348560b208570c18c Mon Sep 17 00:00:00 2001
From: Steven Wu <stevenwu at apple.com>
Date: Mon, 10 Aug 2026 13:01:40 -0700
Subject: [PATCH] =?UTF-8?q?[=F0=9D=98=80=F0=9D=97=BD=F0=9D=97=BF]=20initia?=
=?UTF-8?q?l=20version?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Created using spr 1.3.7
---
llvm/docs/ContentAddressableStorage.md | 10 ++
llvm/docs/ReleaseNotes.md | 6 +
llvm/include/llvm/CAS/ObjectStore.h | 37 +++++-
llvm/include/llvm/CAS/OnDiskGraphDB.h | 15 +++
llvm/lib/CAS/ObjectStore.cpp | 19 ++++
llvm/lib/CAS/OnDiskCAS.cpp | 12 ++
llvm/lib/CAS/OnDiskGraphDB.cpp | 83 ++++++++++++++
llvm/unittests/CAS/ObjectStoreTest.cpp | 150 +++++++++++++++++++++++++
8 files changed, 328 insertions(+), 4 deletions(-)
diff --git a/llvm/docs/ContentAddressableStorage.md b/llvm/docs/ContentAddressableStorage.md
index a42cbe731bf25..702c48bac3169 100644
--- a/llvm/docs/ContentAddressableStorage.md
+++ b/llvm/docs/ContentAddressableStorage.md
@@ -61,6 +61,12 @@ of error handling. The class APIs also provide convenient methods to
access underlying data. The lifetime of the underlying data is equal to
the lifetime of the instance of `ObjectStore` unless explicitly copied.
+To get a lifetime-extended buffer, call `getStandaloneMemoryBuffer()`, which
+returns a `MemoryBuffer` that remains valid after the `ObjectStore` is
+destroyed. A CAS can provide a customized implementation that is cheaper than
+a copy; the on-disk CAS re-reads the object's file where it has one, which
+lets the pages be shared and reclaimed rather than charged to the process.
+
### CASID
`CASID` is the hash identifier for CASObjects. It owns the underlying
@@ -106,6 +112,10 @@ ones.
To add your own implementation, you just need to add a subclass to
`llvm::cas::ObjectStore` and implement all its pure virtual methods.
+`getStandaloneMemoryBufferImpl()` is an optional customization point; it
+copies by default, so override it only if the implementation can hand out
+storage that outlives itself.
+
To be interchangeable with LLVM ObjectStore, the new CAS implementation
needs to conform to following contracts:
diff --git a/llvm/docs/ReleaseNotes.md b/llvm/docs/ReleaseNotes.md
index efd33a907cfa7..dc381334f62b8 100644
--- a/llvm/docs/ReleaseNotes.md
+++ b/llvm/docs/ReleaseNotes.md
@@ -54,6 +54,12 @@ Makes programs 10x faster by doing Special New Thing.
### Changes to LLVM infrastructure
+* `cas::ObjectStore::getMemoryBuffer()` was documented as returning a buffer
+ whose lifetime is independent of the CAS, but the buffer it returns may alias
+ storage the CAS owns and so cannot outlive it. The documentation now matches
+ the behavior, and the new `getStandaloneMemoryBuffer()` provides a buffer that
+ does stay valid after the `ObjectStore` is destroyed.
+
### Changes to building LLVM
### Changes to TableGen
diff --git a/llvm/include/llvm/CAS/ObjectStore.h b/llvm/include/llvm/CAS/ObjectStore.h
index eb61b5f0eb36f..1e2a4ed685ff6 100644
--- a/llvm/include/llvm/CAS/ObjectStore.h
+++ b/llvm/include/llvm/CAS/ObjectStore.h
@@ -82,7 +82,9 @@ class ActionCache;
/// lifetime tradeoffs:
///
/// - \a getData() accesses data without exposing lifetime at all.
-/// - \a getMemoryBuffer() returns a \a MemoryBuffer whose lifetime
+/// - \a getMemoryBuffer() returns a \a MemoryBuffer that may alias storage
+/// owned by the CAS, so it must not outlive the \a ObjectStore.
+/// - \a getStandaloneMemoryBuffer() returns a \a MemoryBuffer whose lifetime
/// is independent of the CAS (it can live longer).
/// - \a getDataString() return StringRef with lifetime is guaranteed to last as
/// long as \a ObjectStore.
@@ -178,14 +180,25 @@ class LLVM_ABI ObjectStore {
return toStringRef(getData(Node));
}
- /// Get a lifetime-extended MemoryBuffer pointing at \p Data.
+ /// Get a MemoryBuffer pointing at \p Data.
///
- /// Depending on the CAS implementation, this may involve in-memory storage
- /// overhead.
+ /// The buffer may alias storage owned by this ObjectStore, in which case it
+ /// is only valid for as long as the store is.
std::unique_ptr<MemoryBuffer>
getMemoryBuffer(ObjectHandle Node, StringRef Name = "",
bool RequiresNullTerminator = true);
+ /// Get a MemoryBuffer for \p Node that stays valid after this ObjectStore is
+ /// destroyed.
+ ///
+ /// May be more expensive than \a getMemoryBuffer(), which is free to alias
+ /// storage the store already has mapped; prefer that one whenever the buffer
+ /// cannot outlive the store. Never returns \c nullptr: copying the data
+ /// always satisfies the lifetime requirement.
+ std::unique_ptr<MemoryBuffer>
+ getStandaloneMemoryBuffer(ObjectHandle Node, StringRef Name = "",
+ bool RequiresNullTerminator = true);
+
/// Read all the refs from object in a SmallVector.
virtual void readRefs(ObjectHandle Node,
SmallVectorImpl<ObjectRef> &Refs) const;
@@ -287,6 +300,17 @@ class LLVM_ABI ObjectStore {
ObjectStore(const CASContext &Context) : Context(Context) {}
private:
+ /// Customization point for \a getStandaloneMemoryBuffer(). The default
+ /// implementation copies the data, which always satisfies the lifetime
+ /// requirement; implementations that can hand out storage outliving
+ /// themselves, e.g. a mapping of a file they do not keep open, should
+ /// override this to avoid the copy. Must not return \c nullptr: fall back
+ /// to \c ObjectStore::getStandaloneMemoryBufferImpl() where the cheaper
+ /// path does not apply.
+ virtual std::unique_ptr<MemoryBuffer>
+ getStandaloneMemoryBufferImpl(ObjectHandle Node, StringRef Name,
+ bool RequiresNullTerminator);
+
const CASContext &Context;
};
@@ -318,6 +342,11 @@ class ObjectProxy {
getMemoryBuffer(StringRef Name = "",
bool RequiresNullTerminator = true) const;
+ /// Get a MemoryBuffer that stays valid after the CAS is destroyed.
+ LLVM_ABI std::unique_ptr<MemoryBuffer>
+ getStandaloneMemoryBuffer(StringRef Name = "",
+ bool RequiresNullTerminator = true) const;
+
/// Get the content of the node. Valid as long as the CAS is valid.
StringRef getData() const { return CAS->getDataString(H); }
diff --git a/llvm/include/llvm/CAS/OnDiskGraphDB.h b/llvm/include/llvm/CAS/OnDiskGraphDB.h
index f994a7134a364..addc39a85598f 100644
--- a/llvm/include/llvm/CAS/OnDiskGraphDB.h
+++ b/llvm/include/llvm/CAS/OnDiskGraphDB.h
@@ -22,6 +22,10 @@
#include "llvm/CAS/OnDiskTrieRawHashMap.h"
#include <atomic>
+namespace llvm {
+class MemoryBuffer;
+} // namespace llvm
+
namespace llvm::cas::ondisk {
/// Standard 8 byte reference inside OnDiskGraphDB.
@@ -354,6 +358,17 @@ class OnDiskGraphDB {
LLVM_ABI FileBackedData
getInternalFileBackedObjectData(ObjectHandle Node) const;
+ /// Get a MemoryBuffer for \p Node's data that stays valid after this
+ /// database is destroyed.
+ ///
+ /// Objects stored in a file of their own are re-read from it rather than
+ /// copied out of this database's mapping, which lets the pages be shared and
+ /// reclaimed rather than charged to this process. The rest are copied. Never
+ /// returns \c nullptr.
+ LLVM_ABI std::unique_ptr<MemoryBuffer>
+ getStandaloneMemoryBuffer(ObjectHandle Node, StringRef Name,
+ bool RequiresNullTerminator) const;
+
/// \returns Total size of stored objects.
///
/// NOTE: There's a possibility that the returned size is not including a
diff --git a/llvm/lib/CAS/ObjectStore.cpp b/llvm/lib/CAS/ObjectStore.cpp
index e52be9711cac3..cc8013a3fea16 100644
--- a/llvm/lib/CAS/ObjectStore.cpp
+++ b/llvm/lib/CAS/ObjectStore.cpp
@@ -75,6 +75,19 @@ ObjectStore::getMemoryBuffer(ObjectHandle Node, StringRef Name,
RequiresNullTerminator);
}
+std::unique_ptr<MemoryBuffer>
+ObjectStore::getStandaloneMemoryBuffer(ObjectHandle Node, StringRef Name,
+ bool RequiresNullTerminator) {
+ return getStandaloneMemoryBufferImpl(Node, Name, RequiresNullTerminator);
+}
+
+std::unique_ptr<MemoryBuffer>
+ObjectStore::getStandaloneMemoryBufferImpl(ObjectHandle Node, StringRef Name,
+ bool RequiresNullTerminator) {
+ return MemoryBuffer::getMemBufferCopy(
+ toStringRef(getData(Node, RequiresNullTerminator)), Name);
+}
+
void ObjectStore::readRefs(ObjectHandle Node,
SmallVectorImpl<ObjectRef> &Refs) const {
consumeError(forEachRef(Node, [&Refs](ObjectRef Ref) -> Error {
@@ -294,3 +307,9 @@ ObjectProxy::getMemoryBuffer(StringRef Name,
bool RequiresNullTerminator) const {
return CAS->getMemoryBuffer(H, Name, RequiresNullTerminator);
}
+
+std::unique_ptr<MemoryBuffer>
+ObjectProxy::getStandaloneMemoryBuffer(StringRef Name,
+ bool RequiresNullTerminator) const {
+ return CAS->getStandaloneMemoryBuffer(H, Name, RequiresNullTerminator);
+}
diff --git a/llvm/lib/CAS/OnDiskCAS.cpp b/llvm/lib/CAS/OnDiskCAS.cpp
index 37c2334b2577f..688f75e999dd1 100644
--- a/llvm/lib/CAS/OnDiskCAS.cpp
+++ b/llvm/lib/CAS/OnDiskCAS.cpp
@@ -17,6 +17,7 @@
#include "llvm/Support/Compiler.h"
#include "llvm/Support/Error.h"
#include "llvm/Support/IOSandbox.h"
+#include "llvm/Support/MemoryBuffer.h"
#include "llvm/Support/Path.h"
using namespace llvm;
@@ -45,6 +46,10 @@ class OnDiskCAS : public BuiltinCAS {
Error exportDataToFile(ObjectHandle Node, StringRef Path) const final;
+ std::unique_ptr<MemoryBuffer>
+ getStandaloneMemoryBufferImpl(ObjectHandle Node, StringRef Name,
+ bool RequiresNullTerminator) final;
+
void print(raw_ostream &OS) const final;
Error validate(bool CheckHash) const final;
@@ -166,6 +171,13 @@ Expected<ObjectRef> OnDiskCAS::storeFromFile(StringRef Path) {
return convertRef(*StoredID);
}
+std::unique_ptr<MemoryBuffer>
+OnDiskCAS::getStandaloneMemoryBufferImpl(ObjectHandle Node, StringRef Name,
+ bool RequiresNullTerminator) {
+ return DB->getStandaloneMemoryBuffer(convertHandle(Node), Name,
+ RequiresNullTerminator);
+}
+
Error OnDiskCAS::exportDataToFile(ObjectHandle Node, StringRef Path) const {
auto FBData = DB->getInternalFileBackedObjectData(convertHandle(Node));
if (!FBData.FileInfo.has_value())
diff --git a/llvm/lib/CAS/OnDiskGraphDB.cpp b/llvm/lib/CAS/OnDiskGraphDB.cpp
index c38dd9151bc08..a817b35f6f588 100644
--- a/llvm/lib/CAS/OnDiskGraphDB.cpp
+++ b/llvm/lib/CAS/OnDiskGraphDB.cpp
@@ -371,6 +371,15 @@ class StandaloneDataInMemory {
OnDiskGraphDB::FileBackedData
getInternalFileBackedObjectData(StringRef RootPath) const;
+ /// Read this object's data from its file again, so the result does not
+ /// reference \a Region and stays valid after this object is gone.
+ ///
+ /// \returns \c nullptr when it does not apply, and the caller is
+ /// expected to copy instead.
+ std::unique_ptr<MemoryBuffer>
+ getStandaloneMemoryBuffer(StringRef RootPath, StringRef Name,
+ bool RequiresNullTerminator) const;
+
StandaloneDataInMemory(std::unique_ptr<sys::fs::mapped_file_region> Region,
TrieRecord::StorageKind SK, FileOffset IndexOffset)
: Region(std::move(Region)), SK(SK), IndexOffset(IndexOffset) {
@@ -1291,6 +1300,23 @@ OnDiskGraphDB::getInternalFileBackedObjectData(ObjectHandle Node) const {
}
}
+std::unique_ptr<MemoryBuffer>
+OnDiskGraphDB::getStandaloneMemoryBuffer(ObjectHandle Node, StringRef Name,
+ bool RequiresNullTerminator) const {
+ // Only an object with a file to itself can be read back on its own; one in
+ // the shared data pool is a subrange of a file holding unrelated objects.
+ auto SDIMOrRecord = getStandaloneDataOrDataRecord(DataPool, Node);
+ if (auto **SDIM =
+ std::get_if<const StandaloneDataInMemory *>(&SDIMOrRecord)) {
+ if (std::unique_ptr<MemoryBuffer> Standalone =
+ (*SDIM)->getStandaloneMemoryBuffer(RootPath, Name,
+ RequiresNullTerminator))
+ return Standalone;
+ }
+
+ return MemoryBuffer::getMemBufferCopy(toStringRef(getObjectData(Node)), Name);
+}
+
Expected<std::optional<ObjectHandle>>
OnDiskGraphDB::load(ObjectID ExternalRef) {
InternalRef Ref = getInternalRef(ExternalRef);
@@ -1463,6 +1489,63 @@ StandaloneDataInMemory::getInternalFileBackedObjectData(
llvm_unreachable("Unknown StorageKind enum");
}
+namespace {
+/// A MemoryBuffer exposing a subrange of another buffer's bytes, under its own
+/// name.
+class AdoptedMemoryBuffer final : public MemoryBuffer {
+public:
+ AdoptedMemoryBuffer(std::unique_ptr<MemoryBuffer> Buffer, StringRef Name,
+ uint64_t Offset, uint64_t Size)
+ : Buffer(std::move(Buffer)), Name(Name.str()) {
+ const char *Start = this->Buffer->getBufferStart() + Offset;
+ init(Start, Start + Size, /*RequiresNullTerminator=*/false);
+ }
+
+ StringRef getBufferIdentifier() const final { return Name; }
+
+ BufferKind getBufferKind() const final { return Buffer->getBufferKind(); }
+
+private:
+ std::unique_ptr<MemoryBuffer> Buffer;
+ std::string Name;
+};
+} // end anonymous namespace
+
+std::unique_ptr<MemoryBuffer> StandaloneDataInMemory::getStandaloneMemoryBuffer(
+ StringRef RootPath, StringRef Name, bool RequiresNullTerminator) const {
+ // A plain leaf's file is exactly the data, with no nul after it to map. The
+ // other kinds have one: a record's own terminator, or the one appended to a
+ // "leaf+0".
+ if (RequiresNullTerminator && SK == TrieRecord::StorageKind::StandaloneLeaf)
+ return nullptr;
+
+ // Read the file again instead of sharing \a Region, whose lifetime is tied
+ // to this object. These files are written once and never modified, so the
+ // second read sees the same bytes. Whether that ends up mapping the file or
+ // copying it is up to MemoryBuffer; either way the result stands alone.
+ SmallString<256> Path;
+ ::getStandalonePath(RootPath, TrieRecord::getStandaloneFilePrefix(SK),
+ IndexOffset, Path);
+ auto BypassSandbox = sys::sandbox::scopedDisable();
+ ErrorOr<std::unique_ptr<MemoryBuffer>> Mapped =
+ MemoryBuffer::getFile(Path, /*IsText=*/false,
+ /*RequiresNullTerminator=*/false,
+ /*IsVolatile=*/false);
+ if (!Mapped)
+ return nullptr;
+
+ // Find the data within the mapping. A leaf's file holds just the data; a
+ // record's also holds its header and refs.
+ OnDiskContent Content = getContent();
+ ArrayRef<char> Data = Content.getData();
+ uint64_t Offset = Content.Record ? Data.data() - Region->data() : 0;
+ if (Offset + Data.size() > (*Mapped)->getBufferSize())
+ return nullptr;
+
+ return std::make_unique<AdoptedMemoryBuffer>(std::move(*Mapped), Name, Offset,
+ Data.size());
+}
+
static Expected<MappedTempFile>
createTempFile(StringRef FinalPath, uint64_t Size, OnDiskCASLogger *Logger) {
auto BypassSandbox = sys::sandbox::scopedDisable();
diff --git a/llvm/unittests/CAS/ObjectStoreTest.cpp b/llvm/unittests/CAS/ObjectStoreTest.cpp
index af248b6de8432..aff7f783ee9ab 100644
--- a/llvm/unittests/CAS/ObjectStoreTest.cpp
+++ b/llvm/unittests/CAS/ObjectStoreTest.cpp
@@ -8,7 +8,10 @@
#include "llvm/CAS/ObjectStore.h"
#include "OnDiskCommonUtils.h"
+#include "llvm/CAS/ActionCache.h"
+#include "llvm/CAS/BuiltinUnifiedCASDatabases.h"
#include "llvm/Config/llvm-config.h"
+#include "llvm/Support/FileSystem.h"
#include "llvm/Support/MemoryBuffer.h"
#include "llvm/Support/Process.h"
#include "llvm/Support/RandomNumberGenerator.h"
@@ -498,3 +501,150 @@ TEST_F(OnDiskCASTest, OnDiskCASDiskSize) {
CAS.reset();
CheckFileSizes(/*Mapped=*/false);
}
+
+TEST_P(CASTest, StandaloneMemoryBufferOutlivesCAS) {
+ // The buffer has to stay readable after the last reference to the store is
+ // gone. Cover both sides of the size threshold that decides whether an
+ // on-disk CAS embeds an object in its shared data pool or gives it a file of
+ // its own, since only the latter can be mapped.
+ for (uint64_t Size : {uint64_t(64), uint64_t(100 * 1024)}) {
+ std::shared_ptr<ObjectStore> CAS = createObjectStore();
+ std::string Data(Size, '\a');
+ Data.front() = 'b';
+ Data.back() = 'e';
+
+ std::optional<ObjectProxy> Proxy;
+ ASSERT_THAT_ERROR(CAS->createProxy({}, Data).moveInto(Proxy), Succeeded());
+ std::unique_ptr<MemoryBuffer> Buffer =
+ Proxy->getStandaloneMemoryBuffer("name");
+ ASSERT_TRUE(Buffer);
+ EXPECT_EQ("name", Buffer->getBufferIdentifier());
+
+ Proxy.reset();
+ CAS.reset();
+
+ // Read every page, not just the ends, so a mapping that lost its backing
+ // store faults here rather than silently passing.
+ ASSERT_EQ(Size, Buffer->getBufferSize());
+ EXPECT_EQ(Data, Buffer->getBuffer());
+ }
+}
+
+TEST_P(CASTest, StandaloneMemoryBufferNullTerminated) {
+ std::shared_ptr<ObjectStore> CAS = createObjectStore();
+ // Cover both sides of the size threshold that decides whether an on-disk CAS
+ // gives an object a file of its own or embeds it in the shared data pool,
+ // and include exact multiples of the page size: a mapping ending on a page
+ // boundary has no zero-filled slack to serve as the terminator.
+ uint64_t PageSize = sys::Process::getPageSizeEstimate();
+ for (uint64_t Size : {uint64_t(64), uint64_t(60000), uint64_t(65535),
+ uint64_t(100 * 1024), PageSize, 4 * PageSize}) {
+ std::string Data(Size, 'z');
+ std::optional<ObjectProxy> Proxy;
+ ASSERT_THAT_ERROR(CAS->createProxy({}, Data).moveInto(Proxy), Succeeded());
+ std::unique_ptr<MemoryBuffer> Buffer = Proxy->getStandaloneMemoryBuffer(
+ "name", /*RequiresNullTerminator=*/true);
+ ASSERT_TRUE(Buffer);
+ ASSERT_EQ(Size, Buffer->getBufferSize());
+ EXPECT_EQ('\0', *Buffer->getBufferEnd());
+ EXPECT_EQ(Data, Buffer->getBuffer());
+ }
+}
+
+TEST_P(CASTest, StandaloneMemoryBufferWithoutNullTerminator) {
+ std::shared_ptr<ObjectStore> CAS = createObjectStore();
+ uint64_t PageSize = sys::Process::getPageSizeEstimate();
+ for (uint64_t Size : {uint64_t(64), uint64_t(60000), uint64_t(65535),
+ uint64_t(100 * 1024), PageSize, 4 * PageSize}) {
+ std::string Data(Size, 'q');
+ Data.front() = 'b';
+ Data.back() = 'e';
+ std::optional<ObjectProxy> Proxy;
+ ASSERT_THAT_ERROR(CAS->createProxy({}, Data).moveInto(Proxy), Succeeded());
+ std::unique_ptr<MemoryBuffer> Buffer = Proxy->getStandaloneMemoryBuffer(
+ "name", /*RequiresNullTerminator=*/false);
+ ASSERT_TRUE(Buffer);
+ ASSERT_EQ(Size, Buffer->getBufferSize());
+ EXPECT_EQ(Data, Buffer->getBuffer());
+ }
+}
+
+TEST_F(OnDiskCASTest, StandaloneMemoryBufferSurvivesDeletedCAS) {
+ // The point of a standalone buffer is that clients can drop the store,
+ // letting its lock go so the CAS can be pruned, and still read what they
+ // loaded. Deleting the whole directory is the strongest form of that: a
+ // mapping keeps the file alive until it is unmapped.
+ unittest::TempDir Temp("on-disk-cas", /*Unique=*/true);
+
+ std::string SmallData(64, 's');
+ std::string BigData(100ULL * 1024, 'b');
+ BigData.back() = 'e';
+
+ std::unique_ptr<MemoryBuffer> SmallBuffer, BigBuffer;
+ {
+ std::pair<std::unique_ptr<ObjectStore>, std::unique_ptr<ActionCache>> DBs;
+ ASSERT_THAT_ERROR(
+ createOnDiskUnifiedCASDatabases(Temp.path()).moveInto(DBs),
+ Succeeded());
+ std::optional<ObjectProxy> Small, Big;
+ ASSERT_THAT_ERROR(DBs.first->createProxy({}, SmallData).moveInto(Small),
+ Succeeded());
+ ASSERT_THAT_ERROR(DBs.first->createProxy({}, BigData).moveInto(Big),
+ Succeeded());
+ SmallBuffer =
+ Small->getStandaloneMemoryBuffer("", /*RequiresNullTerminator=*/false);
+ BigBuffer =
+ Big->getStandaloneMemoryBuffer("", /*RequiresNullTerminator=*/false);
+ ASSERT_TRUE(SmallBuffer);
+ ASSERT_TRUE(BigBuffer);
+ }
+
+ // Windows refuses to delete a file while it is still mapped, so only check
+ // the deleted case where unlinking a mapped file is allowed.
+#ifndef _WIN32
+ ASSERT_EQ(std::error_code(), sys::fs::remove_directories(Temp.path()));
+ ASSERT_FALSE(sys::fs::exists(Temp.path()));
+#endif
+
+ EXPECT_EQ(SmallData, SmallBuffer->getBuffer());
+ EXPECT_EQ(BigData, BigBuffer->getBuffer());
+
+ // The big object gets a file of its own, so it should be mapped rather than
+ // copied -- that is the whole point of not holding the store open. (The
+ // small one is embedded in the shared data pool, which is always copied.)
+ EXPECT_EQ(MemoryBuffer::MemoryBuffer_MMap, BigBuffer->getBufferKind());
+ EXPECT_EQ(MemoryBuffer::MemoryBuffer_Malloc, SmallBuffer->getBufferKind());
+}
+
+TEST_F(OnDiskCASTest, StandaloneMemoryBufferRecordWithRefs) {
+ // An object with refs that is too big for the pool gets a file of its own
+ // too, but one holding a record: a header and the refs, then the data and a
+ // nul. It can still be mapped, at an offset, and the nul it already has
+ // means even a caller wanting a terminator does not force a copy.
+ unittest::TempDir Temp("on-disk-cas", /*Unique=*/true);
+
+ std::string Data(100ULL * 1024, 'r');
+ Data.back() = 'e';
+
+ std::unique_ptr<MemoryBuffer> Buffer;
+ {
+ std::pair<std::unique_ptr<ObjectStore>, std::unique_ptr<ActionCache>> DBs;
+ ASSERT_THAT_ERROR(
+ createOnDiskUnifiedCASDatabases(Temp.path()).moveInto(DBs),
+ Succeeded());
+ std::optional<ObjectProxy> Child, Parent;
+ ASSERT_THAT_ERROR(DBs.first->createProxy({}, "child").moveInto(Child),
+ Succeeded());
+ ASSERT_THAT_ERROR(
+ DBs.first->createProxy({Child->getRef()}, Data).moveInto(Parent),
+ Succeeded());
+ Buffer = Parent->getStandaloneMemoryBuffer("name");
+ ASSERT_TRUE(Buffer);
+ }
+
+ EXPECT_EQ(MemoryBuffer::MemoryBuffer_MMap, Buffer->getBufferKind());
+ EXPECT_EQ("name", Buffer->getBufferIdentifier());
+ ASSERT_EQ(Data.size(), Buffer->getBufferSize());
+ EXPECT_EQ(Data, Buffer->getBuffer());
+ EXPECT_EQ('\0', *Buffer->getBufferEnd());
+}
More information about the llvm-commits
mailing list