[clang-tools-extra] [clangd] Don't count index slabs twice in the memory usage report (PR #242888)
via cfe-commits
cfe-commits at lists.llvm.org
Sun Nov 15 14:07:56 PST 2026
llvmorg-github-actions[bot] wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-clangd
Author: Олександр Федоров (namefreezers)
<details>
<summary>Changes</summary>
In the memory usage report (`$/memoryUsage`), the background index and the
two parts of the dynamic index (`preamble` and `main_file`) each have two
children:
- `slabs`: the per-file symbol, reference and relation slabs that
`FileSymbols` holds;
- `index`: the `estimateMemoryUsage()` of the index that
`FileSymbols::buildIndex()` builds from these slabs.
`buildIndex()` added the size of the symbol and reference slabs to the
backing data size it passes to `Dex` or `MemIndex`, and both include that
size in their estimate. So these slabs were counted under `slabs` and again
under `index`. The slab sizes went into the estimate in
https://reviews.llvm.org/D51539, before https://reviews.llvm.org/D88414 gave
the slabs their own node in the report.
Now the backing data size covers only the storage that `buildIndex()`
creates: the merged symbols and the contiguous refs. The estimates logged
when these indexes are rebuilt change the same way. Until an index is
rebuilt, slabs that `FileSymbols` has already replaced are kept alive by the
index but not counted.
`FileIndexTest.Profile` now gives the main file a declaration: its main-file
index had a non-zero estimate only because the empty slabs were counted.
Fixes clangd/clangd#<!-- -->2724
Assisted-by: Claude Opus 5.5
---
Full diff: https://github.com/llvm/llvm-project/pull/242888.diff
2 Files Affected:
- (modified) clang-tools-extra/clangd/index/FileIndex.cpp (+2-4)
- (modified) clang-tools-extra/clangd/unittests/FileIndexTests.cpp (+26-1)
``````````diff
diff --git a/clang-tools-extra/clangd/index/FileIndex.cpp b/clang-tools-extra/clangd/index/FileIndex.cpp
index 2e005bfe3537e6..73632d941c2b44 100644
--- a/clang-tools-extra/clangd/index/FileIndex.cpp
+++ b/clang-tools-extra/clangd/index/FileIndex.cpp
@@ -374,12 +374,10 @@ FileSymbols::buildIndex(IndexType Type, DuplicateHandling DuplicateHandle,
llvm::sort(AllRelations);
AllRelations.erase(llvm::unique(AllRelations), AllRelations.end());
+ // Only the storage built here counts towards the index's estimated size: the
+ // slabs are shared with FileSymbols, and profile() reports them.
size_t StorageSize =
RefsStorage.size() * sizeof(Ref) + SymsStorage.size() * sizeof(Symbol);
- for (const auto &Slab : SymbolSlabs)
- StorageSize += Slab->bytes();
- for (const auto &RefSlab : RefSlabs)
- StorageSize += RefSlab->bytes();
// Index must keep the slabs and contiguous ranges alive.
switch (Type) {
diff --git a/clang-tools-extra/clangd/unittests/FileIndexTests.cpp b/clang-tools-extra/clangd/unittests/FileIndexTests.cpp
index a92142fbcd7c4a..bcceae579ac699 100644
--- a/clang-tools-extra/clangd/unittests/FileIndexTests.cpp
+++ b/clang-tools-extra/clangd/unittests/FileIndexTests.cpp
@@ -717,7 +717,9 @@ TEST(FileIndexTest, Profile) {
FileIndex FI(true);
auto FileName = testPath("foo.cpp");
- auto AST = TestTU::withHeaderCode("int a;").build();
+ TestTU TU = TestTU::withHeaderCode("int a;");
+ TU.Code = "int b;";
+ auto AST = TU.build();
FI.updateMain(FileName, AST);
FI.updatePreamble(FileName, "v1", AST.getASTContext(), AST.getPreprocessor(),
AST.getPragmaIncludes());
@@ -757,6 +759,29 @@ TEST(FileSymbolsTest, Profile) {
EXPECT_THAT(MT.child("f3").total(), Gt(0U));
}
+TEST(FileSymbolsTest, IndexEstimateExcludesSlabs) {
+ // profile() reports the slabs, so the estimate of an index built from them
+ // must not include them: it can't depend on how long the strings in the
+ // symbol and ref slabs are.
+ auto Estimate = [](IndexType Type, size_t StringSize) {
+ SymbolSlab::Builder Slab;
+ for (int I = 0; I < 100; ++I) {
+ std::string Name = std::to_string(I);
+ std::string Doc = std::string(StringSize, 'x') + Name;
+ Symbol Sym = symbol(Name);
+ Sym.Documentation = Doc;
+ Slab.insert(Sym);
+ }
+ std::string Path = std::string(100 * StringSize, 'x') + ".cc";
+ FileSymbols FS(IndexContents::All, true);
+ FS.update("f1", std::make_unique<SymbolSlab>(std::move(Slab).build()),
+ refSlab(SymbolID("1"), Path.c_str()), nullptr, false);
+ return FS.buildIndex(Type)->estimateMemoryUsage();
+ };
+ for (IndexType Type : {IndexType::Light, IndexType::Heavy})
+ EXPECT_EQ(Estimate(Type, 0), Estimate(Type, 1000));
+}
+
TEST(FileIndexTest, MacrosFromMainFile) {
FileIndex Idx(true);
TestTU TU;
``````````
</details>
https://github.com/llvm/llvm-project/pull/242888
More information about the cfe-commits
mailing list