[PATCH] D51154: [clangd] Log memory usage of DexIndex and MemIndex

[Eric Liu via Phabricator via cfe-commits]

ioeric added a comment.

Do we plan to expose an API in `ClangdServer` to allow C++ API users to track index memory usages?



================
Comment at: clang-tools-extra/clangd/index/FileIndex.cpp:123
+  size_t Bytes = Index.estimateMemoryUsage();
+  for (const auto &Scheme : URISchemes) {
+    // std::string contains chars with sizeof(char) == 1.
----------------
I think the URI scheme names should be negligible.


================
Comment at: clang-tools-extra/clangd/index/dex/DexIndex.cpp:180
+  Bytes += SymbolQuality.size() * sizeof(std::pair<const Symbol *, float>);
+  Bytes += InvertedIndex.size() * sizeof(Token);
+  {
----------------
kbobyrev wrote:
> sammccall wrote:
> > I think you're not counting the size of the actual symbols.
> > This is difficult to do precisely, but avoiding it seems misleading (we have "shared ownership" but it's basically exclusive). What's the plan here?
> As discussed offline: I should put a FIXME and leave it for later.
SymbolSlab provides a pretty good estimation of memory usage, and I think we should try to use if possible (as Sam suggested, it could be misleading without the size of index corpus). IIUC, the challenge here is that SymbolSlabs can be hiden behind `shared_ptr`, and index might not have access to the underlying slabs. If that's the case, I think it would be reasonable to ask callers of `build` to also pass in an estimated size of the hidden slabs?


https://reviews.llvm.org/D51154