[PATCH] D93452: [clangd] Trim memory periodically

[Quentin Chateau via Phabricator via llvm-commits]

qchateau added a comment.

> Sorry to mess you around like this, but I think I might have jumped the gun in asking this to be abstracted away in `Process.h`.

No problem, there is not much code anyway. The thought process behind the "right" solution is by far the most important.

> So I think my favorite option may be back to:
>
> - Have ClangdLSPServer::Options take a nullable `std::function<void()>` to abstract the (optional) platform-specific operation
> - Have ClangdMain contain platform #ifdefs and define+inject the wrapper that calls `malloc_trim`
> - Optionally: have a clangd or cmake flag to disable this even if glibc was detected. This provides a workaround for other allocators if we need it.

I'll go back to this then, and add all the other changes you suggested

-----------

In the meantime, I've done some testing with tcmalloc and jemalloc.

`jemalloc` does very good in this situation, the RSS goes down as soon as the memory is not required anymore:

- RSS goes down when the file is done parsing
- RSS goes down as soon as I close the file in VSCode

`tcmalloc` does fine, but is not ideal, the RSS does not grow to an unreasonable size, but does not go down when we release resources. It basically plateaus at the maximum usage (meaning what you use when parsing N files, N depending on the parallelism level). I'm not an expert on `tcmalloc` but from what I found it's supposed to release resources over time, which I failed to observe. `tcmalloc` does have an extension to actively release memory (`MallocExtension::instance()->ReleaseFreeMemory()`) which works as expected, bringing the RSS down. That being said, I'm not sure we need to do something about it, clangd will indeed waste a significant amount of RAM in this case, but [1] the usage is still reasonable and bounded and [2] it seems known that `tcmalloc` is a greedy allocator.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D93452/new/

https://reviews.llvm.org/D93452