[PATCH] D66031: clangd: use -j for background index pool

[Ben Jackson via Phabricator via cfe-commits]

puremourning created this revision.
puremourning added a reviewer: kadircet.
Herald added subscribers: cfe-commits, arphaman, jkorous, MaskRay, ilya-biryukov.
Herald added a project: clang.

clangd supports a -j option to limit the amount of threads to use for parsing
TUs. However, when using -background-index (the default in later versions of
clangd), the parallelism used by clangd defaults to the hardware_parallelisn,
i.e. number of physical cores.

On shared hardware environments, with large projects, this can significantly
affect performance with no way to tune it down.

This change makes the -j parameter apply equally to parsing and background
index. It's not perfect, because the total number of threads is 2x the -j value,
which may still be unexpected. But at least this change allows users to prevent
clangd using all CPU cores.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D66031

Files:
  clang-tools-extra/clangd/ClangdServer.cpp
  clang-tools-extra/clangd/tool/ClangdMain.cpp


Index: clang-tools-extra/clangd/tool/ClangdMain.cpp
===================================================================
--- clang-tools-extra/clangd/tool/ClangdMain.cpp
+++ clang-tools-extra/clangd/tool/ClangdMain.cpp
@@ -308,7 +308,7 @@
 opt<bool> Sync{
     "sync",
     cat(Misc),
-    desc("Parse on main thread. If set, -j is ignored"),
+    desc("Parse on main thread. If set, -j is only applied to background index"),
     init(false),
     Hidden,
 };
Index: clang-tools-extra/clangd/ClangdServer.cpp
===================================================================
--- clang-tools-extra/clangd/ClangdServer.cpp
+++ clang-tools-extra/clangd/ClangdServer.cpp
@@ -43,6 +43,7 @@
 #include <future>
 #include <memory>
 #include <mutex>
+#include <type_traits>
 
 namespace clang {
 namespace clangd {
@@ -141,10 +142,19 @@
   if (Opts.StaticIndex)
     AddIndex(Opts.StaticIndex);
   if (Opts.BackgroundIndex) {
-    BackgroundIdx = llvm::make_unique<BackgroundIndex>(
-        Context::current().clone(), FSProvider, CDB,
-        BackgroundIndexStorage::createDiskBackedStorageFactory(
-            [&CDB](llvm::StringRef File) { return CDB.getProjectInfo(File); }));
+    auto&& DBSF = BackgroundIndexStorage::createDiskBackedStorageFactory(
+            [&CDB](llvm::StringRef File) { return CDB.getProjectInfo(File); });
+
+    if (Opts.AsyncThreadsCount) {
+      BackgroundIdx = llvm::make_unique<BackgroundIndex>(
+          Context::current().clone(), FSProvider, CDB,
+          std::forward<decltype(DBSF)>(DBSF),
+          Opts.AsyncThreadsCount );
+    } else {
+      BackgroundIdx = llvm::make_unique<BackgroundIndex>(
+          Context::current().clone(), FSProvider, CDB,
+          std::forward<decltype(DBSF)>( DBSF ) );
+    }
     AddIndex(BackgroundIdx.get());
   }
   if (DynamicIdx)


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D66031.214447.patch
Type: text/x-patch
Size: 1816 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20190809/e9ad766f/attachment.bin>