[clang-tools-extra] r342866 - [clangd] Force Dex to respect symbol collector flags

[Kirill Bobyrev via cfe-commits]

Author: omtcyfz
Date: Mon Sep 24 01:45:18 2018
New Revision: 342866

URL: http://llvm.org/viewvc/llvm-project?rev=342866&view=rev
Log:
[clangd] Force Dex to respect symbol collector flags

`Dex` should utilize `FuzzyFindRequest.RestrictForCodeCompletion` flags
and omit symbols not meant for code completion when asked for it.

The measurements below were conducted with setting
`FuzzyFindRequest.RestrictForCodeCompletion` to `true` (so that it's
more realistic). Sadly, the average latency goes down, I suspect that is
mostly because of the empty queries where the number of posting lists is
critical.

| Metrics  | Before | After | Relative difference
| -----  | -----  | -----   | -----
| Cumulative query latency (7000 `FuzzyFindRequest`s over LLVM static index)  | 6182735043 ns    | 7202442053 ns | +16%
| Whole Index size | 81.24 MB    | 81.79 MB | +0.6%

Out of 292252 symbols collected from LLVM codebase 136926 appear to be
restricted for code completion.

Reviewers: ioeric

Differential Revision: https://reviews.llvm.org/D52357

Modified:
    clang-tools-extra/trunk/clangd/index/dex/Dex.cpp
    clang-tools-extra/trunk/unittests/clangd/DexTests.cpp

Modified: clang-tools-extra/trunk/clangd/index/dex/Dex.cpp
URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/index/dex/Dex.cpp?rev=342866&r1=342865&r2=342866&view=diff
==============================================================================
--- clang-tools-extra/trunk/clangd/index/dex/Dex.cpp (original)
+++ clang-tools-extra/trunk/clangd/index/dex/Dex.cpp Mon Sep 24 01:45:18 2018
@@ -22,6 +22,10 @@ namespace dex {
 
 namespace {
 
+// Mark symbols which are can be used for code completion.
+static const Token RestrictedForCodeCompletion =
+    Token(Token::Kind::Sentinel, "Restricted For Code Completion");
+
 // Returns the tokens which are given symbol's characteristics. Currently, the
 // generated tokens only contain fuzzy matching trigrams and symbol's scope,
 // but in the future this will also return path proximity tokens and other
@@ -39,6 +43,8 @@ std::vector<Token> generateSearchTokens(
     for (const auto &ProximityURI :
          generateProximityURIs(Sym.CanonicalDeclaration.FileURI))
       Result.emplace_back(Token::Kind::ProximityURI, ProximityURI);
+  if (Sym.Flags & Symbol::IndexedForCodeCompletion)
+    Result.emplace_back(RestrictedForCodeCompletion);
   return Result;
 }
 
@@ -175,6 +181,10 @@ bool Dex::fuzzyFind(const FuzzyFindReque
     TopLevelChildren.push_back(createOr(move(BoostingIterators)));
   }
 
+  if (Req.RestrictForCodeCompletion)
+    TopLevelChildren.push_back(
+        InvertedIndex.find(RestrictedForCodeCompletion)->second.iterator());
+
   // Use TRUE iterator if both trigrams and scopes from the query are not
   // present in the symbol index.
   auto QueryIterator = TopLevelChildren.empty()

Modified: clang-tools-extra/trunk/unittests/clangd/DexTests.cpp
URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/clangd/DexTests.cpp?rev=342866&r1=342865&r2=342866&view=diff
==============================================================================
--- clang-tools-extra/trunk/unittests/clangd/DexTests.cpp (original)
+++ clang-tools-extra/trunk/unittests/clangd/DexTests.cpp Mon Sep 24 01:45:18 2018
@@ -583,6 +583,20 @@ TEST(DexTest, Lookup) {
   EXPECT_THAT(lookup(*I, SymbolID("ns::nonono")), UnorderedElementsAre());
 }
 
+TEST(DexTest, SymbolIndexOptionsFilter) {
+  auto CodeCompletionSymbol = symbol("Completion");
+  auto NonCodeCompletionSymbol = symbol("NoCompletion");
+  CodeCompletionSymbol.Flags = Symbol::SymbolFlag::IndexedForCodeCompletion;
+  NonCodeCompletionSymbol.Flags = Symbol::SymbolFlag::None;
+  std::vector<Symbol> Symbols{CodeCompletionSymbol, NonCodeCompletionSymbol};
+  Dex I(Symbols, URISchemes);
+  FuzzyFindRequest Req;
+  Req.RestrictForCodeCompletion = false;
+  EXPECT_THAT(match(I, Req), ElementsAre("Completion", "NoCompletion"));
+  Req.RestrictForCodeCompletion = true;
+  EXPECT_THAT(match(I, Req), ElementsAre("Completion"));
+}
+
 TEST(DexTest, ProximityPathsBoosting) {
   auto RootSymbol = symbol("root::abc");
   RootSymbol.CanonicalDeclaration.FileURI = "unittest:///file.h";