<div dir="ltr">Apologies. This is a GCC bug I wasn't familiar with, and my first guess at a fix was wrong. (<a href="https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58541">https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58541</a>)<div>r319604 should fix this.<br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Dec 1, 2017 at 10:20 PM, Galina Kistanova <span dir="ltr"><<a href="mailto:gkistanova@gmail.com" target="_blank">gkistanova@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><span style="font-family:monospace,monospace">Hello Sam,<br><br>This commit broke one of our bots:<br><a href="http://lab.llvm.org:8011/builders/clang-x86_64-linux-abi-test/builds/18908" target="_blank">http://lab.llvm.org:8011/<wbr>builders/clang-x86_64-linux-<wbr>abi-test/builds/18908</a><br><br>. . .<br>FAILED: tools/clang/tools/extra/<wbr>clangd/CMakeFiles/clangDaemon.<wbr>dir/FuzzyMatch.cpp.o <br>/usr/bin/c++ -DGTEST_HAS_RTTI=0 -D_DEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Itools/clang/tools/extra/<wbr>clangd -I/home/buildslave/<wbr>buildslave1a/clang-x86_64-<wbr>linux-abi-test/llvm/tools/<wbr>clang/tools/extra/clangd -I/home/buildslave/<wbr>buildslave1a/clang-x86_64-<wbr>linux-abi-test/llvm/tools/<wbr>clang/include -Itools/clang/include -Iinclude -I/home/buildslave/<wbr>buildslave1a/clang-x86_64-<wbr>linux-abi-test/llvm/include -fPIC -fvisibility-inlines-hidden -std=c++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-<wbr>initializers -pedantic -Wno-long-long -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment -ffunction-sections -fdata-sections -fno-common -Woverloaded-virtual -fno-strict-aliasing -O3 -UNDEBUG -fno-exceptions -fno-rtti -MD -MT tools/clang/tools/extra/<wbr>clangd/CMakeFiles/clangDaemon.<wbr>dir/FuzzyMatch.cpp.o -MF tools/clang/tools/extra/<wbr>clangd/CMakeFiles/clangDaemon.<wbr>dir/FuzzyMatch.cpp.o.d -o tools/clang/tools/extra/<wbr>clangd/CMakeFiles/clangDaemon.<wbr>dir/FuzzyMatch.cpp.o -c /home/buildslave/buildslave1a/<wbr>clang-x86_64-linux-abi-test/<wbr>llvm/tools/clang/tools/extra/<wbr>clangd/FuzzyMatch.cpp<br>/home/buildslave/buildslave1a/<wbr>clang-x86_64-linux-abi-test/<wbr>llvm/tools/clang/tools/extra/<wbr>clangd/FuzzyMatch.cpp:64:25: error: redeclaration ‘clang::clangd::FuzzyMatcher::<wbr>MaxPat’ differs in ‘constexpr’<br> const int FuzzyMatcher::MaxPat;<br> ^<br>In file included from /home/buildslave/buildslave1a/<wbr>clang-x86_64-linux-abi-test/<wbr>llvm/tools/clang/tools/extra/<wbr>clangd/FuzzyMatch.cpp:57:0:<br>/home/buildslave/buildslave1a/<wbr>clang-x86_64-linux-abi-test/<wbr>llvm/tools/clang/tools/extra/<wbr>clangd/FuzzyMatch.h:45:24: error: from previous declaration ‘clang::clangd::FuzzyMatcher::<wbr>MaxPat’<br> constexpr static int MaxPat = 63, MaxWord = 127;<br> ^<br>/home/buildslave/buildslave1a/<wbr>clang-x86_64-linux-abi-test/<wbr>llvm/tools/clang/tools/extra/<wbr>clangd/FuzzyMatch.cpp:64:25: error: declaration of ‘constexpr const int clang::clangd::FuzzyMatcher::<wbr>MaxPat’ outside of class is not definition [-fpermissive]<br> const int FuzzyMatcher::MaxPat;<br> ^<br>/home/buildslave/buildslave1a/<wbr>clang-x86_64-linux-abi-test/<wbr>llvm/tools/clang/tools/extra/<wbr>clangd/FuzzyMatch.cpp:65:25: error: redeclaration ‘clang::clangd::FuzzyMatcher::<wbr>MaxWord’ differs in ‘constexpr’<br> const int FuzzyMatcher::MaxWord;<br> ^<br>In file included from /home/buildslave/buildslave1a/<wbr>clang-x86_64-linux-abi-test/<wbr>llvm/tools/clang/tools/extra/<wbr>clangd/FuzzyMatch.cpp:57:0:<br>/home/buildslave/buildslave1a/<wbr>clang-x86_64-linux-abi-test/<wbr>llvm/tools/clang/tools/extra/<wbr>clangd/FuzzyMatch.h:45:37: error: from previous declaration ‘clang::clangd::FuzzyMatcher::<wbr>MaxWord’<br> constexpr static int MaxPat = 63, MaxWord = 127;<br> <wbr> ^<br>/home/buildslave/buildslave1a/<wbr>clang-x86_64-linux-abi-test/<wbr>llvm/tools/clang/tools/extra/<wbr>clangd/FuzzyMatch.cpp:65:25: error: declaration of ‘constexpr const int clang::clangd::FuzzyMatcher::<wbr>MaxWord’ outside of class is not definition [-fpermissive]<br> const int FuzzyMatcher::MaxWord;<br> ^<br><br>Please have a look?<br><br>Thanks<br><br>Galina<br></span></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Dec 1, 2017 at 9:08 AM, Sam McCall via cfe-commits <span dir="ltr"><<a href="mailto:cfe-commits@lists.llvm.org" target="_blank">cfe-commits@lists.llvm.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Author: sammccall<br>
Date: Fri Dec 1 09:08:02 2017<br>
New Revision: 319557<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=319557&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-pr<wbr>oject?rev=319557&view=rev</a><br>
Log:<br>
[clangd] Fuzzy match scorer<br>
<br>
Summary:<br>
This will be used for rescoring code completion results based on partial<br>
identifiers.<br>
Short-term use:<br>
- we want to limit the number of code completion results returned to<br>
improve performance of global completion. The scorer will be used to<br>
rerank the results to return when the user has applied a filter.<br>
Long-term use case:<br>
- ranking of completion results from in-memory index<br>
- merging of completion results from multiple sources (merging usually<br>
works best when done at the component-score level, rescoring the<br>
fuzzy-match quality avoids different backends needing to have<br>
comparable scores)<br>
<br>
Reviewers: ilya-biryukov<br>
<br>
Subscribers: cfe-commits, mgorny<br>
<br>
Differential Revision: <a href="https://reviews.llvm.org/D40060" rel="noreferrer" target="_blank">https://reviews.llvm.org/D4006<wbr>0</a><br>
<br>
Added:<br>
clang-tools-extra/trunk/clangd<wbr>/FuzzyMatch.cpp<br>
clang-tools-extra/trunk/clangd<wbr>/FuzzyMatch.h<br>
clang-tools-extra/trunk/unitte<wbr>sts/clangd/FuzzyMatchTests.cpp<br>
Modified:<br>
clang-tools-extra/trunk/clangd<wbr>/CMakeLists.txt<br>
clang-tools-extra/trunk/unitte<wbr>sts/clangd/CMakeLists.txt<br>
<br>
Modified: clang-tools-extra/trunk/clangd<wbr>/CMakeLists.txt<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/CMakeLists.txt?rev=319557&r1=319556&r2=319557&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-pr<wbr>oject/clang-tools-extra/trunk/<wbr>clangd/CMakeLists.txt?rev=<wbr>319557&r1=319556&r2=319557&<wbr>view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- clang-tools-extra/trunk/clangd<wbr>/CMakeLists.txt (original)<br>
+++ clang-tools-extra/trunk/clangd<wbr>/CMakeLists.txt Fri Dec 1 09:08:02 2017<br>
@@ -8,6 +8,7 @@ add_clang_library(clangDaemon<br>
ClangdUnit.cpp<br>
ClangdUnitStore.cpp<br>
DraftStore.cpp<br>
+ FuzzyMatch.cpp<br>
GlobalCompilationDatabase.cpp<br>
JSONExpr.cpp<br>
JSONRPCDispatcher.cpp<br>
<br>
Added: clang-tools-extra/trunk/clangd<wbr>/FuzzyMatch.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/FuzzyMatch.cpp?rev=319557&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-pr<wbr>oject/clang-tools-extra/trunk/<wbr>clangd/FuzzyMatch.cpp?rev=<wbr>319557&view=auto</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- clang-tools-extra/trunk/clangd<wbr>/FuzzyMatch.cpp (added)<br>
+++ clang-tools-extra/trunk/clangd<wbr>/FuzzyMatch.cpp Fri Dec 1 09:08:02 2017<br>
@@ -0,0 +1,373 @@<br>
+//===--- FuzzyMatch.h - Approximate identifier matching ---------*- C++-*-===//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===------------------------<wbr>------------------------------<wbr>----------------===//<br>
+//<br>
+// To check for a match between a Pattern ('u_p') and a Word ('unique_ptr'),<br>
+// we consider the possible partial match states:<br>
+//<br>
+// u n i q u e _ p t r<br>
+// +---------------------<br>
+// |A . . . . . . . . . .<br>
+// u|<br>
+// |. . . . . . . . . . .<br>
+// _|<br>
+// |. . . . . . . O . . .<br>
+// p|<br>
+// |. . . . . . . . . . B<br>
+//<br>
+// Each dot represents some prefix of the pattern being matched against some<br>
+// prefix of the word.<br>
+// - A is the initial state: '' matched against ''<br>
+// - O is an intermediate state: 'u_' matched against 'unique_'<br>
+// - B is the target state: 'u_p' matched against 'unique_ptr'<br>
+//<br>
+// We aim to find the best path from A->B.<br>
+// - Moving right (consuming a word character)<br>
+// Always legal: not all word characters must match.<br>
+// - Moving diagonally (consuming both a word and pattern character)<br>
+// Legal if the characters match.<br>
+// - Moving down (consuming a pattern character) is never legal.<br>
+// Never legal: all pattern characters must match something.<br>
+//<br>
+// The scoring is based on heuristics:<br>
+// - when matching a character, apply a bonus or penalty depending on the<br>
+// match quality (does case match, do word segments align, etc)<br>
+// - when skipping a character, apply a penalty if it hurts the match<br>
+// (it starts a word segment, or splits the matched region, etc)<br>
+//<br>
+// These heuristics require the ability to "look backward" one character, to<br>
+// see whether it was matched or not. Therefore the dynamic-programming matrix<br>
+// has an extra dimension (last character matched).<br>
+// Each entry also has an additional flag indicating whether the last-but-one<br>
+// character matched, which is needed to trace back through the scoring table<br>
+// and reconstruct the match.<br>
+//<br>
+// We treat strings as byte-sequences, so only ASCII has first-class support.<br>
+//<br>
+// This algorithm was inspired by VS code's client-side filtering, and aims<br>
+// to be mostly-compatible.<br>
+//<br>
+//===------------------------<wbr>------------------------------<wbr>----------------===//<br>
+<br>
+#include "FuzzyMatch.h"<br>
+#include "llvm/ADT/Optional.h"<br>
+#include "llvm/Support/Format.h"<br>
+<br>
+using namespace llvm;<br>
+using namespace clang::clangd;<br>
+<br>
+const int FuzzyMatcher::MaxPat;<br>
+const int FuzzyMatcher::MaxWord;<br>
+<br>
+static char lower(char C) { return C >= 'A' && C <= 'Z' ? C + ('a' - 'A') : C; }<br>
+// A "negative infinity" score that won't overflow.<br>
+// We use this to mark unreachable states and forbidden solutions.<br>
+// Score field is 15 bits wide, min value is -2^14, we use half of that.<br>
+static constexpr int AwfulScore = -(1 << 13);<br>
+static bool isAwful(int S) { return S < AwfulScore / 2; }<br>
+static constexpr int PerfectBonus = 3; // Perfect per-pattern-char score.<br>
+<br>
+FuzzyMatcher::FuzzyMatcher(St<wbr>ringRef Pattern)<br>
+ : PatN(std::min<int>(MaxPat, Pattern.size())), CaseSensitive(false),<br>
+ ScoreScale(float{1} / (PerfectBonus * PatN)), WordN(0) {<br>
+ memcpy(Pat, Pattern.data(), PatN);<br>
+ for (int I = 0; I < PatN; ++I) {<br>
+ LowPat[I] = lower(Pat[I]);<br>
+ CaseSensitive |= LowPat[I] != Pat[I];<br>
+ }<br>
+ Scores[0][0][Miss] = {0, Miss};<br>
+ Scores[0][0][Match] = {AwfulScore, Miss};<br>
+ for (int P = 0; P <= PatN; ++P)<br>
+ for (int W = 0; W < P; ++W)<br>
+ for (Action A : {Miss, Match})<br>
+ Scores[P][W][A] = {AwfulScore, Miss};<br>
+ calculateRoles(Pat, PatRole, PatN);<br>
+}<br>
+<br>
+Optional<float> FuzzyMatcher::match(StringRef Word) {<br>
+ if (!PatN)<br>
+ return 1;<br>
+ if (!(WordContainsPattern = init(Word)))<br>
+ return None;<br>
+ buildGraph();<br>
+ auto Best = std::max(Scores[PatN][WordN][M<wbr>iss].Score,<br>
+ Scores[PatN][WordN][Match].Sc<wbr>ore);<br>
+ if (isAwful(Best))<br>
+ return None;<br>
+ return ScoreScale * std::min(PerfectBonus * PatN, std::max<int>(0, Best));<br>
+}<br>
+<br>
+// Segmentation of words and patterns.<br>
+// A name like "fooBar_baz" consists of several parts foo, bar, baz.<br>
+// Aligning segmentation of word and pattern improves the fuzzy-match.<br>
+// For example: [lol] matches "LaughingOutLoud" better than "LionPopulation"<br>
+//<br>
+// First we classify each character into types (uppercase, lowercase, etc).<br>
+// Then we look at the sequence: e.g. [upper, lower] is the start of a segment.<br>
+<br>
+// We only distinguish the types of characters that affect segmentation.<br>
+// It's not obvious how to segment digits, we treat them as lowercase letters.<br>
+// As we don't decode UTF-8, we treat bytes over 127 as lowercase too.<br>
+// This means we require exact (case-sensitive) match.<br>
+enum FuzzyMatcher::CharType : char {<br>
+ Empty = 0, // Before-the-start and after-the-end (and control chars).<br>
+ Lower = 1, // Lowercase letters, digits, and non-ASCII bytes.<br>
+ Upper = 2, // Uppercase letters.<br>
+ Punctuation = 3, // ASCII punctuation (including Space)<br>
+};<br>
+<br>
+// We get CharTypes from a lookup table. Each is 2 bits, 4 fit in each byte.<br>
+// The top 6 bits of the char select the byte, the bottom 2 select the offset.<br>
+// e.g. 'q' = 010100 01 = byte 28 (55), bits 3-2 (01) -> Lower.<br>
+constexpr static uint8_t CharTypes[] = {<br>
+ 0x00, 0x00, 0x00, 0x00, // Control characters<br>
+ 0x00, 0x00, 0x00, 0x00, // Control characters<br>
+ 0xff, 0xff, 0xff, 0xff, // Punctuation<br>
+ 0x55, 0x55, 0xf5, 0xff, // Numbers->Lower, more Punctuation.<br>
+ 0xab, 0xaa, 0xaa, 0xaa, // @ and A-O<br>
+ 0xaa, 0xaa, 0xea, 0xff, // P-Z, more Punctuation.<br>
+ 0x57, 0x55, 0x55, 0x55, // ` and a-o<br>
+ 0x55, 0x55, 0xd5, 0x3f, // p-z, Punctuation, DEL.<br>
+ 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, // Bytes over 127 -> Lower.<br>
+ 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, // (probably UTF-8).<br>
+ 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55,<br>
+ 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55,<br>
+};<br>
+<br>
+// Each character's Role is the Head or Tail of a segment, or a Separator.<br>
+// e.g. XMLHttpRequest_Async<br>
+// +--+---+------ +----<br>
+// ^Head ^Tail ^Separator<br>
+enum FuzzyMatcher::CharRole : char {<br>
+ Unknown = 0, // Stray control characters or impossible states.<br>
+ Tail = 1, // Part of a word segment, but not the first character.<br>
+ Head = 2, // The first character of a word segment.<br>
+ Separator = 3, // Punctuation characters that separate word segments.<br>
+};<br>
+<br>
+// The Role can be determined from the Type of a character and its neighbors:<br>
+//<br>
+// Example | Chars | Type | Role<br>
+// ---------+--------------+----<wbr>-<br>
+// F(o)oBar | Foo | Ull | Tail<br>
+// Foo(B)ar | oBa | lUl | Head<br>
+// (f)oo | ^fo | Ell | Head<br>
+// H(T)TP | HTT | UUU | Tail<br>
+//<br>
+// Our lookup table maps a 6 bit key (Prev, Curr, Next) to a 2-bit Role.<br>
+// A byte packs 4 Roles. (Prev, Curr) selects a byte, Next selects the offset.<br>
+// e.g. Lower, Upper, Lower -> 01 10 01 -> byte 6 (aa), bits 3-2 (10) -> Head.<br>
+constexpr static uint8_t CharRoles[] = {<br>
+ // clang-format off<br>
+ // Curr= Empty Lower Upper Separ<br>
+ /* Prev=Empty */ 0x00, 0xaa, 0xaa, 0xff, // At start, Lower|Upper->Head<br>
+ /* Prev=Lower */ 0x00, 0x55, 0xaa, 0xff, // In word, Upper->Head;Lower->Tail<br>
+ /* Prev=Upper */ 0x00, 0x55, 0x59, 0xff, // Ditto, but U(U)U->Tail<br>
+ /* Prev=Separ */ 0x00, 0xaa, 0xaa, 0xff, // After separator, like at start<br>
+ // clang-format on<br>
+};<br>
+<br>
+template <typename T> static T packedLookup(const uint8_t *Data, int I) {<br>
+ return static_cast<T>((Data[I >> 2] >> ((I & 3) * 2)) & 3);<br>
+}<br>
+void FuzzyMatcher::calculateRoles(c<wbr>onst char *Text, CharRole *Out, int N) {<br>
+ // Types holds a sliding window of (Prev, Curr, Next) types.<br>
+ // Initial value is (Empty, Empty, type of Text[0]).<br>
+ int Types = packedLookup<CharType>(CharTyp<wbr>es, Text[0]);<br>
+ // Rotate slides in the type of the next character.<br>
+ auto Rotate = [&](CharType T) { Types = ((Types << 2) | T) & 0x3f; };<br>
+ for (int I = 0; I < N - 1; ++I) {<br>
+ // For each character, rotate in the next, and look up the role.<br>
+ Rotate(packedLookup<CharType>(<wbr>CharTypes, Text[I + 1]));<br>
+ *Out++ = packedLookup<CharRole>(CharRol<wbr>es, Types);<br>
+ }<br>
+ // For the last character, the "next character" is Empty.<br>
+ Rotate(Empty);<br>
+ *Out++ = packedLookup<CharRole>(CharRol<wbr>es, Types);<br>
+}<br>
+<br>
+// Sets up the data structures matching Word.<br>
+// Returns false if we can cheaply determine that no match is possible.<br>
+bool FuzzyMatcher::init(StringRef NewWord) {<br>
+ WordN = std::min<int>(MaxWord, NewWord.size());<br>
+ if (PatN > WordN)<br>
+ return false;<br>
+ memcpy(Word, NewWord.data(), WordN);<br>
+ for (int I = 0; I < WordN; ++I)<br>
+ LowWord[I] = lower(Word[I]);<br>
+<br>
+ // Cheap subsequence check.<br>
+ for (int W = 0, P = 0; P != PatN; ++W) {<br>
+ if (W == WordN)<br>
+ return false;<br>
+ if (LowWord[W] == LowPat[P])<br>
+ ++P;<br>
+ }<br>
+<br>
+ calculateRoles(Word, WordRole, WordN);<br>
+ return true;<br>
+}<br>
+<br>
+// The forwards pass finds the mappings of Pattern onto Word.<br>
+// Score = best score achieved matching Word[..W] against Pat[..P].<br>
+// Unlike other tables, indices range from 0 to N *inclusive*<br>
+// Matched = whether we chose to match Word[W] with Pat[P] or not.<br>
+//<br>
+// Points are mostly assigned to matched characters, with 1 being a good score<br>
+// and 3 being a great one. So we treat the score range as [0, 3 * PatN].<br>
+// This range is not strict: we can apply larger bonuses/penalties, or penalize<br>
+// non-matched characters.<br>
+void FuzzyMatcher::buildGraph() {<br>
+ for (int W = 0; W < WordN; ++W) {<br>
+ Scores[0][W + 1][Miss] = {Scores[0][W][Miss].Score - skipPenalty(W, Miss),<br>
+ Miss};<br>
+ Scores[0][W + 1][Match] = {AwfulScore, Miss};<br>
+ }<br>
+ for (int P = 0; P < PatN; ++P) {<br>
+ for (int W = P; W < WordN; ++W) {<br>
+ auto &Score = Scores[P + 1][W + 1], &PreMiss = Scores[P + 1][W];<br>
+<br>
+ auto MatchMissScore = PreMiss[Match].Score;<br>
+ auto MissMissScore = PreMiss[Miss].Score;<br>
+ if (P < PatN - 1) { // Skipping trailing characters is always free.<br>
+ MatchMissScore -= skipPenalty(W, Match);<br>
+ MissMissScore -= skipPenalty(W, Miss);<br>
+ }<br>
+ Score[Miss] = (MatchMissScore > MissMissScore)<br>
+ ? ScoreInfo{MatchMissScore, Match}<br>
+ : ScoreInfo{MissMissScore, Miss};<br>
+<br>
+ if (LowPat[P] != LowWord[W]) { // No match possible.<br>
+ Score[Match] = {AwfulScore, Miss};<br>
+ } else {<br>
+ auto &PreMatch = Scores[P][W];<br>
+ auto MatchMatchScore = PreMatch[Match].Score + matchBonus(P, W, Match);<br>
+ auto MissMatchScore = PreMatch[Miss].Score + matchBonus(P, W, Miss);<br>
+ Score[Match] = (MatchMatchScore > MissMatchScore)<br>
+ ? ScoreInfo{MatchMatchScore, Match}<br>
+ : ScoreInfo{MissMatchScore, Miss};<br>
+ }<br>
+ }<br>
+ }<br>
+}<br>
+<br>
+int FuzzyMatcher::skipPenalty(int W, Action Last) {<br>
+ int S = 0;<br>
+ if (WordRole[W] == Head) // Skipping a segment.<br>
+ S += 1;<br>
+ if (Last == Match) // Non-consecutive match.<br>
+ S += 2; // We'd rather skip a segment than split our match.<br>
+ return S;<br>
+}<br>
+<br>
+int FuzzyMatcher::matchBonus(int P, int W, Action Last) {<br>
+ assert(LowPat[P] == LowWord[W]);<br>
+ int S = 1;<br>
+ // Bonus: pattern so far is a (case-insensitive) prefix of the word.<br>
+ if (P == W) // We can't skip pattern characters, so we must have matched all.<br>
+ ++S;<br>
+ // Bonus: case matches, or a Head in the pattern aligns with one in the word.<br>
+ if ((Pat[P] == Word[W] && (CaseSensitive || P == W)) ||<br>
+ (PatRole[P] == Head && WordRole[W] == Head))<br>
+ ++S;<br>
+ // Penalty: matching inside a segment (and previous char wasn't matched).<br>
+ if (WordRole[W] == Tail && P && Last == Miss)<br>
+ S -= 3;<br>
+ // Penalty: a Head in the pattern matches in the middle of a word segment.<br>
+ if (PatRole[P] == Head && WordRole[W] == Tail)<br>
+ --S;<br>
+ // Penalty: matching the first pattern character in the middle of a segment.<br>
+ if (P == 0 && WordRole[W] == Tail)<br>
+ S -= 4;<br>
+ assert(S <= PerfectBonus);<br>
+ return S;<br>
+}<br>
+<br>
+llvm::SmallString<256> FuzzyMatcher::dumpLast(llvm::r<wbr>aw_ostream &OS) const {<br>
+ llvm::SmallString<256> Result;<br>
+ OS << "=== Match \"" << StringRef(Word, WordN) << "\" against ["<br>
+ << StringRef(Pat, PatN) << "] ===\n";<br>
+ if (!WordContainsPattern) {<br>
+ OS << "Substring check failed.\n";<br>
+ return Result;<br>
+ } else if (isAwful(std::max(Scores[PatN]<wbr>[WordN][Match].Score,<br>
+ Scores[PatN][WordN][Miss].Scor<wbr>e))) {<br>
+ OS << "Substring check passed, but all matches are forbidden\n";<br>
+ }<br>
+ if (!CaseSensitive)<br>
+ OS << "Lowercase query, so scoring ignores case\n";<br>
+<br>
+ // Traverse Matched table backwards to reconstruct the Pattern/Word mapping.<br>
+ // The Score table has cumulative scores, subtracting along this path gives<br>
+ // us the per-letter scores.<br>
+ Action Last =<br>
+ (Scores[PatN][WordN][Match].Sc<wbr>ore > Scores[PatN][WordN][Miss].Scor<wbr>e)<br>
+ ? Match<br>
+ : Miss;<br>
+ int S[MaxWord];<br>
+ Action A[MaxWord];<br>
+ for (int W = WordN - 1, P = PatN - 1; W >= 0; --W) {<br>
+ A[W] = Last;<br>
+ const auto &Cell = Scores[P + 1][W + 1][Last];<br>
+ if (Last == Match)<br>
+ --P;<br>
+ const auto &Prev = Scores[P + 1][W][Cell.Prev];<br>
+ S[W] = Cell.Score - Prev.Score;<br>
+ Last = Cell.Prev;<br>
+ }<br>
+ for (int I = 0; I < WordN; ++I) {<br>
+ if (A[I] == Match && (I == 0 || A[I - 1] == Miss))<br>
+ Result.push_back('[');<br>
+ if (A[I] == Miss && I > 0 && A[I - 1] == Match)<br>
+ Result.push_back(']');<br>
+ Result.push_back(Word[I]);<br>
+ }<br>
+ if (A[WordN - 1] == Match)<br>
+ Result.push_back(']');<br>
+<br>
+ for (char C : StringRef(Word, WordN))<br>
+ OS << " " << C << " ";<br>
+ OS << "\n";<br>
+ for (int I = 0, J = 0; I < WordN; I++)<br>
+ OS << " " << (A[I] == Match ? Pat[J++] : ' ') << " ";<br>
+ OS << "\n";<br>
+ for (int I = 0; I < WordN; I++)<br>
+ OS << format("%2d ", S[I]);<br>
+ OS << "\n";<br>
+<br>
+ OS << "\nSegmentation:";<br>
+ OS << "\n'" << StringRef(Word, WordN) << "'\n ";<br>
+ for (int I = 0; I < WordN; ++I)<br>
+ OS << "?-+ "[static_cast<int>(WordRole[I]<wbr>)];<br>
+ OS << "\n[" << StringRef(Pat, PatN) << "]\n ";<br>
+ for (int I = 0; I < PatN; ++I)<br>
+ OS << "?-+ "[static_cast<int>(PatRole[I])<wbr>];<br>
+ OS << "\n";<br>
+<br>
+ OS << "\nScoring table (last-Miss, last-Match):\n";<br>
+ OS << " | ";<br>
+ for (char C : StringRef(Word, WordN))<br>
+ OS << " " << C << " ";<br>
+ OS << "\n";<br>
+ OS << "-+----" << std::string(WordN * 4, '-') << "\n";<br>
+ for (int I = 0; I <= PatN; ++I) {<br>
+ for (Action A : {Miss, Match}) {<br>
+ OS << ((I && A == Miss) ? Pat[I - 1] : ' ') << "|";<br>
+ for (int J = 0; J <= WordN; ++J) {<br>
+ if (!isAwful(Scores[I][J][A].Scor<wbr>e))<br>
+ OS << format("%3d%c", Scores[I][J][A].Score,<br>
+ Scores[I][J][A].Prev == Match ? '*' : ' ');<br>
+ else<br>
+ OS << " ";<br>
+ }<br>
+ OS << "\n";<br>
+ }<br>
+ }<br>
+<br>
+ return Result;<br>
+}<br>
<br>
Added: clang-tools-extra/trunk/clangd<wbr>/FuzzyMatch.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/FuzzyMatch.h?rev=319557&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-pr<wbr>oject/clang-tools-extra/trunk/<wbr>clangd/FuzzyMatch.h?rev=319557<wbr>&view=auto</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- clang-tools-extra/trunk/clangd<wbr>/FuzzyMatch.h (added)<br>
+++ clang-tools-extra/trunk/clangd<wbr>/FuzzyMatch.h Fri Dec 1 09:08:02 2017<br>
@@ -0,0 +1,84 @@<br>
+//===--- FuzzyMatch.h - Approximate identifier matching ---------*- C++-*-===//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===------------------------<wbr>------------------------------<wbr>----------------===//<br>
+//<br>
+// This file implements fuzzy-matching of strings against identifiers.<br>
+// It indicates both the existence and quality of a match:<br>
+// 'eb' matches both 'emplace_back' and 'embed', the former has a better score.<br>
+//<br>
+//===------------------------<wbr>------------------------------<wbr>----------------===//<br>
+<br>
+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_<wbr>FUZZYMATCH_H<br>
+#define LLVM_CLANG_TOOLS_EXTRA_CLANGD_<wbr>FUZZYMATCH_H<br>
+<br>
+#include "llvm/ADT/Optional.h"<br>
+#include "llvm/ADT/SmallString.h"<br>
+#include "llvm/ADT/StringRef.h"<br>
+#include "llvm/Support/raw_ostream.h"<br>
+<br>
+namespace clang {<br>
+namespace clangd {<br>
+<br>
+// A matcher capable of matching and scoring strings against a single pattern.<br>
+// It's optimized for matching against many strings - match() does not allocate.<br>
+class FuzzyMatcher {<br>
+public:<br>
+ // Characters beyond MaxPat are ignored.<br>
+ FuzzyMatcher(llvm::StringRef Pattern);<br>
+<br>
+ // If Word matches the pattern, return a score in [0,1] (higher is better).<br>
+ // Characters beyond MaxWord are ignored.<br>
+ llvm::Optional<float> match(llvm::StringRef Word);<br>
+<br>
+ // Dump internal state from the last match() to the stream, for debugging.<br>
+ // Returns the pattern with [] around matched characters, e.g.<br>
+ // [u_p] + "unique_ptr" --> "[u]nique[_p]tr"<br>
+ llvm::SmallString<256> dumpLast(llvm::raw_ostream &) const;<br>
+<br>
+private:<br>
+ // We truncate the pattern and the word to bound the cost of matching.<br>
+ constexpr static int MaxPat = 63, MaxWord = 127;<br>
+ enum CharRole : char; // For segmentation.<br>
+ enum CharType : char; // For segmentation.<br>
+ enum Action { Miss = 0, Match = 1 };<br>
+<br>
+ bool init(llvm::StringRef Word);<br>
+ void buildGraph();<br>
+ void calculateRoles(const char *Text, CharRole *Out, int N);<br>
+ int skipPenalty(int W, Action Last);<br>
+ int matchBonus(int P, int W, Action Last);<br>
+<br>
+ // Pattern data is initialized by the constructor, then constant.<br>
+ char Pat[MaxPat]; // Pattern data<br>
+ int PatN; // Length<br>
+ char LowPat[MaxPat]; // Pattern in lowercase<br>
+ CharRole PatRole[MaxPat]; // Pattern segmentation info<br>
+ bool CaseSensitive; // Case-sensitive match if pattern has uppercase<br>
+ float ScoreScale; // Normalizes scores for the pattern length.<br>
+<br>
+ // Word data is initialized on each call to match(), mostly by init().<br>
+ char Word[MaxWord]; // Word data<br>
+ int WordN; // Length<br>
+ char LowWord[MaxWord]; // Word in lowercase<br>
+ CharRole WordRole[MaxWord]; // Word segmentation info<br>
+ bool WordContainsPattern; // Simple substring check<br>
+<br>
+ // Cumulative best-match score table.<br>
+ // Boundary conditions are filled in by the constructor.<br>
+ // The rest is repopulated for each match(), by buildGraph().<br>
+ struct ScoreInfo {<br>
+ signed int Score : 15;<br>
+ Action Prev : 1;<br>
+ };<br>
+ ScoreInfo Scores[MaxPat + 1][MaxWord + 1][/* Last Action */ 2];<br>
+};<br>
+<br>
+} // namespace clangd<br>
+} // namespace clang<br>
+<br>
+#endif<br>
<br>
Modified: clang-tools-extra/trunk/unitte<wbr>sts/clangd/CMakeLists.txt<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/clangd/CMakeLists.txt?rev=319557&r1=319556&r2=319557&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-pr<wbr>oject/clang-tools-extra/trunk/<wbr>unittests/clangd/CMakeLists.<wbr>txt?rev=319557&r1=319556&r2=<wbr>319557&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- clang-tools-extra/trunk/unitte<wbr>sts/clangd/CMakeLists.txt (original)<br>
+++ clang-tools-extra/trunk/unitte<wbr>sts/clangd/CMakeLists.txt Fri Dec 1 09:08:02 2017<br>
@@ -10,6 +10,7 @@ include_directories(<br>
<br>
add_extra_unittest(<wbr>ClangdTests<br>
ClangdTests.cpp<br>
+ FuzzyMatchTests.cpp<br>
JSONExprTests.cpp<br>
TraceTests.cpp<br>
)<br>
<br>
Added: clang-tools-extra/trunk/unitte<wbr>sts/clangd/FuzzyMatchTests.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/clangd/FuzzyMatchTests.cpp?rev=319557&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-pr<wbr>oject/clang-tools-extra/trunk/<wbr>unittests/clangd/FuzzyMatchTes<wbr>ts.cpp?rev=319557&view=auto</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- clang-tools-extra/trunk/unitte<wbr>sts/clangd/FuzzyMatchTests.cpp (added)<br>
+++ clang-tools-extra/trunk/unitte<wbr>sts/clangd/FuzzyMatchTests.cpp Fri Dec 1 09:08:02 2017<br>
@@ -0,0 +1,252 @@<br>
+//===-- FuzzyMatchTests.cpp - String fuzzy matcher tests --------*- C++ -*-===//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===------------------------<wbr>------------------------------<wbr>----------------===//<br>
+<br>
+#include "FuzzyMatch.h"<br>
+<br>
+#include "llvm/ADT/StringExtras.h"<br>
+#include "gmock/gmock.h"<br>
+#include "gtest/gtest.h"<br>
+<br>
+namespace clang {<br>
+namespace clangd {<br>
+namespace {<br>
+using namespace llvm;<br>
+using testing::Not;<br>
+<br>
+struct ExpectedMatch {<br>
+ ExpectedMatch(StringRef Annotated) : Word(Annotated), Annotated(Annotated) {<br>
+ for (char C : "[]")<br>
+ Word.erase(std::remove(Word.be<wbr>gin(), Word.end(), C), Word.end());<br>
+ }<br>
+ std::string Word;<br>
+ StringRef Annotated;<br>
+};<br>
+raw_ostream &operator<<(raw_ostream &OS, const ExpectedMatch &M) {<br>
+ return OS << "'" << M.Word << "' as " << M.Annotated;<br>
+}<br>
+<br>
+struct MatchesMatcher : public testing::MatcherInterface<Stri<wbr>ngRef> {<br>
+ ExpectedMatch Candidate;<br>
+ MatchesMatcher(ExpectedMatch Candidate) : Candidate(std::move(Candidate)<wbr>) {}<br>
+<br>
+ void DescribeTo(::std::ostream *OS) const override {<br>
+ raw_os_ostream(*OS) << "Matches " << Candidate;<br>
+ }<br>
+<br>
+ bool MatchAndExplain(StringRef Pattern,<br>
+ testing::MatchResultListener *L) const override {<br>
+ std::unique_ptr<raw_ostream> OS(<br>
+ L->stream() ? (raw_ostream *)(new raw_os_ostream(*L->stream()))<br>
+ : new raw_null_ostream());<br>
+ FuzzyMatcher Matcher(Pattern);<br>
+ auto Result = Matcher.match(Candidate.Word);<br>
+ auto AnnotatedMatch = Matcher.dumpLast(*OS << "\n");<br>
+ return Result && AnnotatedMatch == Candidate.Annotated;<br>
+ }<br>
+};<br>
+<br>
+// Accepts patterns that match a given word.<br>
+// Dumps the debug tables on match failure.<br>
+testing::Matcher<StringRef> matches(StringRef M) {<br>
+ return testing::MakeMatcher<StringRef<wbr>>(new MatchesMatcher(M));<br>
+}<br>
+<br>
+TEST(FuzzyMatch, Matches) {<br>
+ EXPECT_THAT("u_p", matches("[u]nique[_p]tr"));<br>
+ EXPECT_THAT("up", matches("[u]nique_[p]tr"));<br>
+ EXPECT_THAT("uq", matches("[u]ni[q]ue_ptr"));<br>
+ EXPECT_THAT("qp", Not(matches("unique_ptr")));<br>
+ EXPECT_THAT("log", Not(matches("SVGFEMorphologyEl<wbr>ement")));<br>
+<br>
+ EXPECT_THAT("tit", matches("win.[tit]"));<br>
+ EXPECT_THAT("title", matches("win.[title]"));<br>
+ EXPECT_THAT("WordCla", matches("[Word]Character[Cla]s<wbr>sifier"));<br>
+ EXPECT_THAT("WordCCla", matches("[WordC]haracter[Cla]s<wbr>sifier"));<br>
+<br>
+ EXPECT_THAT("dete", Not(matches("editor.quickSugge<wbr>stionsDelay")));<br>
+<br>
+ EXPECT_THAT("highlight", matches("editorHover[Highlight<wbr>]"));<br>
+ EXPECT_THAT("hhighlight", matches("editor[H]over[Highlig<wbr>ht]"));<br>
+ EXPECT_THAT("dhhighlight", Not(matches("editorHoverHighli<wbr>ght")));<br>
+<br>
+ EXPECT_THAT("-moz", matches("[-moz]-foo"));<br>
+ EXPECT_THAT("moz", matches("-[moz]-foo"));<br>
+ EXPECT_THAT("moza", matches("-[moz]-[a]nimation"))<wbr>;<br>
+<br>
+ EXPECT_THAT("ab", matches("[ab]A"));<br>
+ EXPECT_THAT("ccm", matches("[c]a[cm]elCase"));<br>
+ EXPECT_THAT("bti", Not(matches("the_black_knight"<wbr>)));<br>
+ EXPECT_THAT("ccm", Not(matches("camelCase")));<br>
+ EXPECT_THAT("cmcm", Not(matches("camelCase")));<br>
+ EXPECT_THAT("BK", matches("the_[b]lack_[k]night"<wbr>));<br>
+ EXPECT_THAT("KeyboardLayout=", Not(matches("KeyboardLayout"))<wbr>);<br>
+ EXPECT_THAT("LLL", matches("SVisual[L]ogger[L]ogs<wbr>[L]ist"));<br>
+ EXPECT_THAT("LLLL", Not(matches("SVilLoLosLi")));<br>
+ EXPECT_THAT("LLLL", Not(matches("SVisualLoggerLogs<wbr>List")));<br>
+ EXPECT_THAT("TEdit", matches("[T]ext[Edit]"));<br>
+ EXPECT_THAT("TEdit", matches("[T]ext[Edit]or"));<br>
+ EXPECT_THAT("TEdit", matches("[Te]xte[dit]"));<br>
+ EXPECT_THAT("TEdit", matches("[t]ext_[edit]"));<br>
+ EXPECT_THAT("TEditDit", matches("[T]ext[Edit]or[D]ecor<wbr>at[i]on[T]ype"));<br>
+ EXPECT_THAT("TEdit", matches("[T]ext[Edit]orDecorat<wbr>ionType"));<br>
+ EXPECT_THAT("Tedit", matches("[T]ext[Edit]"));<br>
+ EXPECT_THAT("ba", Not(matches("?AB?")));<br>
+ EXPECT_THAT("bkn", matches("the_[b]lack_[kn]ight"<wbr>));<br>
+ EXPECT_THAT("bt", matches("the_[b]lack_knigh[t]"<wbr>));<br>
+ EXPECT_THAT("ccm", matches("[c]amelCase[cm]"));<br>
+ EXPECT_THAT("fdm", matches("[f]in[dM]odel"));<br>
+ EXPECT_THAT("fob", matches("[fo]o[b]ar"));<br>
+ EXPECT_THAT("fobz", Not(matches("foobar")));<br>
+ EXPECT_THAT("foobar", matches("[foobar]"));<br>
+ EXPECT_THAT("form", matches("editor.[form]atOnSave<wbr>"));<br>
+ EXPECT_THAT("g p", matches("[G]it:[ P]ull"));<br>
+ EXPECT_THAT("g p", matches("[G]it:[ P]ull"));<br>
+ EXPECT_THAT("gip", matches("[Gi]t: [P]ull"));<br>
+ EXPECT_THAT("gip", matches("[Gi]t: [P]ull"));<br>
+ EXPECT_THAT("gp", matches("[G]it: [P]ull"));<br>
+ EXPECT_THAT("gp", matches("[G]it_Git_[P]ull"));<br>
+ EXPECT_THAT("is", matches("[I]mport[S]tatement")<wbr>);<br>
+ EXPECT_THAT("is", matches("[is]Valid"));<br>
+ EXPECT_THAT("lowrd", matches("[low]Wo[rd]"));<br>
+ EXPECT_THAT("myvable", matches("[myva]ria[ble]"));<br>
+ EXPECT_THAT("no", Not(matches("")));<br>
+ EXPECT_THAT("no", Not(matches("match")));<br>
+ EXPECT_THAT("ob", Not(matches("foobar")));<br>
+ EXPECT_THAT("sl", matches("[S]Visual[L]oggerLogs<wbr>List"));<br>
+ EXPECT_THAT("sllll", matches("[S]Visua[lL]ogger[L]o<wbr>gs[L]ist"));<br>
+ EXPECT_THAT("Three", matches("H[T]ML[HRE]l[e]ment")<wbr>);<br>
+ EXPECT_THAT("Three", matches("[Three]"));<br>
+ EXPECT_THAT("fo", Not(matches("barfoo")));<br>
+ EXPECT_THAT("fo", matches("bar_[fo]o"));<br>
+ EXPECT_THAT("fo", matches("bar_[Fo]o"));<br>
+ EXPECT_THAT("fo", matches("bar [fo]o"));<br>
+ EXPECT_THAT("fo", matches("bar.[fo]o"));<br>
+ EXPECT_THAT("fo", matches("bar/[fo]o"));<br>
+ EXPECT_THAT("fo", matches("bar\\[fo]o"));<br>
+<br>
+ EXPECT_THAT(<br>
+ "aaaaaa",<br>
+ matches("[aaaaaa]aaaaaaaaaaaaa<wbr>aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa<wbr>aaaaaaaaaaaa"<br>
+ "aaaaaaaaaaaaaaaaaaaaaaaaaaaaa<wbr>aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa<wbr>aa"));<br>
+ EXPECT_THAT("baba", Not(matches("ababababab")));<br>
+ EXPECT_THAT("fsfsfs", Not(matches("dsafdsafdsafdsafd<wbr>safdsafdsafasdfdsa")));<br>
+ EXPECT_THAT("fsfsfsfsfsfsfsf",<br>
+ Not(matches("dsafdsafdsafdsafd<wbr>safdsafdsafasdfdsafdsafdsafdsa<wbr>fdsfd"<br>
+ "safdsfdfdfasdnfdsajfndsjnafjn<wbr>dsajlknfdsa")));<br>
+<br>
+ EXPECT_THAT(" g", matches("[ g]roup"));<br>
+ EXPECT_THAT("g", matches(" [g]roup"));<br>
+ EXPECT_THAT("g g", Not(matches(" groupGroup")));<br>
+ EXPECT_THAT("g g", matches(" [g]roup[ G]roup"));<br>
+ EXPECT_THAT(" g g", matches("[ ] [g]roup[ G]roup"));<br>
+ EXPECT_THAT("zz", matches("[zz]Group"));<br>
+ EXPECT_THAT("zzg", matches("[zzG]roup"));<br>
+ EXPECT_THAT("g", matches("zz[G]roup"));<br>
+<br>
+ EXPECT_THAT("aaaa", matches("_a_[aaaa]")); // Prefer consecutive.<br>
+ EXPECT_THAT("printf", matches("s[printf]"));<br>
+ EXPECT_THAT("str", matches("o[str]eam"));<br>
+}<br>
+<br>
+struct RankMatcher : public testing::MatcherInterface<Stri<wbr>ngRef> {<br>
+ std::vector<ExpectedMatch> RankedStrings;<br>
+ RankMatcher(std::initializer_l<wbr>ist<ExpectedMatch> RankedStrings)<br>
+ : RankedStrings(RankedStrings) {}<br>
+<br>
+ void DescribeTo(::std::ostream *OS) const override {<br>
+ raw_os_ostream O(*OS);<br>
+ O << "Ranks strings in order: [";<br>
+ for (const auto &Str : RankedStrings)<br>
+ O << "\n\t" << Str;<br>
+ O << "\n]";<br>
+ }<br>
+<br>
+ bool MatchAndExplain(StringRef Pattern,<br>
+ testing::MatchResultListener *L) const override {<br>
+ std::unique_ptr<raw_ostream> OS(<br>
+ L->stream() ? (raw_ostream *)(new raw_os_ostream(*L->stream()))<br>
+ : new raw_null_ostream());<br>
+ FuzzyMatcher Matcher(Pattern);<br>
+ const ExpectedMatch *LastMatch;<br>
+ Optional<float> LastScore;<br>
+ bool Ok = true;<br>
+ for (const auto &Str : RankedStrings) {<br>
+ auto Score = Matcher.match(Str.Word);<br>
+ if (!Score) {<br>
+ *OS << "\nDoesn't match '" << Str.Word << "'";<br>
+ Matcher.dumpLast(*OS << "\n");<br>
+ Ok = false;<br>
+ } else {<br>
+ std::string Buf;<br>
+ llvm::raw_string_ostream Info(Buf);<br>
+ auto AnnotatedMatch = Matcher.dumpLast(Info);<br>
+<br>
+ if (AnnotatedMatch != Str.Annotated) {<br>
+ *OS << "\nMatched " << Str.Word << " as " << AnnotatedMatch<br>
+ << " instead of " << Str.Annotated << "\n"<br>
+ << Info.str();<br>
+ Ok = false;<br>
+ } else if (LastScore && *LastScore < *Score) {<br>
+ *OS << "\nRanks '" << Str.Word << "'=" << *Score << " above '"<br>
+ << LastMatch->Word << "'=" << *LastScore << "\n"<br>
+ << Info.str();<br>
+ Matcher.match(LastMatch->Word)<wbr>;<br>
+ Matcher.dumpLast(*OS << "\n");<br>
+ Ok = false;<br>
+ }<br>
+ }<br>
+ LastMatch = &Str;<br>
+ LastScore = Score;<br>
+ }<br>
+ return Ok;<br>
+ }<br>
+};<br>
+<br>
+// Accepts patterns that match all the strings and rank them in the given order.<br>
+// Dumps the debug tables on match failure.<br>
+template <typename... T> testing::Matcher<StringRef> ranks(T... RankedStrings) {<br>
+ return testing::MakeMatcher<StringRef<wbr>>(<br>
+ new RankMatcher{ExpectedMatch(Rank<wbr>edStrings)...});<br>
+}<br>
+<br>
+TEST(FuzzyMatch, Ranking) {<br>
+ EXPECT_THAT("eb", ranks("[e]mplace_[b]ack", "[e]m[b]ed"));<br>
+ EXPECT_THAT("cons",<br>
+ ranks("[cons]ole", "[Cons]ole", "ArrayBuffer[Cons]tructor"));<br>
+ EXPECT_THAT("foo", ranks("[foo]", "[Foo]"));<br>
+ EXPECT_THAT("onMess",<br>
+ ranks("[onMess]age", "[onmess]age", "[on]This[M]ega[Es]cape[s]"));<br>
+ EXPECT_THAT("CC", ranks("[C]amel[C]ase", "[c]amel[C]ase"));<br>
+ EXPECT_THAT("cC", ranks("[c]amel[C]ase", "[C]amel[C]ase"));<br>
+ EXPECT_THAT("p", ranks("[p]arse", "[p]osix", "[p]afdsa", "[p]ath", "[p]"));<br>
+ EXPECT_THAT("pa", ranks("[pa]rse", "[pa]th", "[pa]fdsa"));<br>
+ EXPECT_THAT("log", ranks("[log]", "Scroll[Log]icalPosition"));<br>
+ EXPECT_THAT("e", ranks("[e]lse", "Abstract[E]lement"));<br>
+ EXPECT_THAT("workbench.sideb",<br>
+ ranks("[workbench.sideB]ar.loc<wbr>ation",<br>
+ "[workbench.]editor.default[Si<wbr>deB]ySideLayout"));<br>
+ EXPECT_THAT("editor.r", ranks("[editor.r]enderControlC<wbr>haracter",<br>
+ "[editor.]overview[R]ulerlanes<wbr>",<br>
+ "diff[Editor.r]enderSideBySide<wbr>"));<br>
+ EXPECT_THAT("-mo", ranks("[-mo]z-columns", "[-]ms-ime-[mo]de"));<br>
+ EXPECT_THAT("convertModelPosit<wbr>ion",<br>
+ ranks("[convertModelPosition]T<wbr>oViewPosition",<br>
+ "[convert]ViewTo[ModelPosition<wbr>]"));<br>
+ EXPECT_THAT("is", ranks("[is]ValidViewletId", "[i]mport [s]tatement"));<br>
+ EXPECT_THAT("title", ranks("window.[title]",<br>
+ "files.[t]r[i]m[T]rai[l]ingWh<wbr>it[e]space"));<br>
+ EXPECT_THAT("strcpy", ranks("[strcpy]", "[strcpy]_s", "[str]n[cpy]"));<br>
+ EXPECT_THAT("close", ranks("workbench.quickOpen.[cl<wbr>ose]OnFocusOut",<br>
+ "[c]ss.[l]int.imp[o]rt[S]tat[<wbr>e]ment",<br>
+ "[c]<a href="http://ss.co" rel="noreferrer" target="_blank">ss.co</a>[lo]rDecorator[s].[<wbr>e]nable"));<br>
+}<br>
+<br>
+} // namespace<br>
+} // namespace clangd<br>
+} // namespace clang<br>
<br>
<br>
______________________________<wbr>_________________<br>
cfe-commits mailing list<br>
<a href="mailto:cfe-commits@lists.llvm.org" target="_blank">cfe-commits@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/cfe-commits</a><br>
</blockquote></div><br></div>
</blockquote></div><br></div>