[PATCH] D59300: [clangd] Tune the fuzzy-matching algorithm

Wed Mar 13 09:02:32 PDT 2019

ilya-biryukov added a comment.

Here are some initial measurements with the current metrics that we have. We seem to regress significantly in some cases. This is expected, since we only check the prefix matches there.
@ioeric is working on including the non-prefix (more concretely, initialism matches) into the metrics, we'll update when we get those metrics too.

  ==================================================================================================
                                          OVERALL (excl. CROSS_NAMESPACE)
  ==================================================================================================
    Total measurements: 109166 (+0)
    Average latency (ms): 219.219589233 (-7)
    All measurements:
  	MRR: 69.12 (-0.93)	Top-1: 59.54% (-1.20%)	Top-5: 81.21% (-0.48%)	Top-100: 96.06% (-0.10%)
    Full identifiers:
  	MRR: 97.68 (-0.54)	Top-1: 96.62% (-0.96%)	Top-5: 98.96% (-0.07%)	Top-100: 99.15% (-0.01%)
    Filter length 0-5:
  	MRR:      29.11 (+0.00)		62.02 (+0.41)		70.71 (-1.45)		73.41 (-1.18)		75.44 (-1.71)		78.78 (-2.50)
  	Top-1:    17.49% (+0.00%)		49.19% (+0.50%)		59.52% (-1.51%)		62.47% (-1.41%)		65.27% (-2.11%)		69.25% (-3.53%)
  	Top-5:    42.45% (+0.00%)		78.99% (+0.39%)		85.48% (-1.01%)		87.21% (-0.87%)		88.26% (-1.05%)		90.83% (-0.94%)
  	Top-100:  84.62% (+0.00%)		96.75% (+0.15%)		98.03% (-0.16%)		98.22% (-0.23%)		98.29% (-0.31%)		98.47% (-0.23%)
  ==================================================================================================
                                          DEFAULT
  ==================================================================================================
    Total measurements: 52054 (+0)
    Average latency (ms): 251.10333252 (-8)
    All measurements:
  	MRR: 61.26 (-1.13)	Top-1: 51.13% (-1.31%)	Top-5: 74.10% (-0.68%)	Top-100: 92.60% (-0.21%)
    Full identifiers:
  	MRR: 96.31 (-0.24)	Top-1: 95.08% (-0.33%)	Top-5: 97.93% (-0.09%)	Top-100: 98.30% (-0.01%)
    Filter length 0-5:
  	MRR:      16.78 (+0.00)		48.89 (+0.32)		62.07 (-2.05)		66.09 (-1.67)		69.65 (-2.45)		72.49 (-2.23)
  	Top-1:    9.06% (+0.00%)		34.63% (+0.25%)		49.11% (-2.03%)		53.07% (-1.99%)		58.01% (-2.83%)		61.95% (-2.74%)
  	Top-5:    23.57% (+0.00%)		68.10% (+0.52%)		79.78% (-1.35%)		82.73% (-1.26%)		84.63% (-1.66%)		86.08% (-1.19%)
  	Top-100:  70.73% (+0.00%)		93.67% (+0.29%)		96.41% (-0.31%)		96.77% (-0.47%)		96.92% (-0.63%)		97.07% (-0.47%)
  ==================================================================================================
                                          EXPLICIT_MEMBER_ACCESS
  ==================================================================================================
    Total measurements: 30470 (+0)
    Average latency (ms): 114.196846008 (-5)
    All measurements:
  	MRR: 67.95 (-1.05)	Top-1: 58.04% (-1.53%)	Top-5: 80.28% (-0.43%)	Top-100: 98.64% (-0.01%)
    Full identifiers:
  	MRR: 98.19 (-1.45)	Top-1: 96.69% (-2.75%)	Top-5: 99.83% (-0.07%)	Top-100: 99.89% (+0.00%)
    Filter length 0-5:
  	MRR:      27.70 (+0.00)		62.41 (+0.78)		68.45 (-1.11)		70.82 (-0.71)		72.00 (-1.10)		78.54 (-4.29)
  	Top-1:    16.41% (+0.00%)		49.21% (+1.05%)		57.13% (-1.02%)		60.02% (-0.71%)		61.31% (-1.43%)		67.65% (-6.68%)
  	Top-5:    39.39% (+0.00%)		80.05% (+0.52%)		83.26% (-1.18%)		84.73% (-0.69%)		85.38% (-0.69%)		92.24% (-1.05%)
  	Top-100:  94.25% (+0.00%)		99.21% (+0.02%)		99.20% (-0.04%)		99.24% (-0.02%)		99.26% (-0.02%)		99.71% (+0.00%)
  ==================================================================================================
                                          WANT_LOCAL
  ==================================================================================================
    Total measurements: 26642 (+0)
    Average latency (ms): 277.036773682 (-7)
    All measurements:
  	MRR: 85.82 (-0.40)	Top-1: 77.69% (-0.60%)	Top-5: 96.18% (-0.13%)	Top-100: 99.88% (+0.00%)
    Full identifiers:
  	MRR: 99.63 (-0.11)	Top-1: 99.37% (-0.21%)	Top-5: 99.91% (-0.02%)	Top-100: 99.93% (+0.00%)
    Filter length 0-5:
  	MRR:      53.07 (+0.00)		86.94 (+0.19)		90.42 (-0.64)		91.19 (-0.74)		91.51 (-0.94)		92.59 (-0.81)
  	Top-1:    34.01% (+0.00%)		77.25% (+0.34%)		82.89% (-1.01%)		84.30% (-1.06%)		84.99% (-1.45%)		86.95% (-1.23%)
  	Top-5:    80.13% (+0.00%)		98.80% (+0.00%)		99.32% (-0.15%)		99.16% (-0.33%)		99.21% (-0.24%)		99.23% (-0.23%)
  	Top-100:  99.68% (+0.00%)		99.93% (+0.00%)		99.92% (+0.00%)		99.92% (+0.00%)		99.91% (+0.00%)		99.90% (+0.00%)
  ==================================================================================================
                                          CROSS_NAMESPACE
  ==================================================================================================
    Total measurements: 18030 (+0)
    Average latency (ms): 268.525360107 (-6)
    All measurements:
  	MRR: 32.61 (-2.11)	Top-1: 25.29% (-1.85%)	Top-5: 40.29% (-2.63%)	Top-100: 72.80% (-3.47%)
    Full identifiers:
  	MRR: 79.96 (-1.83)	Top-1: 73.22% (-1.77%)	Top-5: 88.01% (-2.14%)	Top-100: 98.71% (+0.00%)
    Filter length 0-5:
  	MRR:      1.21 (+0.00)		11.40 (+0.28)		23.19 (-2.98)		29.33 (-3.42)		39.34 (-3.80)		46.81 (-3.58)
  	Top-1:    0.59% (+0.00%)		5.75% (+0.04%)		14.16% (-2.45%)		19.95% (-2.53%)		29.11% (-3.62%)		36.17% (-3.05%)
  	Top-5:    1.44% (+0.00%)		16.38% (+0.41%)		31.15% (-4.28%)		39.71% (-4.67%)		50.27% (-4.08%)		59.36% (-4.25%)
  	Top-100:  9.44% (+0.00%)		63.63% (+0.96%)		85.02% (-2.19%)		82.54% (-8.21%)		86.24% (-8.50%)		88.93% (-7.92%)

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D59300/new/

https://reviews.llvm.org/D59300