[all-commits] [llvm/llvm-project] 19daa2: [clangd] Rethink how SelectionTree deals with macr...

[Sam McCall via All-commits]

  Branch: refs/heads/master
  Home:   https://github.com/llvm/llvm-project
  Commit: 19daa21f841ad45290c923689ee3d25198651a4c
      https://github.com/llvm/llvm-project/commit/19daa21f841ad45290c923689ee3d25198651a4c
  Author: Sam McCall <sam.mccall at gmail.com>
  Date:   2019-11-29 (Fri, 29 Nov 2019)

  Changed paths:
    M clang-tools-extra/clangd/Selection.cpp
    M clang-tools-extra/clangd/Selection.h
    M clang-tools-extra/clangd/unittests/SelectionTests.cpp
    M clang-tools-extra/clangd/unittests/TweakTests.cpp
    M clang/include/clang/Tooling/Syntax/Tokens.h
    M clang/lib/Tooling/Syntax/Tokens.cpp
    M clang/unittests/Tooling/Syntax/TokensTest.cpp

  Log Message:
  -----------
  [clangd] Rethink how SelectionTree deals with macros and #includes.

Summary:
The exclusive-claim model is successful at resolving conflicts over tokens
between parent/child or siblings. However claims at the spelled-token
level do the wrong thing for macro expansions, where siblings can be
equally associated with the macro invocation.
Moreover, any model that only uses the endpoints in a range can fail when
a macro invocation occurs inside the node.

To address this, we use the existing TokenBuffer in more depth.
Claims are expressed in terms of expanded tokens, so there is no need to worry
about macros, includes etc.

Once we know which expanded tokens were claimed, they are mapped onto
spelled tokens for hit-testing.
This mapping is fairly flexible, currently the handling of macros is
pretty simple (map macro args onto spellings, other macro expansions onto the
macro name token).
This mapping is in principle token-by-token for correctness (though
there's some batching for performance).

The aggregation of the selection enum is now more principled as we need to be
able to aggregate several hit-test results together.

For simplicity i removed the ability to determine selectedness of TUDecl.
(That was originally implemented in 90a5bf92ff97b1, but doesn't seem to be very
important or worth the complexity any longer).

The expandedTokens(SourceLocation) helper could be added locally, but seems to
make sense on TokenBuffer.

Fixes https://github.com/clangd/clangd/issues/202
Fixes https://github.com/clangd/clangd/issues/126

Reviewers: hokein

Subscribers: MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits, ilya-biryukov

Tags: #clang

Differential Revision: https://reviews.llvm.org/D70512