[PATCH] D139926: [clangd] Add semantic tokens for angle brackets

Sat Jan 7 00:14:44 PST 2023

nridge added a comment.

In D139926#4032473 <https://reviews.llvm.org/D139926#4032473>, @ckandeler wrote:

> In D139926#4030782 <https://reviews.llvm.org/D139926#4030782>, @nridge wrote:
>
>> It's true that there is an ambiguity between `<` and `>` as operators, vs. template arg/param list delimiters, but, at least in terms of user understanding of code, my sense is that the highlighting of the **preceding** token should be sufficient to disambiguate -- i.e. it would be some sort of type name in the template case, vs. a variable / literal / punctuation ending an expression in the operator case.
>
> We used to do this sort of heuristic in our old libclang-based implementation, and it turned out to be rather messy, with a surprising amount of exceptions having to be added.

To clarify, I'm not suggesting that any client-side logic use this heuristic, only that it's probably good enough for a human reader to disambiguate without needing to assign angle brackets and comparison operators different colors.

>>> This is needed for clients that would like to visualize matching opening and closing angle brackets, which can be valuable in non-trivial template declarations or instantiations.
>>
>> For this use case, could an editor make use of the recently added operator tokens (https://reviews.llvm.org/D136594) instead, inferring that a `<` token which does not have an `operator` semantic token is a template delimiter?
>
> I have a suspicion that this will lead to false positives for invalid code.

Ah, interesting. I guess in the case of invalid code (no/malformed AST) you wouldn't get AngleBracket tokens either, so this gives you a three-way signal (Operator vs. AngleBracket vs. no semantic token) that's richer than just the two-way signal of Operator vs. no Operator.

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D139926/new/

https://reviews.llvm.org/D139926