[PATCH] D128411: [syntax] Introduce a BaseToken class.
Ilya Biryukov via Phabricator via cfe-commits
cfe-commits at lists.llvm.org
Fri Jun 24 06:15:55 PDT 2022
ilya-biryukov added inline comments.
Comment at: clang/include/clang/Tooling/Syntax/TokenManager.h:33
+ /// The syntax-tree Leaf node holds a Key.
+ using Key = const void *;
+ /// Gets the text of token identified by the key.
I have just realized that we were discussing having opaque index as a key, but there may also be tokens in the syntax tree that are not backed by the `TokenBuffer`.
Those can be synthesized, e.g. imagine someone wants to change `!(A && B)` to `!A || !B`. They will need to synthesize at least the `||` token as it's not in the source code. There is a way to do this now and it prohibits the use of an index to the `TokenBuffer`.
So having the opaque pointer is probably ok for now, it should enable the pseudo-parser to build syntax trees.
We might want to add an operation to synthesize tokens into the `TokenManager` at some point, but that's a discussion for another day.
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
More information about the cfe-commits