[PATCH] D108308: [WIP] Cleanup identifier parsing.

[Corentin Jabot via Phabricator via cfe-commits]

cor3ntin added inline comments.


================
Comment at: clang/include/clang/Lex/Lexer.h:702
   // Helper functions to lex the remainder of a token of the specific type.
-  bool LexIdentifier         (Token &Result, const char *CurPtr);
+  bool LexIdentifierContinue(Token &Result, const char *CurPtr);
   bool LexNumericConstant    (Token &Result, const char *CurPtr);
----------------
aaron.ballman wrote:
> Should this be `LexUnicodeIdentifierContinue()`? If so, perhaps it can also be moved up to line 578 so it's near the "start" function?
> 
> Or does this function handle both Unicode and ASCII identifiers? If so, the comments could probably be updated.
This handles all identifiers - after the first codepoint has been parsed - Which comment are you referring to?


================
Comment at: clang/lib/Lex/Lexer.cpp:1758
+bool Lexer::LexIdentifierContinue(Token &Result, const char *CurPtr) {
+  // Match [_A-Za-z0-9]*, we have already matched [_A-Za-z$]
+  unsigned Size;
----------------
aaron.ballman wrote:
> Is the comment here still accurate? Might be worth rewriting in prose rather than regex?
I don't think the comment was accurate before, I'll find somehing better!


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D108308/new/

https://reviews.llvm.org/D108308