[PATCH] D118471: [clang][Lexer] Make raw and normal lexer behave the same for line comments

[Sam McCall via Phabricator via cfe-commits]

sammccall accepted this revision.
sammccall added inline comments.
This revision is now accepted and ready to land.


================
Comment at: clang/lib/Lex/Lexer.cpp:2387
     // unit.
     LangOpts.LineComment = true;
   }
----------------
sammccall wrote:
> It's much less obvious what this does in raw lexing mode.
> 
> My rough understanding is raw lexing mode is ~stateless and we may choose to recreate the lexer, start lexing at any point etc. So having a stateful flag flip won't actually result in the same text being lexed the same way, because we may not run in the same sequence.
OK, discussed offline:
 - if lexing the whole file in a loop (motivating case), this will DTRT
 - if lexing one token and thtrowing away the lexer, modifying LineComment is a no-op
 - if lexing some range, we're going to be stateful but with possibly-wrong initial state

As I understand, this fixes case 1, leaves 2 broken, and slightly changes the way in which 3 is broken, which seems OK.
Like you say, the right fix is to take the statefulness out of the language extension, which can happen after the branch.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D118471/new/

https://reviews.llvm.org/D118471