[PATCH] D159345: [Clang] Handle non-ASCII after line splicing

[Shafik Yaghmour via Phabricator via cfe-commits]

shafik added inline comments.


================
Comment at: clang/lib/Lex/Lexer.cpp:1757
+  // If a UTF-8 codepoint appears immediately after an escaped new line,
+  // CurPtr may point to the splicing \ at the on the preceding line,
+  // so we need to skip it.
----------------
I think that is what you meant to say


================
Comment at: clang/lib/Lex/Lexer.cpp:1761
+  getCharAndSize(CurPtr, FirstCodeUnitSize);
+  const char *CharStart = CurPtr + FirstCodeUnitSize - 1;
+  const char *UnicodePtr = CharStart;
----------------
Do we need to verify that `FirstCodeUnitSize` is not zero?


================
Comment at: clang/lib/Lex/Lexer.cpp:1798
 
+  ConsumeChar(CurPtr, FirstCodeUnitSize, Tok);
   CurPtr = UnicodePtr;
----------------
So if we get here it mean we have the splice to skip?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D159345/new/

https://reviews.llvm.org/D159345