[all-commits] [llvm/llvm-project] ff7707: [clang][Lexer] Make raw and normal lexer behave th...

[kadir çetinkaya via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: ff77071a4d672fab7c8b30bea8525b89be8596fc
      https://github.com/llvm/llvm-project/commit/ff77071a4d672fab7c8b30bea8525b89be8596fc
  Author: Kadir Cetinkaya <kadircet at google.com>
  Date:   2022-01-31 (Mon, 31 Jan 2022)

  Changed paths:
    M clang/lib/Lex/Lexer.cpp
    M clang/unittests/Lex/LexerTest.cpp

  Log Message:
  -----------
  [clang][Lexer] Make raw and normal lexer behave the same for line comments

Normally there are heruistics in lexer to treat `//*` specially in
language modes that don't have line comments (to emit `/`). Unfortunately this
only applied to the first occurence of a line comment inside the file, as the
subsequent line comments were treated as if language had support for them.

This unfortunately only holds in normal lexing mode, as in raw mode all
occurences of line comments received this treatment, which created discrepancies
when comparing expanded and spelled tokens.

The proper fix would be to just make sure we treat all the line comments with a
subsequent `*` the same way, but it would imply breaking some code that's
accepted by clang today. So instead we introduce the same bug into raw lexing
mode.

Fixes https://github.com/clangd/clangd/issues/1003.

Differential Revision: https://reviews.llvm.org/D118471