[all-commits] [llvm/llvm-project] f66d37: [pseudo] Split greatergreater token.

[Haojian Wu via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: f66d3758bda99e9f57bfdad168212feda18792ae
      https://github.com/llvm/llvm-project/commit/f66d3758bda99e9f57bfdad168212feda18792ae
  Author: Haojian Wu <hokein.wu at gmail.com>
  Date:   2022-03-17 (Thu, 17 Mar 2022)

  Changed paths:
    M clang-tools-extra/pseudo/include/clang-pseudo/Token.h
    M clang-tools-extra/pseudo/lib/Lex.cpp
    M clang-tools-extra/pseudo/lib/cxx.bnf
    M clang-tools-extra/pseudo/unittests/TokenTest.cpp

  Log Message:
  -----------
  [pseudo] Split greatergreater token.

For a >> token (a right shift operator, or a nested template?), the clang
lexer always returns a single greatergreater token, as a result,
the grammar-based GLR parser never try to parse the nested template
case.

We derive a token stream by always splitting the >> token, so that the
GLR parser is able to pursue both options during parsing (usually 1
path fails).

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D121678