[clang] [clang][Preprocessor] Add peekNextPPToken, makes look ahead next token without side-effects (PR #143898)

via cfe-commits cfe-commits at lists.llvm.org
Tue Jun 24 09:26:19 PDT 2025


================
@@ -2302,10 +2302,41 @@ class Preprocessor {
     }
   }
 
-  /// Determine whether the next preprocessor token to be
-  /// lexed is a '('.  If so, consume the token and return true, if not, this
+  /// Check whether the next pp-token is one of the specificed token kind. this
   /// method should have no observable side-effect on the lexed tokens.
-  bool isNextPPTokenLParen();
+  template <tok::TokenKind K, tok::TokenKind... Ks> bool isNextPPTokenOneOf() {
+    // Do some quick tests for rejection cases.
+    std::optional<Token> Val;
+    if (CurLexer)
+      Val = CurLexer->peekNextPPToken();
+    else
+      Val = CurTokenLexer->peekNextPPToken();
+
+    if (!Val) {
+      // We have run off the end.  If it's a source file we don't
+      // examine enclosing ones (C99 5.1.1.2p4).  Otherwise walk up the
+      // macro stack.
+      if (CurPPLexer)
+        return false;
+      for (const IncludeStackInfo &Entry : llvm::reverse(IncludeMacroStack)) {
+        if (Entry.TheLexer)
+          Val = Entry.TheLexer->peekNextPPToken();
+        else
+          Val = Entry.TheTokenLexer->peekNextPPToken();
+
+        if (Val)
+          break;
+
+        // Ran off the end of a source file?
+        if (Entry.ThePPLexer)
+          return false;
+      }
+    }
+
+    // Okay, we found the token and return.  Otherwise we found the end of the
+    // translation unit.
+    return Val->is(K) || (... || Val->is(Ks));
----------------
yronglin wrote:

Agree 100%, we can use `Val->isOneOf(K, Ks...);`, but we need improve the implementation of `Token::isOneOf`, otherwise we will encounter the following error:
```
/llvm-project/clang/include/clang/Lex/Token.h:108:22: error: no matching member function for call to 'isOneOf'
  108 |     return is(K1) || isOneOf(Ks...);
      |                      ^~~~~~~
/llvm-project/clang/include/clang/Lex/Preprocessor.h:2331:17: note: in instantiation of function template specialization 'clang::Token::isOneOf<>' requested here
 2331 |     return Val->isOneOf(K, Ks...);
      |                 ^
/llvm-project/clang/lib/Lex/Preprocessor.cpp:831:38: note: in instantiation of function template specialization 'clang::Preprocessor::isNextPPTokenOneOf<clang::tok::l_paren>' requested here
  831 |         if (!MI->isFunctionLike() || isNextPPTokenOneOf<tok::l_paren>())
      |                                      ^
/llvm-project/clang/include/clang/Lex/Token.h:107:34: note: candidate function template not viable: requires at least argument 'K1', but no arguments were provided
  107 |   template <typename... Ts> bool isOneOf(tok::TokenKind K1, Ts... Ks) const {
      |                                  ^       ~~~~~~~~~~~~~~~~~~~~~~~~~~~
/llvm-project/clang/include/clang/Lex/Token.h:104:8: note: candidate function not viable: requires 2 arguments, but 0 were provided
  104 |   bool isOneOf(tok::TokenKind K1, tok::TokenKind K2) const {
      |        ^       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
```

I have created a PR to address suggestions(https://github.com/llvm/llvm-project/pull/145546), could you help review again? Many thanks!

https://github.com/llvm/llvm-project/pull/143898


More information about the cfe-commits mailing list