[clang] [clang][Lexer] Save and restore MIOpt in `peekNextPPToken()` (PR #180700)

[via cfe-commits]

llvmbot wrote:


<!--LLVM PR SUMMARY COMMENT-->

@llvm/pr-subscribers-clang

Author: Thorsten Klein (thorsten-klein)

<details>
<summary>Changes</summary>

Fixes issue #<!-- -->180155 which is caused by `peekNextPPToken()` not saving and restoring the MIOpt state, corrupting include guard optimization. 

A test is added which uses C++20 mode and include guards to verify headers are only processed once even when peeking ahead at tokens.

---
Full diff: https://github.com/llvm/llvm-project/pull/180700.diff


3 Files Affected:

- (modified) clang/lib/Lex/Lexer.cpp (+2) 
- (added) clang/test/Preprocessor/peek-next-token-miopt.cpp (+23) 
- (added) clang/test/Preprocessor/peek-next-token-miopt.h (+15) 


``````````diff
diff --git a/clang/lib/Lex/Lexer.cpp b/clang/lib/Lex/Lexer.cpp
index 1498657047bd6..2215a2c623bab 100644
--- a/clang/lib/Lex/Lexer.cpp
+++ b/clang/lib/Lex/Lexer.cpp
@@ -3233,6 +3233,7 @@ std::optional<Token> Lexer::peekNextPPToken() {
   bool atStartOfLine = IsAtStartOfLine;
   bool atPhysicalStartOfLine = IsAtPhysicalStartOfLine;
   bool leadingSpace = HasLeadingSpace;
+  MultipleIncludeOpt SavedMIOpt = MIOpt;
 
   Token Tok;
   Lex(Tok);
@@ -3243,6 +3244,7 @@ std::optional<Token> Lexer::peekNextPPToken() {
   HasLeadingSpace = leadingSpace;
   IsAtStartOfLine = atStartOfLine;
   IsAtPhysicalStartOfLine = atPhysicalStartOfLine;
+  MIOpt = SavedMIOpt;
   // Restore the lexer back to non-skipping mode.
   LexingRawMode = false;
 
diff --git a/clang/test/Preprocessor/peek-next-token-miopt.cpp b/clang/test/Preprocessor/peek-next-token-miopt.cpp
new file mode 100644
index 0000000000000..1b6efb33f4038
--- /dev/null
+++ b/clang/test/Preprocessor/peek-next-token-miopt.cpp
@@ -0,0 +1,23 @@
+// RUN: %clang_cc1 -std=c++20 -E %s -verify
+// expected-no-diagnostics
+
+// Test that peekNextPPToken() correctly saves and restores the MIOpt
+// (Multiple Include Optimization) state. This is important when processing
+// C++20 modules where peekNextPPToken() is called to look ahead.
+// Without saving/restoring MIOpt, the include guard detection could be
+// corrupted.
+
+#include "peek-next-token-miopt.h"
+#include "peek-next-token-miopt.h"
+
+// The header should only be processed once due to the include guard.
+// If MIOpt state is corrupted by peekNextPPToken(), this test would fail.
+
+#ifndef TEST_MACRO
+#error "TEST_MACRO should be defined from the header"
+#endif
+
+// Verify the value is 1 (defined once, not redefined)
+#if TEST_MACRO != 1
+#error "TEST_MACRO should be 1"
+#endif
diff --git a/clang/test/Preprocessor/peek-next-token-miopt.h b/clang/test/Preprocessor/peek-next-token-miopt.h
new file mode 100644
index 0000000000000..9460c24b999a2
--- /dev/null
+++ b/clang/test/Preprocessor/peek-next-token-miopt.h
@@ -0,0 +1,15 @@
+// Header file with include guard for peek-next-token-miopt.cpp test
+
+#ifndef PEEK_NEXT_TOKEN_MIOPT_H
+#define PEEK_NEXT_TOKEN_MIOPT_H
+
+// This should only be defined once
+#ifndef TEST_MACRO
+#define TEST_MACRO 1
+#else
+// If the header is included twice, redefine to show the bug
+#undef TEST_MACRO
+#define TEST_MACRO 2
+#endif
+
+#endif // PEEK_NEXT_TOKEN_MIOPT_H

``````````

</details>


https://github.com/llvm/llvm-project/pull/180700