[clang] [Clang] Implement P3034R1 Module Declarations Shouldn’t be Macros (PR #90574)

[Michael Spencer via cfe-commits]

================
@@ -1329,6 +1341,129 @@ bool Preprocessor::LexAfterModuleImport(Token &Result) {
   return true;
 }
 
+/// Lex a token following the 'module' contextual keyword.
+///
+/// [cpp.module]/p2:
+/// The pp-tokens, if any, of a pp-module shall be of the form:
+///       pp-module-name pp-module-partition[opt] pp-tokens[opt]
+///
+/// where the pp-tokens (if any) shall not begin with a ( preprocessing token
+/// and the grammar non-terminals are defined as:
+///       pp-module-name:
+///             pp-module-name-qualifierp[opt] identifier
+///       pp-module-partition:
+///             : pp-module-name-qualifier[opt] identifier
+///       pp-module-name-qualifier:
+///             identifier .
+///             pp-module-name-qualifier identifier .
+/// No identifier in the pp-module-name or pp-module-partition shall currently
+/// be defined as an object-like macro.
+///
+/// [cpp.module]/p3:
+/// Any preprocessing tokens after the module preprocessing token in the module
+/// directive are processed just as in normal text.
+bool Preprocessor::LexAfterModuleDecl(Token &Result) {
+  // Figure out what kind of lexer we actually have.
+  recomputeCurLexerKind();
+  LexUnexpandedToken(Result);
+
+  auto EnterTokens = [this](ArrayRef<Token> Toks, bool DisableMacroExpansion) {
+    auto ToksCopy = std::make_unique<Token[]>(Toks.size());
+    std::copy(Toks.begin(), Toks.end(), ToksCopy.get());
+    EnterTokenStream(std::move(ToksCopy), Toks.size(), DisableMacroExpansion,
+                     /*IsReinject=*/false);
+  };
+
+  // If we not expect an identifier but got an identifier, it's not a part of
+  // module name.
+  if (!ModuleDeclExpectsIdentifier && Result.is(tok::identifier)) {
+    EnterTokens(Result, /*DisableMacroExpansion=*/false);
+    return false;
+  }
+
+  // The token sequence
+  //
+  // export[opt] module identifier (. identifier)*
+  //
+  // indicates a module import directive. We already saw the 'module'
+  // contextual keyword, so now we're looking for the identifiers.
+  if (ModuleDeclExpectsIdentifier && Result.is(tok::identifier)) {
+    auto *MI = getMacroInfo(Result.getIdentifierInfo());
+    if (MI && MI->isObjectLike()) {
+      auto BuildFixItHint =
+          [&](std::string &Replacement) -> std::optional<FixItHint> {
+        EnterTokens(Result, /*DisableMacroExpansion=*/false);
+        if (!CurTokenLexer)
+          return std::nullopt;
+        Token Tok;
+        bool HasUnknownToken = false;
+        llvm::raw_string_ostream OS(Replacement);
+        do {
+          Lex(Tok);
+          if (const char *Punc = tok::getPunctuatorSpelling(Tok.getKind()))
+            OS << Punc;
+          else if (Tok.isLiteral() && Tok.getLiteralData())
+            OS << StringRef(Tok.getLiteralData(), Tok.getLength());
+          else if (auto *II = Tok.getIdentifierInfo())
+            OS << II->getName();
+          else
+            HasUnknownToken = true;
+        } while (CurTokenLexer && !CurTokenLexer->isAtEnd());
----------------
Bigcheese wrote:

I forget now where else I made this comment, but this is probably the 5th or 6th case of expanding a macro into a string in Clang. I'd really like to have a single API for doing this, as they are all slightly different.

https://github.com/llvm/llvm-project/pull/90574