[llvm-branch-commits] [clang] release/21.x: [clang] Allow trivial pp-directives before C++ module directive (#153641) (PR #154077)
via llvm-branch-commits
llvm-branch-commits at lists.llvm.org
Mon Aug 18 02:10:36 PDT 2025
https://github.com/yronglin updated https://github.com/llvm/llvm-project/pull/154077
>From 2deca657c7c728143bb323e94023255a97a27617 Mon Sep 17 00:00:00 2001
From: yronglin <yronglin777 at gmail.com>
Date: Mon, 18 Aug 2025 14:17:35 +0800
Subject: [PATCH 1/2] [clang] Allow trivial pp-directives before C++ module
directive (#153641)
Consider the following code:
```cpp
# 1 __FILE__ 1 3
export module a;
```
According to the wording in
[P1857R3](https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/p1857r3.html):
```
A module directive may only appear as the first preprocessing tokens in a file (excluding the global module fragment.)
```
and the wording in
[[cpp.pre]](https://eel.is/c++draft/cpp.pre#nt:module-file)
```
module-file:
pp-global-module-fragment[opt] pp-module group[opt] pp-private-module-fragment[opt]
```
`#` is the first pp-token in the translation unit, and it was rejected
by clang, but they really should be exempted from this rule. The goal is
to not allow any preprocessor conditionals or most state changes, but
these don't fit that.
State change would mean most semantically observable preprocessor state,
particularly anything that is order dependent. Global flags like being a
system header/module shouldn't matter.
We should exempt a brunch of directives, even though it violates the
current standard wording.
In this patch, we introduce a `TrivialDirectiveTracer` to trace the
**State change** that described above and propose to exempt the
following kind of directive: `#line`, GNU line marker, `#ident`,
`#pragma comment`, `#pragma mark`, `#pragma detect_mismatch`, `#pragma
clang __debug`, `#pragma message`, `#pragma GCC warning`, `#pragma GCC
error`, `#pragma gcc diagnostic`, `#pragma OPENCL EXTENSION`, `#pragma
warning`, `#pragma execution_character_set`, `#pragma clang
assume_nonnull` and builtin macro expansion.
Fixes https://github.com/llvm/llvm-project/issues/145274
---------
Signed-off-by: yronglin <yronglin777 at gmail.com>
(cherry picked from commit e6e874ce8f055f5b8c5d7f8c7fb0afe764d1d350)
---
clang/include/clang/Lex/Lexer.h | 3 -
.../clang/Lex/NoTrivialPPDirectiveTracer.h | 310 ++++++++++++++++++
clang/include/clang/Lex/Preprocessor.h | 12 +
clang/include/clang/Lex/Token.h | 17 +-
clang/include/clang/Sema/Sema.h | 2 +-
clang/lib/Lex/Lexer.cpp | 9 -
clang/lib/Lex/Preprocessor.cpp | 40 ++-
clang/lib/Parse/Parser.cpp | 8 +-
clang/lib/Sema/SemaModule.cpp | 6 +-
clang/test/CXX/module/cpp.pre/module_decl.cpp | 141 +++++++-
clang/unittests/Lex/CMakeLists.txt | 1 +
clang/unittests/Lex/LexerTest.cpp | 4 +-
clang/unittests/Lex/ModuleDeclStateTest.cpp | 128 ++++----
.../Lex/NoTrivialPPDirectiveTracerTest.cpp | 182 ++++++++++
14 files changed, 762 insertions(+), 101 deletions(-)
create mode 100644 clang/include/clang/Lex/NoTrivialPPDirectiveTracer.h
create mode 100644 clang/unittests/Lex/NoTrivialPPDirectiveTracerTest.cpp
diff --git a/clang/include/clang/Lex/Lexer.h b/clang/include/clang/Lex/Lexer.h
index 06971ff87ab96..423f2ffe2f852 100644
--- a/clang/include/clang/Lex/Lexer.h
+++ b/clang/include/clang/Lex/Lexer.h
@@ -143,9 +143,6 @@ class Lexer : public PreprocessorLexer {
/// True if this is the first time we're lexing the input file.
bool IsFirstTimeLexingFile;
- /// True if current lexing token is the first pp-token.
- bool IsFirstPPToken;
-
// NewLinePtr - A pointer to new line character '\n' being lexed. For '\r\n',
// it also points to '\n.'
const char *NewLinePtr;
diff --git a/clang/include/clang/Lex/NoTrivialPPDirectiveTracer.h b/clang/include/clang/Lex/NoTrivialPPDirectiveTracer.h
new file mode 100644
index 0000000000000..9ab3c6a528a1a
--- /dev/null
+++ b/clang/include/clang/Lex/NoTrivialPPDirectiveTracer.h
@@ -0,0 +1,310 @@
+//===--- NoTrivialPPDirectiveTracer.h ---------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file defines the NoTrivialPPDirectiveTracer interface.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_LEX_NO_TRIVIAL_PPDIRECTIVE_TRACER_H
+#define LLVM_CLANG_LEX_NO_TRIVIAL_PPDIRECTIVE_TRACER_H
+
+#include "clang/Lex/PPCallbacks.h"
+
+namespace clang {
+class Preprocessor;
+
+/// Consider the following code:
+///
+/// # 1 __FILE__ 1 3
+/// export module a;
+///
+/// According to the wording in
+/// [P1857R3](https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/p1857r3.html):
+///
+/// A module directive may only appear as the first preprocessing tokens in a
+/// file (excluding the global module fragment.)
+///
+/// and the wording in
+/// [[cpp.pre]](https://eel.is/c++draft/cpp.pre#nt:module-file):
+/// module-file:
+/// pp-global-module-fragment[opt] pp-module group[opt]
+/// pp-private-module-fragment[opt]
+///
+/// `#` is the first pp-token in the translation unit, and it was rejected by
+/// clang, but they really should be exempted from this rule. The goal is to not
+/// allow any preprocessor conditionals or most state changes, but these don't
+/// fit that.
+///
+/// State change would mean most semantically observable preprocessor state,
+/// particularly anything that is order dependent. Global flags like being a
+/// system header/module shouldn't matter.
+///
+/// We should exempt a brunch of directives, even though it violates the current
+/// standard wording.
+///
+/// This class used to trace 'no-trivial' pp-directives in main file, which may
+/// change the preprocessing state.
+///
+/// FIXME: Once the wording of the standard is revised, we need to follow the
+/// wording of the standard. Currently this is just a workaround
+class NoTrivialPPDirectiveTracer : public PPCallbacks {
+ Preprocessor &PP;
+
+ /// Whether preprocessing main file. We only focus on the main file.
+ bool InMainFile = true;
+
+ /// Whether one or more conditional, include or other 'no-trivial'
+ /// pp-directives has seen before.
+ bool SeenNoTrivialPPDirective = false;
+
+ void setSeenNoTrivialPPDirective();
+
+public:
+ NoTrivialPPDirectiveTracer(Preprocessor &P) : PP(P) {}
+
+ bool hasSeenNoTrivialPPDirective() const;
+
+ /// Callback invoked whenever the \p Lexer moves to a different file for
+ /// lexing. Unlike \p FileChanged line number directives and other related
+ /// pragmas do not trigger callbacks to \p LexedFileChanged.
+ ///
+ /// \param FID The \p FileID that the \p Lexer moved to.
+ ///
+ /// \param Reason Whether the \p Lexer entered a new file or exited one.
+ ///
+ /// \param FileType The \p CharacteristicKind of the file the \p Lexer moved
+ /// to.
+ ///
+ /// \param PrevFID The \p FileID the \p Lexer was using before the change.
+ ///
+ /// \param Loc The location where the \p Lexer entered a new file from or the
+ /// location that the \p Lexer moved into after exiting a file.
+ void LexedFileChanged(FileID FID, LexedFileChangeReason Reason,
+ SrcMgr::CharacteristicKind FileType, FileID PrevFID,
+ SourceLocation Loc) override;
+
+ /// Callback invoked whenever an embed directive has been processed,
+ /// regardless of whether the embed will actually find a file.
+ ///
+ /// \param HashLoc The location of the '#' that starts the embed directive.
+ ///
+ /// \param FileName The name of the file being included, as written in the
+ /// source code.
+ ///
+ /// \param IsAngled Whether the file name was enclosed in angle brackets;
+ /// otherwise, it was enclosed in quotes.
+ ///
+ /// \param File The actual file that may be included by this embed directive.
+ ///
+ /// \param Params The parameters used by the directive.
+ void EmbedDirective(SourceLocation HashLoc, StringRef FileName, bool IsAngled,
+ OptionalFileEntryRef File,
+ const LexEmbedParametersResult &Params) override {
+ setSeenNoTrivialPPDirective();
+ }
+
+ /// Callback invoked whenever an inclusion directive of
+ /// any kind (\c \#include, \c \#import, etc.) has been processed, regardless
+ /// of whether the inclusion will actually result in an inclusion.
+ ///
+ /// \param HashLoc The location of the '#' that starts the inclusion
+ /// directive.
+ ///
+ /// \param IncludeTok The token that indicates the kind of inclusion
+ /// directive, e.g., 'include' or 'import'.
+ ///
+ /// \param FileName The name of the file being included, as written in the
+ /// source code.
+ ///
+ /// \param IsAngled Whether the file name was enclosed in angle brackets;
+ /// otherwise, it was enclosed in quotes.
+ ///
+ /// \param FilenameRange The character range of the quotes or angle brackets
+ /// for the written file name.
+ ///
+ /// \param File The actual file that may be included by this inclusion
+ /// directive.
+ ///
+ /// \param SearchPath Contains the search path which was used to find the file
+ /// in the file system. If the file was found via an absolute include path,
+ /// SearchPath will be empty. For framework includes, the SearchPath and
+ /// RelativePath will be split up. For example, if an include of "Some/Some.h"
+ /// is found via the framework path
+ /// "path/to/Frameworks/Some.framework/Headers/Some.h", SearchPath will be
+ /// "path/to/Frameworks/Some.framework/Headers" and RelativePath will be
+ /// "Some.h".
+ ///
+ /// \param RelativePath The path relative to SearchPath, at which the include
+ /// file was found. This is equal to FileName except for framework includes.
+ ///
+ /// \param SuggestedModule The module suggested for this header, if any.
+ ///
+ /// \param ModuleImported Whether this include was translated into import of
+ /// \p SuggestedModule.
+ ///
+ /// \param FileType The characteristic kind, indicates whether a file or
+ /// directory holds normal user code, system code, or system code which is
+ /// implicitly 'extern "C"' in C++ mode.
+ ///
+ void InclusionDirective(SourceLocation HashLoc, const Token &IncludeTok,
+ StringRef FileName, bool IsAngled,
+ CharSourceRange FilenameRange,
+ OptionalFileEntryRef File, StringRef SearchPath,
+ StringRef RelativePath, const Module *SuggestedModule,
+ bool ModuleImported,
+ SrcMgr::CharacteristicKind FileType) override {
+ setSeenNoTrivialPPDirective();
+ }
+
+ /// Callback invoked whenever there was an explicit module-import
+ /// syntax.
+ ///
+ /// \param ImportLoc The location of import directive token.
+ ///
+ /// \param Path The identifiers (and their locations) of the module
+ /// "path", e.g., "std.vector" would be split into "std" and "vector".
+ ///
+ /// \param Imported The imported module; can be null if importing failed.
+ ///
+ void moduleImport(SourceLocation ImportLoc, ModuleIdPath Path,
+ const Module *Imported) override {
+ setSeenNoTrivialPPDirective();
+ }
+
+ /// Callback invoked when the end of the main file is reached.
+ ///
+ /// No subsequent callbacks will be made.
+ void EndOfMainFile() override { setSeenNoTrivialPPDirective(); }
+
+ /// Callback invoked when start reading any pragma directive.
+ void PragmaDirective(SourceLocation Loc,
+ PragmaIntroducerKind Introducer) override {}
+
+ /// Called by Preprocessor::HandleMacroExpandedIdentifier when a
+ /// macro invocation is found.
+ void MacroExpands(const Token &MacroNameTok, const MacroDefinition &MD,
+ SourceRange Range, const MacroArgs *Args) override;
+
+ /// Hook called whenever a macro definition is seen.
+ void MacroDefined(const Token &MacroNameTok,
+ const MacroDirective *MD) override {
+ setSeenNoTrivialPPDirective();
+ }
+
+ /// Hook called whenever a macro \#undef is seen.
+ /// \param MacroNameTok The active Token
+ /// \param MD A MacroDefinition for the named macro.
+ /// \param Undef New MacroDirective if the macro was defined, null otherwise.
+ ///
+ /// MD is released immediately following this callback.
+ void MacroUndefined(const Token &MacroNameTok, const MacroDefinition &MD,
+ const MacroDirective *Undef) override {
+ setSeenNoTrivialPPDirective();
+ }
+
+ /// Hook called whenever the 'defined' operator is seen.
+ /// \param MD The MacroDirective if the name was a macro, null otherwise.
+ void Defined(const Token &MacroNameTok, const MacroDefinition &MD,
+ SourceRange Range) override {
+ setSeenNoTrivialPPDirective();
+ }
+
+ /// Hook called whenever an \#if is seen.
+ /// \param Loc the source location of the directive.
+ /// \param ConditionRange The SourceRange of the expression being tested.
+ /// \param ConditionValue The evaluated value of the condition.
+ ///
+ // FIXME: better to pass in a list (or tree!) of Tokens.
+ void If(SourceLocation Loc, SourceRange ConditionRange,
+ ConditionValueKind ConditionValue) override {
+ setSeenNoTrivialPPDirective();
+ }
+
+ /// Hook called whenever an \#elif is seen.
+ /// \param Loc the source location of the directive.
+ /// \param ConditionRange The SourceRange of the expression being tested.
+ /// \param ConditionValue The evaluated value of the condition.
+ /// \param IfLoc the source location of the \#if/\#ifdef/\#ifndef directive.
+ // FIXME: better to pass in a list (or tree!) of Tokens.
+ void Elif(SourceLocation Loc, SourceRange ConditionRange,
+ ConditionValueKind ConditionValue, SourceLocation IfLoc) override {
+ setSeenNoTrivialPPDirective();
+ }
+
+ /// Hook called whenever an \#ifdef is seen.
+ /// \param Loc the source location of the directive.
+ /// \param MacroNameTok Information on the token being tested.
+ /// \param MD The MacroDefinition if the name was a macro, null otherwise.
+ void Ifdef(SourceLocation Loc, const Token &MacroNameTok,
+ const MacroDefinition &MD) override {
+ setSeenNoTrivialPPDirective();
+ }
+
+ /// Hook called whenever an \#elifdef branch is taken.
+ /// \param Loc the source location of the directive.
+ /// \param MacroNameTok Information on the token being tested.
+ /// \param MD The MacroDefinition if the name was a macro, null otherwise.
+ void Elifdef(SourceLocation Loc, const Token &MacroNameTok,
+ const MacroDefinition &MD) override {
+ setSeenNoTrivialPPDirective();
+ }
+ /// Hook called whenever an \#elifdef is skipped.
+ /// \param Loc the source location of the directive.
+ /// \param ConditionRange The SourceRange of the expression being tested.
+ /// \param IfLoc the source location of the \#if/\#ifdef/\#ifndef directive.
+ // FIXME: better to pass in a list (or tree!) of Tokens.
+ void Elifdef(SourceLocation Loc, SourceRange ConditionRange,
+ SourceLocation IfLoc) override {
+ setSeenNoTrivialPPDirective();
+ }
+
+ /// Hook called whenever an \#ifndef is seen.
+ /// \param Loc the source location of the directive.
+ /// \param MacroNameTok Information on the token being tested.
+ /// \param MD The MacroDefiniton if the name was a macro, null otherwise.
+ void Ifndef(SourceLocation Loc, const Token &MacroNameTok,
+ const MacroDefinition &MD) override {
+ setSeenNoTrivialPPDirective();
+ }
+
+ /// Hook called whenever an \#elifndef branch is taken.
+ /// \param Loc the source location of the directive.
+ /// \param MacroNameTok Information on the token being tested.
+ /// \param MD The MacroDefinition if the name was a macro, null otherwise.
+ void Elifndef(SourceLocation Loc, const Token &MacroNameTok,
+ const MacroDefinition &MD) override {
+ setSeenNoTrivialPPDirective();
+ }
+ /// Hook called whenever an \#elifndef is skipped.
+ /// \param Loc the source location of the directive.
+ /// \param ConditionRange The SourceRange of the expression being tested.
+ /// \param IfLoc the source location of the \#if/\#ifdef/\#ifndef directive.
+ // FIXME: better to pass in a list (or tree!) of Tokens.
+ void Elifndef(SourceLocation Loc, SourceRange ConditionRange,
+ SourceLocation IfLoc) override {
+ setSeenNoTrivialPPDirective();
+ }
+
+ /// Hook called whenever an \#else is seen.
+ /// \param Loc the source location of the directive.
+ /// \param IfLoc the source location of the \#if/\#ifdef/\#ifndef directive.
+ void Else(SourceLocation Loc, SourceLocation IfLoc) override {
+ setSeenNoTrivialPPDirective();
+ }
+
+ /// Hook called whenever an \#endif is seen.
+ /// \param Loc the source location of the directive.
+ /// \param IfLoc the source location of the \#if/\#ifdef/\#ifndef directive.
+ void Endif(SourceLocation Loc, SourceLocation IfLoc) override {
+ setSeenNoTrivialPPDirective();
+ }
+};
+
+} // namespace clang
+
+#endif // LLVM_CLANG_LEX_NO_TRIVIAL_PPDIRECTIVE_TRACER_H
diff --git a/clang/include/clang/Lex/Preprocessor.h b/clang/include/clang/Lex/Preprocessor.h
index 4d82e20e5d4f3..e90564a9739a5 100644
--- a/clang/include/clang/Lex/Preprocessor.h
+++ b/clang/include/clang/Lex/Preprocessor.h
@@ -82,6 +82,7 @@ class PreprocessorLexer;
class PreprocessorOptions;
class ScratchBuffer;
class TargetInfo;
+class NoTrivialPPDirectiveTracer;
namespace Builtin {
class Context;
@@ -353,6 +354,11 @@ class Preprocessor {
/// First pp-token source location in current translation unit.
SourceLocation FirstPPTokenLoc;
+ /// A preprocessor directive tracer to trace whether the preprocessing
+ /// state changed. These changes would mean most semantically observable
+ /// preprocessor state, particularly anything that is order dependent.
+ NoTrivialPPDirectiveTracer *DirTracer = nullptr;
+
/// A position within a C++20 import-seq.
class StdCXXImportSeq {
public:
@@ -609,6 +615,8 @@ class Preprocessor {
return State == NamedModuleImplementation && !getName().contains(':');
}
+ bool isNotAModuleDecl() const { return State == NotAModuleDecl; }
+
StringRef getName() const {
assert(isNamedModule() && "Can't get name from a non named module");
return Name;
@@ -3087,6 +3095,10 @@ class Preprocessor {
bool setDeserializedSafeBufferOptOutMap(
const SmallVectorImpl<SourceLocation> &SrcLocSeqs);
+ /// Whether we've seen pp-directives which may have changed the preprocessing
+ /// state.
+ bool hasSeenNoTrivialPPDirective() const;
+
private:
/// Helper functions to forward lexing to the actual lexer. They all share the
/// same signature.
diff --git a/clang/include/clang/Lex/Token.h b/clang/include/clang/Lex/Token.h
index fc43e72593b94..d9dc5a562d802 100644
--- a/clang/include/clang/Lex/Token.h
+++ b/clang/include/clang/Lex/Token.h
@@ -86,12 +86,12 @@ class Token {
// macro stringizing or charizing operator.
CommaAfterElided = 0x200, // The comma following this token was elided (MS).
IsEditorPlaceholder = 0x400, // This identifier is a placeholder.
-
- IsReinjected = 0x800, // A phase 4 token that was produced before and
- // re-added, e.g. via EnterTokenStream. Annotation
- // tokens are *not* reinjected.
- FirstPPToken = 0x1000, // This token is the first pp token in the
- // translation unit.
+ IsReinjected = 0x800, // A phase 4 token that was produced before and
+ // re-added, e.g. via EnterTokenStream. Annotation
+ // tokens are *not* reinjected.
+ HasSeenNoTrivialPPDirective =
+ 0x1000, // Whether we've seen any 'no-trivial' pp-directives before
+ // current position.
};
tok::TokenKind getKind() const { return Kind; }
@@ -321,8 +321,9 @@ class Token {
/// lexer uses identifier tokens to represent placeholders.
bool isEditorPlaceholder() const { return getFlag(IsEditorPlaceholder); }
- /// Returns true if this token is the first pp-token.
- bool isFirstPPToken() const { return getFlag(FirstPPToken); }
+ bool hasSeenNoTrivialPPDirective() const {
+ return getFlag(HasSeenNoTrivialPPDirective);
+ }
};
/// Information about the conditional stack (\#if directives)
diff --git a/clang/include/clang/Sema/Sema.h b/clang/include/clang/Sema/Sema.h
index b331acbe606b7..7b0be368d67f9 100644
--- a/clang/include/clang/Sema/Sema.h
+++ b/clang/include/clang/Sema/Sema.h
@@ -9836,7 +9836,7 @@ class Sema final : public SemaBase {
SourceLocation ModuleLoc, ModuleDeclKind MDK,
ModuleIdPath Path, ModuleIdPath Partition,
ModuleImportState &ImportState,
- bool IntroducerIsFirstPPToken);
+ bool SeenNoTrivialPPDirective);
/// The parser has processed a global-module-fragment declaration that begins
/// the definition of the global module fragment of the current module unit.
diff --git a/clang/lib/Lex/Lexer.cpp b/clang/lib/Lex/Lexer.cpp
index 1f695b4a8676c..b282a600c0e56 100644
--- a/clang/lib/Lex/Lexer.cpp
+++ b/clang/lib/Lex/Lexer.cpp
@@ -174,8 +174,6 @@ void Lexer::InitLexer(const char *BufStart, const char *BufPtr,
ExtendedTokenMode = 0;
NewLinePtr = nullptr;
-
- IsFirstPPToken = true;
}
/// Lexer constructor - Create a new lexer object for the specified buffer
@@ -3225,7 +3223,6 @@ std::optional<Token> Lexer::peekNextPPToken() {
bool atStartOfLine = IsAtStartOfLine;
bool atPhysicalStartOfLine = IsAtPhysicalStartOfLine;
bool leadingSpace = HasLeadingSpace;
- bool isFirstPPToken = IsFirstPPToken;
Token Tok;
Lex(Tok);
@@ -3236,7 +3233,6 @@ std::optional<Token> Lexer::peekNextPPToken() {
HasLeadingSpace = leadingSpace;
IsAtStartOfLine = atStartOfLine;
IsAtPhysicalStartOfLine = atPhysicalStartOfLine;
- IsFirstPPToken = isFirstPPToken;
// Restore the lexer back to non-skipping mode.
LexingRawMode = false;
@@ -3726,11 +3722,6 @@ bool Lexer::Lex(Token &Result) {
HasLeadingEmptyMacro = false;
}
- if (IsFirstPPToken) {
- Result.setFlag(Token::FirstPPToken);
- IsFirstPPToken = false;
- }
-
bool atPhysicalStartOfLine = IsAtPhysicalStartOfLine;
IsAtPhysicalStartOfLine = false;
bool isRawLex = isLexingRawMode();
diff --git a/clang/lib/Lex/Preprocessor.cpp b/clang/lib/Lex/Preprocessor.cpp
index bcd3ea60ce3da..2120e45dd8f8a 100644
--- a/clang/lib/Lex/Preprocessor.cpp
+++ b/clang/lib/Lex/Preprocessor.cpp
@@ -43,6 +43,7 @@
#include "clang/Lex/MacroArgs.h"
#include "clang/Lex/MacroInfo.h"
#include "clang/Lex/ModuleLoader.h"
+#include "clang/Lex/NoTrivialPPDirectiveTracer.h"
#include "clang/Lex/Pragma.h"
#include "clang/Lex/PreprocessingRecord.h"
#include "clang/Lex/PreprocessorLexer.h"
@@ -247,8 +248,6 @@ void Preprocessor::DumpToken(const Token &Tok, bool DumpFlags) const {
llvm::errs() << " [LeadingSpace]";
if (Tok.isExpandDisabled())
llvm::errs() << " [ExpandDisabled]";
- if (Tok.isFirstPPToken())
- llvm::errs() << " [First pp-token]";
if (Tok.needsCleaning()) {
const char *Start = SourceMgr.getCharacterData(Tok.getLocation());
llvm::errs() << " [UnClean='" << StringRef(Start, Tok.getLength())
@@ -577,8 +576,11 @@ void Preprocessor::EnterMainSourceFile() {
// export module M; // error: module declaration must occur
// // at the start of the translation unit.
if (getLangOpts().CPlusPlusModules) {
+ auto Tracer = std::make_unique<NoTrivialPPDirectiveTracer>(*this);
+ DirTracer = Tracer.get();
+ addPPCallbacks(std::move(Tracer));
std::optional<Token> FirstPPTok = CurLexer->peekNextPPToken();
- if (FirstPPTok && FirstPPTok->isFirstPPToken())
+ if (FirstPPTok)
FirstPPTokenLoc = FirstPPTok->getLocation();
}
}
@@ -940,6 +942,8 @@ void Preprocessor::Lex(Token &Result) {
StdCXXImportSeqState.handleHeaderName();
break;
case tok::kw_export:
+ if (hasSeenNoTrivialPPDirective())
+ Result.setFlag(Token::HasSeenNoTrivialPPDirective);
TrackGMFState.handleExport();
StdCXXImportSeqState.handleExport();
ModuleDeclState.handleExport();
@@ -966,6 +970,8 @@ void Preprocessor::Lex(Token &Result) {
}
break;
} else if (Result.getIdentifierInfo() == getIdentifierInfo("module")) {
+ if (hasSeenNoTrivialPPDirective())
+ Result.setFlag(Token::HasSeenNoTrivialPPDirective);
TrackGMFState.handleModule(StdCXXImportSeqState.afterTopLevelSeq());
ModuleDeclState.handleModule();
break;
@@ -1680,3 +1686,31 @@ const char *Preprocessor::getCheckPoint(FileID FID, const char *Start) const {
return nullptr;
}
+
+bool Preprocessor::hasSeenNoTrivialPPDirective() const {
+ return DirTracer && DirTracer->hasSeenNoTrivialPPDirective();
+}
+
+bool NoTrivialPPDirectiveTracer::hasSeenNoTrivialPPDirective() const {
+ return SeenNoTrivialPPDirective;
+}
+
+void NoTrivialPPDirectiveTracer::setSeenNoTrivialPPDirective() {
+ if (InMainFile && !SeenNoTrivialPPDirective)
+ SeenNoTrivialPPDirective = true;
+}
+
+void NoTrivialPPDirectiveTracer::LexedFileChanged(
+ FileID FID, LexedFileChangeReason Reason,
+ SrcMgr::CharacteristicKind FileType, FileID PrevFID, SourceLocation Loc) {
+ InMainFile = (FID == PP.getSourceManager().getMainFileID());
+}
+
+void NoTrivialPPDirectiveTracer::MacroExpands(const Token &MacroNameTok,
+ const MacroDefinition &MD,
+ SourceRange Range,
+ const MacroArgs *Args) {
+ // FIXME: Does only enable builtin macro expansion make sense?
+ if (!MD.getMacroInfo()->isBuiltinMacro())
+ setSeenNoTrivialPPDirective();
+}
diff --git a/clang/lib/Parse/Parser.cpp b/clang/lib/Parse/Parser.cpp
index 8834bf80c4016..447d0283431ba 100644
--- a/clang/lib/Parse/Parser.cpp
+++ b/clang/lib/Parse/Parser.cpp
@@ -2361,9 +2361,10 @@ Parser::ParseModuleDecl(Sema::ModuleImportState &ImportState) {
// Parse a global-module-fragment, if present.
if (getLangOpts().CPlusPlusModules && Tok.is(tok::semi)) {
SourceLocation SemiLoc = ConsumeToken();
- if (!Introducer.isFirstPPToken()) {
+ if (ImportState != Sema::ModuleImportState::FirstDecl ||
+ Introducer.hasSeenNoTrivialPPDirective()) {
Diag(StartLoc, diag::err_global_module_introducer_not_at_start)
- << SourceRange(StartLoc, SemiLoc);
+ << SourceRange(StartLoc, SemiLoc);
return nullptr;
}
if (MDK == Sema::ModuleDeclKind::Interface) {
@@ -2418,7 +2419,8 @@ Parser::ParseModuleDecl(Sema::ModuleImportState &ImportState) {
ExpectAndConsumeSemi(diag::err_module_expected_semi);
return Actions.ActOnModuleDecl(StartLoc, ModuleLoc, MDK, Path, Partition,
- ImportState, Introducer.isFirstPPToken());
+ ImportState,
+ Introducer.hasSeenNoTrivialPPDirective());
}
Decl *Parser::ParseModuleImport(SourceLocation AtLoc,
diff --git a/clang/lib/Sema/SemaModule.cpp b/clang/lib/Sema/SemaModule.cpp
index 7c982bcd63d73..10a390286ec7a 100644
--- a/clang/lib/Sema/SemaModule.cpp
+++ b/clang/lib/Sema/SemaModule.cpp
@@ -264,10 +264,11 @@ Sema::DeclGroupPtrTy
Sema::ActOnModuleDecl(SourceLocation StartLoc, SourceLocation ModuleLoc,
ModuleDeclKind MDK, ModuleIdPath Path,
ModuleIdPath Partition, ModuleImportState &ImportState,
- bool IntroducerIsFirstPPToken) {
+ bool SeenNoTrivialPPDirective) {
assert(getLangOpts().CPlusPlusModules &&
"should only have module decl in standard C++ modules");
+ bool IsFirstDecl = ImportState == ModuleImportState::FirstDecl;
bool SeenGMF = ImportState == ModuleImportState::GlobalFragment;
// If any of the steps here fail, we count that as invalidating C++20
// module state;
@@ -335,7 +336,8 @@ Sema::ActOnModuleDecl(SourceLocation StartLoc, SourceLocation ModuleLoc,
// In C++20, A module directive may only appear as the first preprocessing
// tokens in a file (excluding the global module fragment.).
- if (getLangOpts().CPlusPlusModules && !IntroducerIsFirstPPToken && !SeenGMF) {
+ if (getLangOpts().CPlusPlusModules &&
+ (!IsFirstDecl || SeenNoTrivialPPDirective) && !SeenGMF) {
Diag(ModuleLoc, diag::err_module_decl_not_at_start);
SourceLocation BeginLoc = PP.getMainFileFirstPPTokenLoc();
Diag(BeginLoc, diag::note_global_module_introducer_missing)
diff --git a/clang/test/CXX/module/cpp.pre/module_decl.cpp b/clang/test/CXX/module/cpp.pre/module_decl.cpp
index 6238347c167ac..5c29aeff1b632 100644
--- a/clang/test/CXX/module/cpp.pre/module_decl.cpp
+++ b/clang/test/CXX/module/cpp.pre/module_decl.cpp
@@ -1,8 +1,147 @@
// RUN: rm -rf %t
// RUN: mkdir -p %t
-// RUN: %clang_cc1 -std=c++20 -emit-module-interface %s -verify -o %t/M.pcm
+// RUN: split-file %s %t
+// RUN: %clang_cc1 -std=c++20 -emit-module-interface %t/line.cpp -verify -o %t/line.pcm
+// RUN: %clang_cc1 -std=c++20 -emit-module-interface %t/gnu_line_marker.cpp -verify -o %t/gnu_line_marker.pcm
+// RUN: %clang_cc1 -std=c++20 -emit-module-interface %t/include.cpp -verify -o %t/include.pcm
+// RUN: %clang_cc1 -std=c++20 -emit-module-interface %t/ident.cpp -verify -o %t/ident.pcm
+// RUN: %clang_cc1 -std=c++20 -emit-module-interface %t/pragma_comment.cpp -verify -o %t/pragma_comment.pcm
+// RUN: %clang_cc1 -std=c++20 -emit-module-interface %t/pragma_mark.cpp -verify -o %t/pragma_mark.pcm
+// RUN: %clang_cc1 -std=c++20 -emit-module-interface %t/pragma_detect_mismatch.cpp -verify -o %t/pragma_detect_mismatch.pcm
+// RUN: %clang_cc1 -std=c++20 -emit-module-interface %t/pragma_clang_debug.cpp -verify -o %t/pragma_clang_debug.pcm
+// RUN: %clang_cc1 -std=c++20 -emit-module-interface %t/pragma_message.cpp -verify -o %t/pragma_message.pcm
+// RUN: %clang_cc1 -std=c++20 -emit-module-interface %t/pragma_gcc_warn.cpp -verify -o %t/pragma_gcc_warn.pcm
+// RUN: %clang_cc1 -std=c++20 -emit-module-interface %t/pragma_gcc_error.cpp -verify -o %t/pragma_gcc_error.pcm
+// RUN: %clang_cc1 -std=c++20 -emit-module-interface %t/pragma_diag_push_pop.cpp -verify -o %t/pragma_diag_push_pop.pcm
+// RUN: %clang_cc1 -std=c++20 -emit-module-interface %t/pragma_diag_ignore.cpp -verify -o %t/pragma_diag_ignore.pcm
+// RUN: %clang_cc1 -std=c++20 -emit-module-interface %t/pragma_opencl_ext.cpp -verify -o %t/pragma_opencl_ext.pcm
+// RUN: %clang_cc1 -std=c++20 -emit-module-interface %t/pragma_push_pop.cpp -verify -o %t/pragma_push_pop.pcm
+// RUN: %clang_cc1 -std=c++20 -emit-module-interface %t/pragma_exec_charset.cpp -verify -o %t/pragma_exec_charset.pcm
+// RUN: %clang_cc1 -std=c++20 -emit-module-interface %t/pragma_clang_assume_nonnull.cpp -verify -o %t/pragma_clang_assume_nonnull.pcm
+// RUN: %clang_cc1 -std=c++20 -emit-module-interface %t/marco_expand.cpp -DMACRO="" -verify -o %t/marco_expand.pcm
+// RUN: %clang_cc1 -std=c++20 -emit-module-interface %t/define.cpp -verify -o %t/define.pcm
+// RUN: %clang_cc1 -std=c++20 -emit-module-interface %t/undef.cpp -verify -o %t/undef.pcm
+// RUN: %clang_cc1 -std=c++20 -emit-module-interface %t/defined.cpp -verify -o %t/defined.pcm
+// RUN: %clang_cc1 -std=c++20 -emit-module-interface %t/has_embed.cpp -verify -o %t/has_embed.pcm
+// RUN: %clang_cc1 -std=c++20 -emit-module-interface %t/has_include.cpp -verify -o %t/has_include.pcm
+//--- header.h
+#ifndef HEADER_H
+#define HEADER_H
+
+#endif // HEADER_H
+
+//--- line.cpp
+// expected-no-diagnostics
+#line 3
+export module M;
+
+//--- gnu_line_marker.cpp
+// expected-no-diagnostics
+# 1 __FILE__ 1 3
+export module M;
+
+//--- include.cpp
+#include "header.h" // expected-note {{add 'module;' to the start of the file to introduce a global module fragment}}
+export module M; // expected-error {{module declaration must occur at the start of the translation unit}}
+
+//--- ident.cpp
+// expected-no-diagnostics
+#ident "$Header:$"
+export module M;
+
+//--- pragma_comment.cpp
+// expected-no-diagnostics
+#pragma comment(lib, "msvcrt.lib")
+export module M;
+
+//--- pragma_mark.cpp
+// expected-no-diagnostics
+#pragma mark LLVM's world
+export module M;
+
+//--- pragma_detect_mismatch.cpp
+// expected-no-diagnostics
+#pragma detect_mismatch("test", "1")
+export module M;
+
+//--- pragma_clang_debug.cpp
+// expected-no-diagnostics
+#pragma clang __debug dump Test
+export module M;
+
+//--- pragma_message.cpp
+#pragma message "test" // expected-warning {{test}}
+export module M;
+
+//--- pragma_gcc_warn.cpp
+#pragma GCC warning "Foo" // expected-warning {{Foo}}
+export module M;
+
+//--- pragma_gcc_error.cpp
+#pragma GCC error "Foo" // expected-error {{Foo}}
+export module M;
+
+//--- pragma_diag_push_pop.cpp
+// expected-no-diagnostics
+#pragma gcc diagnostic push
+#pragma gcc diagnostic pop
+export module M;
+
+//--- pragma_diag_ignore.cpp
+// expected-no-diagnostics
+#pragma GCC diagnostic ignored "-Wframe-larger-than"
+export module M;
+
+//--- pragma_opencl_ext.cpp
+// expected-no-diagnostics
+#pragma OPENCL EXTENSION __cl_clang_variadic_functions : enable
+export module M;
+
+//--- pragma_push_pop.cpp
+// expected-no-diagnostics
+#pragma warning(push)
+#pragma warning(pop)
+export module M;
+
+//--- pragma_exec_charset.cpp
+// expected-no-diagnostics
+#pragma execution_character_set(push, "UTF-8")
+#pragma execution_character_set(pop)
+export module M;
+
+//--- pragma_clang_assume_nonnull.cpp
+// expected-no-diagnostics
+#pragma clang assume_nonnull begin
+#pragma clang assume_nonnull end
+export module M;
+
+//--- marco_expand.cpp
+MACRO // expected-note {{add 'module;' to the start of the file to introduce a global module fragment}}
+export module M; // expected-error {{module declaration must occur at the start of the translation unit}}
+
+//--- define.cpp
// This is a comment
#define I32 int // expected-note {{add 'module;' to the start of the file to introduce a global module fragment}}
export module M; // expected-error {{module declaration must occur at the start of the translation unit}}
export I32 i32;
+
+//--- undef.cpp
+#undef FOO // expected-note {{add 'module;' to the start of the file to introduce a global module fragment}}
+export module M; // expected-error {{module declaration must occur at the start of the translation unit}}
+
+//--- defined.cpp
+#if defined(FOO) // expected-note {{add 'module;' to the start of the file to introduce a global module fragment}}
+#endif
+export module M; // expected-error {{module declaration must occur at the start of the translation unit}}
+
+//--- has_embed.cpp
+#if __has_embed(__FILE__ ext::token(0xB055)) // expected-note {{add 'module;' to the start of the file to introduce a global module fragment}}
+#endif
+export module M; // expected-error {{module declaration must occur at the start of the translation unit}}
+
+//--- has_include.cpp
+#if __has_include(<stdio.h>) || __has_include_next(<stdlib.h>) // expected-note {{add 'module;' to the start of the file to introduce a global module fragment}} \
+ // expected-warning {{#include_next in primary source file; will search from start of include path}}
+#endif
+export module M; // expected-error {{module declaration must occur at the start of the translation unit}}
diff --git a/clang/unittests/Lex/CMakeLists.txt b/clang/unittests/Lex/CMakeLists.txt
index 96ca6dda9cd85..fa5e58f5a8932 100644
--- a/clang/unittests/Lex/CMakeLists.txt
+++ b/clang/unittests/Lex/CMakeLists.txt
@@ -5,6 +5,7 @@ add_clang_unittest(LexTests
LexerTest.cpp
LexHLSLRootSignatureTest.cpp
ModuleDeclStateTest.cpp
+ NoTrivialPPDirectiveTracerTest.cpp
PPCallbacksTest.cpp
PPConditionalDirectiveRecordTest.cpp
PPDependencyDirectivesTest.cpp
diff --git a/clang/unittests/Lex/LexerTest.cpp b/clang/unittests/Lex/LexerTest.cpp
index 86df872f6b7df..bb6404c434485 100644
--- a/clang/unittests/Lex/LexerTest.cpp
+++ b/clang/unittests/Lex/LexerTest.cpp
@@ -796,7 +796,7 @@ TEST_F(LexerTest, CheckFirstPPToken) {
EXPECT_FALSE(Lexer::getRawToken(PP->getMainFileFirstPPTokenLoc(), Tok,
PP->getSourceManager(), PP->getLangOpts(),
/*IgnoreWhiteSpace=*/false));
- EXPECT_TRUE(Tok.isFirstPPToken());
+ EXPECT_TRUE(PP->getMainFileFirstPPTokenLoc() == Tok.getLocation());
EXPECT_TRUE(Tok.is(tok::hash));
}
@@ -812,7 +812,7 @@ TEST_F(LexerTest, CheckFirstPPToken) {
EXPECT_FALSE(Lexer::getRawToken(PP->getMainFileFirstPPTokenLoc(), Tok,
PP->getSourceManager(), PP->getLangOpts(),
/*IgnoreWhiteSpace=*/false));
- EXPECT_TRUE(Tok.isFirstPPToken());
+ EXPECT_TRUE(PP->getMainFileFirstPPTokenLoc() == Tok.getLocation());
EXPECT_TRUE(Tok.is(tok::raw_identifier));
EXPECT_TRUE(Tok.getRawIdentifier() == "FOO");
}
diff --git a/clang/unittests/Lex/ModuleDeclStateTest.cpp b/clang/unittests/Lex/ModuleDeclStateTest.cpp
index 6ecba4de3187c..0c03cfd6d0f89 100644
--- a/clang/unittests/Lex/ModuleDeclStateTest.cpp
+++ b/clang/unittests/Lex/ModuleDeclStateTest.cpp
@@ -61,14 +61,15 @@ class ModuleDeclStateTest : public ::testing::Test {
Target = TargetInfo::CreateTargetInfo(Diags, *TargetOpts);
}
- std::unique_ptr<Preprocessor>
- getPreprocessor(const char *source, Language Lang) {
+ std::unique_ptr<Preprocessor> getPreprocessor(const char *source,
+ Language Lang) {
std::unique_ptr<llvm::MemoryBuffer> Buf =
llvm::MemoryBuffer::getMemBuffer(source);
SourceMgr.setMainFileID(SourceMgr.createFileID(std::move(Buf)));
std::vector<std::string> Includes;
- LangOptions::setLangDefaults(LangOpts, Lang, Target->getTriple(), Includes, LangStandard::lang_cxx20);
+ LangOptions::setLangDefaults(LangOpts, Lang, Target->getTriple(), Includes,
+ LangStandard::lang_cxx20);
LangOpts.CPlusPlusModules = true;
if (Lang != Language::CXX) {
LangOpts.Modules = true;
@@ -113,12 +114,11 @@ export module foo;
std::unique_ptr<Preprocessor> PP = getPreprocessor(source, Language::CXX);
std::initializer_list<bool> ImportKinds = {};
- preprocess(*PP,
- std::make_unique<CheckNamedModuleImportingCB>(*PP, ImportKinds));
-
- auto *Callback =
- static_cast<CheckNamedModuleImportingCB *>(PP->getPPCallbacks());
- EXPECT_EQ(Callback->importNamedModuleNum(), (size_t)0);
+ auto Callback =
+ std::make_unique<CheckNamedModuleImportingCB>(*PP, ImportKinds);
+ CheckNamedModuleImportingCB *CallbackPtr = Callback.get();
+ preprocess(*PP, std::move(Callback));
+ EXPECT_EQ(CallbackPtr->importNamedModuleNum(), (size_t)0);
EXPECT_TRUE(PP->isInNamedModule());
EXPECT_TRUE(PP->isInNamedInterfaceUnit());
EXPECT_FALSE(PP->isInImplementationUnit());
@@ -132,12 +132,11 @@ module foo;
std::unique_ptr<Preprocessor> PP = getPreprocessor(source, Language::CXX);
std::initializer_list<bool> ImportKinds = {};
- preprocess(*PP,
- std::make_unique<CheckNamedModuleImportingCB>(*PP, ImportKinds));
-
- auto *Callback =
- static_cast<CheckNamedModuleImportingCB *>(PP->getPPCallbacks());
- EXPECT_EQ(Callback->importNamedModuleNum(), (size_t)0);
+ auto Callback =
+ std::make_unique<CheckNamedModuleImportingCB>(*PP, ImportKinds);
+ CheckNamedModuleImportingCB *CallbackPtr = Callback.get();
+ preprocess(*PP, std::move(Callback));
+ EXPECT_EQ(CallbackPtr->importNamedModuleNum(), (size_t)0);
EXPECT_TRUE(PP->isInNamedModule());
EXPECT_FALSE(PP->isInNamedInterfaceUnit());
EXPECT_TRUE(PP->isInImplementationUnit());
@@ -151,12 +150,11 @@ module foo:part;
std::unique_ptr<Preprocessor> PP = getPreprocessor(source, Language::CXX);
std::initializer_list<bool> ImportKinds = {};
- preprocess(*PP,
- std::make_unique<CheckNamedModuleImportingCB>(*PP, ImportKinds));
-
- auto *Callback =
- static_cast<CheckNamedModuleImportingCB *>(PP->getPPCallbacks());
- EXPECT_EQ(Callback->importNamedModuleNum(), (size_t)0);
+ auto Callback =
+ std::make_unique<CheckNamedModuleImportingCB>(*PP, ImportKinds);
+ CheckNamedModuleImportingCB *CallbackPtr = Callback.get();
+ preprocess(*PP, std::move(Callback));
+ EXPECT_EQ(CallbackPtr->importNamedModuleNum(), (size_t)0);
EXPECT_TRUE(PP->isInNamedModule());
EXPECT_FALSE(PP->isInNamedInterfaceUnit());
EXPECT_FALSE(PP->isInImplementationUnit());
@@ -170,12 +168,11 @@ export module foo:part;
std::unique_ptr<Preprocessor> PP = getPreprocessor(source, Language::CXX);
std::initializer_list<bool> ImportKinds = {};
- preprocess(*PP,
- std::make_unique<CheckNamedModuleImportingCB>(*PP, ImportKinds));
-
- auto *Callback =
- static_cast<CheckNamedModuleImportingCB *>(PP->getPPCallbacks());
- EXPECT_EQ(Callback->importNamedModuleNum(), (size_t)0);
+ auto Callback =
+ std::make_unique<CheckNamedModuleImportingCB>(*PP, ImportKinds);
+ CheckNamedModuleImportingCB *CallbackPtr = Callback.get();
+ preprocess(*PP, std::move(Callback));
+ EXPECT_EQ(CallbackPtr->importNamedModuleNum(), (size_t)0);
EXPECT_TRUE(PP->isInNamedModule());
EXPECT_TRUE(PP->isInNamedInterfaceUnit());
EXPECT_FALSE(PP->isInImplementationUnit());
@@ -189,12 +186,11 @@ export module foo.dot:part.dot;
std::unique_ptr<Preprocessor> PP = getPreprocessor(source, Language::CXX);
std::initializer_list<bool> ImportKinds = {};
- preprocess(*PP,
- std::make_unique<CheckNamedModuleImportingCB>(*PP, ImportKinds));
-
- auto *Callback =
- static_cast<CheckNamedModuleImportingCB *>(PP->getPPCallbacks());
- EXPECT_EQ(Callback->importNamedModuleNum(), (size_t)0);
+ auto Callback =
+ std::make_unique<CheckNamedModuleImportingCB>(*PP, ImportKinds);
+ CheckNamedModuleImportingCB *CallbackPtr = Callback.get();
+ preprocess(*PP, std::move(Callback));
+ EXPECT_EQ(CallbackPtr->importNamedModuleNum(), (size_t)0);
EXPECT_TRUE(PP->isInNamedModule());
EXPECT_TRUE(PP->isInNamedInterfaceUnit());
EXPECT_FALSE(PP->isInImplementationUnit());
@@ -208,12 +204,11 @@ TEST_F(ModuleDeclStateTest, NotModule) {
std::unique_ptr<Preprocessor> PP = getPreprocessor(source, Language::CXX);
std::initializer_list<bool> ImportKinds = {};
- preprocess(*PP,
- std::make_unique<CheckNamedModuleImportingCB>(*PP, ImportKinds));
-
- auto *Callback =
- static_cast<CheckNamedModuleImportingCB *>(PP->getPPCallbacks());
- EXPECT_EQ(Callback->importNamedModuleNum(), (size_t)0);
+ auto Callback =
+ std::make_unique<CheckNamedModuleImportingCB>(*PP, ImportKinds);
+ CheckNamedModuleImportingCB *CallbackPtr = Callback.get();
+ preprocess(*PP, std::move(Callback));
+ EXPECT_EQ(CallbackPtr->importNamedModuleNum(), (size_t)0);
EXPECT_FALSE(PP->isInNamedModule());
EXPECT_FALSE(PP->isInNamedInterfaceUnit());
EXPECT_FALSE(PP->isInImplementationUnit());
@@ -234,12 +229,11 @@ import :another;
std::unique_ptr<Preprocessor> PP = getPreprocessor(source, Language::CXX);
std::initializer_list<bool> ImportKinds = {true, true};
- preprocess(*PP,
- std::make_unique<CheckNamedModuleImportingCB>(*PP, ImportKinds));
-
- auto *Callback =
- static_cast<CheckNamedModuleImportingCB *>(PP->getPPCallbacks());
- EXPECT_EQ(Callback->importNamedModuleNum(), (size_t)2);
+ auto Callback =
+ std::make_unique<CheckNamedModuleImportingCB>(*PP, ImportKinds);
+ CheckNamedModuleImportingCB *CallbackPtr = Callback.get();
+ preprocess(*PP, std::move(Callback));
+ EXPECT_EQ(CallbackPtr->importNamedModuleNum(), (size_t)2);
EXPECT_TRUE(PP->isInNamedModule());
EXPECT_TRUE(PP->isInNamedInterfaceUnit());
EXPECT_FALSE(PP->isInImplementationUnit());
@@ -261,12 +255,11 @@ import :another;
std::unique_ptr<Preprocessor> PP = getPreprocessor(source, Language::CXX);
std::initializer_list<bool> ImportKinds = {true, true};
- preprocess(*PP,
- std::make_unique<CheckNamedModuleImportingCB>(*PP, ImportKinds));
-
- auto *Callback =
- static_cast<CheckNamedModuleImportingCB *>(PP->getPPCallbacks());
- EXPECT_EQ(Callback->importNamedModuleNum(), (size_t)2);
+ auto Callback =
+ std::make_unique<CheckNamedModuleImportingCB>(*PP, ImportKinds);
+ CheckNamedModuleImportingCB *CallbackPtr = Callback.get();
+ preprocess(*PP, std::move(Callback));
+ EXPECT_EQ(CallbackPtr->importNamedModuleNum(), (size_t)2);
EXPECT_TRUE(PP->isInNamedModule());
EXPECT_TRUE(PP->isInNamedInterfaceUnit());
EXPECT_FALSE(PP->isInImplementationUnit());
@@ -287,12 +280,11 @@ import :another;
std::unique_ptr<Preprocessor> PP = getPreprocessor(source, Language::CXX);
std::initializer_list<bool> ImportKinds = {true};
- preprocess(*PP,
- std::make_unique<CheckNamedModuleImportingCB>(*PP, ImportKinds));
-
- auto *Callback =
- static_cast<CheckNamedModuleImportingCB *>(PP->getPPCallbacks());
- EXPECT_EQ(Callback->importNamedModuleNum(), (size_t)1);
+ auto Callback =
+ std::make_unique<CheckNamedModuleImportingCB>(*PP, ImportKinds);
+ CheckNamedModuleImportingCB *CallbackPtr = Callback.get();
+ preprocess(*PP, std::move(Callback));
+ EXPECT_EQ(CallbackPtr->importNamedModuleNum(), (size_t)1);
EXPECT_FALSE(PP->isInNamedModule());
EXPECT_FALSE(PP->isInNamedInterfaceUnit());
EXPECT_FALSE(PP->isInImplementationUnit());
@@ -305,12 +297,11 @@ TEST_F(ModuleDeclStateTest, ImportAClangNamedModule) {
std::unique_ptr<Preprocessor> PP = getPreprocessor(source, Language::ObjCXX);
std::initializer_list<bool> ImportKinds = {false};
- preprocess(*PP,
- std::make_unique<CheckNamedModuleImportingCB>(*PP, ImportKinds));
-
- auto *Callback =
- static_cast<CheckNamedModuleImportingCB *>(PP->getPPCallbacks());
- EXPECT_EQ(Callback->importNamedModuleNum(), (size_t)1);
+ auto Callback =
+ std::make_unique<CheckNamedModuleImportingCB>(*PP, ImportKinds);
+ CheckNamedModuleImportingCB *CallbackPtr = Callback.get();
+ preprocess(*PP, std::move(Callback));
+ EXPECT_EQ(CallbackPtr->importNamedModuleNum(), (size_t)1);
EXPECT_FALSE(PP->isInNamedModule());
EXPECT_FALSE(PP->isInNamedInterfaceUnit());
EXPECT_FALSE(PP->isInImplementationUnit());
@@ -327,12 +318,11 @@ import M2;
std::unique_ptr<Preprocessor> PP = getPreprocessor(source, Language::ObjCXX);
std::initializer_list<bool> ImportKinds = {false, true, false, true};
- preprocess(*PP,
- std::make_unique<CheckNamedModuleImportingCB>(*PP, ImportKinds));
-
- auto *Callback =
- static_cast<CheckNamedModuleImportingCB *>(PP->getPPCallbacks());
- EXPECT_EQ(Callback->importNamedModuleNum(), (size_t)4);
+ auto Callback =
+ std::make_unique<CheckNamedModuleImportingCB>(*PP, ImportKinds);
+ CheckNamedModuleImportingCB *CallbackPtr = Callback.get();
+ preprocess(*PP, std::move(Callback));
+ EXPECT_EQ(CallbackPtr->importNamedModuleNum(), (size_t)4);
EXPECT_FALSE(PP->isInNamedModule());
EXPECT_FALSE(PP->isInNamedInterfaceUnit());
EXPECT_FALSE(PP->isInImplementationUnit());
diff --git a/clang/unittests/Lex/NoTrivialPPDirectiveTracerTest.cpp b/clang/unittests/Lex/NoTrivialPPDirectiveTracerTest.cpp
new file mode 100644
index 0000000000000..d79c1428e55bb
--- /dev/null
+++ b/clang/unittests/Lex/NoTrivialPPDirectiveTracerTest.cpp
@@ -0,0 +1,182 @@
+//===- unittests/Lex/NoTrivialPPDirectiveTracerTest.cpp -------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/Basic/Diagnostic.h"
+#include "clang/Basic/DiagnosticOptions.h"
+#include "clang/Basic/FileManager.h"
+#include "clang/Basic/LangOptions.h"
+#include "clang/Basic/SourceManager.h"
+#include "clang/Basic/TargetInfo.h"
+#include "clang/Basic/TargetOptions.h"
+#include "clang/Lex/HeaderSearch.h"
+#include "clang/Lex/HeaderSearchOptions.h"
+#include "clang/Lex/ModuleLoader.h"
+#include "clang/Lex/Preprocessor.h"
+#include "clang/Lex/PreprocessorOptions.h"
+#include "gtest/gtest.h"
+#include <cstddef>
+#include <initializer_list>
+
+using namespace clang;
+
+namespace {
+class NoTrivialPPDirectiveTracerTest : public ::testing::Test {
+protected:
+ NoTrivialPPDirectiveTracerTest()
+ : VFS(llvm::makeIntrusiveRefCnt<llvm::vfs::InMemoryFileSystem>()),
+ FileMgr(FileMgrOpts, VFS),
+ Diags(DiagnosticIDs::create(), DiagOpts, new IgnoringDiagConsumer()),
+ SourceMgr(Diags, FileMgr), TargetOpts(new TargetOptions) {
+ TargetOpts->Triple = "x86_64-unknown-linux-gnu";
+ Target = TargetInfo::CreateTargetInfo(Diags, *TargetOpts);
+ }
+
+ void addFile(const char *source, StringRef Filename) {
+ VFS->addFile(Filename, 0, llvm::MemoryBuffer::getMemBuffer(source),
+ /*User=*/std::nullopt,
+ /*Group=*/std::nullopt,
+ llvm::sys::fs::file_type::regular_file);
+ }
+
+ std::unique_ptr<Preprocessor> getPreprocessor(const char *source,
+ Language Lang) {
+ std::unique_ptr<llvm::MemoryBuffer> Buf =
+ llvm::MemoryBuffer::getMemBuffer(source);
+ SourceMgr.setMainFileID(SourceMgr.createFileID(std::move(Buf)));
+
+ std::vector<std::string> Includes;
+ LangOptions::setLangDefaults(LangOpts, Lang, Target->getTriple(), Includes,
+ LangStandard::lang_cxx20);
+ LangOpts.CPlusPlusModules = true;
+ if (Lang != Language::CXX) {
+ LangOpts.Modules = true;
+ LangOpts.ImplicitModules = true;
+ }
+
+ HeaderInfo.emplace(HSOpts, SourceMgr, Diags, LangOpts, Target.get());
+
+ auto DE = FileMgr.getOptionalDirectoryRef(".");
+ assert(DE);
+ auto DL = DirectoryLookup(*DE, SrcMgr::C_User, /*isFramework=*/false);
+ HeaderInfo->AddSearchPath(DL, /*isAngled=*/false);
+
+ return std::make_unique<Preprocessor>(PPOpts, Diags, LangOpts, SourceMgr,
+ *HeaderInfo, ModLoader,
+ /*IILookup=*/nullptr,
+ /*OwnsHeaderSearch=*/false);
+ }
+
+ IntrusiveRefCntPtr<llvm::vfs::InMemoryFileSystem> VFS;
+ FileSystemOptions FileMgrOpts;
+ FileManager FileMgr;
+ DiagnosticOptions DiagOpts;
+ DiagnosticsEngine Diags;
+ SourceManager SourceMgr;
+ std::shared_ptr<TargetOptions> TargetOpts;
+ IntrusiveRefCntPtr<TargetInfo> Target;
+ LangOptions LangOpts;
+ TrivialModuleLoader ModLoader;
+ HeaderSearchOptions HSOpts;
+ std::optional<HeaderSearch> HeaderInfo;
+ PreprocessorOptions PPOpts;
+};
+
+TEST_F(NoTrivialPPDirectiveTracerTest, TrivialDirective) {
+ const char *source = R"(
+ #line 7
+ # 1 __FILE__ 1 3
+ #ident "$Header:$"
+ #pragma comment(lib, "msvcrt.lib")
+ #pragma mark LLVM's world
+ #pragma detect_mismatch("test", "1")
+ #pragma clang __debug dump Test
+ #pragma message "test"
+ #pragma GCC warning "Foo"
+ #pragma GCC error "Foo"
+ #pragma gcc diagnostic push
+ #pragma gcc diagnostic pop
+ #pragma GCC diagnostic ignored "-Wframe-larger-than"
+ #pragma OPENCL EXTENSION __cl_clang_variadic_functions : enable
+ #pragma warning(push)
+ #pragma warning(pop)
+ #pragma execution_character_set(push, "UTF-8")
+ #pragma execution_character_set(pop)
+ #pragma clang assume_nonnull begin
+ #pragma clang assume_nonnull end
+ int foo;
+ )";
+ std::unique_ptr<Preprocessor> PP = getPreprocessor(source, Language::CXX);
+ PP->Initialize(*Target);
+ PP->EnterMainSourceFile();
+ Token Tok;
+ PP->Lex(Tok);
+ EXPECT_FALSE(PP->hasSeenNoTrivialPPDirective());
+}
+
+TEST_F(NoTrivialPPDirectiveTracerTest, IncludeDirective) {
+ const char *source = R"(
+ #include "header.h"
+ int foo;
+ )";
+ const char *header = R"(
+ #ifndef HEADER_H
+ #define HEADER_H
+ #endif // HEADER_H
+ )";
+ std::unique_ptr<Preprocessor> PP = getPreprocessor(source, Language::CXX);
+ addFile(header, "header.h");
+ PP->Initialize(*Target);
+ PP->EnterMainSourceFile();
+ Token Tok;
+ PP->Lex(Tok);
+ EXPECT_TRUE(PP->hasSeenNoTrivialPPDirective());
+}
+
+TEST_F(NoTrivialPPDirectiveTracerTest, DefineDirective) {
+ const char *source = R"(
+ #define FOO
+ int foo;
+ )";
+ std::unique_ptr<Preprocessor> PP = getPreprocessor(source, Language::CXX);
+ PP->Initialize(*Target);
+ PP->EnterMainSourceFile();
+ Token Tok;
+ PP->Lex(Tok);
+ EXPECT_TRUE(PP->hasSeenNoTrivialPPDirective());
+}
+
+TEST_F(NoTrivialPPDirectiveTracerTest, UnDefineDirective) {
+ const char *source = R"(
+ #undef FOO
+ int foo;
+ )";
+ std::unique_ptr<Preprocessor> PP = getPreprocessor(source, Language::CXX);
+ PP->Initialize(*Target);
+ PP->setPredefines("#define FOO");
+ PP->EnterMainSourceFile();
+ Token Tok;
+ PP->Lex(Tok);
+ EXPECT_TRUE(PP->hasSeenNoTrivialPPDirective());
+}
+
+TEST_F(NoTrivialPPDirectiveTracerTest, IfDefinedDirective) {
+ const char *source = R"(
+ #if defined(FOO)
+ #endif
+ int foo;
+ )";
+ std::unique_ptr<Preprocessor> PP = getPreprocessor(source, Language::CXX);
+ PP->Initialize(*Target);
+ PP->setPredefines("#define FOO");
+ PP->EnterMainSourceFile();
+ Token Tok;
+ PP->Lex(Tok);
+ EXPECT_TRUE(PP->hasSeenNoTrivialPPDirective());
+}
+
+} // namespace
>From 591a19f83d2b1a79043cbc01f86437219054512e Mon Sep 17 00:00:00 2001
From: yronglin <yronglin777 at gmail.com>
Date: Mon, 18 Aug 2025 17:09:43 +0800
Subject: [PATCH 2/2] Fix NoTrivialPPDirectiveTracerTest.cpp build issue
Signed-off-by: yronglin <yronglin777 at gmail.com>
---
clang/unittests/Lex/NoTrivialPPDirectiveTracerTest.cpp | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/clang/unittests/Lex/NoTrivialPPDirectiveTracerTest.cpp b/clang/unittests/Lex/NoTrivialPPDirectiveTracerTest.cpp
index d79c1428e55bb..8b546fe2a40e2 100644
--- a/clang/unittests/Lex/NoTrivialPPDirectiveTracerTest.cpp
+++ b/clang/unittests/Lex/NoTrivialPPDirectiveTracerTest.cpp
@@ -29,8 +29,8 @@ class NoTrivialPPDirectiveTracerTest : public ::testing::Test {
protected:
NoTrivialPPDirectiveTracerTest()
: VFS(llvm::makeIntrusiveRefCnt<llvm::vfs::InMemoryFileSystem>()),
- FileMgr(FileMgrOpts, VFS),
- Diags(DiagnosticIDs::create(), DiagOpts, new IgnoringDiagConsumer()),
+ FileMgr(FileMgrOpts, VFS), DiagID(new DiagnosticIDs()),
+ Diags(DiagID, DiagOpts, new IgnoringDiagConsumer()),
SourceMgr(Diags, FileMgr), TargetOpts(new TargetOptions) {
TargetOpts->Triple = "x86_64-unknown-linux-gnu";
Target = TargetInfo::CreateTargetInfo(Diags, *TargetOpts);
@@ -74,6 +74,7 @@ class NoTrivialPPDirectiveTracerTest : public ::testing::Test {
IntrusiveRefCntPtr<llvm::vfs::InMemoryFileSystem> VFS;
FileSystemOptions FileMgrOpts;
FileManager FileMgr;
+ IntrusiveRefCntPtr<DiagnosticIDs> DiagID;
DiagnosticOptions DiagOpts;
DiagnosticsEngine Diags;
SourceManager SourceMgr;
More information about the llvm-branch-commits
mailing list