r302309 - Add support for building modules from preprocessed source.
Richard Smith via cfe-commits
cfe-commits at lists.llvm.org
Fri May 5 15:18:51 PDT 2017
Author: rsmith
Date: Fri May 5 17:18:51 2017
New Revision: 302309
URL: http://llvm.org/viewvc/llvm-project?rev=302309&view=rev
Log:
Add support for building modules from preprocessed source.
To support this, an optional marker "#pragma clang module contents" is
recognized in module map files, and the rest of the module map file from that
point onwards is treated as the source of the module. Preprocessing a module
map produces the input module followed by the marker and then the preprocessed
contents of the module.
Ignoring line markers, a preprocessed module might look like this:
module A {
header "a.h"
}
#pragma clang module contents
#pragma clang module begin A
// ... a.h ...
#pragma clang module end
The preprocessed output generates line markers, which are not accepted by the
module map parser, so -x c++-module-map-cpp-output should be used to compile
such outputs.
A couple of major parts do not work yet:
1) The files that are listed in the module map must exist on disk, in order to
build the on-disk header -> module lookup table in the PCM file. To fix
this, we need the preprocessed output to track the file size and other stat
information we might use to build the lookup table.
2) Declaration ownership semantics don't work properly yet, since mapping from
a source location to a module relies on mapping from FileIDs to modules,
which we can't do if module transitions can occur in the middle of a file.
Modified:
cfe/trunk/include/clang/Frontend/FrontendAction.h
cfe/trunk/include/clang/Lex/HeaderSearch.h
cfe/trunk/include/clang/Lex/ModuleMap.h
cfe/trunk/lib/Frontend/FrontendAction.cpp
cfe/trunk/lib/Frontend/FrontendActions.cpp
cfe/trunk/lib/Frontend/Rewrite/FrontendActions.cpp
cfe/trunk/lib/Lex/HeaderSearch.cpp
cfe/trunk/lib/Lex/ModuleMap.cpp
cfe/trunk/lib/Sema/SemaDecl.cpp
cfe/trunk/test/Modules/preprocess-module.cpp
Modified: cfe/trunk/include/clang/Frontend/FrontendAction.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Frontend/FrontendAction.h?rev=302309&r1=302308&r2=302309&view=diff
==============================================================================
--- cfe/trunk/include/clang/Frontend/FrontendAction.h (original)
+++ cfe/trunk/include/clang/Frontend/FrontendAction.h Fri May 5 17:18:51 2017
@@ -146,6 +146,8 @@ public:
return *CurrentASTUnit;
}
+ Module *getCurrentModule() const;
+
std::unique_ptr<ASTUnit> takeCurrentASTUnit() {
return std::move(CurrentASTUnit);
}
Modified: cfe/trunk/include/clang/Lex/HeaderSearch.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Lex/HeaderSearch.h?rev=302309&r1=302308&r2=302309&view=diff
==============================================================================
--- cfe/trunk/include/clang/Lex/HeaderSearch.h (original)
+++ cfe/trunk/include/clang/Lex/HeaderSearch.h Fri May 5 17:18:51 2017
@@ -538,9 +538,15 @@ public:
///
/// \param File The module map file.
/// \param IsSystem Whether this file is in a system header directory.
+ /// \param ID If the module map file is already mapped (perhaps as part of
+ /// processing a preprocessed module), the ID of the file.
+ /// \param Offset [inout] An offset within ID to start parsing. On exit,
+ /// filled by the end of the parsed contents (either EOF or the
+ /// location of an end-of-module-map pragma).
///
/// \returns true if an error occurred, false otherwise.
- bool loadModuleMapFile(const FileEntry *File, bool IsSystem);
+ bool loadModuleMapFile(const FileEntry *File, bool IsSystem,
+ FileID ID = FileID(), unsigned *Offset = nullptr);
/// \brief Collect the set of all known, top-level modules.
///
@@ -686,7 +692,9 @@ private:
LoadModuleMapResult loadModuleMapFileImpl(const FileEntry *File,
bool IsSystem,
- const DirectoryEntry *Dir);
+ const DirectoryEntry *Dir,
+ FileID ID = FileID(),
+ unsigned *Offset = nullptr);
/// \brief Try to load the module map file in the given directory.
///
Modified: cfe/trunk/include/clang/Lex/ModuleMap.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Lex/ModuleMap.h?rev=302309&r1=302308&r2=302309&view=diff
==============================================================================
--- cfe/trunk/include/clang/Lex/ModuleMap.h (original)
+++ cfe/trunk/include/clang/Lex/ModuleMap.h Fri May 5 17:18:51 2017
@@ -546,14 +546,20 @@ public:
/// \param HomeDir The directory in which relative paths within this module
/// map file will be resolved.
///
+ /// \param ID The FileID of the file to process, if we've already entered it.
+ ///
+ /// \param Offset [inout] On input the offset at which to start parsing. On
+ /// output, the offset at which the module map terminated.
+ ///
/// \param ExternModuleLoc The location of the "extern module" declaration
/// that caused us to load this module map file, if any.
///
/// \returns true if an error occurred, false otherwise.
bool parseModuleMapFile(const FileEntry *File, bool IsSystem,
- const DirectoryEntry *HomeDir,
+ const DirectoryEntry *HomeDir, FileID ID = FileID(),
+ unsigned *Offset = nullptr,
SourceLocation ExternModuleLoc = SourceLocation());
-
+
/// \brief Dump the contents of the module map, for debugging purposes.
void dump();
Modified: cfe/trunk/lib/Frontend/FrontendAction.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/FrontendAction.cpp?rev=302309&r1=302308&r2=302309&view=diff
==============================================================================
--- cfe/trunk/lib/Frontend/FrontendAction.cpp (original)
+++ cfe/trunk/lib/Frontend/FrontendAction.cpp Fri May 5 17:18:51 2017
@@ -136,6 +136,12 @@ void FrontendAction::setCurrentInput(con
CurrentASTUnit = std::move(AST);
}
+Module *FrontendAction::getCurrentModule() const {
+ CompilerInstance &CI = getCompilerInstance();
+ return CI.getPreprocessor().getHeaderSearchInfo().lookupModule(
+ CI.getLangOpts().CurrentModule, /*AllowSearch*/false);
+}
+
std::unique_ptr<ASTConsumer>
FrontendAction::CreateWrappedASTConsumer(CompilerInstance &CI,
StringRef InFile) {
@@ -188,16 +194,25 @@ FrontendAction::CreateWrappedASTConsumer
return llvm::make_unique<MultiplexConsumer>(std::move(Consumers));
}
-// For preprocessed files, if the first line is the linemarker and specifies
-// the original source file name, use that name as the input file name.
-static bool ReadOriginalFileName(CompilerInstance &CI, std::string &InputFile)
-{
- bool Invalid = false;
+/// For preprocessed files, if the first line is the linemarker and specifies
+/// the original source file name, use that name as the input file name.
+/// Returns the location of the first token after the line marker directive.
+///
+/// \param CI The compiler instance.
+/// \param InputFile Populated with the filename from the line marker.
+/// \param AddLineNote If \c true, add a line note corresponding to this line
+/// directive. Only use this if the directive will not actually be
+/// visited by the preprocessor.
+static SourceLocation ReadOriginalFileName(CompilerInstance &CI,
+ std::string &InputFile,
+ bool AddLineNote = false) {
auto &SourceMgr = CI.getSourceManager();
auto MainFileID = SourceMgr.getMainFileID();
+
+ bool Invalid = false;
const auto *MainFileBuf = SourceMgr.getBuffer(MainFileID, &Invalid);
if (Invalid)
- return false;
+ return SourceLocation();
std::unique_ptr<Lexer> RawLexer(
new Lexer(MainFileID, MainFileBuf, SourceMgr, CI.getLangOpts()));
@@ -209,19 +224,37 @@ static bool ReadOriginalFileName(Compile
// we use FILENAME as the input file name.
Token T;
if (RawLexer->LexFromRawLexer(T) || T.getKind() != tok::hash)
- return false;
+ return SourceLocation();
if (RawLexer->LexFromRawLexer(T) || T.isAtStartOfLine() ||
T.getKind() != tok::numeric_constant)
- return false;
+ return SourceLocation();
+
+ unsigned LineNo;
+ SourceLocation LineNoLoc = T.getLocation();
+ if (AddLineNote) {
+ llvm::SmallString<16> Buffer;
+ if (Lexer::getSpelling(LineNoLoc, Buffer, SourceMgr, CI.getLangOpts())
+ .getAsInteger(10, LineNo))
+ return SourceLocation();
+ }
+
RawLexer->LexFromRawLexer(T);
if (T.isAtStartOfLine() || T.getKind() != tok::string_literal)
- return false;
+ return SourceLocation();
StringLiteralParser Literal(T, CI.getPreprocessor());
if (Literal.hadError)
- return false;
+ return SourceLocation();
+ RawLexer->LexFromRawLexer(T);
+ if (T.isNot(tok::eof) && !T.isAtStartOfLine())
+ return SourceLocation();
InputFile = Literal.GetString().str();
- return true;
+
+ if (AddLineNote)
+ CI.getSourceManager().AddLineNote(
+ LineNoLoc, LineNo, SourceMgr.getLineTableFilenameID(InputFile));
+
+ return T.getLocation();
}
static SmallVectorImpl<char> &
@@ -339,42 +372,44 @@ collectModuleHeaderIncludes(const LangOp
return std::error_code();
}
-/// Parse a module map and compute the corresponding real input buffer that
-/// should be used to build the module described by that module map and the
-/// current module name.
-static std::unique_ptr<llvm::MemoryBuffer>
-getInputBufferForModuleMap(CompilerInstance &CI, StringRef Filename,
- bool IsSystem) {
- // Find the module map file.
- const FileEntry *ModuleMap =
- CI.getFileManager().getFile(Filename, /*openFile*/true);
- if (!ModuleMap) {
- CI.getDiagnostics().Report(diag::err_module_map_not_found)
- << Filename;
- return nullptr;
- }
+static bool
+loadModuleMapForModuleBuild(CompilerInstance &CI, StringRef Filename,
+ bool IsSystem, bool IsPreprocessed,
+ unsigned &Offset) {
+ auto &SrcMgr = CI.getSourceManager();
+ HeaderSearch &HS = CI.getPreprocessor().getHeaderSearchInfo();
- // Find the module map file from which it was generated, if different.
- const FileEntry *OriginalModuleMap = ModuleMap;
- StringRef OriginalModuleMapName = CI.getFrontendOpts().OriginalModuleMap;
- if (!OriginalModuleMapName.empty()) {
- OriginalModuleMap = CI.getFileManager().getFile(OriginalModuleMapName,
- /*openFile*/ true);
- if (!OriginalModuleMap) {
- CI.getDiagnostics().Report(diag::err_module_map_not_found)
- << OriginalModuleMapName;
- return nullptr;
- }
+ // Map the current input to a file.
+ FileID ModuleMapID = SrcMgr.getMainFileID();
+ const FileEntry *ModuleMap = SrcMgr.getFileEntryForID(ModuleMapID);
+
+ // If the module map is preprocessed, handle the initial line marker;
+ // line directives are not part of the module map syntax in general.
+ Offset = 0;
+ if (IsPreprocessed) {
+ std::string PresumedModuleMapFile;
+ SourceLocation EndOfLineMarker =
+ ReadOriginalFileName(CI, PresumedModuleMapFile, /*AddLineNote*/true);
+ if (EndOfLineMarker.isValid())
+ Offset = CI.getSourceManager().getDecomposedLoc(EndOfLineMarker).second;
+ // FIXME: Use PresumedModuleMapFile as the MODULE_MAP_FILE in the PCM.
}
-
- // Parse the module map file.
- HeaderSearch &HS = CI.getPreprocessor().getHeaderSearchInfo();
- if (HS.loadModuleMapFile(ModuleMap, IsSystem))
- return nullptr;
-
+
+ // Load the module map file.
+ if (HS.loadModuleMapFile(ModuleMap, IsSystem, ModuleMapID, &Offset))
+ return true;
+
+ if (SrcMgr.getBuffer(ModuleMapID)->getBufferSize() == Offset)
+ Offset = 0;
+
+ return false;
+}
+
+static Module *prepareToBuildModule(CompilerInstance &CI,
+ StringRef ModuleMapFilename) {
if (CI.getLangOpts().CurrentModule.empty()) {
CI.getDiagnostics().Report(diag::err_missing_module_name);
-
+
// FIXME: Eventually, we could consider asking whether there was just
// a single module described in the module map, and use that as a
// default. Then it would be fairly trivial to just "compile" a module
@@ -382,21 +417,14 @@ getInputBufferForModuleMap(CompilerInsta
return nullptr;
}
- // If we're being run from the command-line, the module build stack will not
- // have been filled in yet, so complete it now in order to allow us to detect
- // module cycles.
- SourceManager &SourceMgr = CI.getSourceManager();
- if (SourceMgr.getModuleBuildStack().empty())
- SourceMgr.pushModuleBuildStack(CI.getLangOpts().CurrentModule,
- FullSourceLoc(SourceLocation(), SourceMgr));
-
// Dig out the module definition.
+ HeaderSearch &HS = CI.getPreprocessor().getHeaderSearchInfo();
Module *M = HS.lookupModule(CI.getLangOpts().CurrentModule,
/*AllowSearch=*/false);
if (!M) {
CI.getDiagnostics().Report(diag::err_missing_module)
- << CI.getLangOpts().CurrentModule << Filename;
-
+ << CI.getLangOpts().CurrentModule << ModuleMapFilename;
+
return nullptr;
}
@@ -417,11 +445,45 @@ getInputBufferForModuleMap(CompilerInsta
return nullptr;
}
- if (OriginalModuleMap != ModuleMap) {
- M->IsInferred = true;
- HS.getModuleMap().setInferredModuleAllowedBy(M, OriginalModuleMap);
+ // Inform the preprocessor that includes from within the input buffer should
+ // be resolved relative to the build directory of the module map file.
+ CI.getPreprocessor().setMainFileDir(M->Directory);
+
+ // If the module was inferred from a different module map (via an expanded
+ // umbrella module definition), track that fact.
+ // FIXME: It would be preferable to fill this in as part of processing
+ // the module map, rather than adding it after the fact.
+ StringRef OriginalModuleMapName = CI.getFrontendOpts().OriginalModuleMap;
+ if (!OriginalModuleMapName.empty()) {
+ auto *OriginalModuleMap =
+ CI.getFileManager().getFile(OriginalModuleMapName,
+ /*openFile*/ true);
+ if (!OriginalModuleMap) {
+ CI.getDiagnostics().Report(diag::err_module_map_not_found)
+ << OriginalModuleMapName;
+ return nullptr;
+ }
+ if (OriginalModuleMap != CI.getSourceManager().getFileEntryForID(
+ CI.getSourceManager().getMainFileID())) {
+ M->IsInferred = true;
+ CI.getPreprocessor().getHeaderSearchInfo().getModuleMap()
+ .setInferredModuleAllowedBy(M, OriginalModuleMap);
+ }
}
+ // If we're being run from the command-line, the module build stack will not
+ // have been filled in yet, so complete it now in order to allow us to detect
+ // module cycles.
+ SourceManager &SourceMgr = CI.getSourceManager();
+ if (SourceMgr.getModuleBuildStack().empty())
+ SourceMgr.pushModuleBuildStack(CI.getLangOpts().CurrentModule,
+ FullSourceLoc(SourceLocation(), SourceMgr));
+ return M;
+}
+
+/// Compute the input buffer that should be used to build the specified module.
+static std::unique_ptr<llvm::MemoryBuffer>
+getInputBufferForModule(CompilerInstance &CI, Module *M) {
FileManager &FileMgr = CI.getFileManager();
// Collect the set of #includes we need to build the module.
@@ -441,10 +503,6 @@ getInputBufferForModuleMap(CompilerInsta
return nullptr;
}
- // Inform the preprocessor that includes from within the input buffer should
- // be resolved relative to the build directory of the module map file.
- CI.getPreprocessor().setMainFileDir(M->Directory);
-
return llvm::MemoryBuffer::getMemBufferCopy(
HeaderContents, Module::getModuleInputBufferName());
}
@@ -457,7 +515,6 @@ bool FrontendAction::BeginSourceFile(Com
setCompilerInstance(&CI);
StringRef InputFile = Input.getFile();
- FrontendInputFile FileToProcess = Input;
bool HasBegunSourceFile = false;
if (!BeginInvocation(CI))
goto failure;
@@ -597,36 +654,45 @@ bool FrontendAction::BeginSourceFile(Com
&CI.getPreprocessor());
HasBegunSourceFile = true;
+ // Initialize the main file entry.
+ if (!CI.InitializeSourceManager(Input))
+ goto failure;
+
// For module map files, we first parse the module map and synthesize a
// "<module-includes>" buffer before more conventional processing.
if (Input.getKind().getFormat() == InputKind::ModuleMap) {
CI.getLangOpts().setCompilingModule(LangOptions::CMK_ModuleMap);
- auto Buffer = getInputBufferForModuleMap(CI, InputFile, Input.isSystem());
- if (!Buffer)
+ unsigned OffsetToContents;
+ if (loadModuleMapForModuleBuild(CI, Input.getFile(), Input.isSystem(),
+ Input.isPreprocessed(), OffsetToContents))
+ goto failure;
+
+ auto *CurrentModule = prepareToBuildModule(CI, Input.getFile());
+ if (!CurrentModule)
goto failure;
- Module *CurrentModule =
- CI.getPreprocessor().getHeaderSearchInfo().lookupModule(
- CI.getLangOpts().CurrentModule,
- /*AllowSearch=*/false);
- assert(CurrentModule && "no module info for current module");
-
- // The input that we end up processing is the generated buffer, not the
- // module map file itself.
- FileToProcess = FrontendInputFile(
- Buffer.release(), Input.getKind().withFormat(InputKind::Source),
- CurrentModule->IsSystem);
+ if (OffsetToContents)
+ // If the module contents are in the same file, skip to them.
+ CI.getPreprocessor().setSkipMainFilePreamble(OffsetToContents, true);
+ else {
+ // Otherwise, convert the module description to a suitable input buffer.
+ auto Buffer = getInputBufferForModule(CI, CurrentModule);
+ if (!Buffer)
+ goto failure;
+
+ // Reinitialize the main file entry to refer to the new input.
+ if (!CI.InitializeSourceManager(FrontendInputFile(
+ Buffer.release(), Input.getKind().withFormat(InputKind::Source),
+ CurrentModule->IsSystem)))
+ goto failure;
+ }
}
// Initialize the action.
if (!BeginSourceFileAction(CI, InputFile))
goto failure;
- // Initialize the main file entry.
- if (!CI.InitializeSourceManager(FileToProcess))
- goto failure;
-
// Create the AST context and consumer unless this is a preprocessor only
// action.
if (!usesPreprocessorOnly()) {
@@ -636,13 +702,12 @@ bool FrontendAction::BeginSourceFile(Com
// For preprocessed files, check if the first line specifies the original
// source file name with a linemarker.
- std::string OrigFile;
+ std::string PresumedInputFile = InputFile;
if (Input.isPreprocessed())
- if (ReadOriginalFileName(CI, OrigFile))
- InputFile = OrigFile;
+ ReadOriginalFileName(CI, PresumedInputFile);
std::unique_ptr<ASTConsumer> Consumer =
- CreateWrappedASTConsumer(CI, InputFile);
+ CreateWrappedASTConsumer(CI, PresumedInputFile);
if (!Consumer)
goto failure;
Modified: cfe/trunk/lib/Frontend/FrontendActions.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/FrontendActions.cpp?rev=302309&r1=302308&r2=302309&view=diff
==============================================================================
--- cfe/trunk/lib/Frontend/FrontendActions.cpp (original)
+++ cfe/trunk/lib/Frontend/FrontendActions.cpp Fri May 5 17:18:51 2017
@@ -542,6 +542,18 @@ void PrintPreprocessedAction::ExecuteAct
CI.createDefaultOutputFile(BinaryMode, getCurrentFile());
if (!OS) return;
+ // If we're preprocessing a module map, start by dumping the contents of the
+ // module itself before switching to the input buffer.
+ auto &Input = getCurrentInput();
+ if (Input.getKind().getFormat() == InputKind::ModuleMap) {
+ if (Input.isFile())
+ (*OS) << "# 1 \"" << Input.getFile() << "\"\n";
+ // FIXME: Include additional information here so that we don't need the
+ // original source files to exist on disk.
+ getCurrentModule()->print(*OS);
+ (*OS) << "#pragma clang module contents\n";
+ }
+
DoPrintPreprocessedInput(CI.getPreprocessor(), OS.get(),
CI.getPreprocessorOutputOpts());
}
Modified: cfe/trunk/lib/Frontend/Rewrite/FrontendActions.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/Rewrite/FrontendActions.cpp?rev=302309&r1=302308&r2=302309&view=diff
==============================================================================
--- cfe/trunk/lib/Frontend/Rewrite/FrontendActions.cpp (original)
+++ cfe/trunk/lib/Frontend/Rewrite/FrontendActions.cpp Fri May 5 17:18:51 2017
@@ -196,6 +196,18 @@ void RewriteIncludesAction::ExecuteActio
CI.createDefaultOutputFile(true, getCurrentFile());
if (!OS) return;
+ // If we're preprocessing a module map, start by dumping the contents of the
+ // module itself before switching to the input buffer.
+ auto &Input = getCurrentInput();
+ if (Input.getKind().getFormat() == InputKind::ModuleMap) {
+ if (Input.isFile())
+ (*OS) << "# 1 \"" << Input.getFile() << "\"\n";
+ // FIXME: Include additional information here so that we don't need the
+ // original source files to exist on disk.
+ getCurrentModule()->print(*OS);
+ (*OS) << "#pragma clang module contents\n";
+ }
+
RewriteIncludesInInput(CI.getPreprocessor(), OS.get(),
CI.getPreprocessorOutputOpts());
}
Modified: cfe/trunk/lib/Lex/HeaderSearch.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/HeaderSearch.cpp?rev=302309&r1=302308&r2=302309&view=diff
==============================================================================
--- cfe/trunk/lib/Lex/HeaderSearch.cpp (original)
+++ cfe/trunk/lib/Lex/HeaderSearch.cpp Fri May 5 17:18:51 2017
@@ -1325,7 +1325,8 @@ static const FileEntry *getPrivateModule
return FileMgr.getFile(PrivateFilename);
}
-bool HeaderSearch::loadModuleMapFile(const FileEntry *File, bool IsSystem) {
+bool HeaderSearch::loadModuleMapFile(const FileEntry *File, bool IsSystem,
+ FileID ID, unsigned *Offset) {
// Find the directory for the module. For frameworks, that may require going
// up from the 'Modules' directory.
const DirectoryEntry *Dir = nullptr;
@@ -1344,7 +1345,7 @@ bool HeaderSearch::loadModuleMapFile(con
}
}
- switch (loadModuleMapFileImpl(File, IsSystem, Dir)) {
+ switch (loadModuleMapFileImpl(File, IsSystem, Dir, ID, Offset)) {
case LMM_AlreadyLoaded:
case LMM_NewlyLoaded:
return false;
@@ -1357,7 +1358,8 @@ bool HeaderSearch::loadModuleMapFile(con
HeaderSearch::LoadModuleMapResult
HeaderSearch::loadModuleMapFileImpl(const FileEntry *File, bool IsSystem,
- const DirectoryEntry *Dir) {
+ const DirectoryEntry *Dir, FileID ID,
+ unsigned *Offset) {
assert(File && "expected FileEntry");
// Check whether we've already loaded this module map, and mark it as being
@@ -1366,7 +1368,7 @@ HeaderSearch::loadModuleMapFileImpl(cons
if (!AddResult.second)
return AddResult.first->second ? LMM_AlreadyLoaded : LMM_InvalidModuleMap;
- if (ModMap.parseModuleMapFile(File, IsSystem, Dir)) {
+ if (ModMap.parseModuleMapFile(File, IsSystem, Dir, ID, Offset)) {
LoadedModuleMaps[File] = false;
return LMM_InvalidModuleMap;
}
Modified: cfe/trunk/lib/Lex/ModuleMap.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/ModuleMap.cpp?rev=302309&r1=302308&r2=302309&view=diff
==============================================================================
--- cfe/trunk/lib/Lex/ModuleMap.cpp (original)
+++ cfe/trunk/lib/Lex/ModuleMap.cpp Fri May 5 17:18:51 2017
@@ -1132,14 +1132,17 @@ namespace clang {
}
bool parseModuleMapFile();
+
+ bool terminatedByDirective() { return false; }
+ SourceLocation getLocation() { return Tok.getLocation(); }
};
}
SourceLocation ModuleMapParser::consumeToken() {
-retry:
SourceLocation Result = Tok.getLocation();
+
+retry:
Tok.clear();
-
Token LToken;
L.LexFromRawLexer(LToken);
Tok.Location = LToken.getLocation().getRawEncoding();
@@ -1232,9 +1235,28 @@ retry:
case tok::comment:
goto retry;
-
+
+ case tok::hash:
+ // A module map can be terminated prematurely by
+ // #pragma clang module contents
+ // When building the module, we'll treat the rest of the file as the
+ // contents of the module.
+ {
+ auto NextIsIdent = [&](StringRef Str) -> bool {
+ L.LexFromRawLexer(LToken);
+ return !LToken.isAtStartOfLine() && LToken.is(tok::raw_identifier) &&
+ LToken.getRawIdentifier() == Str;
+ };
+ if (NextIsIdent("pragma") && NextIsIdent("clang") &&
+ NextIsIdent("module") && NextIsIdent("contents")) {
+ Tok.Kind = MMToken::EndOfFile;
+ break;
+ }
+ }
+ LLVM_FALLTHROUGH;
+
default:
- Diags.Report(LToken.getLocation(), diag::err_mmap_unknown_token);
+ Diags.Report(Tok.getLocation(), diag::err_mmap_unknown_token);
HadError = true;
goto retry;
}
@@ -1682,7 +1704,8 @@ void ModuleMapParser::parseExternModuleD
File, /*IsSystem=*/false,
Map.HeaderInfo.getHeaderSearchOpts().ModuleMapFileHomeIsCwd
? Directory
- : File->getDir(), ExternLoc);
+ : File->getDir(),
+ FileID(), nullptr, ExternLoc);
}
/// Whether to add the requirement \p Feature to the module \p M.
@@ -2522,28 +2545,45 @@ bool ModuleMapParser::parseModuleMapFile
}
bool ModuleMap::parseModuleMapFile(const FileEntry *File, bool IsSystem,
- const DirectoryEntry *Dir,
+ const DirectoryEntry *Dir, FileID ID,
+ unsigned *Offset,
SourceLocation ExternModuleLoc) {
+ assert(Target && "Missing target information");
llvm::DenseMap<const FileEntry *, bool>::iterator Known
= ParsedModuleMap.find(File);
if (Known != ParsedModuleMap.end())
return Known->second;
+ // If the module map file wasn't already entered, do so now.
+ if (ID.isInvalid()) {
+ auto FileCharacter = IsSystem ? SrcMgr::C_System : SrcMgr::C_User;
+ ID = SourceMgr.createFileID(File, ExternModuleLoc, FileCharacter);
+ }
+
assert(Target && "Missing target information");
- auto FileCharacter = IsSystem ? SrcMgr::C_System : SrcMgr::C_User;
- FileID ID = SourceMgr.createFileID(File, ExternModuleLoc, FileCharacter);
const llvm::MemoryBuffer *Buffer = SourceMgr.getBuffer(ID);
if (!Buffer)
return ParsedModuleMap[File] = true;
+ assert((!Offset || *Offset <= Buffer->getBufferSize()) &&
+ "invalid buffer offset");
// Parse this module map file.
- Lexer L(ID, SourceMgr.getBuffer(ID), SourceMgr, MMapLangOpts);
+ Lexer L(SourceMgr.getLocForStartOfFile(ID), MMapLangOpts,
+ Buffer->getBufferStart(),
+ Buffer->getBufferStart() + (Offset ? *Offset : 0),
+ Buffer->getBufferEnd());
SourceLocation Start = L.getSourceLocation();
ModuleMapParser Parser(L, SourceMgr, Target, Diags, *this, File, Dir,
BuiltinIncludeDir, IsSystem);
bool Result = Parser.parseModuleMapFile();
ParsedModuleMap[File] = Result;
+ if (Offset) {
+ auto Loc = SourceMgr.getDecomposedLoc(Parser.getLocation());
+ assert(Loc.first == ID && "stopped in a different file?");
+ *Offset = Loc.second;
+ }
+
// Notify callbacks that we parsed it.
for (const auto &Cb : Callbacks)
Cb->moduleMapFileRead(Start, *File, IsSystem);
Modified: cfe/trunk/lib/Sema/SemaDecl.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Sema/SemaDecl.cpp?rev=302309&r1=302308&r2=302309&view=diff
==============================================================================
--- cfe/trunk/lib/Sema/SemaDecl.cpp (original)
+++ cfe/trunk/lib/Sema/SemaDecl.cpp Fri May 5 17:18:51 2017
@@ -15902,7 +15902,7 @@ void Sema::ActOnModuleBegin(SourceLocati
VisibleModules.setVisible(Mod, DirectiveLoc);
}
-void Sema::ActOnModuleEnd(SourceLocation EofLoc, Module *Mod) {
+void Sema::ActOnModuleEnd(SourceLocation EomLoc, Module *Mod) {
if (getLangOpts().ModulesLocalVisibility) {
VisibleModules = std::move(ModuleScopes.back().OuterVisibleModules);
// Leaving a module hides namespace names, so our visible namespace cache
@@ -15914,12 +15914,19 @@ void Sema::ActOnModuleEnd(SourceLocation
"left the wrong module scope");
ModuleScopes.pop_back();
- // We got to the end of processing a #include of a local module. Create an
+ // We got to the end of processing a local module. Create an
// ImportDecl as we would for an imported module.
- FileID File = getSourceManager().getFileID(EofLoc);
- assert(File != getSourceManager().getMainFileID() &&
- "end of submodule in main source file");
- SourceLocation DirectiveLoc = getSourceManager().getIncludeLoc(File);
+ FileID File = getSourceManager().getFileID(EomLoc);
+ SourceLocation DirectiveLoc;
+ if (EomLoc == getSourceManager().getLocForEndOfFile(File)) {
+ // We reached the end of a #included module header. Use the #include loc.
+ assert(File != getSourceManager().getMainFileID() &&
+ "end of submodule in main source file");
+ DirectiveLoc = getSourceManager().getIncludeLoc(File);
+ } else {
+ // We reached an EOM pragma. Use the pragma location.
+ DirectiveLoc = EomLoc;
+ }
BuildModuleInclude(DirectiveLoc, Mod);
}
Modified: cfe/trunk/test/Modules/preprocess-module.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/preprocess-module.cpp?rev=302309&r1=302308&r2=302309&view=diff
==============================================================================
--- cfe/trunk/test/Modules/preprocess-module.cpp (original)
+++ cfe/trunk/test/Modules/preprocess-module.cpp Fri May 5 17:18:51 2017
@@ -1,10 +1,35 @@
// RUN: rm -rf %t
+// RUN: mkdir %t
// RUN: not %clang_cc1 -fmodules -fmodule-name=file -I%S/Inputs/preprocess -x c++-module-map %S/Inputs/preprocess/module.modulemap -E 2>&1 | FileCheck %s --check-prefix=MISSING-FWD
// MISSING-FWD: module 'fwd' is needed
-// RUN: %clang_cc1 -fmodules -fmodule-name=file -fmodules-cache-path=%t -I%S/Inputs/preprocess -x c++-module-map %S/Inputs/preprocess/module.modulemap -E | FileCheck %s --check-prefix=CHECK --check-prefix=NO-REWRITE
-// RUN: %clang_cc1 -fmodules -fmodule-name=file -fmodules-cache-path=%t -I%S/Inputs/preprocess -x c++-module-map %S/Inputs/preprocess/module.modulemap -E -frewrite-includes | FileCheck %s --check-prefix=CHECK --check-prefix=REWRITE
+// RUN: %clang_cc1 -fmodules -fmodule-name=fwd -I%S/Inputs/preprocess -x c++-module-map %S/Inputs/preprocess/module.modulemap -emit-module -o %t/fwd.pcm
+
+// Check that we can preprocess modules, and get the expected output.
+// RUN: %clang_cc1 -fmodules -fmodule-name=file -fmodule-file=%t/fwd.pcm -I%S/Inputs/preprocess -x c++-module-map %S/Inputs/preprocess/module.modulemap -E -o %t/no-rewrite.ii
+// RUN: %clang_cc1 -fmodules -fmodule-name=file -fmodule-file=%t/fwd.pcm -I%S/Inputs/preprocess -x c++-module-map %S/Inputs/preprocess/module.modulemap -E -frewrite-includes -o %t/rewrite.ii
+//
+// RUN: FileCheck %s --input-file %t/no-rewrite.ii --check-prefix=CHECK --check-prefix=NO-REWRITE
+// RUN: FileCheck %s --input-file %t/rewrite.ii --check-prefix=CHECK --check-prefix=REWRITE
+
+// Check that we can build a module from the preprocessed output.
+// FIXME: For now, we need the headers to exist.
+// RUN: touch %t/file.h %t/file2.h
+// RUN: %clang_cc1 -fmodules -fmodule-name=file -fmodule-file=%t/fwd.pcm -x c++-module-map-cpp-output %t/no-rewrite.ii -emit-module -o %t/no-rewrite.pcm
+// RUN: %clang_cc1 -fmodules -fmodule-name=file -fmodule-file=%t/fwd.pcm -x c++-module-map-cpp-output %t/rewrite.ii -emit-module -o %t/rewrite.pcm
+
+// Check the module we built works.
+// RUN: %clang_cc1 -fmodules -fmodule-file=%t/no-rewrite.pcm %s -verify
+// RUN: %clang_cc1 -fmodules -fmodule-file=%t/rewrite.pcm %s -verify
+
+
+// == module map
+// CHECK: # 1 "{{.*}}module.modulemap"
+// CHECK: module file {
+// CHECK: header "file.h"
+// CHECK: header "file2.h"
+// CHECK: }
// == file.h
// CHECK: # 1 "<module-includes>"
@@ -63,3 +88,14 @@
// REWRITE: #pragma clang module end
// CHECK: # 3 "<module-includes>" 2
// NO-REWRITE: #pragma clang module end
+
+
+// expected-no-diagnostics
+
+// FIXME: This should be rejected: we have not imported the submodule defining it yet.
+__FILE *a;
+
+#pragma clang module import file
+
+FILE *b;
+int x = file2;
More information about the cfe-commits
mailing list