r304346 - [modules] When compiling a preprocessed module map, look for headers relative

Galina Kistanova via cfe-commits cfe-commits at lists.llvm.org
Thu Jun 1 11:10:23 PDT 2017


Hello Richard,

This commit broke tests on few of our builders:

Failing Tests (2):
    Clang :: Modules/preprocess-module.cpp
    Clang :: Modules/preprocess-nested.cpp

http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/builds/10199
and
http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-win/builds/2819

Please have a look at this?

Also I see that email notifications on these failures were sent.

Thanks

Galina


On Wed, May 31, 2017 at 1:56 PM, Richard Smith via cfe-commits <
cfe-commits at lists.llvm.org> wrote:

> Author: rsmith
> Date: Wed May 31 15:56:55 2017
> New Revision: 304346
>
> URL: http://llvm.org/viewvc/llvm-project?rev=304346&view=rev
> Log:
> [modules] When compiling a preprocessed module map, look for headers
> relative
> to the original module map.
>
> Also use the path and name of the original module map when emitting that
> information into the .pcm file. The upshot of this is that the produced
> .pcm
> file will track information for headers in their original locations (where
> the
> module was preprocessed), not relative to whatever directory the
> preprocessed
> module map was in when it was built.
>
> Modified:
>     cfe/trunk/include/clang/Basic/Module.h
>     cfe/trunk/include/clang/Lex/HeaderSearch.h
>     cfe/trunk/lib/Frontend/FrontendAction.cpp
>     cfe/trunk/lib/Lex/HeaderSearch.cpp
>     cfe/trunk/lib/Serialization/ASTWriter.cpp
>     cfe/trunk/test/Modules/preprocess-module.cpp
>     cfe/trunk/test/Modules/preprocess-nested.cpp
>
> Modified: cfe/trunk/include/clang/Basic/Module.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/
> clang/Basic/Module.h?rev=304346&r1=304345&r2=304346&view=diff
> ============================================================
> ==================
> --- cfe/trunk/include/clang/Basic/Module.h (original)
> +++ cfe/trunk/include/clang/Basic/Module.h Wed May 31 15:56:55 2017
> @@ -83,6 +83,10 @@ public:
>    /// are found.
>    const DirectoryEntry *Directory;
>
> +  /// \brief The presumed file name for the module map defining this
> module.
> +  /// Only non-empty when building from preprocessed source.
> +  std::string PresumedModuleMapFile;
> +
>    /// \brief The umbrella header or directory.
>    llvm::PointerUnion<const DirectoryEntry *, const FileEntry *> Umbrella;
>
>
> Modified: cfe/trunk/include/clang/Lex/HeaderSearch.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/
> clang/Lex/HeaderSearch.h?rev=304346&r1=304345&r2=304346&view=diff
> ============================================================
> ==================
> --- cfe/trunk/include/clang/Lex/HeaderSearch.h (original)
> +++ cfe/trunk/include/clang/Lex/HeaderSearch.h Wed May 31 15:56:55 2017
> @@ -543,10 +543,13 @@ public:
>    /// \param Offset [inout] An offset within ID to start parsing. On exit,
>    ///        filled by the end of the parsed contents (either EOF or the
>    ///        location of an end-of-module-map pragma).
> -  ///
> +  /// \param OriginalModuleMapFile The original path to the module map
> file,
> +  ///        used to resolve paths within the module (this is required
> when
> +  ///        building the module from preprocessed source).
>    /// \returns true if an error occurred, false otherwise.
>    bool loadModuleMapFile(const FileEntry *File, bool IsSystem,
> -                         FileID ID = FileID(), unsigned *Offset =
> nullptr);
> +                         FileID ID = FileID(), unsigned *Offset = nullptr,
> +                         StringRef OriginalModuleMapFile = StringRef());
>
>    /// \brief Collect the set of all known, top-level modules.
>    ///
>
> Modified: cfe/trunk/lib/Frontend/FrontendAction.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/
> Frontend/FrontendAction.cpp?rev=304346&r1=304345&r2=304346&view=diff
> ============================================================
> ==================
> --- cfe/trunk/lib/Frontend/FrontendAction.cpp (original)
> +++ cfe/trunk/lib/Frontend/FrontendAction.cpp Wed May 31 15:56:55 2017
> @@ -373,10 +373,11 @@ collectModuleHeaderIncludes(const LangOp
>    return std::error_code();
>  }
>
> -static bool
> -loadModuleMapForModuleBuild(CompilerInstance &CI, StringRef Filename,
> -                            bool IsSystem, bool IsPreprocessed,
> -                            unsigned &Offset) {
> +static bool loadModuleMapForModuleBuild(CompilerInstance &CI,
> +                                        StringRef Filename, bool IsSystem,
> +                                        bool IsPreprocessed,
> +                                        std::string
> &PresumedModuleMapFile,
> +                                        unsigned &Offset) {
>    auto &SrcMgr = CI.getSourceManager();
>    HeaderSearch &HS = CI.getPreprocessor().getHeaderSearchInfo();
>
> @@ -388,16 +389,15 @@ loadModuleMapForModuleBuild(CompilerInst
>    // line directives are not part of the module map syntax in general.
>    Offset = 0;
>    if (IsPreprocessed) {
> -    std::string PresumedModuleMapFile;
>      SourceLocation EndOfLineMarker =
>          ReadOriginalFileName(CI, PresumedModuleMapFile,
> /*AddLineNote*/true);
>      if (EndOfLineMarker.isValid())
>        Offset = CI.getSourceManager().getDecomposedLoc(
> EndOfLineMarker).second;
> -    // FIXME: Use PresumedModuleMapFile as the MODULE_MAP_FILE in the PCM.
>    }
>
>    // Load the module map file.
> -  if (HS.loadModuleMapFile(ModuleMap, IsSystem, ModuleMapID, &Offset))
> +  if (HS.loadModuleMapFile(ModuleMap, IsSystem, ModuleMapID, &Offset,
> +                           PresumedModuleMapFile))
>      return true;
>
>    if (SrcMgr.getBuffer(ModuleMapID)->getBufferSize() == Offset)
> @@ -664,15 +664,19 @@ bool FrontendAction::BeginSourceFile(Com
>    if (Input.getKind().getFormat() == InputKind::ModuleMap) {
>      CI.getLangOpts().setCompilingModule(LangOptions::CMK_ModuleMap);
>
> +    std::string PresumedModuleMapFile;
>      unsigned OffsetToContents;
>      if (loadModuleMapForModuleBuild(CI, Input.getFile(),
> Input.isSystem(),
> -                                    Input.isPreprocessed(),
> OffsetToContents))
> +                                    Input.isPreprocessed(),
> +                                    PresumedModuleMapFile,
> OffsetToContents))
>        goto failure;
>
>      auto *CurrentModule = prepareToBuildModule(CI, Input.getFile());
>      if (!CurrentModule)
>        goto failure;
>
> +    CurrentModule->PresumedModuleMapFile = PresumedModuleMapFile;
> +
>      if (OffsetToContents)
>        // If the module contents are in the same file, skip to them.
>        CI.getPreprocessor().setSkipMainFilePreamble(OffsetToContents,
> true);
>
> Modified: cfe/trunk/lib/Lex/HeaderSearch.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/
> HeaderSearch.cpp?rev=304346&r1=304345&r2=304346&view=diff
> ============================================================
> ==================
> --- cfe/trunk/lib/Lex/HeaderSearch.cpp (original)
> +++ cfe/trunk/lib/Lex/HeaderSearch.cpp Wed May 31 15:56:55 2017
> @@ -1326,14 +1326,27 @@ static const FileEntry *getPrivateModule
>  }
>
>  bool HeaderSearch::loadModuleMapFile(const FileEntry *File, bool
> IsSystem,
> -                                     FileID ID, unsigned *Offset) {
> +                                     FileID ID, unsigned *Offset,
> +                                     StringRef OriginalModuleMapFile) {
>    // Find the directory for the module. For frameworks, that may require
> going
>    // up from the 'Modules' directory.
>    const DirectoryEntry *Dir = nullptr;
>    if (getHeaderSearchOpts().ModuleMapFileHomeIsCwd)
>      Dir = FileMgr.getDirectory(".");
>    else {
> -    Dir = File->getDir();
> +    if (!OriginalModuleMapFile.empty()) {
> +      // We're building a preprocessed module map. Find or invent the
> directory
> +      // that it originally occupied.
> +      Dir = FileMgr.getDirectory(
> +          llvm::sys::path::parent_path(OriginalModuleMapFile));
> +      if (!Dir) {
> +        auto *FakeFile = FileMgr.getVirtualFile(OriginalModuleMapFile,
> 0, 0);
> +        Dir = FakeFile->getDir();
> +      }
> +    } else {
> +      Dir = File->getDir();
> +    }
> +
>      StringRef DirName(Dir->getName());
>      if (llvm::sys::path::filename(DirName) == "Modules") {
>        DirName = llvm::sys::path::parent_path(DirName);
>
> Modified: cfe/trunk/lib/Serialization/ASTWriter.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/
> Serialization/ASTWriter.cpp?rev=304346&r1=304345&r2=304346&view=diff
> ============================================================
> ==================
> --- cfe/trunk/lib/Serialization/ASTWriter.cpp (original)
> +++ cfe/trunk/lib/Serialization/ASTWriter.cpp Wed May 31 15:56:55 2017
> @@ -1422,8 +1422,8 @@ void ASTWriter::WriteControlBlock(Prepro
>      Stream.EmitRecordWithBlob(MetadataAbbrevCode, Record,
>                                getClangFullRepositoryVersion());
>    }
> -  if (WritingModule) {
>
> +  if (WritingModule) {
>      // Module name
>      auto Abbrev = std::make_shared<BitCodeAbbrev>();
>      Abbrev->Add(BitCodeAbbrevOp(MODULE_NAME));
> @@ -1466,9 +1466,10 @@ void ASTWriter::WriteControlBlock(Prepro
>      Record.clear();
>
>      auto &Map = PP.getHeaderSearchInfo().getModuleMap();
> -
> -    // Primary module map file.
> -    AddPath(Map.getModuleMapFileForUniquing(WritingModule)->getName(),
> Record);
> +    AddPath(WritingModule->PresumedModuleMapFile.empty()
> +                ? Map.getModuleMapFileForUniquing(
> WritingModule)->getName()
> +                : StringRef(WritingModule->PresumedModuleMapFile),
> +            Record);
>
>      // Additional module map files.
>      if (auto *AdditionalModMaps =
>
> Modified: cfe/trunk/test/Modules/preprocess-module.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/
> Modules/preprocess-module.cpp?rev=304346&r1=304345&r2=304346&view=diff
> ============================================================
> ==================
> --- cfe/trunk/test/Modules/preprocess-module.cpp (original)
> +++ cfe/trunk/test/Modules/preprocess-module.cpp Wed May 31 15:56:55 2017
> @@ -14,8 +14,6 @@
>  // RUN: FileCheck %s --input-file %t/rewrite.ii    --check-prefix=CHECK
> --check-prefix=REWRITE
>
>  // Check that we can build a module from the preprocessed output.
> -// FIXME: For now, we need the headers to exist.
> -// RUN: touch %t/file.h %t/file2.h
>  // RUN: %clang_cc1 -fmodules -fmodule-name=file -fmodule-file=%t/fwd.pcm
> -x c++-module-map-cpp-output %t/no-rewrite.ii -emit-module -o
> %t/no-rewrite.pcm
>  // RUN: %clang_cc1 -fmodules -fmodule-name=file -fmodule-file=%t/fwd.pcm
> -x c++-module-map-cpp-output %t/rewrite.ii -emit-module -o %t/rewrite.pcm
>
> @@ -27,6 +25,8 @@
>  // Check the module we built works.
>  // RUN: %clang_cc1 -fmodules -fmodule-file=%t/no-rewrite.pcm %s -I%t
> -verify -fno-modules-error-recovery
>  // RUN: %clang_cc1 -fmodules -fmodule-file=%t/rewrite.pcm %s -I%t -verify
> -fno-modules-error-recovery -DREWRITE
> +// RUN: %clang_cc1 -fmodules -fmodule-file=%t/no-rewrite.pcm %s -I%t
> -verify -fno-modules-error-recovery -DINCLUDE -I%S/Inputs/preprocess
> +// RUN: %clang_cc1 -fmodules -fmodule-file=%t/rewrite.pcm %s -I%t -verify
> -fno-modules-error-recovery -DREWRITE -DINCLUDE -I%S/Inputs/preprocess
>
>
>  // == module map
> @@ -102,7 +102,11 @@ __FILE *a; // expected-error {{declarati
>  // expected-note at no-rewrite.ii:1 {{here}}
>  #endif
>
> +#ifdef INCLUDE
> +#include "file.h"
> +#else
>  #pragma clang module import file
> +#endif
>
>  FILE *b;
> -int x = file2;
> +int x = file2; // ok, found in file2.h, even under -DINCLUDE
>
> Modified: cfe/trunk/test/Modules/preprocess-nested.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/
> Modules/preprocess-nested.cpp?rev=304346&r1=304345&r2=304346&view=diff
> ============================================================
> ==================
> --- cfe/trunk/test/Modules/preprocess-nested.cpp (original)
> +++ cfe/trunk/test/Modules/preprocess-nested.cpp Wed May 31 15:56:55 2017
> @@ -8,8 +8,6 @@
>  // RUN: FileCheck %s --input-file %t/rewrite.ii    --check-prefix=CHECK
> --check-prefix=REWRITE
>
>  // Check that we can build a module from the preprocessed output.
> -// FIXME: For now, the files need to exist.
> -// RUN: touch %t/a.h %t/b.h %t/c.h
>  // RUN: %clang_cc1 -fmodules -fmodules-local-submodule-visibility
> -fmodule-name=nested -x c++-module-map-cpp-output %t/no-rewrite.ii
> -emit-module -o %t/no-rewrite.pcm
>  // RUN: %clang_cc1 -fmodules -fmodules-local-submodule-visibility
> -fmodule-name=nested -x c++-module-map-cpp-output %t/rewrite.ii
> -emit-module -o %t/rewrite.pcm
>
>
>
> _______________________________________________
> cfe-commits mailing list
> cfe-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20170601/0d6d4e91/attachment-0001.html>


More information about the cfe-commits mailing list