r298278 - Reapply "Modules: Cache PCMs in memory and avoid a use-after-free"
Duncan P. N. Exon Smith via cfe-commits
cfe-commits at lists.llvm.org
Tue Mar 21 11:26:30 PDT 2017
Looking.
> On 2017-Mar-21, at 11:25, Russell Gallop <russell.gallop at gmail.com> wrote:
>
> The unit test "MemoryBufferCacheTest.addBuffer" seems to be intermittently failing on Windows. I can see this on the buildbot:
> http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/builds/7086/steps/test/logs/stdio <http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/builds/7086/steps/test/logs/stdio>
>
> Please could you take a look?
>
> Thanks
> Russ
>
> Note: Google Test filter = MemoryBufferCacheTest.addBuffer
> [==========] Running 1 test from 1 test case.
> [----------] Global test environment set-up.
> [----------] 1 test from MemoryBufferCacheTest
> [ RUN ] MemoryBufferCacheTest.addBuffer
> <snip>\llvm\tools\clang\unittests\Basic\MemoryBufferCacheTest.cpp(53): error: Expected: (RawB2) != (B2.get()), actual: 000000075B1FDB50 vs 000000075B1FDB50
> [ FAILED ] MemoryBufferCacheTest.addBuffer (1 ms)
> [----------] 1 test from MemoryBufferCacheTest (1 ms total)
>
> [----------] Global test environment tear-down
> [==========] 1 test from 1 test case ran. (1 ms total)
> [ PASSED ] 0 tests.
> [ FAILED ] 1 test, listed below:
> [ FAILED ] MemoryBufferCacheTest.addBuffer
>
> 1 FAILED TEST
>
> On 20 March 2017 at 17:58, Duncan P. N. Exon Smith via cfe-commits <cfe-commits at lists.llvm.org <mailto:cfe-commits at lists.llvm.org>> wrote:
> Author: dexonsmith
> Date: Mon Mar 20 12:58:26 2017
> New Revision: 298278
>
> URL: http://llvm.org/viewvc/llvm-project?rev=298278&view=rev <http://llvm.org/viewvc/llvm-project?rev=298278&view=rev>
> Log:
> Reapply "Modules: Cache PCMs in memory and avoid a use-after-free"
>
> This reverts commit r298185, effectively reapplying r298165, after fixing the
> new unit tests (PR32338). The memory buffer generator doesn't null-terminate
> the MemoryBuffer it creates; this version of the commit informs getMemBuffer
> about that to avoid the assert.
>
> Original commit message follows:
>
> ----
>
> Clang's internal build system for implicit modules uses lock files to
> ensure that after a process writes a PCM it will read the same one back
> in (without contention from other -cc1 commands). Since PCMs are read
> from disk repeatedly while invalidating, building, and importing, the
> lock is not released quickly. Furthermore, the LockFileManager is not
> robust in every environment. Other -cc1 commands can stall until
> timeout (after about eight minutes).
>
> This commit changes the lock file from being necessary for correctness
> to a (possibly dubious) performance hack. The remaining benefit is to
> reduce duplicate work in competing -cc1 commands which depend on the
> same module. Follow-up commits will change the internal build system to
> continue after a timeout, and reduce the timeout. Perhaps we should
> reconsider blocking at all.
>
> This also fixes a use-after-free, when one part of a compilation
> validates a PCM and starts using it, and another tries to swap out the
> PCM for something new.
>
> The PCMCache is a new type called MemoryBufferCache, which saves memory
> buffers based on their filename. Its ownership is shared by the
> CompilerInstance and ModuleManager.
>
> - The ModuleManager stores PCMs there that it loads from disk, never
> touching the disk if the cache is hot.
>
> - When modules fail to validate, they're removed from the cache.
>
> - When a CompilerInstance is spawned to build a new module, each
> already-loaded PCM is assumed to be valid, and is frozen to avoid
> the use-after-free.
>
> - Any newly-built module is written directly to the cache to avoid the
> round-trip to the filesystem, making lock files unnecessary for
> correctness.
>
> Original patch by Manman Ren; most testcases by Adrian Prantl!
>
> Added:
> cfe/trunk/include/clang/Basic/MemoryBufferCache.h
> cfe/trunk/lib/Basic/MemoryBufferCache.cpp
> cfe/trunk/test/Modules/Inputs/system-out-of-date/X.h
> cfe/trunk/test/Modules/Inputs/system-out-of-date/Y.h
> cfe/trunk/test/Modules/Inputs/system-out-of-date/Z.h
> cfe/trunk/test/Modules/Inputs/system-out-of-date/module.map
> cfe/trunk/test/Modules/Inputs/warning-mismatch/Mismatch.h
> cfe/trunk/test/Modules/Inputs/warning-mismatch/System.h
> cfe/trunk/test/Modules/Inputs/warning-mismatch/module.modulemap
> cfe/trunk/test/Modules/outofdate-rebuild.m
> cfe/trunk/test/Modules/system-out-of-date-test.m
> cfe/trunk/test/Modules/warning-mismatch.m
> cfe/trunk/unittests/Basic/MemoryBufferCacheTest.cpp
> Modified:
> cfe/trunk/include/clang/Basic/DiagnosticSerializationKinds.td
> cfe/trunk/include/clang/Frontend/ASTUnit.h
> cfe/trunk/include/clang/Frontend/CompilerInstance.h
> cfe/trunk/include/clang/Lex/Preprocessor.h
> cfe/trunk/include/clang/Serialization/ASTReader.h
> cfe/trunk/include/clang/Serialization/ASTWriter.h
> cfe/trunk/include/clang/Serialization/Module.h
> cfe/trunk/include/clang/Serialization/ModuleManager.h
> cfe/trunk/lib/Basic/CMakeLists.txt
> cfe/trunk/lib/Frontend/ASTUnit.cpp
> cfe/trunk/lib/Frontend/CompilerInstance.cpp
> cfe/trunk/lib/Lex/Preprocessor.cpp
> cfe/trunk/lib/Serialization/ASTReader.cpp
> cfe/trunk/lib/Serialization/ASTWriter.cpp
> cfe/trunk/lib/Serialization/GeneratePCH.cpp
> cfe/trunk/lib/Serialization/ModuleManager.cpp
> cfe/trunk/unittests/Basic/CMakeLists.txt
> cfe/trunk/unittests/Basic/SourceManagerTest.cpp
> cfe/trunk/unittests/Lex/LexerTest.cpp
> cfe/trunk/unittests/Lex/PPCallbacksTest.cpp
> cfe/trunk/unittests/Lex/PPConditionalDirectiveRecordTest.cpp
>
> Modified: cfe/trunk/include/clang/Basic/DiagnosticSerializationKinds.td
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/DiagnosticSerializationKinds.td?rev=298278&r1=298277&r2=298278&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/DiagnosticSerializationKinds.td?rev=298278&r1=298277&r2=298278&view=diff>
> ==============================================================================
> --- cfe/trunk/include/clang/Basic/DiagnosticSerializationKinds.td (original)
> +++ cfe/trunk/include/clang/Basic/DiagnosticSerializationKinds.td Mon Mar 20 12:58:26 2017
> @@ -176,6 +176,11 @@ def warn_duplicate_module_file_extension
> "duplicate module file extension block name '%0'">,
> InGroup<ModuleFileExtension>;
>
> +def warn_module_system_bit_conflict : Warning<
> + "module file '%0' was validated as a system module and is now being imported "
> + "as a non-system module; any difference in diagnostic options will be ignored">,
> + InGroup<ModuleConflict>;
> +
> } // let CategoryName
> } // let Component
>
>
> Added: cfe/trunk/include/clang/Basic/MemoryBufferCache.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/MemoryBufferCache.h?rev=298278&view=auto <http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/MemoryBufferCache.h?rev=298278&view=auto>
> ==============================================================================
> --- cfe/trunk/include/clang/Basic/MemoryBufferCache.h (added)
> +++ cfe/trunk/include/clang/Basic/MemoryBufferCache.h Mon Mar 20 12:58:26 2017
> @@ -0,0 +1,80 @@
> +//===- MemoryBufferCache.h - Cache for loaded memory buffers ----*- C++ -*-===//
> +//
> +// The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +
> +#ifndef LLVM_CLANG_BASIC_MEMORYBUFFERCACHE_H
> +#define LLVM_CLANG_BASIC_MEMORYBUFFERCACHE_H
> +
> +#include "llvm/ADT/IntrusiveRefCntPtr.h"
> +#include "llvm/ADT/StringMap.h"
> +#include <memory>
> +
> +namespace llvm {
> +class MemoryBuffer;
> +} // end namespace llvm
> +
> +namespace clang {
> +
> +/// Manage memory buffers across multiple users.
> +///
> +/// Ensures that multiple users have a consistent view of each buffer. This is
> +/// used by \a CompilerInstance when building PCMs to ensure that each \a
> +/// ModuleManager sees the same files.
> +///
> +/// \a finalizeCurrentBuffers() should be called before creating a new user.
> +/// This locks in the current buffers, ensuring that no buffer that has already
> +/// been accessed can be purged, preventing use-after-frees.
> +class MemoryBufferCache : public llvm::RefCountedBase<MemoryBufferCache> {
> + struct BufferEntry {
> + std::unique_ptr<llvm::MemoryBuffer> Buffer;
> +
> + /// Track the timeline of when this was added to the cache.
> + unsigned Index;
> + };
> +
> + /// Cache of buffers.
> + llvm::StringMap<BufferEntry> Buffers;
> +
> + /// Monotonically increasing index.
> + unsigned NextIndex = 0;
> +
> + /// Bumped to prevent "older" buffers from being removed.
> + unsigned FirstRemovableIndex = 0;
> +
> +public:
> + /// Store the Buffer under the Filename.
> + ///
> + /// \pre There is not already buffer is not already in the cache.
> + /// \return a reference to the buffer as a convenience.
> + llvm::MemoryBuffer &addBuffer(llvm::StringRef Filename,
> + std::unique_ptr<llvm::MemoryBuffer> Buffer);
> +
> + /// Try to remove a buffer from the cache.
> + ///
> + /// \return false on success, iff \c !isBufferFinal().
> + bool tryToRemoveBuffer(llvm::StringRef Filename);
> +
> + /// Get a pointer to the buffer if it exists; else nullptr.
> + llvm::MemoryBuffer *lookupBuffer(llvm::StringRef Filename);
> +
> + /// Check whether the buffer is final.
> + ///
> + /// \return true iff \a finalizeCurrentBuffers() has been called since the
> + /// buffer was added. This prevents buffers from being removed.
> + bool isBufferFinal(llvm::StringRef Filename);
> +
> + /// Finalize the current buffers in the cache.
> + ///
> + /// Should be called when creating a new user to ensure previous uses aren't
> + /// invalidated.
> + void finalizeCurrentBuffers();
> +};
> +
> +} // end namespace clang
> +
> +#endif // LLVM_CLANG_BASIC_MEMORYBUFFERCACHE_H
>
> Modified: cfe/trunk/include/clang/Frontend/ASTUnit.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Frontend/ASTUnit.h?rev=298278&r1=298277&r2=298278&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Frontend/ASTUnit.h?rev=298278&r1=298277&r2=298278&view=diff>
> ==============================================================================
> --- cfe/trunk/include/clang/Frontend/ASTUnit.h (original)
> +++ cfe/trunk/include/clang/Frontend/ASTUnit.h Mon Mar 20 12:58:26 2017
> @@ -51,6 +51,7 @@ class DiagnosticsEngine;
> class FileEntry;
> class FileManager;
> class HeaderSearch;
> +class MemoryBufferCache;
> class Preprocessor;
> class PCHContainerOperations;
> class PCHContainerReader;
> @@ -84,6 +85,7 @@ private:
> IntrusiveRefCntPtr<DiagnosticsEngine> Diagnostics;
> IntrusiveRefCntPtr<FileManager> FileMgr;
> IntrusiveRefCntPtr<SourceManager> SourceMgr;
> + IntrusiveRefCntPtr<MemoryBufferCache> PCMCache;
> std::unique_ptr<HeaderSearch> HeaderInfo;
> IntrusiveRefCntPtr<TargetInfo> Target;
> std::shared_ptr<Preprocessor> PP;
>
> Modified: cfe/trunk/include/clang/Frontend/CompilerInstance.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Frontend/CompilerInstance.h?rev=298278&r1=298277&r2=298278&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Frontend/CompilerInstance.h?rev=298278&r1=298277&r2=298278&view=diff>
> ==============================================================================
> --- cfe/trunk/include/clang/Frontend/CompilerInstance.h (original)
> +++ cfe/trunk/include/clang/Frontend/CompilerInstance.h Mon Mar 20 12:58:26 2017
> @@ -44,6 +44,7 @@ class ExternalASTSource;
> class FileEntry;
> class FileManager;
> class FrontendAction;
> +class MemoryBufferCache;
> class Module;
> class Preprocessor;
> class Sema;
> @@ -90,6 +91,9 @@ class CompilerInstance : public ModuleLo
> /// The source manager.
> IntrusiveRefCntPtr<SourceManager> SourceMgr;
>
> + /// The cache of PCM files.
> + IntrusiveRefCntPtr<MemoryBufferCache> PCMCache;
> +
> /// The preprocessor.
> std::shared_ptr<Preprocessor> PP;
>
> @@ -178,7 +182,7 @@ public:
> explicit CompilerInstance(
> std::shared_ptr<PCHContainerOperations> PCHContainerOps =
> std::make_shared<PCHContainerOperations>(),
> - bool BuildingModule = false);
> + MemoryBufferCache *SharedPCMCache = nullptr);
> ~CompilerInstance() override;
>
> /// @name High-Level Operations
> @@ -783,6 +787,8 @@ public:
> }
>
> void setExternalSemaSource(IntrusiveRefCntPtr<ExternalSemaSource> ESS);
> +
> + MemoryBufferCache &getPCMCache() const { return *PCMCache; }
> };
>
> } // end namespace clang
>
> Modified: cfe/trunk/include/clang/Lex/Preprocessor.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Lex/Preprocessor.h?rev=298278&r1=298277&r2=298278&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Lex/Preprocessor.h?rev=298278&r1=298277&r2=298278&view=diff>
> ==============================================================================
> --- cfe/trunk/include/clang/Lex/Preprocessor.h (original)
> +++ cfe/trunk/include/clang/Lex/Preprocessor.h Mon Mar 20 12:58:26 2017
> @@ -47,6 +47,7 @@ class ExternalPreprocessorSource;
> class FileManager;
> class FileEntry;
> class HeaderSearch;
> +class MemoryBufferCache;
> class PragmaNamespace;
> class PragmaHandler;
> class CommentHandler;
> @@ -102,6 +103,7 @@ class Preprocessor {
> const TargetInfo *AuxTarget;
> FileManager &FileMgr;
> SourceManager &SourceMgr;
> + MemoryBufferCache &PCMCache;
> std::unique_ptr<ScratchBuffer> ScratchBuf;
> HeaderSearch &HeaderInfo;
> ModuleLoader &TheModuleLoader;
> @@ -652,6 +654,7 @@ class Preprocessor {
> public:
> Preprocessor(std::shared_ptr<PreprocessorOptions> PPOpts,
> DiagnosticsEngine &diags, LangOptions &opts, SourceManager &SM,
> + MemoryBufferCache &PCMCache,
> HeaderSearch &Headers, ModuleLoader &TheModuleLoader,
> IdentifierInfoLookup *IILookup = nullptr,
> bool OwnsHeaderSearch = false,
> @@ -691,6 +694,7 @@ public:
> const TargetInfo *getAuxTargetInfo() const { return AuxTarget; }
> FileManager &getFileManager() const { return FileMgr; }
> SourceManager &getSourceManager() const { return SourceMgr; }
> + MemoryBufferCache &getPCMCache() const { return PCMCache; }
> HeaderSearch &getHeaderSearchInfo() const { return HeaderInfo; }
>
> IdentifierTable &getIdentifierTable() { return Identifiers; }
>
> Modified: cfe/trunk/include/clang/Serialization/ASTReader.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Serialization/ASTReader.h?rev=298278&r1=298277&r2=298278&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Serialization/ASTReader.h?rev=298278&r1=298277&r2=298278&view=diff>
> ==============================================================================
> --- cfe/trunk/include/clang/Serialization/ASTReader.h (original)
> +++ cfe/trunk/include/clang/Serialization/ASTReader.h Mon Mar 20 12:58:26 2017
> @@ -408,6 +408,9 @@ private:
> /// \brief The module manager which manages modules and their dependencies
> ModuleManager ModuleMgr;
>
> + /// The cache that manages memory buffers for PCM files.
> + MemoryBufferCache &PCMCache;
> +
> /// \brief A dummy identifier resolver used to merge TU-scope declarations in
> /// C, for the cases where we don't have a Sema object to provide a real
> /// identifier resolver.
>
> Modified: cfe/trunk/include/clang/Serialization/ASTWriter.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Serialization/ASTWriter.h?rev=298278&r1=298277&r2=298278&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Serialization/ASTWriter.h?rev=298278&r1=298277&r2=298278&view=diff>
> ==============================================================================
> --- cfe/trunk/include/clang/Serialization/ASTWriter.h (original)
> +++ cfe/trunk/include/clang/Serialization/ASTWriter.h Mon Mar 20 12:58:26 2017
> @@ -54,6 +54,7 @@ class MacroInfo;
> class OpaqueValueExpr;
> class OpenCLOptions;
> class ASTReader;
> +class MemoryBufferCache;
> class Module;
> class ModuleFileExtension;
> class ModuleFileExtensionWriter;
> @@ -109,6 +110,9 @@ private:
> /// The buffer associated with the bitstream.
> const SmallVectorImpl<char> &Buffer;
>
> + /// \brief The PCM manager which manages memory buffers for pcm files.
> + MemoryBufferCache &PCMCache;
> +
> /// \brief The ASTContext we're writing.
> ASTContext *Context = nullptr;
>
> @@ -512,6 +516,7 @@ public:
> /// \brief Create a new precompiled header writer that outputs to
> /// the given bitstream.
> ASTWriter(llvm::BitstreamWriter &Stream, SmallVectorImpl<char> &Buffer,
> + MemoryBufferCache &PCMCache,
> ArrayRef<std::shared_ptr<ModuleFileExtension>> Extensions,
> bool IncludeTimestamps = true);
> ~ASTWriter() override;
>
> Modified: cfe/trunk/include/clang/Serialization/Module.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Serialization/Module.h?rev=298278&r1=298277&r2=298278&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Serialization/Module.h?rev=298278&r1=298277&r2=298278&view=diff>
> ==============================================================================
> --- cfe/trunk/include/clang/Serialization/Module.h (original)
> +++ cfe/trunk/include/clang/Serialization/Module.h Mon Mar 20 12:58:26 2017
> @@ -163,9 +163,9 @@ public:
> /// \brief The generation of which this module file is a part.
> unsigned Generation;
>
> - /// \brief The memory buffer that stores the data associated with
> - /// this AST file.
> - std::unique_ptr<llvm::MemoryBuffer> Buffer;
> + /// The memory buffer that stores the data associated with
> + /// this AST file, owned by the PCMCache in the ModuleManager.
> + llvm::MemoryBuffer *Buffer;
>
> /// \brief The size of this file, in bits.
> uint64_t SizeInBits = 0;
>
> Modified: cfe/trunk/include/clang/Serialization/ModuleManager.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Serialization/ModuleManager.h?rev=298278&r1=298277&r2=298278&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Serialization/ModuleManager.h?rev=298278&r1=298277&r2=298278&view=diff>
> ==============================================================================
> --- cfe/trunk/include/clang/Serialization/ModuleManager.h (original)
> +++ cfe/trunk/include/clang/Serialization/ModuleManager.h Mon Mar 20 12:58:26 2017
> @@ -24,6 +24,7 @@
> namespace clang {
>
> class GlobalModuleIndex;
> +class MemoryBufferCache;
> class ModuleMap;
> class PCHContainerReader;
>
> @@ -51,6 +52,9 @@ class ModuleManager {
> /// FileEntry *.
> FileManager &FileMgr;
>
> + /// Cache of PCM files.
> + IntrusiveRefCntPtr<MemoryBufferCache> PCMCache;
> +
> /// \brief Knows how to unwrap module containers.
> const PCHContainerReader &PCHContainerRdr;
>
> @@ -123,7 +127,7 @@ public:
> ModuleReverseIterator;
> typedef std::pair<uint32_t, StringRef> ModuleOffset;
>
> - explicit ModuleManager(FileManager &FileMgr,
> + explicit ModuleManager(FileManager &FileMgr, MemoryBufferCache &PCMCache,
> const PCHContainerReader &PCHContainerRdr);
> ~ModuleManager();
>
> @@ -290,6 +294,8 @@ public:
>
> /// \brief View the graphviz representation of the module graph.
> void viewGraph();
> +
> + MemoryBufferCache &getPCMCache() const { return *PCMCache; }
> };
>
> } } // end namespace clang::serialization
>
> Modified: cfe/trunk/lib/Basic/CMakeLists.txt
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Basic/CMakeLists.txt?rev=298278&r1=298277&r2=298278&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Basic/CMakeLists.txt?rev=298278&r1=298277&r2=298278&view=diff>
> ==============================================================================
> --- cfe/trunk/lib/Basic/CMakeLists.txt (original)
> +++ cfe/trunk/lib/Basic/CMakeLists.txt Mon Mar 20 12:58:26 2017
> @@ -74,6 +74,7 @@ add_clang_library(clangBasic
> FileSystemStatCache.cpp
> IdentifierTable.cpp
> LangOptions.cpp
> + MemoryBufferCache.cpp
> Module.cpp
> ObjCRuntime.cpp
> OpenMPKinds.cpp
>
> Added: cfe/trunk/lib/Basic/MemoryBufferCache.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Basic/MemoryBufferCache.cpp?rev=298278&view=auto <http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Basic/MemoryBufferCache.cpp?rev=298278&view=auto>
> ==============================================================================
> --- cfe/trunk/lib/Basic/MemoryBufferCache.cpp (added)
> +++ cfe/trunk/lib/Basic/MemoryBufferCache.cpp Mon Mar 20 12:58:26 2017
> @@ -0,0 +1,48 @@
> +//===- MemoryBufferCache.cpp - Cache for loaded memory buffers ------------===//
> +//
> +// The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +
> +#include "clang/Basic/MemoryBufferCache.h"
> +#include "llvm/Support/MemoryBuffer.h"
> +
> +using namespace clang;
> +
> +llvm::MemoryBuffer &
> +MemoryBufferCache::addBuffer(llvm::StringRef Filename,
> + std::unique_ptr<llvm::MemoryBuffer> Buffer) {
> + auto Insertion =
> + Buffers.insert({Filename, BufferEntry{std::move(Buffer), NextIndex++}});
> + assert(Insertion.second && "Already has a buffer");
> + return *Insertion.first->second.Buffer;
> +}
> +
> +llvm::MemoryBuffer *MemoryBufferCache::lookupBuffer(llvm::StringRef Filename) {
> + auto I = Buffers.find(Filename);
> + if (I == Buffers.end())
> + return nullptr;
> + return I->second.Buffer.get();
> +}
> +
> +bool MemoryBufferCache::isBufferFinal(llvm::StringRef Filename) {
> + auto I = Buffers.find(Filename);
> + if (I == Buffers.end())
> + return false;
> + return I->second.Index < FirstRemovableIndex;
> +}
> +
> +bool MemoryBufferCache::tryToRemoveBuffer(llvm::StringRef Filename) {
> + auto I = Buffers.find(Filename);
> + assert(I != Buffers.end() && "No buffer to remove...");
> + if (I->second.Index < FirstRemovableIndex)
> + return true;
> +
> + Buffers.erase(I);
> + return false;
> +}
> +
> +void MemoryBufferCache::finalizeCurrentBuffers() { FirstRemovableIndex = NextIndex; }
>
> Modified: cfe/trunk/lib/Frontend/ASTUnit.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/ASTUnit.cpp?rev=298278&r1=298277&r2=298278&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/ASTUnit.cpp?rev=298278&r1=298277&r2=298278&view=diff>
> ==============================================================================
> --- cfe/trunk/lib/Frontend/ASTUnit.cpp (original)
> +++ cfe/trunk/lib/Frontend/ASTUnit.cpp Mon Mar 20 12:58:26 2017
> @@ -18,6 +18,7 @@
> #include "clang/AST/StmtVisitor.h"
> #include "clang/AST/TypeOrdering.h"
> #include "clang/Basic/Diagnostic.h"
> +#include "clang/Basic/MemoryBufferCache.h"
> #include "clang/Basic/TargetInfo.h"
> #include "clang/Basic/TargetOptions.h"
> #include "clang/Basic/VirtualFileSystem.h"
> @@ -185,7 +186,8 @@ struct ASTUnit::ASTWriterData {
> llvm::BitstreamWriter Stream;
> ASTWriter Writer;
>
> - ASTWriterData() : Stream(Buffer), Writer(Stream, Buffer, {}) {}
> + ASTWriterData(MemoryBufferCache &PCMCache)
> + : Stream(Buffer), Writer(Stream, Buffer, PCMCache, {}) {}
> };
>
> void ASTUnit::clearFileLevelDecls() {
> @@ -681,6 +683,7 @@ std::unique_ptr<ASTUnit> ASTUnit::LoadFr
> AST->SourceMgr = new SourceManager(AST->getDiagnostics(),
> AST->getFileManager(),
> UserFilesAreVolatile);
> + AST->PCMCache = new MemoryBufferCache;
> AST->HSOpts = std::make_shared<HeaderSearchOptions>();
> AST->HSOpts->ModuleFormat = PCHContainerRdr.getFormat();
> AST->HeaderInfo.reset(new HeaderSearch(AST->HSOpts,
> @@ -701,7 +704,7 @@ std::unique_ptr<ASTUnit> ASTUnit::LoadFr
>
> AST->PP = std::make_shared<Preprocessor>(
> std::move(PPOpts), AST->getDiagnostics(), AST->ASTFileLangOpts,
> - AST->getSourceManager(), HeaderInfo, *AST,
> + AST->getSourceManager(), *AST->PCMCache, HeaderInfo, *AST,
> /*IILookup=*/nullptr,
> /*OwnsHeaderSearch=*/false);
> Preprocessor &PP = *AST->PP;
> @@ -1727,6 +1730,7 @@ ASTUnit::create(std::shared_ptr<Compiler
> AST->UserFilesAreVolatile = UserFilesAreVolatile;
> AST->SourceMgr = new SourceManager(AST->getDiagnostics(), *AST->FileMgr,
> UserFilesAreVolatile);
> + AST->PCMCache = new MemoryBufferCache;
>
> return AST;
> }
> @@ -1997,6 +2001,7 @@ ASTUnit *ASTUnit::LoadFromCommandLine(
> if (!VFS)
> return nullptr;
> AST->FileMgr = new FileManager(AST->FileSystemOpts, VFS);
> + AST->PCMCache = new MemoryBufferCache;
> AST->OnlyLocalDecls = OnlyLocalDecls;
> AST->CaptureDiagnostics = CaptureDiagnostics;
> AST->TUKind = TUKind;
> @@ -2008,7 +2013,7 @@ ASTUnit *ASTUnit::LoadFromCommandLine(
> AST->StoredDiagnostics.swap(StoredDiagnostics);
> AST->Invocation = CI;
> if (ForSerialization)
> - AST->WriterData.reset(new ASTWriterData());
> + AST->WriterData.reset(new ASTWriterData(*AST->PCMCache));
> // Zero out now to ease cleanup during crash recovery.
> CI = nullptr;
> Diags = nullptr;
> @@ -2523,7 +2528,8 @@ bool ASTUnit::serialize(raw_ostream &OS)
>
> SmallString<128> Buffer;
> llvm::BitstreamWriter Stream(Buffer);
> - ASTWriter Writer(Stream, Buffer, {});
> + MemoryBufferCache PCMCache;
> + ASTWriter Writer(Stream, Buffer, PCMCache, {});
> return serializeUnit(Writer, Buffer, getSema(), hasErrors, OS);
> }
>
>
> Modified: cfe/trunk/lib/Frontend/CompilerInstance.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/CompilerInstance.cpp?rev=298278&r1=298277&r2=298278&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/CompilerInstance.cpp?rev=298278&r1=298277&r2=298278&view=diff>
> ==============================================================================
> --- cfe/trunk/lib/Frontend/CompilerInstance.cpp (original)
> +++ cfe/trunk/lib/Frontend/CompilerInstance.cpp Mon Mar 20 12:58:26 2017
> @@ -13,6 +13,7 @@
> #include "clang/AST/Decl.h"
> #include "clang/Basic/Diagnostic.h"
> #include "clang/Basic/FileManager.h"
> +#include "clang/Basic/MemoryBufferCache.h"
> #include "clang/Basic/SourceManager.h"
> #include "clang/Basic/TargetInfo.h"
> #include "clang/Basic/Version.h"
> @@ -55,9 +56,15 @@ using namespace clang;
>
> CompilerInstance::CompilerInstance(
> std::shared_ptr<PCHContainerOperations> PCHContainerOps,
> - bool BuildingModule)
> - : ModuleLoader(BuildingModule), Invocation(new CompilerInvocation()),
> - ThePCHContainerOperations(std::move(PCHContainerOps)) {}
> + MemoryBufferCache *SharedPCMCache)
> + : ModuleLoader(/* BuildingModule = */ SharedPCMCache),
> + Invocation(new CompilerInvocation()),
> + PCMCache(SharedPCMCache ? SharedPCMCache : new MemoryBufferCache),
> + ThePCHContainerOperations(std::move(PCHContainerOps)) {
> + // Don't allow this to invalidate buffers in use by others.
> + if (SharedPCMCache)
> + getPCMCache().finalizeCurrentBuffers();
> +}
>
> CompilerInstance::~CompilerInstance() {
> assert(OutputFiles.empty() && "Still output files in flight?");
> @@ -128,6 +135,8 @@ IntrusiveRefCntPtr<ASTReader> CompilerIn
> return ModuleManager;
> }
> void CompilerInstance::setModuleManager(IntrusiveRefCntPtr<ASTReader> Reader) {
> + assert(PCMCache.get() == &Reader->getModuleManager().getPCMCache() &&
> + "Expected ASTReader to use the same PCM cache");
> ModuleManager = std::move(Reader);
> }
>
> @@ -370,7 +379,7 @@ void CompilerInstance::createPreprocesso
> getDiagnostics(), getLangOpts(), &getTarget());
> PP = std::make_shared<Preprocessor>(
> Invocation->getPreprocessorOptsPtr(), getDiagnostics(), getLangOpts(),
> - getSourceManager(), *HeaderInfo, *this, PTHMgr,
> + getSourceManager(), getPCMCache(), *HeaderInfo, *this, PTHMgr,
> /*OwnsHeaderSearch=*/true, TUKind);
> PP->Initialize(getTarget(), getAuxTarget());
>
> @@ -1073,9 +1082,11 @@ static bool compileModuleImpl(CompilerIn
> Invocation->getModuleHash() && "Module hash mismatch!");
>
> // Construct a compiler instance that will be used to actually create the
> - // module.
> + // module. Since we're sharing a PCMCache,
> + // CompilerInstance::CompilerInstance is responsible for finalizing the
> + // buffers to prevent use-after-frees.
> CompilerInstance Instance(ImportingInstance.getPCHContainerOperations(),
> - /*BuildingModule=*/true);
> + &ImportingInstance.getPreprocessor().getPCMCache());
> auto &Inv = *Invocation;
> Instance.setInvocation(std::move(Invocation));
>
>
> Modified: cfe/trunk/lib/Lex/Preprocessor.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/Preprocessor.cpp?rev=298278&r1=298277&r2=298278&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/Preprocessor.cpp?rev=298278&r1=298277&r2=298278&view=diff>
> ==============================================================================
> --- cfe/trunk/lib/Lex/Preprocessor.cpp (original)
> +++ cfe/trunk/lib/Lex/Preprocessor.cpp Mon Mar 20 12:58:26 2017
> @@ -70,15 +70,15 @@ ExternalPreprocessorSource::~ExternalPre
>
> Preprocessor::Preprocessor(std::shared_ptr<PreprocessorOptions> PPOpts,
> DiagnosticsEngine &diags, LangOptions &opts,
> - SourceManager &SM, HeaderSearch &Headers,
> - ModuleLoader &TheModuleLoader,
> + SourceManager &SM, MemoryBufferCache &PCMCache,
> + HeaderSearch &Headers, ModuleLoader &TheModuleLoader,
> IdentifierInfoLookup *IILookup, bool OwnsHeaders,
> TranslationUnitKind TUKind)
> : PPOpts(std::move(PPOpts)), Diags(&diags), LangOpts(opts), Target(nullptr),
> AuxTarget(nullptr), FileMgr(Headers.getFileMgr()), SourceMgr(SM),
> - ScratchBuf(new ScratchBuffer(SourceMgr)), HeaderInfo(Headers),
> - TheModuleLoader(TheModuleLoader), ExternalSource(nullptr),
> - Identifiers(opts, IILookup),
> + PCMCache(PCMCache), ScratchBuf(new ScratchBuffer(SourceMgr)),
> + HeaderInfo(Headers), TheModuleLoader(TheModuleLoader),
> + ExternalSource(nullptr), Identifiers(opts, IILookup),
> PragmaHandlers(new PragmaNamespace(StringRef())),
> IncrementalProcessing(false), TUKind(TUKind), CodeComplete(nullptr),
> CodeCompletionFile(nullptr), CodeCompletionOffset(0),
>
> Modified: cfe/trunk/lib/Serialization/ASTReader.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Serialization/ASTReader.cpp?rev=298278&r1=298277&r2=298278&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Serialization/ASTReader.cpp?rev=298278&r1=298277&r2=298278&view=diff>
> ==============================================================================
> --- cfe/trunk/lib/Serialization/ASTReader.cpp (original)
> +++ cfe/trunk/lib/Serialization/ASTReader.cpp Mon Mar 20 12:58:26 2017
> @@ -37,6 +37,7 @@
> #include "clang/Basic/FileManager.h"
> #include "clang/Basic/FileSystemOptions.h"
> #include "clang/Basic/LangOptions.h"
> +#include "clang/Basic/MemoryBufferCache.h"
> #include "clang/Basic/ObjCRuntime.h"
> #include "clang/Basic/OperatorKinds.h"
> #include "clang/Basic/Sanitizers.h"
> @@ -463,19 +464,9 @@ static bool checkDiagnosticMappings(Diag
> return checkDiagnosticGroupMappings(StoredDiags, Diags, Complain);
> }
>
> -bool PCHValidator::ReadDiagnosticOptions(
> - IntrusiveRefCntPtr<DiagnosticOptions> DiagOpts, bool Complain) {
> - DiagnosticsEngine &ExistingDiags = PP.getDiagnostics();
> - IntrusiveRefCntPtr<DiagnosticIDs> DiagIDs(ExistingDiags.getDiagnosticIDs());
> - IntrusiveRefCntPtr<DiagnosticsEngine> Diags(
> - new DiagnosticsEngine(DiagIDs, DiagOpts.get()));
> - // This should never fail, because we would have processed these options
> - // before writing them to an ASTFile.
> - ProcessWarningOptions(*Diags, *DiagOpts, /*Report*/false);
> -
> - ModuleManager &ModuleMgr = Reader.getModuleManager();
> - assert(ModuleMgr.size() >= 1 && "what ASTFile is this then");
> -
> +/// Return the top import module if it is implicit, nullptr otherwise.
> +static Module *getTopImportImplicitModule(ModuleManager &ModuleMgr,
> + Preprocessor &PP) {
> // If the original import came from a file explicitly generated by the user,
> // don't check the diagnostic mappings.
> // FIXME: currently this is approximated by checking whether this is not a
> @@ -487,17 +478,37 @@ bool PCHValidator::ReadDiagnosticOptions
> while (!TopImport->ImportedBy.empty())
> TopImport = TopImport->ImportedBy[0];
> if (TopImport->Kind != MK_ImplicitModule)
> - return false;
> + return nullptr;
>
> StringRef ModuleName = TopImport->ModuleName;
> assert(!ModuleName.empty() && "diagnostic options read before module name");
>
> Module *M = PP.getHeaderSearchInfo().lookupModule(ModuleName);
> assert(M && "missing module");
> + return M;
> +}
> +
> +bool PCHValidator::ReadDiagnosticOptions(
> + IntrusiveRefCntPtr<DiagnosticOptions> DiagOpts, bool Complain) {
> + DiagnosticsEngine &ExistingDiags = PP.getDiagnostics();
> + IntrusiveRefCntPtr<DiagnosticIDs> DiagIDs(ExistingDiags.getDiagnosticIDs());
> + IntrusiveRefCntPtr<DiagnosticsEngine> Diags(
> + new DiagnosticsEngine(DiagIDs, DiagOpts.get()));
> + // This should never fail, because we would have processed these options
> + // before writing them to an ASTFile.
> + ProcessWarningOptions(*Diags, *DiagOpts, /*Report*/false);
> +
> + ModuleManager &ModuleMgr = Reader.getModuleManager();
> + assert(ModuleMgr.size() >= 1 && "what ASTFile is this then");
> +
> + Module *TopM = getTopImportImplicitModule(ModuleMgr, PP);
> + if (!TopM)
> + return false;
>
> // FIXME: if the diagnostics are incompatible, save a DiagnosticOptions that
> // contains the union of their flags.
> - return checkDiagnosticMappings(*Diags, ExistingDiags, M->IsSystem, Complain);
> + return checkDiagnosticMappings(*Diags, ExistingDiags, TopM->IsSystem,
> + Complain);
> }
>
> /// \brief Collect the macro definitions provided by the given preprocessor
> @@ -4064,12 +4075,41 @@ ASTReader::readUnhashedControlBlock(Modu
> Listener.get(),
> WasImportedBy ? false : HSOpts.ModulesValidateDiagnosticOptions);
>
> + // If F was directly imported by another module, it's implicitly validated by
> + // the importing module.
> if (DisableValidation || WasImportedBy ||
> (AllowConfigurationMismatch && Result == ConfigurationMismatch))
> return Success;
>
> - if (Result == Failure)
> + if (Result == Failure) {
> Error("malformed block record in AST file");
> + return Failure;
> + }
> +
> + if (Result == OutOfDate && F.Kind == MK_ImplicitModule) {
> + // If this module has already been finalized in the PCMCache, we're stuck
> + // with it; we can only load a single version of each module.
> + //
> + // This can happen when a module is imported in two contexts: in one, as a
> + // user module; in another, as a system module (due to an import from
> + // another module marked with the [system] flag). It usually indicates a
> + // bug in the module map: this module should also be marked with [system].
> + //
> + // If -Wno-system-headers (the default), and the first import is as a
> + // system module, then validation will fail during the as-user import,
> + // since -Werror flags won't have been validated. However, it's reasonable
> + // to treat this consistently as a system module.
> + //
> + // If -Wsystem-headers, the PCM on disk was built with
> + // -Wno-system-headers, and the first import is as a user module, then
> + // validation will fail during the as-system import since the PCM on disk
> + // doesn't guarantee that -Werror was respected. However, the -Werror
> + // flags were checked during the initial as-user import.
> + if (PCMCache.isBufferFinal(F.FileName)) {
> + Diag(diag::warn_module_system_bit_conflict) << F.FileName;
> + return Success;
> + }
> + }
>
> return Result;
> }
> @@ -4122,7 +4162,7 @@ ASTReader::ASTReadResult ASTReader::read
> if (Listener && ValidateDiagnosticOptions &&
> !AllowCompatibleConfigurationMismatch &&
> ParseDiagnosticOptions(Record, Complain, *Listener))
> - return OutOfDate;
> + Result = OutOfDate; // Don't return early. Read the signature.
> break;
> }
> case DIAG_PRAGMA_MAPPINGS:
> @@ -7301,7 +7341,7 @@ LLVM_DUMP_METHOD void ASTReader::dump()
> /// by heap-backed versus mmap'ed memory.
> void ASTReader::getMemoryBufferSizes(MemoryBufferSizes &sizes) const {
> for (ModuleFile &I : ModuleMgr) {
> - if (llvm::MemoryBuffer *buf = I.Buffer.get()) {
> + if (llvm::MemoryBuffer *buf = I.Buffer) {
> size_t bytes = buf->getBufferSize();
> switch (buf->getBufferKind()) {
> case llvm::MemoryBuffer::MemoryBuffer_Malloc:
> @@ -9628,8 +9668,10 @@ ASTReader::ASTReader(Preprocessor &PP, A
> : cast<ASTReaderListener>(new PCHValidator(PP, *this))),
> SourceMgr(PP.getSourceManager()), FileMgr(PP.getFileManager()),
> PCHContainerRdr(PCHContainerRdr), Diags(PP.getDiagnostics()), PP(PP),
> - Context(Context), ModuleMgr(PP.getFileManager(), PCHContainerRdr),
> - DummyIdResolver(PP), ReadTimer(std::move(ReadTimer)), isysroot(isysroot),
> + Context(Context),
> + ModuleMgr(PP.getFileManager(), PP.getPCMCache(), PCHContainerRdr),
> + PCMCache(PP.getPCMCache()), DummyIdResolver(PP),
> + ReadTimer(std::move(ReadTimer)), isysroot(isysroot),
> DisableValidation(DisableValidation),
> AllowASTWithCompilerErrors(AllowASTWithCompilerErrors),
> AllowConfigurationMismatch(AllowConfigurationMismatch),
>
> Modified: cfe/trunk/lib/Serialization/ASTWriter.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Serialization/ASTWriter.cpp?rev=298278&r1=298277&r2=298278&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Serialization/ASTWriter.cpp?rev=298278&r1=298277&r2=298278&view=diff>
> ==============================================================================
> --- cfe/trunk/lib/Serialization/ASTWriter.cpp (original)
> +++ cfe/trunk/lib/Serialization/ASTWriter.cpp Mon Mar 20 12:58:26 2017
> @@ -35,6 +35,7 @@
> #include "clang/Basic/FileSystemOptions.h"
> #include "clang/Basic/LLVM.h"
> #include "clang/Basic/LangOptions.h"
> +#include "clang/Basic/MemoryBufferCache.h"
> #include "clang/Basic/Module.h"
> #include "clang/Basic/ObjCRuntime.h"
> #include "clang/Basic/SourceManager.h"
> @@ -4304,10 +4305,11 @@ void ASTWriter::SetSelectorOffset(Select
> }
>
> ASTWriter::ASTWriter(llvm::BitstreamWriter &Stream,
> - SmallVectorImpl<char> &Buffer,
> + SmallVectorImpl<char> &Buffer, MemoryBufferCache &PCMCache,
> ArrayRef<std::shared_ptr<ModuleFileExtension>> Extensions,
> bool IncludeTimestamps)
> - : Stream(Stream), Buffer(Buffer), IncludeTimestamps(IncludeTimestamps) {
> + : Stream(Stream), Buffer(Buffer), PCMCache(PCMCache),
> + IncludeTimestamps(IncludeTimestamps) {
> for (const auto &Ext : Extensions) {
> if (auto Writer = Ext->createExtensionWriter(*this))
> ModuleFileExtensionWriters.push_back(std::move(Writer));
> @@ -4354,6 +4356,12 @@ ASTFileSignature ASTWriter::WriteAST(Sem
> this->BaseDirectory.clear();
>
> WritingAST = false;
> + if (SemaRef.Context.getLangOpts().ImplicitModules && WritingModule) {
> + // Construct MemoryBuffer and update buffer manager.
> + PCMCache.addBuffer(OutputFile,
> + llvm::MemoryBuffer::getMemBufferCopy(
> + StringRef(Buffer.begin(), Buffer.size())));
> + }
> return Signature;
> }
>
>
> Modified: cfe/trunk/lib/Serialization/GeneratePCH.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Serialization/GeneratePCH.cpp?rev=298278&r1=298277&r2=298278&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Serialization/GeneratePCH.cpp?rev=298278&r1=298277&r2=298278&view=diff>
> ==============================================================================
> --- cfe/trunk/lib/Serialization/GeneratePCH.cpp (original)
> +++ cfe/trunk/lib/Serialization/GeneratePCH.cpp Mon Mar 20 12:58:26 2017
> @@ -28,7 +28,8 @@ PCHGenerator::PCHGenerator(
> bool AllowASTWithErrors, bool IncludeTimestamps)
> : PP(PP), OutputFile(OutputFile), isysroot(isysroot.str()),
> SemaPtr(nullptr), Buffer(Buffer), Stream(Buffer->Data),
> - Writer(Stream, Buffer->Data, Extensions, IncludeTimestamps),
> + Writer(Stream, Buffer->Data, PP.getPCMCache(), Extensions,
> + IncludeTimestamps),
> AllowASTWithErrors(AllowASTWithErrors) {
> Buffer->IsComplete = false;
> }
>
> Modified: cfe/trunk/lib/Serialization/ModuleManager.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Serialization/ModuleManager.cpp?rev=298278&r1=298277&r2=298278&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Serialization/ModuleManager.cpp?rev=298278&r1=298277&r2=298278&view=diff>
> ==============================================================================
> --- cfe/trunk/lib/Serialization/ModuleManager.cpp (original)
> +++ cfe/trunk/lib/Serialization/ModuleManager.cpp Mon Mar 20 12:58:26 2017
> @@ -12,6 +12,7 @@
> //
> //===----------------------------------------------------------------------===//
> #include "clang/Serialization/ModuleManager.h"
> +#include "clang/Basic/MemoryBufferCache.h"
> #include "clang/Frontend/PCHContainerOperations.h"
> #include "clang/Lex/HeaderSearch.h"
> #include "clang/Lex/ModuleMap.h"
> @@ -137,7 +138,9 @@ ModuleManager::addModule(StringRef FileN
> // Load the contents of the module
> if (std::unique_ptr<llvm::MemoryBuffer> Buffer = lookupBuffer(FileName)) {
> // The buffer was already provided for us.
> - NewModule->Buffer = std::move(Buffer);
> + NewModule->Buffer = &PCMCache->addBuffer(FileName, std::move(Buffer));
> + } else if (llvm::MemoryBuffer *Buffer = PCMCache->lookupBuffer(FileName)) {
> + NewModule->Buffer = Buffer;
> } else {
> // Open the AST file.
> llvm::ErrorOr<std::unique_ptr<llvm::MemoryBuffer>> Buf((std::error_code()));
> @@ -158,7 +161,7 @@ ModuleManager::addModule(StringRef FileN
> return Missing;
> }
>
> - NewModule->Buffer = std::move(*Buf);
> + NewModule->Buffer = &PCMCache->addBuffer(FileName, std::move(*Buf));
> }
>
> // Initialize the stream.
> @@ -167,8 +170,13 @@ ModuleManager::addModule(StringRef FileN
> // Read the signature eagerly now so that we can check it. Avoid calling
> // ReadSignature unless there's something to check though.
> if (ExpectedSignature && checkSignature(ReadSignature(NewModule->Data),
> - ExpectedSignature, ErrorStr))
> + ExpectedSignature, ErrorStr)) {
> + // Try to remove the buffer. If it can't be removed, then it was already
> + // validated by this process.
> + if (!PCMCache->tryToRemoveBuffer(NewModule->FileName))
> + FileMgr.invalidateCache(NewModule->File);
> return OutOfDate;
> + }
>
> // We're keeping this module. Store it everywhere.
> Module = Modules[Entry] = NewModule.get();
> @@ -235,7 +243,12 @@ void ModuleManager::removeModules(
> // Files that didn't make it through ReadASTCore successfully will be
> // rebuilt (or there was an error). Invalidate them so that we can load the
> // new files that will be renamed over the old ones.
> - if (LoadedSuccessfully.count(&*victim) == 0)
> + //
> + // The PCMCache tracks whether the module was succesfully loaded in another
> + // thread/context; in that case, it won't need to be rebuilt (and we can't
> + // safely invalidate it anyway).
> + if (LoadedSuccessfully.count(&*victim) == 0 &&
> + !PCMCache->tryToRemoveBuffer(victim->FileName))
> FileMgr.invalidateCache(victim->File);
> }
>
> @@ -292,10 +305,10 @@ void ModuleManager::moduleFileAccepted(M
> ModulesInCommonWithGlobalIndex.push_back(MF);
> }
>
> -ModuleManager::ModuleManager(FileManager &FileMgr,
> +ModuleManager::ModuleManager(FileManager &FileMgr, MemoryBufferCache &PCMCache,
> const PCHContainerReader &PCHContainerRdr)
> - : FileMgr(FileMgr), PCHContainerRdr(PCHContainerRdr), GlobalIndex(),
> - FirstVisitState(nullptr) {}
> + : FileMgr(FileMgr), PCMCache(&PCMCache), PCHContainerRdr(PCHContainerRdr),
> + GlobalIndex(), FirstVisitState(nullptr) {}
>
> ModuleManager::~ModuleManager() { delete FirstVisitState; }
>
>
> Added: cfe/trunk/test/Modules/Inputs/system-out-of-date/X.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/Inputs/system-out-of-date/X.h?rev=298278&view=auto <http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/Inputs/system-out-of-date/X.h?rev=298278&view=auto>
> ==============================================================================
> --- cfe/trunk/test/Modules/Inputs/system-out-of-date/X.h (added)
> +++ cfe/trunk/test/Modules/Inputs/system-out-of-date/X.h Mon Mar 20 12:58:26 2017
> @@ -0,0 +1 @@
> +#import <Y.h>
>
> Added: cfe/trunk/test/Modules/Inputs/system-out-of-date/Y.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/Inputs/system-out-of-date/Y.h?rev=298278&view=auto <http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/Inputs/system-out-of-date/Y.h?rev=298278&view=auto>
> ==============================================================================
> --- cfe/trunk/test/Modules/Inputs/system-out-of-date/Y.h (added)
> +++ cfe/trunk/test/Modules/Inputs/system-out-of-date/Y.h Mon Mar 20 12:58:26 2017
> @@ -0,0 +1 @@
> +//empty
>
> Added: cfe/trunk/test/Modules/Inputs/system-out-of-date/Z.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/Inputs/system-out-of-date/Z.h?rev=298278&view=auto <http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/Inputs/system-out-of-date/Z.h?rev=298278&view=auto>
> ==============================================================================
> --- cfe/trunk/test/Modules/Inputs/system-out-of-date/Z.h (added)
> +++ cfe/trunk/test/Modules/Inputs/system-out-of-date/Z.h Mon Mar 20 12:58:26 2017
> @@ -0,0 +1 @@
> +#import <Y.h>
>
> Added: cfe/trunk/test/Modules/Inputs/system-out-of-date/module.map
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/Inputs/system-out-of-date/module.map?rev=298278&view=auto <http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/Inputs/system-out-of-date/module.map?rev=298278&view=auto>
> ==============================================================================
> --- cfe/trunk/test/Modules/Inputs/system-out-of-date/module.map (added)
> +++ cfe/trunk/test/Modules/Inputs/system-out-of-date/module.map Mon Mar 20 12:58:26 2017
> @@ -0,0 +1,12 @@
> +module X [system] {
> + header "X.h" // imports Y
> + export *
> +}
> +module Y {
> + header "Y.h"
> + export *
> +}
> +module Z {
> + header "Z.h" // imports Y
> + export *
> +}
>
> Added: cfe/trunk/test/Modules/Inputs/warning-mismatch/Mismatch.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/Inputs/warning-mismatch/Mismatch.h?rev=298278&view=auto <http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/Inputs/warning-mismatch/Mismatch.h?rev=298278&view=auto>
> ==============================================================================
> --- cfe/trunk/test/Modules/Inputs/warning-mismatch/Mismatch.h (added)
> +++ cfe/trunk/test/Modules/Inputs/warning-mismatch/Mismatch.h Mon Mar 20 12:58:26 2017
> @@ -0,0 +1 @@
> +struct Mismatch { int i; };
>
> Added: cfe/trunk/test/Modules/Inputs/warning-mismatch/System.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/Inputs/warning-mismatch/System.h?rev=298278&view=auto <http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/Inputs/warning-mismatch/System.h?rev=298278&view=auto>
> ==============================================================================
> --- cfe/trunk/test/Modules/Inputs/warning-mismatch/System.h (added)
> +++ cfe/trunk/test/Modules/Inputs/warning-mismatch/System.h Mon Mar 20 12:58:26 2017
> @@ -0,0 +1,2 @@
> +#import "Mismatch.h"
> +struct System { int i; };
>
> Added: cfe/trunk/test/Modules/Inputs/warning-mismatch/module.modulemap
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/Inputs/warning-mismatch/module.modulemap?rev=298278&view=auto <http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/Inputs/warning-mismatch/module.modulemap?rev=298278&view=auto>
> ==============================================================================
> --- cfe/trunk/test/Modules/Inputs/warning-mismatch/module.modulemap (added)
> +++ cfe/trunk/test/Modules/Inputs/warning-mismatch/module.modulemap Mon Mar 20 12:58:26 2017
> @@ -0,0 +1,7 @@
> +module System [system] {
> + header "System.h"
> +}
> +
> +module Mismatch {
> + header "Mismatch.h"
> +}
>
> Added: cfe/trunk/test/Modules/outofdate-rebuild.m
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/outofdate-rebuild.m?rev=298278&view=auto <http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/outofdate-rebuild.m?rev=298278&view=auto>
> ==============================================================================
> --- cfe/trunk/test/Modules/outofdate-rebuild.m (added)
> +++ cfe/trunk/test/Modules/outofdate-rebuild.m Mon Mar 20 12:58:26 2017
> @@ -0,0 +1,15 @@
> +// RUN: rm -rf %t.cache
> +// RUN: echo "@import CoreText;" > %t.m
> +// RUN: %clang_cc1 -fdisable-module-hash -fmodules-cache-path=%t.cache \
> +// RUN: -fmodules -fimplicit-module-maps -I%S/Inputs/outofdate-rebuild %s \
> +// RUN: -fsyntax-only
> +// RUN: %clang_cc1 -DMISMATCH -Werror -fdisable-module-hash \
> +// RUN: -fmodules-cache-path=%t.cache -fmodules -fimplicit-module-maps \
> +// RUN: -I%S/Inputs/outofdate-rebuild %t.m -fsyntax-only
> +// RUN: %clang_cc1 -fdisable-module-hash -fmodules-cache-path=%t.cache \
> +// RUN: -fmodules -fimplicit-module-maps -I%S/Inputs/outofdate-rebuild %s \
> +// RUN: -fsyntax-only
> +
> +// This testcase reproduces a use-after-free in when ModuleManager removes an
> +// entry from the PCMCache without notifying its parent ASTReader.
> + at import Cocoa;
>
> Added: cfe/trunk/test/Modules/system-out-of-date-test.m
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/system-out-of-date-test.m?rev=298278&view=auto <http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/system-out-of-date-test.m?rev=298278&view=auto>
> ==============================================================================
> --- cfe/trunk/test/Modules/system-out-of-date-test.m (added)
> +++ cfe/trunk/test/Modules/system-out-of-date-test.m Mon Mar 20 12:58:26 2017
> @@ -0,0 +1,17 @@
> +// RUN: rm -rf %t.cache
> +// RUN: echo '@import X;' | \
> +// RUN: %clang_cc1 -fmodules -fimplicit-module-maps \
> +// RUN: -fmodules-cache-path=%t.cache -I%S/Inputs/system-out-of-date \
> +// RUN: -fsyntax-only -x objective-c -
> +//
> +// Build something with different diagnostic options.
> +// RUN: %clang_cc1 -fmodules -fimplicit-module-maps \
> +// RUN: -fmodules-cache-path=%t.cache -I%S/Inputs/system-out-of-date \
> +// RUN: -fsyntax-only %s -Wnon-modular-include-in-framework-module \
> +// RUN: -Werror=non-modular-include-in-framework-module 2>&1 \
> +// RUN: | FileCheck %s
> + at import X;
> +
> +#import <Z.h>
> +// CHECK: While building module 'Z' imported from
> +// CHECK: {{.*}}Y-{{.*}}pcm' was validated as a system module and is now being imported as a non-system module
>
> Added: cfe/trunk/test/Modules/warning-mismatch.m
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/warning-mismatch.m?rev=298278&view=auto <http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/warning-mismatch.m?rev=298278&view=auto>
> ==============================================================================
> --- cfe/trunk/test/Modules/warning-mismatch.m (added)
> +++ cfe/trunk/test/Modules/warning-mismatch.m Mon Mar 20 12:58:26 2017
> @@ -0,0 +1,13 @@
> +// RUN: rm -rf %t.cache
> +// RUN: echo "@import Mismatch;" >%t.m
> +// RUN: %clang_cc1 -Wno-system-headers -fdisable-module-hash \
> +// RUN: -fmodules-cache-path=%t.cache -fmodules -fimplicit-module-maps \
> +// RUN: -I%S/Inputs/warning-mismatch %t.m -fsyntax-only
> +// RUN: %clang_cc1 -Wsystem-headers -fdisable-module-hash \
> +// RUN: -fmodules-cache-path=%t.cache -fmodules -fimplicit-module-maps \
> +// RUN: -I%S/Inputs/warning-mismatch %s -fsyntax-only
> +
> +// This testcase triggers a warning flag mismatch in an already validated
> +// header.
> + at import Mismatch;
> + at import System;
>
> Modified: cfe/trunk/unittests/Basic/CMakeLists.txt
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Basic/CMakeLists.txt?rev=298278&r1=298277&r2=298278&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Basic/CMakeLists.txt?rev=298278&r1=298277&r2=298278&view=diff>
> ==============================================================================
> --- cfe/trunk/unittests/Basic/CMakeLists.txt (original)
> +++ cfe/trunk/unittests/Basic/CMakeLists.txt Mon Mar 20 12:58:26 2017
> @@ -6,6 +6,7 @@ add_clang_unittest(BasicTests
> CharInfoTest.cpp
> DiagnosticTest.cpp
> FileManagerTest.cpp
> + MemoryBufferCacheTest.cpp
> SourceManagerTest.cpp
> VirtualFileSystemTest.cpp
> )
>
> Added: cfe/trunk/unittests/Basic/MemoryBufferCacheTest.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Basic/MemoryBufferCacheTest.cpp?rev=298278&view=auto <http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Basic/MemoryBufferCacheTest.cpp?rev=298278&view=auto>
> ==============================================================================
> --- cfe/trunk/unittests/Basic/MemoryBufferCacheTest.cpp (added)
> +++ cfe/trunk/unittests/Basic/MemoryBufferCacheTest.cpp Mon Mar 20 12:58:26 2017
> @@ -0,0 +1,95 @@
> +//===- MemoryBufferCacheTest.cpp - MemoryBufferCache tests ----------------===//
> +//
> +// The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +
> +#include "clang/Basic/MemoryBufferCache.h"
> +#include "llvm/Support/MemoryBuffer.h"
> +#include "gtest/gtest.h"
> +
> +using namespace llvm;
> +using namespace clang;
> +
> +namespace {
> +
> +std::unique_ptr<MemoryBuffer> getBuffer(int I) {
> + SmallVector<char, 8> Bytes;
> + raw_svector_ostream(Bytes) << "data:" << I;
> + return MemoryBuffer::getMemBuffer(StringRef(Bytes.data(), Bytes.size()), "",
> + /* RequiresNullTerminator = */ false);
> +}
> +
> +TEST(MemoryBufferCacheTest, addBuffer) {
> + auto B1 = getBuffer(1);
> + auto B2 = getBuffer(2);
> + auto B3 = getBuffer(3);
> + auto *RawB1 = B1.get();
> + auto *RawB2 = B2.get();
> + auto *RawB3 = B3.get();
> +
> + // Add a few buffers.
> + MemoryBufferCache Cache;
> + EXPECT_EQ(RawB1, &Cache.addBuffer("1", std::move(B1)));
> + EXPECT_EQ(RawB2, &Cache.addBuffer("2", std::move(B2)));
> + EXPECT_EQ(RawB3, &Cache.addBuffer("3", std::move(B3)));
> + EXPECT_EQ(RawB1, Cache.lookupBuffer("1"));
> + EXPECT_EQ(RawB2, Cache.lookupBuffer("2"));
> + EXPECT_EQ(RawB3, Cache.lookupBuffer("3"));
> + EXPECT_FALSE(Cache.isBufferFinal("1"));
> + EXPECT_FALSE(Cache.isBufferFinal("2"));
> + EXPECT_FALSE(Cache.isBufferFinal("3"));
> +
> + // Remove the middle buffer.
> + EXPECT_FALSE(Cache.tryToRemoveBuffer("2"));
> + EXPECT_EQ(nullptr, Cache.lookupBuffer("2"));
> + EXPECT_FALSE(Cache.isBufferFinal("2"));
> +
> + // Replace the middle buffer.
> + B2 = getBuffer(2);
> + ASSERT_NE(RawB2, B2.get());
> + RawB2 = B2.get();
> + EXPECT_EQ(RawB2, &Cache.addBuffer("2", std::move(B2)));
> +
> + // Check that nothing is final.
> + EXPECT_FALSE(Cache.isBufferFinal("1"));
> + EXPECT_FALSE(Cache.isBufferFinal("2"));
> + EXPECT_FALSE(Cache.isBufferFinal("3"));
> +}
> +
> +TEST(MemoryBufferCacheTest, finalizeCurrentBuffers) {
> + // Add a buffer.
> + MemoryBufferCache Cache;
> + auto B1 = getBuffer(1);
> + auto *RawB1 = B1.get();
> + Cache.addBuffer("1", std::move(B1));
> + ASSERT_FALSE(Cache.isBufferFinal("1"));
> +
> + // Finalize it.
> + Cache.finalizeCurrentBuffers();
> + EXPECT_TRUE(Cache.isBufferFinal("1"));
> + EXPECT_TRUE(Cache.tryToRemoveBuffer("1"));
> + EXPECT_EQ(RawB1, Cache.lookupBuffer("1"));
> + EXPECT_TRUE(Cache.isBufferFinal("1"));
> +
> + // Repeat.
> + auto B2 = getBuffer(2);
> + auto *RawB2 = B2.get();
> + Cache.addBuffer("2", std::move(B2));
> + EXPECT_FALSE(Cache.isBufferFinal("2"));
> +
> + Cache.finalizeCurrentBuffers();
> + EXPECT_TRUE(Cache.isBufferFinal("1"));
> + EXPECT_TRUE(Cache.isBufferFinal("2"));
> + EXPECT_TRUE(Cache.tryToRemoveBuffer("1"));
> + EXPECT_TRUE(Cache.tryToRemoveBuffer("2"));
> + EXPECT_EQ(RawB1, Cache.lookupBuffer("1"));
> + EXPECT_EQ(RawB2, Cache.lookupBuffer("2"));
> + EXPECT_TRUE(Cache.isBufferFinal("1"));
> + EXPECT_TRUE(Cache.isBufferFinal("2"));
> +}
> +
> +} // namespace
>
> Modified: cfe/trunk/unittests/Basic/SourceManagerTest.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Basic/SourceManagerTest.cpp?rev=298278&r1=298277&r2=298278&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Basic/SourceManagerTest.cpp?rev=298278&r1=298277&r2=298278&view=diff>
> ==============================================================================
> --- cfe/trunk/unittests/Basic/SourceManagerTest.cpp (original)
> +++ cfe/trunk/unittests/Basic/SourceManagerTest.cpp Mon Mar 20 12:58:26 2017
> @@ -12,6 +12,7 @@
> #include "clang/Basic/DiagnosticOptions.h"
> #include "clang/Basic/FileManager.h"
> #include "clang/Basic/LangOptions.h"
> +#include "clang/Basic/MemoryBufferCache.h"
> #include "clang/Basic/TargetInfo.h"
> #include "clang/Basic/TargetOptions.h"
> #include "clang/Lex/HeaderSearch.h"
> @@ -78,10 +79,11 @@ TEST_F(SourceManagerTest, isBeforeInTran
> SourceMgr.setMainFileID(mainFileID);
>
> VoidModuleLoader ModLoader;
> + MemoryBufferCache PCMCache;
> HeaderSearch HeaderInfo(std::make_shared<HeaderSearchOptions>(), SourceMgr,
> Diags, LangOpts, &*Target);
> Preprocessor PP(std::make_shared<PreprocessorOptions>(), Diags, LangOpts,
> - SourceMgr, HeaderInfo, ModLoader,
> + SourceMgr, PCMCache, HeaderInfo, ModLoader,
> /*IILookup =*/nullptr,
> /*OwnsHeaderSearch =*/false);
> PP.Initialize(*Target);
> @@ -198,10 +200,11 @@ TEST_F(SourceManagerTest, getMacroArgExp
> SourceMgr.overrideFileContents(headerFile, std::move(HeaderBuf));
>
> VoidModuleLoader ModLoader;
> + MemoryBufferCache PCMCache;
> HeaderSearch HeaderInfo(std::make_shared<HeaderSearchOptions>(), SourceMgr,
> Diags, LangOpts, &*Target);
> Preprocessor PP(std::make_shared<PreprocessorOptions>(), Diags, LangOpts,
> - SourceMgr, HeaderInfo, ModLoader,
> + SourceMgr, PCMCache, HeaderInfo, ModLoader,
> /*IILookup =*/nullptr,
> /*OwnsHeaderSearch =*/false);
> PP.Initialize(*Target);
> @@ -298,10 +301,11 @@ TEST_F(SourceManagerTest, isBeforeInTran
> SourceMgr.overrideFileContents(headerFile, std::move(HeaderBuf));
>
> VoidModuleLoader ModLoader;
> + MemoryBufferCache PCMCache;
> HeaderSearch HeaderInfo(std::make_shared<HeaderSearchOptions>(), SourceMgr,
> Diags, LangOpts, &*Target);
> Preprocessor PP(std::make_shared<PreprocessorOptions>(), Diags, LangOpts,
> - SourceMgr, HeaderInfo, ModLoader,
> + SourceMgr, PCMCache, HeaderInfo, ModLoader,
> /*IILookup =*/nullptr,
> /*OwnsHeaderSearch =*/false);
> PP.Initialize(*Target);
>
> Modified: cfe/trunk/unittests/Lex/LexerTest.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Lex/LexerTest.cpp?rev=298278&r1=298277&r2=298278&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Lex/LexerTest.cpp?rev=298278&r1=298277&r2=298278&view=diff>
> ==============================================================================
> --- cfe/trunk/unittests/Lex/LexerTest.cpp (original)
> +++ cfe/trunk/unittests/Lex/LexerTest.cpp Mon Mar 20 12:58:26 2017
> @@ -12,6 +12,7 @@
> #include "clang/Basic/DiagnosticOptions.h"
> #include "clang/Basic/FileManager.h"
> #include "clang/Basic/LangOptions.h"
> +#include "clang/Basic/MemoryBufferCache.h"
> #include "clang/Basic/SourceManager.h"
> #include "clang/Basic/TargetInfo.h"
> #include "clang/Basic/TargetOptions.h"
> @@ -64,10 +65,12 @@ protected:
> SourceMgr.setMainFileID(SourceMgr.createFileID(std::move(Buf)));
>
> VoidModuleLoader ModLoader;
> + MemoryBufferCache PCMCache;
> HeaderSearch HeaderInfo(std::make_shared<HeaderSearchOptions>(), SourceMgr,
> Diags, LangOpts, Target.get());
> Preprocessor PP(std::make_shared<PreprocessorOptions>(), Diags, LangOpts,
> - SourceMgr, HeaderInfo, ModLoader, /*IILookup =*/nullptr,
> + SourceMgr, PCMCache, HeaderInfo, ModLoader,
> + /*IILookup =*/nullptr,
> /*OwnsHeaderSearch =*/false);
> PP.Initialize(*Target);
> PP.EnterMainSourceFile();
>
> Modified: cfe/trunk/unittests/Lex/PPCallbacksTest.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Lex/PPCallbacksTest.cpp?rev=298278&r1=298277&r2=298278&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Lex/PPCallbacksTest.cpp?rev=298278&r1=298277&r2=298278&view=diff>
> ==============================================================================
> --- cfe/trunk/unittests/Lex/PPCallbacksTest.cpp (original)
> +++ cfe/trunk/unittests/Lex/PPCallbacksTest.cpp Mon Mar 20 12:58:26 2017
> @@ -14,6 +14,7 @@
> #include "clang/Basic/DiagnosticOptions.h"
> #include "clang/Basic/FileManager.h"
> #include "clang/Basic/LangOptions.h"
> +#include "clang/Basic/MemoryBufferCache.h"
> #include "clang/Basic/SourceManager.h"
> #include "clang/Basic/TargetInfo.h"
> #include "clang/Basic/TargetOptions.h"
> @@ -161,13 +162,14 @@ protected:
> SourceMgr.setMainFileID(SourceMgr.createFileID(std::move(Buf)));
>
> VoidModuleLoader ModLoader;
> + MemoryBufferCache PCMCache;
>
> HeaderSearch HeaderInfo(std::make_shared<HeaderSearchOptions>(), SourceMgr,
> Diags, LangOpts, Target.get());
> AddFakeHeader(HeaderInfo, HeaderPath, SystemHeader);
>
> Preprocessor PP(std::make_shared<PreprocessorOptions>(), Diags, LangOpts,
> - SourceMgr, HeaderInfo, ModLoader,
> + SourceMgr, PCMCache, HeaderInfo, ModLoader,
> /*IILookup =*/nullptr,
> /*OwnsHeaderSearch =*/false);
> PP.Initialize(*Target);
> @@ -198,11 +200,12 @@ protected:
> SourceMgr.setMainFileID(SourceMgr.createFileID(std::move(SourceBuf)));
>
> VoidModuleLoader ModLoader;
> + MemoryBufferCache PCMCache;
> HeaderSearch HeaderInfo(std::make_shared<HeaderSearchOptions>(), SourceMgr,
> Diags, OpenCLLangOpts, Target.get());
>
> Preprocessor PP(std::make_shared<PreprocessorOptions>(), Diags,
> - OpenCLLangOpts, SourceMgr, HeaderInfo, ModLoader,
> + OpenCLLangOpts, SourceMgr, PCMCache, HeaderInfo, ModLoader,
> /*IILookup =*/nullptr,
> /*OwnsHeaderSearch =*/false);
> PP.Initialize(*Target);
>
> Modified: cfe/trunk/unittests/Lex/PPConditionalDirectiveRecordTest.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Lex/PPConditionalDirectiveRecordTest.cpp?rev=298278&r1=298277&r2=298278&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Lex/PPConditionalDirectiveRecordTest.cpp?rev=298278&r1=298277&r2=298278&view=diff>
> ==============================================================================
> --- cfe/trunk/unittests/Lex/PPConditionalDirectiveRecordTest.cpp (original)
> +++ cfe/trunk/unittests/Lex/PPConditionalDirectiveRecordTest.cpp Mon Mar 20 12:58:26 2017
> @@ -12,6 +12,7 @@
> #include "clang/Basic/DiagnosticOptions.h"
> #include "clang/Basic/FileManager.h"
> #include "clang/Basic/LangOptions.h"
> +#include "clang/Basic/MemoryBufferCache.h"
> #include "clang/Basic/SourceManager.h"
> #include "clang/Basic/TargetInfo.h"
> #include "clang/Basic/TargetOptions.h"
> @@ -93,10 +94,11 @@ TEST_F(PPConditionalDirectiveRecordTest,
> SourceMgr.setMainFileID(SourceMgr.createFileID(std::move(Buf)));
>
> VoidModuleLoader ModLoader;
> + MemoryBufferCache PCMCache;
> HeaderSearch HeaderInfo(std::make_shared<HeaderSearchOptions>(), SourceMgr,
> Diags, LangOpts, Target.get());
> Preprocessor PP(std::make_shared<PreprocessorOptions>(), Diags, LangOpts,
> - SourceMgr, HeaderInfo, ModLoader,
> + SourceMgr, PCMCache, HeaderInfo, ModLoader,
> /*IILookup =*/nullptr,
> /*OwnsHeaderSearch =*/false);
> PP.Initialize(*Target);
>
>
> _______________________________________________
> cfe-commits mailing list
> cfe-commits at lists.llvm.org <mailto:cfe-commits at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits <http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20170321/39b5b261/attachment-0001.html>
More information about the cfe-commits
mailing list