<div dir="ltr"><span style="font-size:12.8px">The unit test "MemoryBufferCacheTest.</span><wbr style="font-size:12.8px"><span style="font-size:12.8px">addBuffer" seems to be intermittently failing on Windows. I can see this on the buildbot:</span><div style="font-size:12.8px"><a href="http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/builds/7086/steps/test/logs/stdio" target="_blank">http://lab.llvm.org:8011/<wbr>builders/llvm-clang-lld-x86_<wbr>64-scei-ps4-windows10pro-fast/<wbr>builds/7086/steps/test/logs/<wbr>stdio</a><br><div><br></div><div>Please could you take a look?</div><div><br></div><div>Thanks</div><div>Russ<br><div><br></div><div><div>Note: Google Test filter = MemoryBufferCacheTest.<wbr>addBuffer</div><div>[==========] Running 1 test from 1 test case.</div><div>[----------] Global test environment set-up.</div><div>[----------] 1 test from MemoryBufferCacheTest</div><div>[ RUN ] MemoryBufferCacheTest.<wbr>addBuffer</div><div><snip>\llvm\tools\clang\<wbr>unittests\Basic\<wbr>MemoryBufferCacheTest.cpp(53): error: Expected: (RawB2) != (B2.get()), actual: 000000075B1FDB50 vs 000000075B1FDB50</div><div>[ FAILED ] MemoryBufferCacheTest.<wbr>addBuffer (1 ms)</div><div>[----------] 1 test from MemoryBufferCacheTest (1 ms total)</div><div><br></div><div>[----------] Global test environment tear-down</div><div>[==========] 1 test from 1 test case ran. (1 ms total)</div><div>[ PASSED ] 0 tests.</div><div>[ FAILED ] 1 test, listed below:</div><div>[ FAILED ] MemoryBufferCacheTest.<wbr>addBuffer</div><div><br></div><div> 1 FAILED TEST</div></div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On 20 March 2017 at 17:58, Duncan P. N. Exon Smith via cfe-commits <span dir="ltr"><<a href="mailto:cfe-commits@lists.llvm.org" target="_blank">cfe-commits@lists.llvm.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Author: dexonsmith<br>
Date: Mon Mar 20 12:58:26 2017<br>
New Revision: 298278<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=298278&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project?rev=298278&view=rev</a><br>
Log:<br>
Reapply "Modules: Cache PCMs in memory and avoid a use-after-free"<br>
<br>
This reverts commit r298185, effectively reapplying r298165, after fixing the<br>
new unit tests (PR32338). The memory buffer generator doesn't null-terminate<br>
the MemoryBuffer it creates; this version of the commit informs getMemBuffer<br>
about that to avoid the assert.<br>
<br>
Original commit message follows:<br>
<br>
----<br>
<br>
Clang's internal build system for implicit modules uses lock files to<br>
ensure that after a process writes a PCM it will read the same one back<br>
in (without contention from other -cc1 commands). Since PCMs are read<br>
from disk repeatedly while invalidating, building, and importing, the<br>
lock is not released quickly. Furthermore, the LockFileManager is not<br>
robust in every environment. Other -cc1 commands can stall until<br>
timeout (after about eight minutes).<br>
<br>
This commit changes the lock file from being necessary for correctness<br>
to a (possibly dubious) performance hack. The remaining benefit is to<br>
reduce duplicate work in competing -cc1 commands which depend on the<br>
same module. Follow-up commits will change the internal build system to<br>
continue after a timeout, and reduce the timeout. Perhaps we should<br>
reconsider blocking at all.<br>
<br>
This also fixes a use-after-free, when one part of a compilation<br>
validates a PCM and starts using it, and another tries to swap out the<br>
PCM for something new.<br>
<br>
The PCMCache is a new type called MemoryBufferCache, which saves memory<br>
buffers based on their filename. Its ownership is shared by the<br>
CompilerInstance and ModuleManager.<br>
<br>
- The ModuleManager stores PCMs there that it loads from disk, never<br>
touching the disk if the cache is hot.<br>
<br>
- When modules fail to validate, they're removed from the cache.<br>
<br>
- When a CompilerInstance is spawned to build a new module, each<br>
already-loaded PCM is assumed to be valid, and is frozen to avoid<br>
the use-after-free.<br>
<br>
- Any newly-built module is written directly to the cache to avoid the<br>
round-trip to the filesystem, making lock files unnecessary for<br>
correctness.<br>
<br>
Original patch by Manman Ren; most testcases by Adrian Prantl!<br>
<br>
Added:<br>
cfe/trunk/include/clang/Basic/<wbr>MemoryBufferCache.h<br>
cfe/trunk/lib/Basic/<wbr>MemoryBufferCache.cpp<br>
cfe/trunk/test/Modules/Inputs/<wbr>system-out-of-date/X.h<br>
cfe/trunk/test/Modules/Inputs/<wbr>system-out-of-date/Y.h<br>
cfe/trunk/test/Modules/Inputs/<wbr>system-out-of-date/Z.h<br>
cfe/trunk/test/Modules/Inputs/<wbr>system-out-of-date/module.map<br>
cfe/trunk/test/Modules/Inputs/<wbr>warning-mismatch/Mismatch.h<br>
cfe/trunk/test/Modules/Inputs/<wbr>warning-mismatch/System.h<br>
cfe/trunk/test/Modules/Inputs/<wbr>warning-mismatch/module.<wbr>modulemap<br>
cfe/trunk/test/Modules/<wbr>outofdate-rebuild.m<br>
cfe/trunk/test/Modules/system-<wbr>out-of-date-test.m<br>
cfe/trunk/test/Modules/<wbr>warning-mismatch.m<br>
cfe/trunk/unittests/Basic/<wbr>MemoryBufferCacheTest.cpp<br>
Modified:<br>
cfe/trunk/include/clang/Basic/<wbr>DiagnosticSerializationKinds.<wbr>td<br>
cfe/trunk/include/clang/<wbr>Frontend/ASTUnit.h<br>
cfe/trunk/include/clang/<wbr>Frontend/CompilerInstance.h<br>
cfe/trunk/include/clang/Lex/<wbr>Preprocessor.h<br>
cfe/trunk/include/clang/<wbr>Serialization/ASTReader.h<br>
cfe/trunk/include/clang/<wbr>Serialization/ASTWriter.h<br>
cfe/trunk/include/clang/<wbr>Serialization/Module.h<br>
cfe/trunk/include/clang/<wbr>Serialization/ModuleManager.h<br>
cfe/trunk/lib/Basic/<wbr>CMakeLists.txt<br>
cfe/trunk/lib/Frontend/<wbr>ASTUnit.cpp<br>
cfe/trunk/lib/Frontend/<wbr>CompilerInstance.cpp<br>
cfe/trunk/lib/Lex/<wbr>Preprocessor.cpp<br>
cfe/trunk/lib/Serialization/<wbr>ASTReader.cpp<br>
cfe/trunk/lib/Serialization/<wbr>ASTWriter.cpp<br>
cfe/trunk/lib/Serialization/<wbr>GeneratePCH.cpp<br>
cfe/trunk/lib/Serialization/<wbr>ModuleManager.cpp<br>
cfe/trunk/unittests/Basic/<wbr>CMakeLists.txt<br>
cfe/trunk/unittests/Basic/<wbr>SourceManagerTest.cpp<br>
cfe/trunk/unittests/Lex/<wbr>LexerTest.cpp<br>
cfe/trunk/unittests/Lex/<wbr>PPCallbacksTest.cpp<br>
cfe/trunk/unittests/Lex/<wbr>PPConditionalDirectiveRecordTe<wbr>st.cpp<br>
<br>
Modified: cfe/trunk/include/clang/Basic/<wbr>DiagnosticSerializationKinds.<wbr>td<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/DiagnosticSerializationKinds.td?rev=298278&r1=298277&r2=298278&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/include/<wbr>clang/Basic/<wbr>DiagnosticSerializationKinds.<wbr>td?rev=298278&r1=298277&r2=<wbr>298278&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/include/clang/Basic/<wbr>DiagnosticSerializationKinds.<wbr>td (original)<br>
+++ cfe/trunk/include/clang/Basic/<wbr>DiagnosticSerializationKinds.<wbr>td Mon Mar 20 12:58:26 2017<br>
@@ -176,6 +176,11 @@ def warn_duplicate_module_file_<wbr>extension<br>
"duplicate module file extension block name '%0'">,<br>
InGroup<ModuleFileExtension>;<br>
<br>
+def warn_module_system_bit_<wbr>conflict : Warning<<br>
+ "module file '%0' was validated as a system module and is now being imported "<br>
+ "as a non-system module; any difference in diagnostic options will be ignored">,<br>
+ InGroup<ModuleConflict>;<br>
+<br>
} // let CategoryName<br>
} // let Component<br>
<br>
<br>
Added: cfe/trunk/include/clang/Basic/<wbr>MemoryBufferCache.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/MemoryBufferCache.h?rev=298278&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/include/<wbr>clang/Basic/MemoryBufferCache.<wbr>h?rev=298278&view=auto</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/include/clang/Basic/<wbr>MemoryBufferCache.h (added)<br>
+++ cfe/trunk/include/clang/Basic/<wbr>MemoryBufferCache.h Mon Mar 20 12:58:26 2017<br>
@@ -0,0 +1,80 @@<br>
+//===- MemoryBufferCache.h - Cache for loaded memory buffers ----*- C++ -*-===//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===------------------------<wbr>------------------------------<wbr>----------------===//<br>
+<br>
+#ifndef LLVM_CLANG_BASIC_<wbr>MEMORYBUFFERCACHE_H<br>
+#define LLVM_CLANG_BASIC_<wbr>MEMORYBUFFERCACHE_H<br>
+<br>
+#include "llvm/ADT/IntrusiveRefCntPtr.<wbr>h"<br>
+#include "llvm/ADT/StringMap.h"<br>
+#include <memory><br>
+<br>
+namespace llvm {<br>
+class MemoryBuffer;<br>
+} // end namespace llvm<br>
+<br>
+namespace clang {<br>
+<br>
+/// Manage memory buffers across multiple users.<br>
+///<br>
+/// Ensures that multiple users have a consistent view of each buffer. This is<br>
+/// used by \a CompilerInstance when building PCMs to ensure that each \a<br>
+/// ModuleManager sees the same files.<br>
+///<br>
+/// \a finalizeCurrentBuffers() should be called before creating a new user.<br>
+/// This locks in the current buffers, ensuring that no buffer that has already<br>
+/// been accessed can be purged, preventing use-after-frees.<br>
+class MemoryBufferCache : public llvm::RefCountedBase<<wbr>MemoryBufferCache> {<br>
+ struct BufferEntry {<br>
+ std::unique_ptr<llvm::<wbr>MemoryBuffer> Buffer;<br>
+<br>
+ /// Track the timeline of when this was added to the cache.<br>
+ unsigned Index;<br>
+ };<br>
+<br>
+ /// Cache of buffers.<br>
+ llvm::StringMap<BufferEntry> Buffers;<br>
+<br>
+ /// Monotonically increasing index.<br>
+ unsigned NextIndex = 0;<br>
+<br>
+ /// Bumped to prevent "older" buffers from being removed.<br>
+ unsigned FirstRemovableIndex = 0;<br>
+<br>
+public:<br>
+ /// Store the Buffer under the Filename.<br>
+ ///<br>
+ /// \pre There is not already buffer is not already in the cache.<br>
+ /// \return a reference to the buffer as a convenience.<br>
+ llvm::MemoryBuffer &addBuffer(llvm::StringRef Filename,<br>
+ std::unique_ptr<llvm::<wbr>MemoryBuffer> Buffer);<br>
+<br>
+ /// Try to remove a buffer from the cache.<br>
+ ///<br>
+ /// \return false on success, iff \c !isBufferFinal().<br>
+ bool tryToRemoveBuffer(llvm::<wbr>StringRef Filename);<br>
+<br>
+ /// Get a pointer to the buffer if it exists; else nullptr.<br>
+ llvm::MemoryBuffer *lookupBuffer(llvm::StringRef Filename);<br>
+<br>
+ /// Check whether the buffer is final.<br>
+ ///<br>
+ /// \return true iff \a finalizeCurrentBuffers() has been called since the<br>
+ /// buffer was added. This prevents buffers from being removed.<br>
+ bool isBufferFinal(llvm::StringRef Filename);<br>
+<br>
+ /// Finalize the current buffers in the cache.<br>
+ ///<br>
+ /// Should be called when creating a new user to ensure previous uses aren't<br>
+ /// invalidated.<br>
+ void finalizeCurrentBuffers();<br>
+};<br>
+<br>
+} // end namespace clang<br>
+<br>
+#endif // LLVM_CLANG_BASIC_<wbr>MEMORYBUFFERCACHE_H<br>
<br>
Modified: cfe/trunk/include/clang/<wbr>Frontend/ASTUnit.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Frontend/ASTUnit.h?rev=298278&r1=298277&r2=298278&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/include/<wbr>clang/Frontend/ASTUnit.h?rev=<wbr>298278&r1=298277&r2=298278&<wbr>view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/include/clang/<wbr>Frontend/ASTUnit.h (original)<br>
+++ cfe/trunk/include/clang/<wbr>Frontend/ASTUnit.h Mon Mar 20 12:58:26 2017<br>
@@ -51,6 +51,7 @@ class DiagnosticsEngine;<br>
class FileEntry;<br>
class FileManager;<br>
class HeaderSearch;<br>
+class MemoryBufferCache;<br>
class Preprocessor;<br>
class PCHContainerOperations;<br>
class PCHContainerReader;<br>
@@ -84,6 +85,7 @@ private:<br>
IntrusiveRefCntPtr<<wbr>DiagnosticsEngine> Diagnostics;<br>
IntrusiveRefCntPtr<<wbr>FileManager> FileMgr;<br>
IntrusiveRefCntPtr<<wbr>SourceManager> SourceMgr;<br>
+ IntrusiveRefCntPtr<<wbr>MemoryBufferCache> PCMCache;<br>
std::unique_ptr<HeaderSearch> HeaderInfo;<br>
IntrusiveRefCntPtr<TargetInfo> Target;<br>
std::shared_ptr<Preprocessor> PP;<br>
<br>
Modified: cfe/trunk/include/clang/<wbr>Frontend/CompilerInstance.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Frontend/CompilerInstance.h?rev=298278&r1=298277&r2=298278&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/include/<wbr>clang/Frontend/<wbr>CompilerInstance.h?rev=298278&<wbr>r1=298277&r2=298278&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/include/clang/<wbr>Frontend/CompilerInstance.h (original)<br>
+++ cfe/trunk/include/clang/<wbr>Frontend/CompilerInstance.h Mon Mar 20 12:58:26 2017<br>
@@ -44,6 +44,7 @@ class ExternalASTSource;<br>
class FileEntry;<br>
class FileManager;<br>
class FrontendAction;<br>
+class MemoryBufferCache;<br>
class Module;<br>
class Preprocessor;<br>
class Sema;<br>
@@ -90,6 +91,9 @@ class CompilerInstance : public ModuleLo<br>
/// The source manager.<br>
IntrusiveRefCntPtr<<wbr>SourceManager> SourceMgr;<br>
<br>
+ /// The cache of PCM files.<br>
+ IntrusiveRefCntPtr<<wbr>MemoryBufferCache> PCMCache;<br>
+<br>
/// The preprocessor.<br>
std::shared_ptr<Preprocessor> PP;<br>
<br>
@@ -178,7 +182,7 @@ public:<br>
explicit CompilerInstance(<br>
std::shared_ptr<<wbr>PCHContainerOperations> PCHContainerOps =<br>
std::make_shared<<wbr>PCHContainerOperations>(),<br>
- bool BuildingModule = false);<br>
+ MemoryBufferCache *SharedPCMCache = nullptr);<br>
~CompilerInstance() override;<br>
<br>
/// @name High-Level Operations<br>
@@ -783,6 +787,8 @@ public:<br>
}<br>
<br>
void setExternalSemaSource(<wbr>IntrusiveRefCntPtr<<wbr>ExternalSemaSource> ESS);<br>
+<br>
+ MemoryBufferCache &getPCMCache() const { return *PCMCache; }<br>
};<br>
<br>
} // end namespace clang<br>
<br>
Modified: cfe/trunk/include/clang/Lex/<wbr>Preprocessor.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Lex/Preprocessor.h?rev=298278&r1=298277&r2=298278&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/include/<wbr>clang/Lex/Preprocessor.h?rev=<wbr>298278&r1=298277&r2=298278&<wbr>view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/include/clang/Lex/<wbr>Preprocessor.h (original)<br>
+++ cfe/trunk/include/clang/Lex/<wbr>Preprocessor.h Mon Mar 20 12:58:26 2017<br>
@@ -47,6 +47,7 @@ class ExternalPreprocessorSource;<br>
class FileManager;<br>
class FileEntry;<br>
class HeaderSearch;<br>
+class MemoryBufferCache;<br>
class PragmaNamespace;<br>
class PragmaHandler;<br>
class CommentHandler;<br>
@@ -102,6 +103,7 @@ class Preprocessor {<br>
const TargetInfo *AuxTarget;<br>
FileManager &FileMgr;<br>
SourceManager &SourceMgr;<br>
+ MemoryBufferCache &PCMCache;<br>
std::unique_ptr<ScratchBuffer> ScratchBuf;<br>
HeaderSearch &HeaderInfo;<br>
ModuleLoader &TheModuleLoader;<br>
@@ -652,6 +654,7 @@ class Preprocessor {<br>
public:<br>
Preprocessor(std::shared_ptr<<wbr>PreprocessorOptions> PPOpts,<br>
DiagnosticsEngine &diags, LangOptions &opts, SourceManager &SM,<br>
+ MemoryBufferCache &PCMCache,<br>
HeaderSearch &Headers, ModuleLoader &TheModuleLoader,<br>
IdentifierInfoLookup *IILookup = nullptr,<br>
bool OwnsHeaderSearch = false,<br>
@@ -691,6 +694,7 @@ public:<br>
const TargetInfo *getAuxTargetInfo() const { return AuxTarget; }<br>
FileManager &getFileManager() const { return FileMgr; }<br>
SourceManager &getSourceManager() const { return SourceMgr; }<br>
+ MemoryBufferCache &getPCMCache() const { return PCMCache; }<br>
HeaderSearch &getHeaderSearchInfo() const { return HeaderInfo; }<br>
<br>
IdentifierTable &getIdentifierTable() { return Identifiers; }<br>
<br>
Modified: cfe/trunk/include/clang/<wbr>Serialization/ASTReader.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Serialization/ASTReader.h?rev=298278&r1=298277&r2=298278&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/include/<wbr>clang/Serialization/ASTReader.<wbr>h?rev=298278&r1=298277&r2=<wbr>298278&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/include/clang/<wbr>Serialization/ASTReader.h (original)<br>
+++ cfe/trunk/include/clang/<wbr>Serialization/ASTReader.h Mon Mar 20 12:58:26 2017<br>
@@ -408,6 +408,9 @@ private:<br>
/// \brief The module manager which manages modules and their dependencies<br>
ModuleManager ModuleMgr;<br>
<br>
+ /// The cache that manages memory buffers for PCM files.<br>
+ MemoryBufferCache &PCMCache;<br>
+<br>
/// \brief A dummy identifier resolver used to merge TU-scope declarations in<br>
/// C, for the cases where we don't have a Sema object to provide a real<br>
/// identifier resolver.<br>
<br>
Modified: cfe/trunk/include/clang/<wbr>Serialization/ASTWriter.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Serialization/ASTWriter.h?rev=298278&r1=298277&r2=298278&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/include/<wbr>clang/Serialization/ASTWriter.<wbr>h?rev=298278&r1=298277&r2=<wbr>298278&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/include/clang/<wbr>Serialization/ASTWriter.h (original)<br>
+++ cfe/trunk/include/clang/<wbr>Serialization/ASTWriter.h Mon Mar 20 12:58:26 2017<br>
@@ -54,6 +54,7 @@ class MacroInfo;<br>
class OpaqueValueExpr;<br>
class OpenCLOptions;<br>
class ASTReader;<br>
+class MemoryBufferCache;<br>
class Module;<br>
class ModuleFileExtension;<br>
class ModuleFileExtensionWriter;<br>
@@ -109,6 +110,9 @@ private:<br>
/// The buffer associated with the bitstream.<br>
const SmallVectorImpl<char> &Buffer;<br>
<br>
+ /// \brief The PCM manager which manages memory buffers for pcm files.<br>
+ MemoryBufferCache &PCMCache;<br>
+<br>
/// \brief The ASTContext we're writing.<br>
ASTContext *Context = nullptr;<br>
<br>
@@ -512,6 +516,7 @@ public:<br>
/// \brief Create a new precompiled header writer that outputs to<br>
/// the given bitstream.<br>
ASTWriter(llvm::<wbr>BitstreamWriter &Stream, SmallVectorImpl<char> &Buffer,<br>
+ MemoryBufferCache &PCMCache,<br>
ArrayRef<std::shared_ptr<<wbr>ModuleFileExtension>> Extensions,<br>
bool IncludeTimestamps = true);<br>
~ASTWriter() override;<br>
<br>
Modified: cfe/trunk/include/clang/<wbr>Serialization/Module.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Serialization/Module.h?rev=298278&r1=298277&r2=298278&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/include/<wbr>clang/Serialization/Module.h?<wbr>rev=298278&r1=298277&r2=<wbr>298278&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/include/clang/<wbr>Serialization/Module.h (original)<br>
+++ cfe/trunk/include/clang/<wbr>Serialization/Module.h Mon Mar 20 12:58:26 2017<br>
@@ -163,9 +163,9 @@ public:<br>
/// \brief The generation of which this module file is a part.<br>
unsigned Generation;<br>
<br>
- /// \brief The memory buffer that stores the data associated with<br>
- /// this AST file.<br>
- std::unique_ptr<llvm::<wbr>MemoryBuffer> Buffer;<br>
+ /// The memory buffer that stores the data associated with<br>
+ /// this AST file, owned by the PCMCache in the ModuleManager.<br>
+ llvm::MemoryBuffer *Buffer;<br>
<br>
/// \brief The size of this file, in bits.<br>
uint64_t SizeInBits = 0;<br>
<br>
Modified: cfe/trunk/include/clang/<wbr>Serialization/ModuleManager.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Serialization/ModuleManager.h?rev=298278&r1=298277&r2=298278&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/include/<wbr>clang/Serialization/<wbr>ModuleManager.h?rev=298278&r1=<wbr>298277&r2=298278&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/include/clang/<wbr>Serialization/ModuleManager.h (original)<br>
+++ cfe/trunk/include/clang/<wbr>Serialization/ModuleManager.h Mon Mar 20 12:58:26 2017<br>
@@ -24,6 +24,7 @@<br>
namespace clang {<br>
<br>
class GlobalModuleIndex;<br>
+class MemoryBufferCache;<br>
class ModuleMap;<br>
class PCHContainerReader;<br>
<br>
@@ -51,6 +52,9 @@ class ModuleManager {<br>
/// FileEntry *.<br>
FileManager &FileMgr;<br>
<br>
+ /// Cache of PCM files.<br>
+ IntrusiveRefCntPtr<<wbr>MemoryBufferCache> PCMCache;<br>
+<br>
/// \brief Knows how to unwrap module containers.<br>
const PCHContainerReader &PCHContainerRdr;<br>
<br>
@@ -123,7 +127,7 @@ public:<br>
ModuleReverseIterator;<br>
typedef std::pair<uint32_t, StringRef> ModuleOffset;<br>
<br>
- explicit ModuleManager(FileManager &FileMgr,<br>
+ explicit ModuleManager(FileManager &FileMgr, MemoryBufferCache &PCMCache,<br>
const PCHContainerReader &PCHContainerRdr);<br>
~ModuleManager();<br>
<br>
@@ -290,6 +294,8 @@ public:<br>
<br>
/// \brief View the graphviz representation of the module graph.<br>
void viewGraph();<br>
+<br>
+ MemoryBufferCache &getPCMCache() const { return *PCMCache; }<br>
};<br>
<br>
} } // end namespace clang::serialization<br>
<br>
Modified: cfe/trunk/lib/Basic/<wbr>CMakeLists.txt<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Basic/CMakeLists.txt?rev=298278&r1=298277&r2=298278&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/lib/Basic/<wbr>CMakeLists.txt?rev=298278&r1=<wbr>298277&r2=298278&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/lib/Basic/<wbr>CMakeLists.txt (original)<br>
+++ cfe/trunk/lib/Basic/<wbr>CMakeLists.txt Mon Mar 20 12:58:26 2017<br>
@@ -74,6 +74,7 @@ add_clang_library(clangBasic<br>
FileSystemStatCache.cpp<br>
IdentifierTable.cpp<br>
LangOptions.cpp<br>
+ MemoryBufferCache.cpp<br>
Module.cpp<br>
ObjCRuntime.cpp<br>
OpenMPKinds.cpp<br>
<br>
Added: cfe/trunk/lib/Basic/<wbr>MemoryBufferCache.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Basic/MemoryBufferCache.cpp?rev=298278&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/lib/Basic/<wbr>MemoryBufferCache.cpp?rev=<wbr>298278&view=auto</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/lib/Basic/<wbr>MemoryBufferCache.cpp (added)<br>
+++ cfe/trunk/lib/Basic/<wbr>MemoryBufferCache.cpp Mon Mar 20 12:58:26 2017<br>
@@ -0,0 +1,48 @@<br>
+//===- MemoryBufferCache.cpp - Cache for loaded memory buffers ------------===//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===------------------------<wbr>------------------------------<wbr>----------------===//<br>
+<br>
+#include "clang/Basic/<wbr>MemoryBufferCache.h"<br>
+#include "llvm/Support/MemoryBuffer.h"<br>
+<br>
+using namespace clang;<br>
+<br>
+llvm::MemoryBuffer &<br>
+MemoryBufferCache::addBuffer(<wbr>llvm::StringRef Filename,<br>
+ std::unique_ptr<llvm::<wbr>MemoryBuffer> Buffer) {<br>
+ auto Insertion =<br>
+ Buffers.insert({Filename, BufferEntry{std::move(Buffer), NextIndex++}});<br>
+ assert(Insertion.second && "Already has a buffer");<br>
+ return *Insertion.first->second.<wbr>Buffer;<br>
+}<br>
+<br>
+llvm::MemoryBuffer *MemoryBufferCache::<wbr>lookupBuffer(llvm::StringRef Filename) {<br>
+ auto I = Buffers.find(Filename);<br>
+ if (I == Buffers.end())<br>
+ return nullptr;<br>
+ return I->second.Buffer.get();<br>
+}<br>
+<br>
+bool MemoryBufferCache::<wbr>isBufferFinal(llvm::StringRef Filename) {<br>
+ auto I = Buffers.find(Filename);<br>
+ if (I == Buffers.end())<br>
+ return false;<br>
+ return I->second.Index < FirstRemovableIndex;<br>
+}<br>
+<br>
+bool MemoryBufferCache::<wbr>tryToRemoveBuffer(llvm::<wbr>StringRef Filename) {<br>
+ auto I = Buffers.find(Filename);<br>
+ assert(I != Buffers.end() && "No buffer to remove...");<br>
+ if (I->second.Index < FirstRemovableIndex)<br>
+ return true;<br>
+<br>
+ Buffers.erase(I);<br>
+ return false;<br>
+}<br>
+<br>
+void MemoryBufferCache::<wbr>finalizeCurrentBuffers() { FirstRemovableIndex = NextIndex; }<br>
<br>
Modified: cfe/trunk/lib/Frontend/<wbr>ASTUnit.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/ASTUnit.cpp?rev=298278&r1=298277&r2=298278&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/lib/<wbr>Frontend/ASTUnit.cpp?rev=<wbr>298278&r1=298277&r2=298278&<wbr>view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/lib/Frontend/<wbr>ASTUnit.cpp (original)<br>
+++ cfe/trunk/lib/Frontend/<wbr>ASTUnit.cpp Mon Mar 20 12:58:26 2017<br>
@@ -18,6 +18,7 @@<br>
#include "clang/AST/StmtVisitor.h"<br>
#include "clang/AST/TypeOrdering.h"<br>
#include "clang/Basic/Diagnostic.h"<br>
+#include "clang/Basic/<wbr>MemoryBufferCache.h"<br>
#include "clang/Basic/TargetInfo.h"<br>
#include "clang/Basic/TargetOptions.h"<br>
#include "clang/Basic/<wbr>VirtualFileSystem.h"<br>
@@ -185,7 +186,8 @@ struct ASTUnit::ASTWriterData {<br>
llvm::BitstreamWriter Stream;<br>
ASTWriter Writer;<br>
<br>
- ASTWriterData() : Stream(Buffer), Writer(Stream, Buffer, {}) {}<br>
+ ASTWriterData(<wbr>MemoryBufferCache &PCMCache)<br>
+ : Stream(Buffer), Writer(Stream, Buffer, PCMCache, {}) {}<br>
};<br>
<br>
void ASTUnit::clearFileLevelDecls() {<br>
@@ -681,6 +683,7 @@ std::unique_ptr<ASTUnit> ASTUnit::LoadFr<br>
AST->SourceMgr = new SourceManager(AST-><wbr>getDiagnostics(),<br>
AST->getFileManager(),<br>
UserFilesAreVolatile);<br>
+ AST->PCMCache = new MemoryBufferCache;<br>
AST->HSOpts = std::make_shared<<wbr>HeaderSearchOptions>();<br>
AST->HSOpts->ModuleFormat = PCHContainerRdr.getFormat();<br>
AST->HeaderInfo.reset(new HeaderSearch(AST->HSOpts,<br>
@@ -701,7 +704,7 @@ std::unique_ptr<ASTUnit> ASTUnit::LoadFr<br>
<br>
AST->PP = std::make_shared<Preprocessor><wbr>(<br>
std::move(PPOpts), AST->getDiagnostics(), AST->ASTFileLangOpts,<br>
- AST->getSourceManager(), HeaderInfo, *AST,<br>
+ AST->getSourceManager(), *AST->PCMCache, HeaderInfo, *AST,<br>
/*IILookup=*/nullptr,<br>
/*OwnsHeaderSearch=*/false);<br>
Preprocessor &PP = *AST->PP;<br>
@@ -1727,6 +1730,7 @@ ASTUnit::create(std::shared_<wbr>ptr<Compiler<br>
AST->UserFilesAreVolatile = UserFilesAreVolatile;<br>
AST->SourceMgr = new SourceManager(AST-><wbr>getDiagnostics(), *AST->FileMgr,<br>
UserFilesAreVolatile);<br>
+ AST->PCMCache = new MemoryBufferCache;<br>
<br>
return AST;<br>
}<br>
@@ -1997,6 +2001,7 @@ ASTUnit *ASTUnit::LoadFromCommandLine(<br>
if (!VFS)<br>
return nullptr;<br>
AST->FileMgr = new FileManager(AST-><wbr>FileSystemOpts, VFS);<br>
+ AST->PCMCache = new MemoryBufferCache;<br>
AST->OnlyLocalDecls = OnlyLocalDecls;<br>
AST->CaptureDiagnostics = CaptureDiagnostics;<br>
AST->TUKind = TUKind;<br>
@@ -2008,7 +2013,7 @@ ASTUnit *ASTUnit::LoadFromCommandLine(<br>
AST->StoredDiagnostics.swap(<wbr>StoredDiagnostics);<br>
AST->Invocation = CI;<br>
if (ForSerialization)<br>
- AST->WriterData.reset(new ASTWriterData());<br>
+ AST->WriterData.reset(new ASTWriterData(*AST->PCMCache))<wbr>;<br>
// Zero out now to ease cleanup during crash recovery.<br>
CI = nullptr;<br>
Diags = nullptr;<br>
@@ -2523,7 +2528,8 @@ bool ASTUnit::serialize(raw_ostream &OS)<br>
<br>
SmallString<128> Buffer;<br>
llvm::BitstreamWriter Stream(Buffer);<br>
- ASTWriter Writer(Stream, Buffer, {});<br>
+ MemoryBufferCache PCMCache;<br>
+ ASTWriter Writer(Stream, Buffer, PCMCache, {});<br>
return serializeUnit(Writer, Buffer, getSema(), hasErrors, OS);<br>
}<br>
<br>
<br>
Modified: cfe/trunk/lib/Frontend/<wbr>CompilerInstance.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/CompilerInstance.cpp?rev=298278&r1=298277&r2=298278&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/lib/<wbr>Frontend/CompilerInstance.cpp?<wbr>rev=298278&r1=298277&r2=<wbr>298278&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/lib/Frontend/<wbr>CompilerInstance.cpp (original)<br>
+++ cfe/trunk/lib/Frontend/<wbr>CompilerInstance.cpp Mon Mar 20 12:58:26 2017<br>
@@ -13,6 +13,7 @@<br>
#include "clang/AST/Decl.h"<br>
#include "clang/Basic/Diagnostic.h"<br>
#include "clang/Basic/FileManager.h"<br>
+#include "clang/Basic/<wbr>MemoryBufferCache.h"<br>
#include "clang/Basic/SourceManager.h"<br>
#include "clang/Basic/TargetInfo.h"<br>
#include "clang/Basic/Version.h"<br>
@@ -55,9 +56,15 @@ using namespace clang;<br>
<br>
CompilerInstance::<wbr>CompilerInstance(<br>
std::shared_ptr<<wbr>PCHContainerOperations> PCHContainerOps,<br>
- bool BuildingModule)<br>
- : ModuleLoader(BuildingModule), Invocation(new CompilerInvocation()),<br>
- ThePCHContainerOperations(std:<wbr>:move(PCHContainerOps)) {}<br>
+ MemoryBufferCache *SharedPCMCache)<br>
+ : ModuleLoader(/* BuildingModule = */ SharedPCMCache),<br>
+ Invocation(new CompilerInvocation()),<br>
+ PCMCache(SharedPCMCache ? SharedPCMCache : new MemoryBufferCache),<br>
+ ThePCHContainerOperations(std:<wbr>:move(PCHContainerOps)) {<br>
+ // Don't allow this to invalidate buffers in use by others.<br>
+ if (SharedPCMCache)<br>
+ getPCMCache().<wbr>finalizeCurrentBuffers();<br>
+}<br>
<br>
CompilerInstance::~<wbr>CompilerInstance() {<br>
assert(OutputFiles.empty() && "Still output files in flight?");<br>
@@ -128,6 +135,8 @@ IntrusiveRefCntPtr<ASTReader> CompilerIn<br>
return ModuleManager;<br>
}<br>
void CompilerInstance::<wbr>setModuleManager(<wbr>IntrusiveRefCntPtr<ASTReader> Reader) {<br>
+ assert(PCMCache.get() == &Reader->getModuleManager().<wbr>getPCMCache() &&<br>
+ "Expected ASTReader to use the same PCM cache");<br>
ModuleManager = std::move(Reader);<br>
}<br>
<br>
@@ -370,7 +379,7 @@ void CompilerInstance::<wbr>createPreprocesso<br>
getDiagnostics(), getLangOpts(), &getTarget());<br>
PP = std::make_shared<Preprocessor><wbr>(<br>
Invocation-><wbr>getPreprocessorOptsPtr(), getDiagnostics(), getLangOpts(),<br>
- getSourceManager(), *HeaderInfo, *this, PTHMgr,<br>
+ getSourceManager(), getPCMCache(), *HeaderInfo, *this, PTHMgr,<br>
/*OwnsHeaderSearch=*/true, TUKind);<br>
PP->Initialize(getTarget(), getAuxTarget());<br>
<br>
@@ -1073,9 +1082,11 @@ static bool compileModuleImpl(CompilerIn<br>
Invocation->getModuleHash() && "Module hash mismatch!");<br>
<br>
// Construct a compiler instance that will be used to actually create the<br>
- // module.<br>
+ // module. Since we're sharing a PCMCache,<br>
+ // CompilerInstance::<wbr>CompilerInstance is responsible for finalizing the<br>
+ // buffers to prevent use-after-frees.<br>
CompilerInstance Instance(ImportingInstance.<wbr>getPCHContainerOperations(),<br>
- /*BuildingModule=*/true);<br>
+ &ImportingInstance.<wbr>getPreprocessor().getPCMCache(<wbr>));<br>
auto &Inv = *Invocation;<br>
Instance.setInvocation(std::<wbr>move(Invocation));<br>
<br>
<br>
Modified: cfe/trunk/lib/Lex/<wbr>Preprocessor.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/Preprocessor.cpp?rev=298278&r1=298277&r2=298278&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/lib/Lex/<wbr>Preprocessor.cpp?rev=298278&<wbr>r1=298277&r2=298278&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/lib/Lex/<wbr>Preprocessor.cpp (original)<br>
+++ cfe/trunk/lib/Lex/<wbr>Preprocessor.cpp Mon Mar 20 12:58:26 2017<br>
@@ -70,15 +70,15 @@ ExternalPreprocessorSource::~<wbr>ExternalPre<br>
<br>
Preprocessor::Preprocessor(<wbr>std::shared_ptr<<wbr>PreprocessorOptions> PPOpts,<br>
DiagnosticsEngine &diags, LangOptions &opts,<br>
- SourceManager &SM, HeaderSearch &Headers,<br>
- ModuleLoader &TheModuleLoader,<br>
+ SourceManager &SM, MemoryBufferCache &PCMCache,<br>
+ HeaderSearch &Headers, ModuleLoader &TheModuleLoader,<br>
IdentifierInfoLookup *IILookup, bool OwnsHeaders,<br>
TranslationUnitKind TUKind)<br>
: PPOpts(std::move(PPOpts)), Diags(&diags), LangOpts(opts), Target(nullptr),<br>
AuxTarget(nullptr), FileMgr(Headers.getFileMgr()), SourceMgr(SM),<br>
- ScratchBuf(new ScratchBuffer(SourceMgr)), HeaderInfo(Headers),<br>
- TheModuleLoader(<wbr>TheModuleLoader), ExternalSource(nullptr),<br>
- Identifiers(opts, IILookup),<br>
+ PCMCache(PCMCache), ScratchBuf(new ScratchBuffer(SourceMgr)),<br>
+ HeaderInfo(Headers), TheModuleLoader(<wbr>TheModuleLoader),<br>
+ ExternalSource(nullptr), Identifiers(opts, IILookup),<br>
PragmaHandlers(new PragmaNamespace(StringRef())),<br>
IncrementalProcessing(false), TUKind(TUKind), CodeComplete(nullptr),<br>
CodeCompletionFile(nullptr), CodeCompletionOffset(0),<br>
<br>
Modified: cfe/trunk/lib/Serialization/<wbr>ASTReader.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Serialization/ASTReader.cpp?rev=298278&r1=298277&r2=298278&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/lib/<wbr>Serialization/ASTReader.cpp?<wbr>rev=298278&r1=298277&r2=<wbr>298278&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/lib/Serialization/<wbr>ASTReader.cpp (original)<br>
+++ cfe/trunk/lib/Serialization/<wbr>ASTReader.cpp Mon Mar 20 12:58:26 2017<br>
@@ -37,6 +37,7 @@<br>
#include "clang/Basic/FileManager.h"<br>
#include "clang/Basic/<wbr>FileSystemOptions.h"<br>
#include "clang/Basic/LangOptions.h"<br>
+#include "clang/Basic/<wbr>MemoryBufferCache.h"<br>
#include "clang/Basic/ObjCRuntime.h"<br>
#include "clang/Basic/OperatorKinds.h"<br>
#include "clang/Basic/Sanitizers.h"<br>
@@ -463,19 +464,9 @@ static bool checkDiagnosticMappings(Diag<br>
return checkDiagnosticGroupMappings(<wbr>StoredDiags, Diags, Complain);<br>
}<br>
<br>
-bool PCHValidator::<wbr>ReadDiagnosticOptions(<br>
- IntrusiveRefCntPtr<<wbr>DiagnosticOptions> DiagOpts, bool Complain) {<br>
- DiagnosticsEngine &ExistingDiags = PP.getDiagnostics();<br>
- IntrusiveRefCntPtr<<wbr>DiagnosticIDs> DiagIDs(ExistingDiags.<wbr>getDiagnosticIDs());<br>
- IntrusiveRefCntPtr<<wbr>DiagnosticsEngine> Diags(<br>
- new DiagnosticsEngine(DiagIDs, DiagOpts.get()));<br>
- // This should never fail, because we would have processed these options<br>
- // before writing them to an ASTFile.<br>
- ProcessWarningOptions(*Diags, *DiagOpts, /*Report*/false);<br>
-<br>
- ModuleManager &ModuleMgr = Reader.getModuleManager();<br>
- assert(ModuleMgr.size() >= 1 && "what ASTFile is this then");<br>
-<br>
+/// Return the top import module if it is implicit, nullptr otherwise.<br>
+static Module *getTopImportImplicitModule(<wbr>ModuleManager &ModuleMgr,<br>
+ Preprocessor &PP) {<br>
// If the original import came from a file explicitly generated by the user,<br>
// don't check the diagnostic mappings.<br>
// FIXME: currently this is approximated by checking whether this is not a<br>
@@ -487,17 +478,37 @@ bool PCHValidator::<wbr>ReadDiagnosticOptions<br>
while (!TopImport->ImportedBy.empty(<wbr>))<br>
TopImport = TopImport->ImportedBy[0];<br>
if (TopImport->Kind != MK_ImplicitModule)<br>
- return false;<br>
+ return nullptr;<br>
<br>
StringRef ModuleName = TopImport->ModuleName;<br>
assert(!ModuleName.empty() && "diagnostic options read before module name");<br>
<br>
Module *M = PP.getHeaderSearchInfo().<wbr>lookupModule(ModuleName);<br>
assert(M && "missing module");<br>
+ return M;<br>
+}<br>
+<br>
+bool PCHValidator::<wbr>ReadDiagnosticOptions(<br>
+ IntrusiveRefCntPtr<<wbr>DiagnosticOptions> DiagOpts, bool Complain) {<br>
+ DiagnosticsEngine &ExistingDiags = PP.getDiagnostics();<br>
+ IntrusiveRefCntPtr<<wbr>DiagnosticIDs> DiagIDs(ExistingDiags.<wbr>getDiagnosticIDs());<br>
+ IntrusiveRefCntPtr<<wbr>DiagnosticsEngine> Diags(<br>
+ new DiagnosticsEngine(DiagIDs, DiagOpts.get()));<br>
+ // This should never fail, because we would have processed these options<br>
+ // before writing them to an ASTFile.<br>
+ ProcessWarningOptions(*Diags, *DiagOpts, /*Report*/false);<br>
+<br>
+ ModuleManager &ModuleMgr = Reader.getModuleManager();<br>
+ assert(ModuleMgr.size() >= 1 && "what ASTFile is this then");<br>
+<br>
+ Module *TopM = getTopImportImplicitModule(<wbr>ModuleMgr, PP);<br>
+ if (!TopM)<br>
+ return false;<br>
<br>
// FIXME: if the diagnostics are incompatible, save a DiagnosticOptions that<br>
// contains the union of their flags.<br>
- return checkDiagnosticMappings(*<wbr>Diags, ExistingDiags, M->IsSystem, Complain);<br>
+ return checkDiagnosticMappings(*<wbr>Diags, ExistingDiags, TopM->IsSystem,<br>
+ Complain);<br>
}<br>
<br>
/// \brief Collect the macro definitions provided by the given preprocessor<br>
@@ -4064,12 +4075,41 @@ ASTReader::<wbr>readUnhashedControlBlock(Modu<br>
Listener.get(),<br>
WasImportedBy ? false : HSOpts.<wbr>ModulesValidateDiagnosticOptio<wbr>ns);<br>
<br>
+ // If F was directly imported by another module, it's implicitly validated by<br>
+ // the importing module.<br>
if (DisableValidation || WasImportedBy ||<br>
(AllowConfigurationMismatch && Result == ConfigurationMismatch))<br>
return Success;<br>
<br>
- if (Result == Failure)<br>
+ if (Result == Failure) {<br>
Error("malformed block record in AST file");<br>
+ return Failure;<br>
+ }<br>
+<br>
+ if (Result == OutOfDate && F.Kind == MK_ImplicitModule) {<br>
+ // If this module has already been finalized in the PCMCache, we're stuck<br>
+ // with it; we can only load a single version of each module.<br>
+ //<br>
+ // This can happen when a module is imported in two contexts: in one, as a<br>
+ // user module; in another, as a system module (due to an import from<br>
+ // another module marked with the [system] flag). It usually indicates a<br>
+ // bug in the module map: this module should also be marked with [system].<br>
+ //<br>
+ // If -Wno-system-headers (the default), and the first import is as a<br>
+ // system module, then validation will fail during the as-user import,<br>
+ // since -Werror flags won't have been validated. However, it's reasonable<br>
+ // to treat this consistently as a system module.<br>
+ //<br>
+ // If -Wsystem-headers, the PCM on disk was built with<br>
+ // -Wno-system-headers, and the first import is as a user module, then<br>
+ // validation will fail during the as-system import since the PCM on disk<br>
+ // doesn't guarantee that -Werror was respected. However, the -Werror<br>
+ // flags were checked during the initial as-user import.<br>
+ if (PCMCache.isBufferFinal(F.<wbr>FileName)) {<br>
+ Diag(diag::warn_module_system_<wbr>bit_conflict) << F.FileName;<br>
+ return Success;<br>
+ }<br>
+ }<br>
<br>
return Result;<br>
}<br>
@@ -4122,7 +4162,7 @@ ASTReader::ASTReadResult ASTReader::read<br>
if (Listener && ValidateDiagnosticOptions &&<br>
!<wbr>AllowCompatibleConfigurationMi<wbr>smatch &&<br>
ParseDiagnosticOptions(Record, Complain, *Listener))<br>
- return OutOfDate;<br>
+ Result = OutOfDate; // Don't return early. Read the signature.<br>
break;<br>
}<br>
case DIAG_PRAGMA_MAPPINGS:<br>
@@ -7301,7 +7341,7 @@ LLVM_DUMP_METHOD void ASTReader::dump()<br>
/// by heap-backed versus mmap'ed memory.<br>
void ASTReader::<wbr>getMemoryBufferSizes(<wbr>MemoryBufferSizes &sizes) const {<br>
for (ModuleFile &I : ModuleMgr) {<br>
- if (llvm::MemoryBuffer *buf = I.Buffer.get()) {<br>
+ if (llvm::MemoryBuffer *buf = I.Buffer) {<br>
size_t bytes = buf->getBufferSize();<br>
switch (buf->getBufferKind()) {<br>
case llvm::MemoryBuffer::<wbr>MemoryBuffer_Malloc:<br>
@@ -9628,8 +9668,10 @@ ASTReader::ASTReader(<wbr>Preprocessor &PP, A<br>
: cast<ASTReaderListener>(new PCHValidator(PP, *this))),<br>
SourceMgr(PP.getSourceManager(<wbr>)), FileMgr(PP.getFileManager()),<br>
PCHContainerRdr(<wbr>PCHContainerRdr), Diags(PP.getDiagnostics()), PP(PP),<br>
- Context(Context), ModuleMgr(PP.getFileManager(), PCHContainerRdr),<br>
- DummyIdResolver(PP), ReadTimer(std::move(ReadTimer)<wbr>), isysroot(isysroot),<br>
+ Context(Context),<br>
+ ModuleMgr(PP.getFileManager(), PP.getPCMCache(), PCHContainerRdr),<br>
+ PCMCache(PP.getPCMCache()), DummyIdResolver(PP),<br>
+ ReadTimer(std::move(ReadTimer)<wbr>), isysroot(isysroot),<br>
DisableValidation(<wbr>DisableValidation),<br>
AllowASTWithCompilerErrors(<wbr>AllowASTWithCompilerErrors),<br>
AllowConfigurationMismatch(<wbr>AllowConfigurationMismatch),<br>
<br>
Modified: cfe/trunk/lib/Serialization/<wbr>ASTWriter.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Serialization/ASTWriter.cpp?rev=298278&r1=298277&r2=298278&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/lib/<wbr>Serialization/ASTWriter.cpp?<wbr>rev=298278&r1=298277&r2=<wbr>298278&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/lib/Serialization/<wbr>ASTWriter.cpp (original)<br>
+++ cfe/trunk/lib/Serialization/<wbr>ASTWriter.cpp Mon Mar 20 12:58:26 2017<br>
@@ -35,6 +35,7 @@<br>
#include "clang/Basic/<wbr>FileSystemOptions.h"<br>
#include "clang/Basic/LLVM.h"<br>
#include "clang/Basic/LangOptions.h"<br>
+#include "clang/Basic/<wbr>MemoryBufferCache.h"<br>
#include "clang/Basic/Module.h"<br>
#include "clang/Basic/ObjCRuntime.h"<br>
#include "clang/Basic/SourceManager.h"<br>
@@ -4304,10 +4305,11 @@ void ASTWriter::SetSelectorOffset(<wbr>Select<br>
}<br>
<br>
ASTWriter::ASTWriter(llvm::<wbr>BitstreamWriter &Stream,<br>
- SmallVectorImpl<char> &Buffer,<br>
+ SmallVectorImpl<char> &Buffer, MemoryBufferCache &PCMCache,<br>
ArrayRef<std::shared_ptr<<wbr>ModuleFileExtension>> Extensions,<br>
bool IncludeTimestamps)<br>
- : Stream(Stream), Buffer(Buffer), IncludeTimestamps(<wbr>IncludeTimestamps) {<br>
+ : Stream(Stream), Buffer(Buffer), PCMCache(PCMCache),<br>
+ IncludeTimestamps(<wbr>IncludeTimestamps) {<br>
for (const auto &Ext : Extensions) {<br>
if (auto Writer = Ext->createExtensionWriter(*<wbr>this))<br>
ModuleFileExtensionWriters.<wbr>push_back(std::move(Writer));<br>
@@ -4354,6 +4356,12 @@ ASTFileSignature ASTWriter::WriteAST(Sem<br>
this->BaseDirectory.clear();<br>
<br>
WritingAST = false;<br>
+ if (SemaRef.Context.getLangOpts()<wbr>.ImplicitModules && WritingModule) {<br>
+ // Construct MemoryBuffer and update buffer manager.<br>
+ PCMCache.addBuffer(OutputFile,<br>
+ llvm::MemoryBuffer::<wbr>getMemBufferCopy(<br>
+ StringRef(Buffer.begin(), Buffer.size())));<br>
+ }<br>
return Signature;<br>
}<br>
<br>
<br>
Modified: cfe/trunk/lib/Serialization/<wbr>GeneratePCH.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Serialization/GeneratePCH.cpp?rev=298278&r1=298277&r2=298278&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/lib/<wbr>Serialization/GeneratePCH.cpp?<wbr>rev=298278&r1=298277&r2=<wbr>298278&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/lib/Serialization/<wbr>GeneratePCH.cpp (original)<br>
+++ cfe/trunk/lib/Serialization/<wbr>GeneratePCH.cpp Mon Mar 20 12:58:26 2017<br>
@@ -28,7 +28,8 @@ PCHGenerator::PCHGenerator(<br>
bool AllowASTWithErrors, bool IncludeTimestamps)<br>
: PP(PP), OutputFile(OutputFile), isysroot(isysroot.str()),<br>
SemaPtr(nullptr), Buffer(Buffer), Stream(Buffer->Data),<br>
- Writer(Stream, Buffer->Data, Extensions, IncludeTimestamps),<br>
+ Writer(Stream, Buffer->Data, PP.getPCMCache(), Extensions,<br>
+ IncludeTimestamps),<br>
AllowASTWithErrors(<wbr>AllowASTWithErrors) {<br>
Buffer->IsComplete = false;<br>
}<br>
<br>
Modified: cfe/trunk/lib/Serialization/<wbr>ModuleManager.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Serialization/ModuleManager.cpp?rev=298278&r1=298277&r2=298278&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/lib/<wbr>Serialization/ModuleManager.<wbr>cpp?rev=298278&r1=298277&r2=<wbr>298278&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/lib/Serialization/<wbr>ModuleManager.cpp (original)<br>
+++ cfe/trunk/lib/Serialization/<wbr>ModuleManager.cpp Mon Mar 20 12:58:26 2017<br>
@@ -12,6 +12,7 @@<br>
//<br>
//===-------------------------<wbr>------------------------------<wbr>---------------===//<br>
#include "clang/Serialization/<wbr>ModuleManager.h"<br>
+#include "clang/Basic/<wbr>MemoryBufferCache.h"<br>
#include "clang/Frontend/<wbr>PCHContainerOperations.h"<br>
#include "clang/Lex/HeaderSearch.h"<br>
#include "clang/Lex/ModuleMap.h"<br>
@@ -137,7 +138,9 @@ ModuleManager::addModule(<wbr>StringRef FileN<br>
// Load the contents of the module<br>
if (std::unique_ptr<llvm::<wbr>MemoryBuffer> Buffer = lookupBuffer(FileName)) {<br>
// The buffer was already provided for us.<br>
- NewModule->Buffer = std::move(Buffer);<br>
+ NewModule->Buffer = &PCMCache->addBuffer(FileName, std::move(Buffer));<br>
+ } else if (llvm::MemoryBuffer *Buffer = PCMCache->lookupBuffer(<wbr>FileName)) {<br>
+ NewModule->Buffer = Buffer;<br>
} else {<br>
// Open the AST file.<br>
llvm::ErrorOr<std::unique_ptr<<wbr>llvm::MemoryBuffer>> Buf((std::error_code()));<br>
@@ -158,7 +161,7 @@ ModuleManager::addModule(<wbr>StringRef FileN<br>
return Missing;<br>
}<br>
<br>
- NewModule->Buffer = std::move(*Buf);<br>
+ NewModule->Buffer = &PCMCache->addBuffer(FileName, std::move(*Buf));<br>
}<br>
<br>
// Initialize the stream.<br>
@@ -167,8 +170,13 @@ ModuleManager::addModule(<wbr>StringRef FileN<br>
// Read the signature eagerly now so that we can check it. Avoid calling<br>
// ReadSignature unless there's something to check though.<br>
if (ExpectedSignature && checkSignature(ReadSignature(<wbr>NewModule->Data),<br>
- ExpectedSignature, ErrorStr))<br>
+ ExpectedSignature, ErrorStr)) {<br>
+ // Try to remove the buffer. If it can't be removed, then it was already<br>
+ // validated by this process.<br>
+ if (!PCMCache->tryToRemoveBuffer(<wbr>NewModule->FileName))<br>
+ FileMgr.invalidateCache(<wbr>NewModule->File);<br>
return OutOfDate;<br>
+ }<br>
<br>
// We're keeping this module. Store it everywhere.<br>
Module = Modules[Entry] = NewModule.get();<br>
@@ -235,7 +243,12 @@ void ModuleManager::removeModules(<br>
// Files that didn't make it through ReadASTCore successfully will be<br>
// rebuilt (or there was an error). Invalidate them so that we can load the<br>
// new files that will be renamed over the old ones.<br>
- if (LoadedSuccessfully.count(&*<wbr>victim) == 0)<br>
+ //<br>
+ // The PCMCache tracks whether the module was succesfully loaded in another<br>
+ // thread/context; in that case, it won't need to be rebuilt (and we can't<br>
+ // safely invalidate it anyway).<br>
+ if (LoadedSuccessfully.count(&*<wbr>victim) == 0 &&<br>
+ !PCMCache->tryToRemoveBuffer(<wbr>victim->FileName))<br>
FileMgr.invalidateCache(<wbr>victim->File);<br>
}<br>
<br>
@@ -292,10 +305,10 @@ void ModuleManager::<wbr>moduleFileAccepted(M<br>
ModulesInCommonWithGlobalIndex<wbr>.push_back(MF);<br>
}<br>
<br>
-ModuleManager::ModuleManager(<wbr>FileManager &FileMgr,<br>
+ModuleManager::ModuleManager(<wbr>FileManager &FileMgr, MemoryBufferCache &PCMCache,<br>
const PCHContainerReader &PCHContainerRdr)<br>
- : FileMgr(FileMgr), PCHContainerRdr(<wbr>PCHContainerRdr), GlobalIndex(),<br>
- FirstVisitState(nullptr) {}<br>
+ : FileMgr(FileMgr), PCMCache(&PCMCache), PCHContainerRdr(<wbr>PCHContainerRdr),<br>
+ GlobalIndex(), FirstVisitState(nullptr) {}<br>
<br>
ModuleManager::~ModuleManager(<wbr>) { delete FirstVisitState; }<br>
<br>
<br>
Added: cfe/trunk/test/Modules/Inputs/<wbr>system-out-of-date/X.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/Inputs/system-out-of-date/X.h?rev=298278&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/test/<wbr>Modules/Inputs/system-out-of-<wbr>date/X.h?rev=298278&view=auto</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/test/Modules/Inputs/<wbr>system-out-of-date/X.h (added)<br>
+++ cfe/trunk/test/Modules/Inputs/<wbr>system-out-of-date/X.h Mon Mar 20 12:58:26 2017<br>
@@ -0,0 +1 @@<br>
+#import <Y.h><br>
<br>
Added: cfe/trunk/test/Modules/Inputs/<wbr>system-out-of-date/Y.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/Inputs/system-out-of-date/Y.h?rev=298278&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/test/<wbr>Modules/Inputs/system-out-of-<wbr>date/Y.h?rev=298278&view=auto</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/test/Modules/Inputs/<wbr>system-out-of-date/Y.h (added)<br>
+++ cfe/trunk/test/Modules/Inputs/<wbr>system-out-of-date/Y.h Mon Mar 20 12:58:26 2017<br>
@@ -0,0 +1 @@<br>
+//empty<br>
<br>
Added: cfe/trunk/test/Modules/Inputs/<wbr>system-out-of-date/Z.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/Inputs/system-out-of-date/Z.h?rev=298278&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/test/<wbr>Modules/Inputs/system-out-of-<wbr>date/Z.h?rev=298278&view=auto</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/test/Modules/Inputs/<wbr>system-out-of-date/Z.h (added)<br>
+++ cfe/trunk/test/Modules/Inputs/<wbr>system-out-of-date/Z.h Mon Mar 20 12:58:26 2017<br>
@@ -0,0 +1 @@<br>
+#import <Y.h><br>
<br>
Added: cfe/trunk/test/Modules/Inputs/<wbr>system-out-of-date/module.map<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/Inputs/system-out-of-date/module.map?rev=298278&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/test/<wbr>Modules/Inputs/system-out-of-<wbr>date/module.map?rev=298278&<wbr>view=auto</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/test/Modules/Inputs/<wbr>system-out-of-date/module.map (added)<br>
+++ cfe/trunk/test/Modules/Inputs/<wbr>system-out-of-date/module.map Mon Mar 20 12:58:26 2017<br>
@@ -0,0 +1,12 @@<br>
+module X [system] {<br>
+ header "X.h" // imports Y<br>
+ export *<br>
+}<br>
+module Y {<br>
+ header "Y.h"<br>
+ export *<br>
+}<br>
+module Z {<br>
+ header "Z.h" // imports Y<br>
+ export *<br>
+}<br>
<br>
Added: cfe/trunk/test/Modules/Inputs/<wbr>warning-mismatch/Mismatch.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/Inputs/warning-mismatch/Mismatch.h?rev=298278&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/test/<wbr>Modules/Inputs/warning-<wbr>mismatch/Mismatch.h?rev=<wbr>298278&view=auto</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/test/Modules/Inputs/<wbr>warning-mismatch/Mismatch.h (added)<br>
+++ cfe/trunk/test/Modules/Inputs/<wbr>warning-mismatch/Mismatch.h Mon Mar 20 12:58:26 2017<br>
@@ -0,0 +1 @@<br>
+struct Mismatch { int i; };<br>
<br>
Added: cfe/trunk/test/Modules/Inputs/<wbr>warning-mismatch/System.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/Inputs/warning-mismatch/System.h?rev=298278&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/test/<wbr>Modules/Inputs/warning-<wbr>mismatch/System.h?rev=298278&<wbr>view=auto</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/test/Modules/Inputs/<wbr>warning-mismatch/System.h (added)<br>
+++ cfe/trunk/test/Modules/Inputs/<wbr>warning-mismatch/System.h Mon Mar 20 12:58:26 2017<br>
@@ -0,0 +1,2 @@<br>
+#import "Mismatch.h"<br>
+struct System { int i; };<br>
<br>
Added: cfe/trunk/test/Modules/Inputs/<wbr>warning-mismatch/module.<wbr>modulemap<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/Inputs/warning-mismatch/module.modulemap?rev=298278&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/test/<wbr>Modules/Inputs/warning-<wbr>mismatch/module.modulemap?rev=<wbr>298278&view=auto</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/test/Modules/Inputs/<wbr>warning-mismatch/module.<wbr>modulemap (added)<br>
+++ cfe/trunk/test/Modules/Inputs/<wbr>warning-mismatch/module.<wbr>modulemap Mon Mar 20 12:58:26 2017<br>
@@ -0,0 +1,7 @@<br>
+module System [system] {<br>
+ header "System.h"<br>
+}<br>
+<br>
+module Mismatch {<br>
+ header "Mismatch.h"<br>
+}<br>
<br>
Added: cfe/trunk/test/Modules/<wbr>outofdate-rebuild.m<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/outofdate-rebuild.m?rev=298278&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/test/<wbr>Modules/outofdate-rebuild.m?<wbr>rev=298278&view=auto</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/test/Modules/<wbr>outofdate-rebuild.m (added)<br>
+++ cfe/trunk/test/Modules/<wbr>outofdate-rebuild.m Mon Mar 20 12:58:26 2017<br>
@@ -0,0 +1,15 @@<br>
+// RUN: rm -rf %t.cache<br>
+// RUN: echo "@import CoreText;" > %t.m<br>
+// RUN: %clang_cc1 -fdisable-module-hash -fmodules-cache-path=%t.cache \<br>
+// RUN: -fmodules -fimplicit-module-maps -I%S/Inputs/outofdate-rebuild %s \<br>
+// RUN: -fsyntax-only<br>
+// RUN: %clang_cc1 -DMISMATCH -Werror -fdisable-module-hash \<br>
+// RUN: -fmodules-cache-path=%t.cache -fmodules -fimplicit-module-maps \<br>
+// RUN: -I%S/Inputs/outofdate-rebuild %t.m -fsyntax-only<br>
+// RUN: %clang_cc1 -fdisable-module-hash -fmodules-cache-path=%t.cache \<br>
+// RUN: -fmodules -fimplicit-module-maps -I%S/Inputs/outofdate-rebuild %s \<br>
+// RUN: -fsyntax-only<br>
+<br>
+// This testcase reproduces a use-after-free in when ModuleManager removes an<br>
+// entry from the PCMCache without notifying its parent ASTReader.<br>
+@import Cocoa;<br>
<br>
Added: cfe/trunk/test/Modules/system-<wbr>out-of-date-test.m<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/system-out-of-date-test.m?rev=298278&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/test/<wbr>Modules/system-out-of-date-<wbr>test.m?rev=298278&view=auto</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/test/Modules/system-<wbr>out-of-date-test.m (added)<br>
+++ cfe/trunk/test/Modules/system-<wbr>out-of-date-test.m Mon Mar 20 12:58:26 2017<br>
@@ -0,0 +1,17 @@<br>
+// RUN: rm -rf %t.cache<br>
+// RUN: echo '@import X;' | \<br>
+// RUN: %clang_cc1 -fmodules -fimplicit-module-maps \<br>
+// RUN: -fmodules-cache-path=%t.cache -I%S/Inputs/system-out-of-date \<br>
+// RUN: -fsyntax-only -x objective-c -<br>
+//<br>
+// Build something with different diagnostic options.<br>
+// RUN: %clang_cc1 -fmodules -fimplicit-module-maps \<br>
+// RUN: -fmodules-cache-path=%t.cache -I%S/Inputs/system-out-of-date \<br>
+// RUN: -fsyntax-only %s -Wnon-modular-include-in-<wbr>framework-module \<br>
+// RUN: -Werror=non-modular-include-<wbr>in-framework-module 2>&1 \<br>
+// RUN: | FileCheck %s<br>
+@import X;<br>
+<br>
+#import <Z.h><br>
+// CHECK: While building module 'Z' imported from<br>
+// CHECK: {{.*}}Y-{{.*}}pcm' was validated as a system module and is now being imported as a non-system module<br>
<br>
Added: cfe/trunk/test/Modules/<wbr>warning-mismatch.m<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/warning-mismatch.m?rev=298278&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/test/<wbr>Modules/warning-mismatch.m?<wbr>rev=298278&view=auto</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/test/Modules/<wbr>warning-mismatch.m (added)<br>
+++ cfe/trunk/test/Modules/<wbr>warning-mismatch.m Mon Mar 20 12:58:26 2017<br>
@@ -0,0 +1,13 @@<br>
+// RUN: rm -rf %t.cache<br>
+// RUN: echo "@import Mismatch;" >%t.m<br>
+// RUN: %clang_cc1 -Wno-system-headers -fdisable-module-hash \<br>
+// RUN: -fmodules-cache-path=%t.cache -fmodules -fimplicit-module-maps \<br>
+// RUN: -I%S/Inputs/warning-mismatch %t.m -fsyntax-only<br>
+// RUN: %clang_cc1 -Wsystem-headers -fdisable-module-hash \<br>
+// RUN: -fmodules-cache-path=%t.cache -fmodules -fimplicit-module-maps \<br>
+// RUN: -I%S/Inputs/warning-mismatch %s -fsyntax-only<br>
+<br>
+// This testcase triggers a warning flag mismatch in an already validated<br>
+// header.<br>
+@import Mismatch;<br>
+@import System;<br>
<br>
Modified: cfe/trunk/unittests/Basic/<wbr>CMakeLists.txt<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Basic/CMakeLists.txt?rev=298278&r1=298277&r2=298278&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/unittests/<wbr>Basic/CMakeLists.txt?rev=<wbr>298278&r1=298277&r2=298278&<wbr>view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/unittests/Basic/<wbr>CMakeLists.txt (original)<br>
+++ cfe/trunk/unittests/Basic/<wbr>CMakeLists.txt Mon Mar 20 12:58:26 2017<br>
@@ -6,6 +6,7 @@ add_clang_unittest(BasicTests<br>
CharInfoTest.cpp<br>
DiagnosticTest.cpp<br>
FileManagerTest.cpp<br>
+ MemoryBufferCacheTest.cpp<br>
SourceManagerTest.cpp<br>
VirtualFileSystemTest.cpp<br>
)<br>
<br>
Added: cfe/trunk/unittests/Basic/<wbr>MemoryBufferCacheTest.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Basic/MemoryBufferCacheTest.cpp?rev=298278&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/unittests/<wbr>Basic/MemoryBufferCacheTest.<wbr>cpp?rev=298278&view=auto</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/unittests/Basic/<wbr>MemoryBufferCacheTest.cpp (added)<br>
+++ cfe/trunk/unittests/Basic/<wbr>MemoryBufferCacheTest.cpp Mon Mar 20 12:58:26 2017<br>
@@ -0,0 +1,95 @@<br>
+//===- MemoryBufferCacheTest.cpp - MemoryBufferCache tests ----------------===//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===------------------------<wbr>------------------------------<wbr>----------------===//<br>
+<br>
+#include "clang/Basic/<wbr>MemoryBufferCache.h"<br>
+#include "llvm/Support/MemoryBuffer.h"<br>
+#include "gtest/gtest.h"<br>
+<br>
+using namespace llvm;<br>
+using namespace clang;<br>
+<br>
+namespace {<br>
+<br>
+std::unique_ptr<MemoryBuffer> getBuffer(int I) {<br>
+ SmallVector<char, 8> Bytes;<br>
+ raw_svector_ostream(Bytes) << "data:" << I;<br>
+ return MemoryBuffer::getMemBuffer(<wbr>StringRef(Bytes.data(), Bytes.size()), "",<br>
+ /* RequiresNullTerminator = */ false);<br>
+}<br>
+<br>
+TEST(MemoryBufferCacheTest, addBuffer) {<br>
+ auto B1 = getBuffer(1);<br>
+ auto B2 = getBuffer(2);<br>
+ auto B3 = getBuffer(3);<br>
+ auto *RawB1 = B1.get();<br>
+ auto *RawB2 = B2.get();<br>
+ auto *RawB3 = B3.get();<br>
+<br>
+ // Add a few buffers.<br>
+ MemoryBufferCache Cache;<br>
+ EXPECT_EQ(RawB1, &Cache.addBuffer("1", std::move(B1)));<br>
+ EXPECT_EQ(RawB2, &Cache.addBuffer("2", std::move(B2)));<br>
+ EXPECT_EQ(RawB3, &Cache.addBuffer("3", std::move(B3)));<br>
+ EXPECT_EQ(RawB1, Cache.lookupBuffer("1"));<br>
+ EXPECT_EQ(RawB2, Cache.lookupBuffer("2"));<br>
+ EXPECT_EQ(RawB3, Cache.lookupBuffer("3"));<br>
+ EXPECT_FALSE(Cache.<wbr>isBufferFinal("1"));<br>
+ EXPECT_FALSE(Cache.<wbr>isBufferFinal("2"));<br>
+ EXPECT_FALSE(Cache.<wbr>isBufferFinal("3"));<br>
+<br>
+ // Remove the middle buffer.<br>
+ EXPECT_FALSE(Cache.<wbr>tryToRemoveBuffer("2"));<br>
+ EXPECT_EQ(nullptr, Cache.lookupBuffer("2"));<br>
+ EXPECT_FALSE(Cache.<wbr>isBufferFinal("2"));<br>
+<br>
+ // Replace the middle buffer.<br>
+ B2 = getBuffer(2);<br>
+ ASSERT_NE(RawB2, B2.get());<br>
+ RawB2 = B2.get();<br>
+ EXPECT_EQ(RawB2, &Cache.addBuffer("2", std::move(B2)));<br>
+<br>
+ // Check that nothing is final.<br>
+ EXPECT_FALSE(Cache.<wbr>isBufferFinal("1"));<br>
+ EXPECT_FALSE(Cache.<wbr>isBufferFinal("2"));<br>
+ EXPECT_FALSE(Cache.<wbr>isBufferFinal("3"));<br>
+}<br>
+<br>
+TEST(MemoryBufferCacheTest, finalizeCurrentBuffers) {<br>
+ // Add a buffer.<br>
+ MemoryBufferCache Cache;<br>
+ auto B1 = getBuffer(1);<br>
+ auto *RawB1 = B1.get();<br>
+ Cache.addBuffer("1", std::move(B1));<br>
+ ASSERT_FALSE(Cache.<wbr>isBufferFinal("1"));<br>
+<br>
+ // Finalize it.<br>
+ Cache.finalizeCurrentBuffers()<wbr>;<br>
+ EXPECT_TRUE(Cache.<wbr>isBufferFinal("1"));<br>
+ EXPECT_TRUE(Cache.<wbr>tryToRemoveBuffer("1"));<br>
+ EXPECT_EQ(RawB1, Cache.lookupBuffer("1"));<br>
+ EXPECT_TRUE(Cache.<wbr>isBufferFinal("1"));<br>
+<br>
+ // Repeat.<br>
+ auto B2 = getBuffer(2);<br>
+ auto *RawB2 = B2.get();<br>
+ Cache.addBuffer("2", std::move(B2));<br>
+ EXPECT_FALSE(Cache.<wbr>isBufferFinal("2"));<br>
+<br>
+ Cache.finalizeCurrentBuffers()<wbr>;<br>
+ EXPECT_TRUE(Cache.<wbr>isBufferFinal("1"));<br>
+ EXPECT_TRUE(Cache.<wbr>isBufferFinal("2"));<br>
+ EXPECT_TRUE(Cache.<wbr>tryToRemoveBuffer("1"));<br>
+ EXPECT_TRUE(Cache.<wbr>tryToRemoveBuffer("2"));<br>
+ EXPECT_EQ(RawB1, Cache.lookupBuffer("1"));<br>
+ EXPECT_EQ(RawB2, Cache.lookupBuffer("2"));<br>
+ EXPECT_TRUE(Cache.<wbr>isBufferFinal("1"));<br>
+ EXPECT_TRUE(Cache.<wbr>isBufferFinal("2"));<br>
+}<br>
+<br>
+} // namespace<br>
<br>
Modified: cfe/trunk/unittests/Basic/<wbr>SourceManagerTest.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Basic/SourceManagerTest.cpp?rev=298278&r1=298277&r2=298278&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/unittests/<wbr>Basic/SourceManagerTest.cpp?<wbr>rev=298278&r1=298277&r2=<wbr>298278&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/unittests/Basic/<wbr>SourceManagerTest.cpp (original)<br>
+++ cfe/trunk/unittests/Basic/<wbr>SourceManagerTest.cpp Mon Mar 20 12:58:26 2017<br>
@@ -12,6 +12,7 @@<br>
#include "clang/Basic/<wbr>DiagnosticOptions.h"<br>
#include "clang/Basic/FileManager.h"<br>
#include "clang/Basic/LangOptions.h"<br>
+#include "clang/Basic/<wbr>MemoryBufferCache.h"<br>
#include "clang/Basic/TargetInfo.h"<br>
#include "clang/Basic/TargetOptions.h"<br>
#include "clang/Lex/HeaderSearch.h"<br>
@@ -78,10 +79,11 @@ TEST_F(SourceManagerTest, isBeforeInTran<br>
SourceMgr.setMainFileID(<wbr>mainFileID);<br>
<br>
VoidModuleLoader ModLoader;<br>
+ MemoryBufferCache PCMCache;<br>
HeaderSearch HeaderInfo(std::make_shared<<wbr>HeaderSearchOptions>(), SourceMgr,<br>
Diags, LangOpts, &*Target);<br>
Preprocessor PP(std::make_shared<<wbr>PreprocessorOptions>(), Diags, LangOpts,<br>
- SourceMgr, HeaderInfo, ModLoader,<br>
+ SourceMgr, PCMCache, HeaderInfo, ModLoader,<br>
/*IILookup =*/nullptr,<br>
/*OwnsHeaderSearch =*/false);<br>
PP.Initialize(*Target);<br>
@@ -198,10 +200,11 @@ TEST_F(SourceManagerTest, getMacroArgExp<br>
SourceMgr.<wbr>overrideFileContents(<wbr>headerFile, std::move(HeaderBuf));<br>
<br>
VoidModuleLoader ModLoader;<br>
+ MemoryBufferCache PCMCache;<br>
HeaderSearch HeaderInfo(std::make_shared<<wbr>HeaderSearchOptions>(), SourceMgr,<br>
Diags, LangOpts, &*Target);<br>
Preprocessor PP(std::make_shared<<wbr>PreprocessorOptions>(), Diags, LangOpts,<br>
- SourceMgr, HeaderInfo, ModLoader,<br>
+ SourceMgr, PCMCache, HeaderInfo, ModLoader,<br>
/*IILookup =*/nullptr,<br>
/*OwnsHeaderSearch =*/false);<br>
PP.Initialize(*Target);<br>
@@ -298,10 +301,11 @@ TEST_F(SourceManagerTest, isBeforeInTran<br>
SourceMgr.<wbr>overrideFileContents(<wbr>headerFile, std::move(HeaderBuf));<br>
<br>
VoidModuleLoader ModLoader;<br>
+ MemoryBufferCache PCMCache;<br>
HeaderSearch HeaderInfo(std::make_shared<<wbr>HeaderSearchOptions>(), SourceMgr,<br>
Diags, LangOpts, &*Target);<br>
Preprocessor PP(std::make_shared<<wbr>PreprocessorOptions>(), Diags, LangOpts,<br>
- SourceMgr, HeaderInfo, ModLoader,<br>
+ SourceMgr, PCMCache, HeaderInfo, ModLoader,<br>
/*IILookup =*/nullptr,<br>
/*OwnsHeaderSearch =*/false);<br>
PP.Initialize(*Target);<br>
<br>
Modified: cfe/trunk/unittests/Lex/<wbr>LexerTest.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Lex/LexerTest.cpp?rev=298278&r1=298277&r2=298278&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/unittests/<wbr>Lex/LexerTest.cpp?rev=298278&<wbr>r1=298277&r2=298278&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/unittests/Lex/<wbr>LexerTest.cpp (original)<br>
+++ cfe/trunk/unittests/Lex/<wbr>LexerTest.cpp Mon Mar 20 12:58:26 2017<br>
@@ -12,6 +12,7 @@<br>
#include "clang/Basic/<wbr>DiagnosticOptions.h"<br>
#include "clang/Basic/FileManager.h"<br>
#include "clang/Basic/LangOptions.h"<br>
+#include "clang/Basic/<wbr>MemoryBufferCache.h"<br>
#include "clang/Basic/SourceManager.h"<br>
#include "clang/Basic/TargetInfo.h"<br>
#include "clang/Basic/TargetOptions.h"<br>
@@ -64,10 +65,12 @@ protected:<br>
SourceMgr.setMainFileID(<wbr>SourceMgr.createFileID(std::<wbr>move(Buf)));<br>
<br>
VoidModuleLoader ModLoader;<br>
+ MemoryBufferCache PCMCache;<br>
HeaderSearch HeaderInfo(std::make_shared<<wbr>HeaderSearchOptions>(), SourceMgr,<br>
Diags, LangOpts, Target.get());<br>
Preprocessor PP(std::make_shared<<wbr>PreprocessorOptions>(), Diags, LangOpts,<br>
- SourceMgr, HeaderInfo, ModLoader, /*IILookup =*/nullptr,<br>
+ SourceMgr, PCMCache, HeaderInfo, ModLoader,<br>
+ /*IILookup =*/nullptr,<br>
/*OwnsHeaderSearch =*/false);<br>
PP.Initialize(*Target);<br>
PP.EnterMainSourceFile();<br>
<br>
Modified: cfe/trunk/unittests/Lex/<wbr>PPCallbacksTest.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Lex/PPCallbacksTest.cpp?rev=298278&r1=298277&r2=298278&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/unittests/<wbr>Lex/PPCallbacksTest.cpp?rev=<wbr>298278&r1=298277&r2=298278&<wbr>view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/unittests/Lex/<wbr>PPCallbacksTest.cpp (original)<br>
+++ cfe/trunk/unittests/Lex/<wbr>PPCallbacksTest.cpp Mon Mar 20 12:58:26 2017<br>
@@ -14,6 +14,7 @@<br>
#include "clang/Basic/<wbr>DiagnosticOptions.h"<br>
#include "clang/Basic/FileManager.h"<br>
#include "clang/Basic/LangOptions.h"<br>
+#include "clang/Basic/<wbr>MemoryBufferCache.h"<br>
#include "clang/Basic/SourceManager.h"<br>
#include "clang/Basic/TargetInfo.h"<br>
#include "clang/Basic/TargetOptions.h"<br>
@@ -161,13 +162,14 @@ protected:<br>
SourceMgr.setMainFileID(<wbr>SourceMgr.createFileID(std::<wbr>move(Buf)));<br>
<br>
VoidModuleLoader ModLoader;<br>
+ MemoryBufferCache PCMCache;<br>
<br>
HeaderSearch HeaderInfo(std::make_shared<<wbr>HeaderSearchOptions>(), SourceMgr,<br>
Diags, LangOpts, Target.get());<br>
AddFakeHeader(HeaderInfo, HeaderPath, SystemHeader);<br>
<br>
Preprocessor PP(std::make_shared<<wbr>PreprocessorOptions>(), Diags, LangOpts,<br>
- SourceMgr, HeaderInfo, ModLoader,<br>
+ SourceMgr, PCMCache, HeaderInfo, ModLoader,<br>
/*IILookup =*/nullptr,<br>
/*OwnsHeaderSearch =*/false);<br>
PP.Initialize(*Target);<br>
@@ -198,11 +200,12 @@ protected:<br>
SourceMgr.setMainFileID(<wbr>SourceMgr.createFileID(std::<wbr>move(SourceBuf)));<br>
<br>
VoidModuleLoader ModLoader;<br>
+ MemoryBufferCache PCMCache;<br>
HeaderSearch HeaderInfo(std::make_shared<<wbr>HeaderSearchOptions>(), SourceMgr,<br>
Diags, OpenCLLangOpts, Target.get());<br>
<br>
Preprocessor PP(std::make_shared<<wbr>PreprocessorOptions>(), Diags,<br>
- OpenCLLangOpts, SourceMgr, HeaderInfo, ModLoader,<br>
+ OpenCLLangOpts, SourceMgr, PCMCache, HeaderInfo, ModLoader,<br>
/*IILookup =*/nullptr,<br>
/*OwnsHeaderSearch =*/false);<br>
PP.Initialize(*Target);<br>
<br>
Modified: cfe/trunk/unittests/Lex/<wbr>PPConditionalDirectiveRecordTe<wbr>st.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Lex/PPConditionalDirectiveRecordTest.cpp?rev=298278&r1=298277&r2=298278&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/cfe/trunk/unittests/<wbr>Lex/<wbr>PPConditionalDirectiveRecordTe<wbr>st.cpp?rev=298278&r1=298277&<wbr>r2=298278&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- cfe/trunk/unittests/Lex/<wbr>PPConditionalDirectiveRecordTe<wbr>st.cpp (original)<br>
+++ cfe/trunk/unittests/Lex/<wbr>PPConditionalDirectiveRecordTe<wbr>st.cpp Mon Mar 20 12:58:26 2017<br>
@@ -12,6 +12,7 @@<br>
#include "clang/Basic/<wbr>DiagnosticOptions.h"<br>
#include "clang/Basic/FileManager.h"<br>
#include "clang/Basic/LangOptions.h"<br>
+#include "clang/Basic/<wbr>MemoryBufferCache.h"<br>
#include "clang/Basic/SourceManager.h"<br>
#include "clang/Basic/TargetInfo.h"<br>
#include "clang/Basic/TargetOptions.h"<br>
@@ -93,10 +94,11 @@ TEST_F(<wbr>PPConditionalDirectiveRecordTe<wbr>st,<br>
SourceMgr.setMainFileID(<wbr>SourceMgr.createFileID(std::<wbr>move(Buf)));<br>
<br>
VoidModuleLoader ModLoader;<br>
+ MemoryBufferCache PCMCache;<br>
HeaderSearch HeaderInfo(std::make_shared<<wbr>HeaderSearchOptions>(), SourceMgr,<br>
Diags, LangOpts, Target.get());<br>
Preprocessor PP(std::make_shared<<wbr>PreprocessorOptions>(), Diags, LangOpts,<br>
- SourceMgr, HeaderInfo, ModLoader,<br>
+ SourceMgr, PCMCache, HeaderInfo, ModLoader,<br>
/*IILookup =*/nullptr,<br>
/*OwnsHeaderSearch =*/false);<br>
PP.Initialize(*Target);<br>
<br>
<br>
______________________________<wbr>_________________<br>
cfe-commits mailing list<br>
<a href="mailto:cfe-commits@lists.llvm.org">cfe-commits@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/cfe-commits</a><br>
</blockquote></div><br></div>