<div dir="ltr">There was yet another uninit read:<div><br></div><div>********************<br>Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. <br>FAIL: Clang-Unit :: CrossTU/./CrossTUTests/CrossTranslationUnit.RespectsLoadThreshold (13891 of 15323)<br>******************** TEST 'Clang-Unit :: CrossTU/./CrossTUTests/CrossTranslationUnit.RespectsLoadThreshold' FAILED ********************<br>Note: Google Test filter = CrossTranslationUnit.RespectsLoadThreshold<br>[==========] Running 1 test from 1 test case.<br>[----------] Global test environment set-up.<br>[----------] 1 test from CrossTranslationUnit<br>[ RUN ] CrossTranslationUnit.RespectsLoadThreshold<br>==8561==WARNING: MemorySanitizer: use-of-uninitialized-value<br> #0 0x1010708 in ~LoadGuard /b/sanitizer-x86_64-linux-bootstrap-msan/build/llvm/tools/clang/include/clang/CrossTU/CrossTranslationUnit.h:288:11<br> #1 0x1010708 in clang::cross_tu::CrossTranslationUnitContext::loadExternalAST(llvm::StringRef, llvm::StringRef, llvm::StringRef, bool) /b/sanitizer-x86_64-linux-bootstrap-msan/build/llvm/tools/clang/lib/CrossTU/CrossTranslationUnit.cpp:499:1<br> #2 0x100babc in llvm::Expected<clang::FunctionDecl const*> clang::cross_tu::CrossTranslationUnitContext::getCrossTUDefinitionImpl<clang::FunctionDecl>(clang::FunctionDecl const*, llvm::StringRef, llvm::StringRef, bool) /b/sanitizer-x86_64-linux-bootstrap-msan/build/llvm/tools/clang/lib/CrossTU/CrossTranslationUnit.cpp:241:7<br> #3 0x100b476 in clang::cross_tu::CrossTranslationUnitContext::getCrossTUDefinition(clang::FunctionDecl const*, llvm::StringRef, llvm::StringRef, bool) /b/sanitizer-x86_64-linux-bootstrap-msan/build/llvm/tools/clang/lib/CrossTU/CrossTranslationUnit.cpp:307:10<br></div><div><br></div><div><br></div><div>I tried fixing that one in r367912. Please verify that that's what you meant as well.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Aug 5, 2019 at 11:22 AM Nico Weber <<a href="mailto:thakis@chromium.org">thakis@chromium.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">The problem is probably this part from the diff:<div><br></div><div>- unsigned NumASTLoaded{0u};<br>+<br>+ /// The number successfully loaded ASTs. Used to indicate, and - with the<br>+ /// appropriate threshold value - limit the memory usage of the<br>+ /// CrossTranslationUnitContext.<br>+ unsigned NumASTLoaded;<br></div><div><br></div><div><br></div><div>i.e. you removed the initialization of NumASTLoaded. Was there a reason for that? I've put it back for now in r367875, but please verify that the code now does what you intended it to do – maybe you were planning to initialize this somewhere else.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Aug 5, 2019 at 10:03 AM Nico Weber <<a href="mailto:thakis@chromium.org" target="_blank">thakis@chromium.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">The msan bot doesn't like this, it reports an uninitialized read a t clang/lib/CrossTU/CrossTranslationUnit.cpp :<div><br></div><div><a href="http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/34087/steps/check-clang%20msan/logs/stdio" target="_blank">http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/34087/steps/check-clang%20msan/logs/stdio</a><br><div><br></div><div>********************<br>Testing: 0 <br>FAIL: Clang :: Analysis/ctu-unknown-parts-in-triples.cpp (492 of 15321)<br>******************** TEST 'Clang :: Analysis/ctu-unknown-parts-in-triples.cpp' FAILED ********************<br>Script:<br>--<br>: 'RUN: at line 4'; rm -rf /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/tools/clang/test/Analysis/Output/ctu-unknown-parts-in-triples.cpp.tmp && mkdir /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/tools/clang/test/Analysis/Output/ctu-unknown-parts-in-triples.cpp.tmp<br>: 'RUN: at line 5'; mkdir -p /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/tools/clang/test/Analysis/Output/ctu-unknown-parts-in-triples.cpp.tmp/ctudir<br>: 'RUN: at line 6'; /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/clang -cc1 -internal-isystem /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/lib/clang/10.0.0/include -nostdsysteminc -triple x86_64-pc-linux-gnu -emit-pch -o /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/tools/clang/test/Analysis/Output/ctu-unknown-parts-in-triples.cpp.tmp/ctudir/ctu-other.cpp.ast /b/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/test/Analysis/Inputs/ctu-other.cpp<br>: 'RUN: at line 8'; cp /b/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/test/Analysis/Inputs/ctu-other.cpp.externalDefMap.txt /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/tools/clang/test/Analysis/Output/ctu-unknown-parts-in-triples.cpp.tmp/ctudir/externalDefMap.txt<br>: 'RUN: at line 9'; /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/clang -cc1 -internal-isystem /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/lib/clang/10.0.0/include -nostdsysteminc -analyze -analyzer-constraints=range -triple x86_64-unknown-linux-gnu -analyzer-checker=core,debug.ExprInspection -analyzer-config experimental-enable-naive-ctu-analysis=true -analyzer-config ctu-dir=/b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/tools/clang/test/Analysis/Output/ctu-unknown-parts-in-triples.cpp.tmp/ctudir -Werror=ctu -verify /b/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/test/Analysis/ctu-unknown-parts-in-triples.cpp<br>--<br>Exit Code: 77<br><br>Command Output (stderr):<br>--<br>==5072==WARNING: MemorySanitizer: use-of-uninitialized-value<br> #0 0xb05c3c4 in clang::cross_tu::CrossTranslationUnitContext::loadExternalAST(llvm::StringRef, llvm::StringRef, llvm::StringRef, bool) /b/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/lib/CrossTU/CrossTranslationUnit.cpp:467:7<br> #1 0xb053a98 in llvm::Expected<clang::FunctionDecl const*> clang::cross_tu::CrossTranslationUnitContext::getCrossTUDefinitionImpl<clang::FunctionDecl>(clang::FunctionDecl const*, llvm::StringRef, llvm::StringRef, bool) /b/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/lib/CrossTU/CrossTranslationUnit.cpp:241:7<br> #2 0xb053466 in clang::cross_tu::CrossTranslationUnitContext::getCrossTUDefinition(clang::FunctionDecl const*, llvm::StringRef, llvm::StringRef, bool) /b/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/lib/CrossTU/CrossTranslationUnit.cpp:307:10<br> #3 0xadb69f5 in clang::ento::AnyFunctionCall::getRuntimeDefinition() const /b/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/lib/StaticAnalyzer/Core/CallEvent.cpp:575:14<br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Aug 5, 2019 at 7:05 AM Endre Fulop via cfe-commits <<a href="mailto:cfe-commits@lists.llvm.org" target="_blank">cfe-commits@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Author: gamesh411<br>
Date: Mon Aug 5 04:06:41 2019<br>
New Revision: 367829<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=367829&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project?rev=367829&view=rev</a><br>
Log:<br>
[CrossTU][NFCI] Refactor loadExternalAST function<br>
<br>
Summary:<br>
Refactor loadExternalAST method of CrossTranslationUnitContext in order to<br>
reduce maintenance burden and so that features are easier to add in the future.<br>
<br>
Reviewers: martong<br>
<br>
Subscribers: rnkovacs, dkrupp, Szelethus, cfe-commits<br>
<br>
Tags: #clang<br>
<br>
Differential Revision: <a href="https://reviews.llvm.org/D64753" rel="noreferrer" target="_blank">https://reviews.llvm.org/D64753</a><br>
<br>
Modified:<br>
cfe/trunk/include/clang/CrossTU/CrossTranslationUnit.h<br>
cfe/trunk/lib/CrossTU/CrossTranslationUnit.cpp<br>
<br>
Modified: cfe/trunk/include/clang/CrossTU/CrossTranslationUnit.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/CrossTU/CrossTranslationUnit.h?rev=367829&r1=367828&r2=367829&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/CrossTU/CrossTranslationUnit.h?rev=367829&r1=367828&r2=367829&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/include/clang/CrossTU/CrossTranslationUnit.h (original)<br>
+++ cfe/trunk/include/clang/CrossTU/CrossTranslationUnit.h Mon Aug 5 04:06:41 2019<br>
@@ -192,11 +192,11 @@ private:<br>
template <typename T><br>
llvm::Expected<const T *> importDefinitionImpl(const T *D, ASTUnit *Unit);<br>
<br>
- llvm::StringMap<std::unique_ptr<clang::ASTUnit>> FileASTUnitMap;<br>
- llvm::StringMap<clang::ASTUnit *> NameASTUnitMap;<br>
- llvm::StringMap<std::string> NameFileMap;<br>
- llvm::DenseMap<TranslationUnitDecl *, std::unique_ptr<ASTImporter>><br>
- ASTUnitImporterMap;<br>
+ using ImporterMapTy =<br>
+ llvm::DenseMap<TranslationUnitDecl *, std::unique_ptr<ASTImporter>>;<br>
+<br>
+ ImporterMapTy ASTUnitImporterMap;<br>
+<br>
CompilerInstance &CI;<br>
ASTContext &Context;<br>
std::shared_ptr<ASTImporterSharedState> ImporterSharedSt;<br>
@@ -209,10 +209,105 @@ private:<br>
/// imported the FileID.<br>
ImportedFileIDMap ImportedFileIDs;<br>
<br>
+ /// Functor for loading ASTUnits from AST-dump files.<br>
+ class ASTFileLoader {<br>
+ public:<br>
+ ASTFileLoader(const CompilerInstance &CI);<br>
+ std::unique_ptr<ASTUnit> operator()(StringRef ASTFilePath);<br>
+<br>
+ private:<br>
+ const CompilerInstance &CI;<br>
+ };<br>
+<br>
+ /// Storage for ASTUnits, cached access, and providing searchability are the<br>
+ /// concerns of ASTUnitStorage class.<br>
+ class ASTUnitStorage {<br>
+ public:<br>
+ ASTUnitStorage(const CompilerInstance &CI);<br>
+ /// Loads an ASTUnit for a function.<br>
+ ///<br>
+ /// \param FuncitionName USR name of the function.<br>
+ /// \param CrossTUDir Path to the directory used to store CTU related files.<br>
+ /// \param IndexName Name of the file inside \p CrossTUDir which maps<br>
+ /// function USR names to file paths. These files contain the corresponding<br>
+ /// AST-dumps.<br>
+ ///<br>
+ /// \return An Expected instance which contains the ASTUnit pointer or the<br>
+ /// error occured during the load.<br>
+ llvm::Expected<ASTUnit *> getASTUnitForFunction(StringRef FunctionName,<br>
+ StringRef CrossTUDir,<br>
+ StringRef IndexName);<br>
+ /// Identifies the path of the file which can be used to load the ASTUnit<br>
+ /// for a given function.<br>
+ ///<br>
+ /// \param FuncitionName USR name of the function.<br>
+ /// \param CrossTUDir Path to the directory used to store CTU related files.<br>
+ /// \param IndexName Name of the file inside \p CrossTUDir which maps<br>
+ /// function USR names to file paths. These files contain the corresponding<br>
+ /// AST-dumps.<br>
+ ///<br>
+ /// \return An Expected instance containing the filepath.<br>
+ llvm::Expected<std::string> getFileForFunction(StringRef FunctionName,<br>
+ StringRef CrossTUDir,<br>
+ StringRef IndexName);<br>
+<br>
+ private:<br>
+ llvm::Error ensureCTUIndexLoaded(StringRef CrossTUDir, StringRef IndexName);<br>
+ llvm::Expected<ASTUnit *> getASTUnitForFile(StringRef FileName);<br>
+<br>
+ template <typename... T> using BaseMapTy = llvm::StringMap<T...>;<br>
+ using OwningMapTy = BaseMapTy<std::unique_ptr<clang::ASTUnit>>;<br>
+ using NonOwningMapTy = BaseMapTy<clang::ASTUnit *>;<br>
+<br>
+ OwningMapTy FileASTUnitMap;<br>
+ NonOwningMapTy NameASTUnitMap;<br>
+<br>
+ using IndexMapTy = BaseMapTy<std::string>;<br>
+ IndexMapTy NameFileMap;<br>
+<br>
+ ASTFileLoader FileAccessor;<br>
+ };<br>
+<br>
+ ASTUnitStorage ASTStorage;<br>
+<br>
/// \p CTULoadTreshold should serve as an upper limit to the number of TUs<br>
/// imported in order to reduce the memory footprint of CTU analysis.<br>
const unsigned CTULoadThreshold;<br>
- unsigned NumASTLoaded{0u};<br>
+<br>
+ /// The number successfully loaded ASTs. Used to indicate, and - with the<br>
+ /// appropriate threshold value - limit the memory usage of the<br>
+ /// CrossTranslationUnitContext.<br>
+ unsigned NumASTLoaded;<br>
+<br>
+ /// RAII counter to signal 'threshold reached' condition, and to increment the<br>
+ /// NumASTLoaded counter upon a successful load.<br>
+ class LoadGuard {<br>
+ public:<br>
+ LoadGuard(unsigned Limit, unsigned &Counter)<br>
+ : Counter(Counter), Enabled(Counter < Limit){};<br>
+ ~LoadGuard() {<br>
+ if (StoreSuccess)<br>
+ ++Counter;<br>
+ }<br>
+ /// Flag the LoadGuard instance as successful, meaning that the load<br>
+ /// operation succeeded, and the memory footprint of the AST storage<br>
+ /// actually increased. In this case, \p Counter should be incremented upon<br>
+ /// destruction.<br>
+ void storedSuccessfully() { StoreSuccess = true; }<br>
+ /// Indicates, whether a new load operation is permitted, it is within the<br>
+ /// threshold.<br>
+ operator bool() const { return Enabled; };<br>
+<br>
+ private:<br>
+ /// The number of ASTs actually imported. LoadGuard does not own the<br>
+ /// counter, just uses on given to it at construction time.<br>
+ unsigned &Counter;<br>
+ /// Indicates whether a load operation can begin, which is equivalent to the<br>
+ /// 'threshold not reached' condition.<br>
+ bool Enabled;<br>
+ /// Shows the state of the current load operation.<br>
+ bool StoreSuccess;<br>
+ };<br>
};<br>
<br>
} // namespace cross_tu<br>
<br>
Modified: cfe/trunk/lib/CrossTU/CrossTranslationUnit.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CrossTU/CrossTranslationUnit.cpp?rev=367829&r1=367828&r2=367829&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CrossTU/CrossTranslationUnit.cpp?rev=367829&r1=367828&r2=367829&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/lib/CrossTU/CrossTranslationUnit.cpp (original)<br>
+++ cfe/trunk/lib/CrossTU/CrossTranslationUnit.cpp Mon Aug 5 04:06:41 2019<br>
@@ -188,7 +188,7 @@ template <typename T> static bool hasBod<br>
}<br>
<br>
CrossTranslationUnitContext::CrossTranslationUnitContext(CompilerInstance &CI)<br>
- : CI(CI), Context(CI.getASTContext()),<br>
+ : CI(CI), Context(CI.getASTContext()), ASTStorage(CI),<br>
CTULoadThreshold(CI.getAnalyzerOpts()->CTUImportThreshold) {}<br>
<br>
CrossTranslationUnitContext::~CrossTranslationUnitContext() {}<br>
@@ -237,8 +237,8 @@ llvm::Expected<const T *> CrossTranslati<br>
if (LookupName.empty())<br>
return llvm::make_error<IndexError>(<br>
index_error_code::failed_to_generate_usr);<br>
- llvm::Expected<ASTUnit *> ASTUnitOrError = loadExternalAST(<br>
- LookupName, CrossTUDir, IndexName, DisplayCTUProgress);<br>
+ llvm::Expected<ASTUnit *> ASTUnitOrError =<br>
+ loadExternalAST(LookupName, CrossTUDir, IndexName, DisplayCTUProgress);<br>
if (!ASTUnitOrError)<br>
return ASTUnitOrError.takeError();<br>
ASTUnit *Unit = *ASTUnitOrError;<br>
@@ -340,6 +340,118 @@ void CrossTranslationUnitContext::emitCr<br>
}<br>
}<br>
<br>
+CrossTranslationUnitContext::ASTFileLoader::ASTFileLoader(<br>
+ const CompilerInstance &CI)<br>
+ : CI(CI) {}<br>
+<br>
+std::unique_ptr<ASTUnit><br>
+CrossTranslationUnitContext::ASTFileLoader::operator()(StringRef ASTFilePath) {<br>
+ // Load AST from ast-dump.<br>
+ IntrusiveRefCntPtr<DiagnosticOptions> DiagOpts = new DiagnosticOptions();<br>
+ TextDiagnosticPrinter *DiagClient =<br>
+ new TextDiagnosticPrinter(llvm::errs(), &*DiagOpts);<br>
+ IntrusiveRefCntPtr<DiagnosticIDs> DiagID(new DiagnosticIDs());<br>
+ IntrusiveRefCntPtr<DiagnosticsEngine> Diags(<br>
+ new DiagnosticsEngine(DiagID, &*DiagOpts, DiagClient));<br>
+<br>
+ return ASTUnit::LoadFromASTFile(<br>
+ ASTFilePath, CI.getPCHContainerOperations()->getRawReader(),<br>
+ ASTUnit::LoadEverything, Diags, CI.getFileSystemOpts());<br>
+}<br>
+<br>
+CrossTranslationUnitContext::ASTUnitStorage::ASTUnitStorage(<br>
+ const CompilerInstance &CI)<br>
+ : FileAccessor(CI) {}<br>
+<br>
+llvm::Expected<ASTUnit *><br>
+CrossTranslationUnitContext::ASTUnitStorage::getASTUnitForFile(StringRef FileName) {<br>
+ // Try the cache first.<br>
+ auto ASTCacheEntry = FileASTUnitMap.find(FileName);<br>
+ if (ASTCacheEntry == FileASTUnitMap.end()) {<br>
+ // Load the ASTUnit from the pre-dumped AST file specified by ASTFileName.<br>
+ std::unique_ptr<ASTUnit> LoadedUnit = FileAccessor(FileName);<br>
+<br>
+ // Need the raw pointer and the unique_ptr as well.<br>
+ ASTUnit* Unit = LoadedUnit.get();<br>
+<br>
+ // Update the cache.<br>
+ FileASTUnitMap[FileName] = std::move(LoadedUnit);<br>
+ return Unit;<br>
+<br>
+ } else {<br>
+ // Found in the cache.<br>
+ return ASTCacheEntry->second.get();<br>
+ }<br>
+}<br>
+<br>
+llvm::Expected<ASTUnit *><br>
+CrossTranslationUnitContext::ASTUnitStorage::getASTUnitForFunction(<br>
+ StringRef FunctionName, StringRef CrossTUDir, StringRef IndexName) {<br>
+ // Try the cache first.<br>
+ auto ASTCacheEntry = NameASTUnitMap.find(FunctionName);<br>
+ if (ASTCacheEntry == NameASTUnitMap.end()) {<br>
+ // Load the ASTUnit from the pre-dumped AST file specified by ASTFileName.<br>
+<br>
+ // Ensure that the Index is loaded, as we need to search in it.<br>
+ if (llvm::Error IndexLoadError =<br>
+ ensureCTUIndexLoaded(CrossTUDir, IndexName))<br>
+ return std::move(IndexLoadError);<br>
+<br>
+ // Check if there is and entry in the index for the function.<br>
+ if (!NameFileMap.count(FunctionName)) {<br>
+ ++NumNotInOtherTU;<br>
+ return llvm::make_error<IndexError>(index_error_code::missing_definition);<br>
+ }<br>
+<br>
+ // Search in the index for the filename where the definition of FuncitonName<br>
+ // resides.<br>
+ if (llvm::Expected<ASTUnit *> FoundForFile =<br>
+ getASTUnitForFile(NameFileMap[FunctionName])) {<br>
+<br>
+ // Update the cache.<br>
+ NameASTUnitMap[FunctionName] = *FoundForFile;<br>
+ return *FoundForFile;<br>
+<br>
+ } else {<br>
+ return FoundForFile.takeError();<br>
+ }<br>
+ } else {<br>
+ // Found in the cache.<br>
+ return ASTCacheEntry->second;<br>
+ }<br>
+}<br>
+<br>
+llvm::Expected<std::string><br>
+CrossTranslationUnitContext::ASTUnitStorage::getFileForFunction(<br>
+ StringRef FunctionName, StringRef CrossTUDir, StringRef IndexName) {<br>
+ if (llvm::Error IndexLoadError = ensureCTUIndexLoaded(CrossTUDir, IndexName))<br>
+ return std::move(IndexLoadError);<br>
+ return NameFileMap[FunctionName];<br>
+}<br>
+<br>
+llvm::Error CrossTranslationUnitContext::ASTUnitStorage::ensureCTUIndexLoaded(<br>
+ StringRef CrossTUDir, StringRef IndexName) {<br>
+ // Dont initialize if the map is filled.<br>
+ if (!NameFileMap.empty())<br>
+ return llvm::Error::success();<br>
+<br>
+ // Get the absolute path to the index file.<br>
+ SmallString<256> IndexFile = CrossTUDir;<br>
+ if (llvm::sys::path::is_absolute(IndexName))<br>
+ IndexFile = IndexName;<br>
+ else<br>
+ llvm::sys::path::append(IndexFile, IndexName);<br>
+<br>
+ if (auto IndexMapping = parseCrossTUIndex(IndexFile, CrossTUDir)) {<br>
+ // Initialize member map.<br>
+ NameFileMap = *IndexMapping;<br>
+ return llvm::Error::success();<br>
+ } else {<br>
+ // Error while parsing CrossTU index file.<br>
+ return IndexMapping.takeError();<br>
+ };<br>
+}<br>
+<br>
llvm::Expected<ASTUnit *> CrossTranslationUnitContext::loadExternalAST(<br>
StringRef LookupName, StringRef CrossTUDir, StringRef IndexName,<br>
bool DisplayCTUProgress) {<br>
@@ -348,64 +460,41 @@ llvm::Expected<ASTUnit *> CrossTranslati<br>
// translation units contains decls with the same lookup name an<br>
// error will be returned.<br>
<br>
- if (NumASTLoaded >= CTULoadThreshold) {<br>
+ // RAII incrementing counter is used to count successful loads.<br>
+ LoadGuard LoadOperation(CTULoadThreshold, NumASTLoaded);<br>
+<br>
+ // If import threshold is reached, don't import anything.<br>
+ if (!LoadOperation) {<br>
++NumASTLoadThresholdReached;<br>
return llvm::make_error<IndexError>(<br>
index_error_code::load_threshold_reached);<br>
}<br>
<br>
- ASTUnit *Unit = nullptr;<br>
- auto NameUnitCacheEntry = NameASTUnitMap.find(LookupName);<br>
- if (NameUnitCacheEntry == NameASTUnitMap.end()) {<br>
- if (NameFileMap.empty()) {<br>
- SmallString<256> IndexFile = CrossTUDir;<br>
- if (llvm::sys::path::is_absolute(IndexName))<br>
- IndexFile = IndexName;<br>
- else<br>
- llvm::sys::path::append(IndexFile, IndexName);<br>
- llvm::Expected<llvm::StringMap<std::string>> IndexOrErr =<br>
- parseCrossTUIndex(IndexFile, CrossTUDir);<br>
- if (IndexOrErr)<br>
- NameFileMap = *IndexOrErr;<br>
- else<br>
- return IndexOrErr.takeError();<br>
- }<br>
+ // Try to get the value from the heavily cached storage.<br>
+ llvm::Expected<ASTUnit *> Unit =<br>
+ ASTStorage.getASTUnitForFunction(LookupName, CrossTUDir, IndexName);<br>
<br>
- auto It = NameFileMap.find(LookupName);<br>
- if (It == NameFileMap.end()) {<br>
- ++NumNotInOtherTU;<br>
- return llvm::make_error<IndexError>(index_error_code::missing_definition);<br>
- }<br>
- StringRef ASTFileName = It->second;<br>
- auto ASTCacheEntry = FileASTUnitMap.find(ASTFileName);<br>
- if (ASTCacheEntry == FileASTUnitMap.end()) {<br>
- IntrusiveRefCntPtr<DiagnosticOptions> DiagOpts = new DiagnosticOptions();<br>
- TextDiagnosticPrinter *DiagClient =<br>
- new TextDiagnosticPrinter(llvm::errs(), &*DiagOpts);<br>
- IntrusiveRefCntPtr<DiagnosticIDs> DiagID(new DiagnosticIDs());<br>
- IntrusiveRefCntPtr<DiagnosticsEngine> Diags(<br>
- new DiagnosticsEngine(DiagID, &*DiagOpts, DiagClient));<br>
-<br>
- std::unique_ptr<ASTUnit> LoadedUnit(ASTUnit::LoadFromASTFile(<br>
- ASTFileName, CI.getPCHContainerOperations()->getRawReader(),<br>
- ASTUnit::LoadEverything, Diags, CI.getFileSystemOpts()));<br>
- Unit = LoadedUnit.get();<br>
- FileASTUnitMap[ASTFileName] = std::move(LoadedUnit);<br>
- ++NumASTLoaded;<br>
- if (DisplayCTUProgress) {<br>
- llvm::errs() << "CTU loaded AST file: "<br>
- << ASTFileName << "\n";<br>
- }<br>
- } else {<br>
- Unit = ASTCacheEntry->second.get();<br>
- }<br>
- NameASTUnitMap[LookupName] = Unit;<br>
- } else {<br>
- Unit = NameUnitCacheEntry->second;<br>
- }<br>
if (!Unit)<br>
+ return Unit.takeError();<br>
+<br>
+ // Check whether the backing pointer of the Expected is a nullptr.<br>
+ if (!*Unit)<br>
return llvm::make_error<IndexError>(<br>
index_error_code::failed_to_get_external_ast);<br>
+<br>
+ // The backing pointer is not null, loading was successful. If anything goes<br>
+ // wrong from this point on, the AST is already stored, so the load part is<br>
+ // finished.<br>
+ LoadOperation.storedSuccessfully();<br>
+<br>
+ if (DisplayCTUProgress) {<br>
+ if (llvm::Expected<std::string> FileName =<br>
+ ASTStorage.getFileForFunction(LookupName, CrossTUDir, IndexName))<br>
+ llvm::errs() << "CTU loaded AST file: " << *FileName << "\n";<br>
+ else<br>
+ return FileName.takeError();<br>
+ }<br>
+<br>
return Unit;<br>
}<br>
<br>
<br>
<br>
_______________________________________________<br>
cfe-commits mailing list<br>
<a href="mailto:cfe-commits@lists.llvm.org" target="_blank">cfe-commits@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits</a><br>
</blockquote></div>
</blockquote></div>
</blockquote></div>