<div dir="ltr"><div>There's a '-run-synchronously' flag forgotten in the test.</div><div>Still, it shouldn't panic.</div><div><br></div>I'll take a look at it tomorrow, thanks for reverting the commit.</div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, May 15, 2017 at 8:29 PM, Adam Nemet <span dir="ltr"><<a href="mailto:anemet@apple.com" target="_blank">anemet@apple.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word">Even after Simon’s fix there is still a test failure (<a href="http://lab.llvm.org:8080/green/job/clang-stage1-configure-RA_check/31377/consoleFull#18373900728254eaf0-7326-4999-85b0-388101f2d404" target="_blank">http://lab.llvm.org:8080/<wbr>green/job/clang-stage1-<wbr>configure-RA_check/31377/<wbr>consoleFull#<wbr>18373900728254eaf0-7326-4999-<wbr>85b0-388101f2d404</a>). Reverted the original commit and the fix in r303093 and r303094.<div><br><div><blockquote type="cite"><div>On May 15, 2017, at 10:06 AM, Adam Nemet <<a href="mailto:anemet@apple.com" target="_blank">anemet@apple.com</a>> wrote:</div><br class="m_-3724680780602895101Apple-interchange-newline"><div><div style="word-wrap:break-word">Hi Simon,<div><br></div><div>Is r303078 supposed to fix this?</div><div><br></div><div>Adam</div><div><br><div><blockquote type="cite"><div>On May 15, 2017, at 9:28 AM, Adam Nemet <<a href="mailto:anemet@apple.com" target="_blank">anemet@apple.com</a>> wrote:</div><br class="m_-3724680780602895101Apple-interchange-newline"><div><div style="word-wrap:break-word">This broke the build <a href="http://green.lab.llvm.org/green/job/clang-stage1-configure-RA/34305/" target="_blank">http://green.lab.llvm.<wbr>org/green/job/clang-stage1-<wbr>configure-RA/34305/</a>, please take a look.<div><br></div><div>Adam</div><div><br><div><blockquote type="cite"><div>On May 15, 2017, at 7:17 AM, Ilya Biryukov via cfe-commits <<a href="mailto:cfe-commits@lists.llvm.org" target="_blank">cfe-commits@lists.llvm.org</a>> wrote:</div><br class="m_-3724680780602895101Apple-interchange-newline"><div><div>Author: ibiryukov<br>Date: Mon May 15 09:17:35 2017<br>New Revision: 303067<br><br>URL: <a href="http://llvm.org/viewvc/llvm-project?rev=303067&view=rev" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project?rev=303067&view=rev</a><br>Log:<br>[ClangD] Refactor clangd into separate components<br><br>Summary: Major refactoring to split LSP implementation, Clang API calls and threading(mostly synchronization)<br><br>Reviewers: bkramer, krasimir<br><br>Reviewed By: bkramer<br><br>Subscribers: cfe-commits, mgorny, klimek<br><br>Tags: #clang-tools-extra<br><br>Differential Revision: <a href="https://reviews.llvm.org/D33047" target="_blank">https://reviews.llvm.org/<wbr>D33047</a><br><br>Added:<br> clang-tools-extra/trunk/<wbr>clangd/ClangdLSPServer.cpp<br> clang-tools-extra/trunk/<wbr>clangd/ClangdLSPServer.h<br> clang-tools-extra/trunk/<wbr>clangd/ClangdServer.cpp<br> - copied, changed from r303060, clang-tools-extra/trunk/<wbr>clangd/ASTManager.cpp<br> clang-tools-extra/trunk/<wbr>clangd/ClangdServer.h<br> - copied, changed from r303063, clang-tools-extra/trunk/<wbr>clangd/ASTManager.h<br> clang-tools-extra/trunk/<wbr>clangd/ClangdUnit.cpp<br> clang-tools-extra/trunk/<wbr>clangd/ClangdUnit.h<br> clang-tools-extra/trunk/<wbr>clangd/ClangdUnitStore.cpp<br> clang-tools-extra/trunk/<wbr>clangd/ClangdUnitStore.h<br> clang-tools-extra/trunk/<wbr>clangd/DraftStore.cpp<br> clang-tools-extra/trunk/<wbr>clangd/DraftStore.h<br> - copied, changed from r303060, clang-tools-extra/trunk/<wbr>clangd/DocumentStore.h<br> clang-tools-extra/trunk/<wbr>clangd/<wbr>GlobalCompilationDatabase.cpp<br> clang-tools-extra/trunk/<wbr>clangd/<wbr>GlobalCompilationDatabase.h<br> clang-tools-extra/trunk/<wbr>clangd/Path.h<br>Removed:<br> clang-tools-extra/trunk/<wbr>clangd/ASTManager.cpp<br> clang-tools-extra/trunk/<wbr>clangd/ASTManager.h<br> clang-tools-extra/trunk/<wbr>clangd/DocumentStore.h<br>Modified:<br> clang-tools-extra/trunk/<wbr>clangd/CMakeLists.txt<br> clang-tools-extra/trunk/<wbr>clangd/ClangdMain.cpp<br> clang-tools-extra/trunk/<wbr>clangd/ProtocolHandlers.cpp<br> clang-tools-extra/trunk/<wbr>clangd/ProtocolHandlers.h<br><br>Removed: clang-tools-extra/trunk/<wbr>clangd/ASTManager.cpp<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ASTManager.cpp?rev=303066&view=auto" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/clang-tools-extra/<wbr>trunk/clangd/ASTManager.cpp?<wbr>rev=303066&view=auto</a><br>==============================<wbr>==============================<wbr>==================<br>--- clang-tools-extra/trunk/<wbr>clangd/ASTManager.cpp (original)<br>+++ clang-tools-extra/trunk/<wbr>clangd/ASTManager.cpp (removed)<br>@@ -1,440 +0,0 @@<br>-//===--- ASTManager.cpp - Clang AST manager ------------------------------<wbr>-===//<br>-//<br>-// The LLVM Compiler Infrastructure<br>-//<br>-// This file is distributed under the University of Illinois Open Source<br>-// License. See LICENSE.TXT for details.<br>-//<br>-//===------------------------<wbr>------------------------------<wbr>----------------===//<br>-<br>-#include "ASTManager.h"<br>-#include "JSONRPCDispatcher.h"<br>-#include "Protocol.h"<br>-#include "clang/Frontend/ASTUnit.h"<br>-#include "clang/Frontend/<wbr>CompilerInstance.h"<br>-#include "clang/Tooling/<wbr>CompilationDatabase.h"<br>-#include "llvm/Support/Format.h"<br>-#include "llvm/Support/Path.h"<br>-#include <mutex><br>-#include <thread><br>-using namespace clang;<br>-using namespace clangd;<br>-<br>-void DocData::setAST(std::unique_<wbr>ptr<ASTUnit> AST) {<br>- this->AST = std::move(AST);<br>-}<br>-<br>-ASTUnit *DocData::getAST() const { return AST.get(); }<br>-<br>-void DocData::cacheFixIts(<wbr>DiagnosticToReplacementMap FixIts) {<br>- this->FixIts = std::move(FixIts);<br>-}<br>-<br>-std::vector<clang::tooling::<wbr>Replacement><br>-DocData::getFixIts(const clangd::Diagnostic &D) const {<br>- auto it = FixIts.find(D);<br>- if (it != FixIts.end())<br>- return it->second;<br>- return {};<br>-}<br>-<br>-ASTManagerRequest::<wbr>ASTManagerRequest(<wbr>ASTManagerRequestType Type,<br>- <wbr> std::string File,<br>- <wbr> DocVersion Version)<br>- : Type(Type), File(File), Version(Version) {}<br>-<br>-/// Retrieve a copy of the contents of every file in the store, for feeding into<br>-/// ASTUnit.<br>-static std::vector<ASTUnit::<wbr>RemappedFile><br>-getRemappedFiles(const DocumentStore &Docs) {<br>- // FIXME: Use VFS instead. This would allow us to get rid of the chdir below.<br>- std::vector<ASTUnit::<wbr>RemappedFile> RemappedFiles;<br>- for (const auto &P : Docs.getAllDocuments()) {<br>- StringRef FileName = P.first;<br>- RemappedFiles.push_back(<wbr>ASTUnit::RemappedFile(<br>- FileName,<br>- llvm::MemoryBuffer::<wbr>getMemBufferCopy(P.second, FileName).release()));<br>- }<br>- return RemappedFiles;<br>-}<br>-<br>-/// Convert from clang diagnostic level to LSP severity.<br>-static int getSeverity(DiagnosticsEngine:<wbr>:Level L) {<br>- switch (L) {<br>- case DiagnosticsEngine::Remark:<br>- return 4;<br>- case DiagnosticsEngine::Note:<br>- return 3;<br>- case DiagnosticsEngine::Warning:<br>- return 2;<br>- case DiagnosticsEngine::Fatal:<br>- case DiagnosticsEngine::Error:<br>- return 1;<br>- case DiagnosticsEngine::Ignored:<br>- return 0;<br>- }<br>- llvm_unreachable("Unknown diagnostic level!");<br>-}<br>-<br>-static CompletionItemKind getKind(CXCursorKind K) {<br>- switch (K) {<br>- case CXCursor_MacroInstantiation:<br>- case CXCursor_MacroDefinition:<br>- return CompletionItemKind::Text;<br>- case CXCursor_CXXMethod:<br>- return CompletionItemKind::Method;<br>- case CXCursor_FunctionDecl:<br>- case CXCursor_FunctionTemplate:<br>- return CompletionItemKind::Function;<br>- case CXCursor_Constructor:<br>- case CXCursor_Destructor:<br>- return CompletionItemKind::<wbr>Constructor;<br>- case CXCursor_FieldDecl:<br>- return CompletionItemKind::Field;<br>- case CXCursor_VarDecl:<br>- case CXCursor_ParmDecl:<br>- return CompletionItemKind::Variable;<br>- case CXCursor_ClassDecl:<br>- case CXCursor_StructDecl:<br>- case CXCursor_UnionDecl:<br>- case CXCursor_ClassTemplate:<br>- case CXCursor_<wbr>ClassTemplatePartialSpecializa<wbr>tion:<br>- return CompletionItemKind::Class;<br>- case CXCursor_Namespace:<br>- case CXCursor_NamespaceAlias:<br>- case CXCursor_NamespaceRef:<br>- return CompletionItemKind::Module;<br>- case CXCursor_EnumConstantDecl:<br>- return CompletionItemKind::Value;<br>- case CXCursor_EnumDecl:<br>- return CompletionItemKind::Enum;<br>- case CXCursor_TypeAliasDecl:<br>- case CXCursor_<wbr>TypeAliasTemplateDecl:<br>- case CXCursor_TypedefDecl:<br>- case CXCursor_MemberRef:<br>- case CXCursor_TypeRef:<br>- return CompletionItemKind::Reference;<br>- default:<br>- return CompletionItemKind::Missing;<br>- }<br>-}<br>-<br>-ASTManager::ASTManager(<wbr>JSONOutput &Output, DocumentStore &Store,<br>- bool RunSynchronously)<br>- : Output(Output), Store(Store), RunSynchronously(<wbr>RunSynchronously),<br>- PCHs(std::make_shared<<wbr>PCHContainerOperations>()),<br>- ClangWorker([this]() { runWorker(); }) {}<br>-<br>-void ASTManager::runWorker() {<br>- while (true) {<br>- ASTManagerRequest Request;<br>-<br>- // Pick request from the queue<br>- {<br>- std::unique_lock<std::<wbr>mutex> Lock(RequestLock);<br>- // Wait for more requests.<br>- ClangRequestCV.wait(Lock,<br>- [<wbr>this] { return !RequestQueue.empty() || Done; });<br>- if (Done)<br>- return;<br>- assert(!RequestQueue.<wbr>empty() && "RequestQueue was empty");<br>-<br>- Request = std::move(RequestQueue.back())<wbr>;<br>- RequestQueue.pop_back();<br>-<br>- // Skip outdated requests<br>- if (Request.Version != DocVersions.find(Request.File)<wbr>->second) {<br>- Output.log("Version for " + Twine(Request.File) +<br>- " in request is outdated, skipping request\n");<br>- continue;<br>- }<br>- } // unlock RequestLock<br>-<br>- handleRequest(Request.Type, Request.File);<br>- }<br>-}<br>-<br>-void ASTManager::queueOrRun(<wbr>ASTManagerRequestType RequestType, StringRef File) {<br>- if (RunSynchronously) {<br>- handleRequest(RequestType, File);<br>- return;<br>- }<br>-<br>- std::lock_guard<std::mutex> Guard(RequestLock);<br>- // We increment the version of the added document immediately and schedule<br>- // the requested operation to be run on a worker thread<br>- DocVersion version = ++DocVersions[File];<br>- RequestQueue.push_back(<wbr>ASTManagerRequest(RequestType, File, version));<br>- ClangRequestCV.notify_one();<br>-}<br>-<br>-void ASTManager::handleRequest(<wbr>ASTManagerRequestType RequestType,<br>- <wbr>StringRef File) {<br>- switch (RequestType) {<br>- case ASTManagerRequestType::<wbr>ParseAndPublishDiagnostics:<br>- <wbr>parseFileAndPublishDiagnostics<wbr>(File);<br>- break;<br>- case ASTManagerRequestType::<wbr>RemoveDocData: {<br>- std::lock_guard<std::mutex> Lock(ClangObjectLock);<br>- auto DocDataIt = DocDatas.find(File);<br>- // We could get the remove request before parsing for the document is<br>- // started, just do nothing in that case, parsing request will be discarded<br>- // because it has a lower version value<br>- if (DocDataIt == DocDatas.end())<br>- return;<br>- DocDatas.erase(DocDataIt);<br>- break;<br>- } // unlock ClangObjectLock<br>- }<br>-}<br>-<br>-void ASTManager::<wbr>parseFileAndPublishDiagnostics<wbr>(StringRef File) {<br>- std::unique_lock<std::mutex> ClangObjectLockGuard(<wbr>ClangObjectLock);<br>-<br>- auto &DocData = DocDatas[File];<br>- ASTUnit *Unit = DocData.getAST();<br>- if (!Unit) {<br>- auto newAST = createASTUnitForFile(File, this->Store);<br>- Unit = newAST.get();<br>-<br>- DocData.setAST(std::move(<wbr>newAST));<br>- } else {<br>- // Do a reparse if this wasn't the first parse.<br>- // FIXME: This might have the wrong working directory if it changed in the<br>- // meantime.<br>- Unit->Reparse(PCHs, getRemappedFiles(this->Store))<wbr>;<br>- }<br>-<br>- if (!Unit)<br>- return;<br>-<br>- // Send the diagnotics to the editor.<br>- // FIXME: If the diagnostic comes from a different file, do we want to<br>- // show them all? Right now we drop everything not coming from the<br>- // main file.<br>- std::string Diagnostics;<br>- DocData::<wbr>DiagnosticToReplacementMap LocalFixIts; // Temporary storage<br>- for (ASTUnit::stored_diag_iterator D = Unit->stored_diag_begin(),<br>- <wbr> DEnd = Unit->stored_diag_end();<br>- D != DEnd; ++D) {<br>- if (!D->getLocation().isValid() ||<br>- !D->getLocation().<wbr>getManager().isInMainFile(D-><wbr>getLocation()))<br>- continue;<br>- Position P;<br>- P.line = D->getLocation().<wbr>getSpellingLineNumber() - 1;<br>- P.character = D->getLocation().<wbr>getSpellingColumnNumber();<br>- Range R = {P, P};<br>- Diagnostics +=<br>- R"({"range":)" + Range::unparse(R) +<br>- R"(,"severity":)" + std::to_string(getSeverity(D-><wbr>getLevel())) +<br>- R"(,"message":")" + llvm::yaml::escape(D-><wbr>getMessage()) +<br>- R"("},)";<br>-<br>- // We convert to Replacements to become independent of the SourceManager.<br>- clangd::Diagnostic Diag = {R, getSeverity(D->getLevel()), D->getMessage()};<br>- auto &FixItsForDiagnostic = LocalFixIts[Diag];<br>- for (const FixItHint &Fix : D->getFixIts()) {<br>- FixItsForDiagnostic.push_<wbr>back(clang::tooling::<wbr>Replacement(<br>- Unit-><wbr>getSourceManager(), Fix.RemoveRange, Fix.CodeToInsert));<br>- }<br>- }<br>-<br>- // Put FixIts into place.<br>- DocData.cacheFixIts(std::<wbr>move(LocalFixIts));<br>-<br>- ClangObjectLockGuard.unlock()<wbr>;<br>- // No accesses to clang objects are allowed after this point.<br>-<br>- // Publish diagnostics.<br>- if (!Diagnostics.empty())<br>- Diagnostics.pop_back(); // Drop trailing comma.<br>- Output.writeMessage(<br>- R"({"jsonrpc":"2.0","<wbr>method":"textDocument/<wbr>publishDiagnostics","params":{<wbr>"uri":")" +<br>- URI::fromFile(File).uri + R"(","diagnostics":[)" + Diagnostics + R"(]}})");<br>-}<br>-<br>-ASTManager::~ASTManager() {<br>- {<br>- std::lock_guard<std::mutex> Guard(RequestLock);<br>- // Wake up the clang worker thread, then exit.<br>- Done = true;<br>- ClangRequestCV.notify_one()<wbr>;<br>- } // unlock DocDataLock<br>- ClangWorker.join();<br>-}<br>-<br>-void ASTManager::onDocumentAdd(<wbr>StringRef File) {<br>- queueOrRun(<wbr>ASTManagerRequestType::<wbr>ParseAndPublishDiagnostics, File);<br>-}<br>-<br>-void ASTManager::onDocumentRemove(<wbr>StringRef File) {<br>- queueOrRun(<wbr>ASTManagerRequestType::<wbr>RemoveDocData, File);<br>-}<br>-<br>-tooling::CompilationDatabase *<br>-ASTManager::<wbr>getOrCreateCompilationDatabase<wbr>ForFile(StringRef File) {<br>- namespace path = llvm::sys::path;<br>-<br>- assert((path::is_absolute(<wbr>File, path::Style::posix) ||<br>- path::is_absolute(<wbr>File, path::Style::windows)) &&<br>- "path must be absolute");<br>-<br>- for (auto Path = path::parent_path(File); !Path.empty();<br>- Path = path::parent_path(Path)) {<br>-<br>- auto CachedIt = CompilationDatabases.find(<wbr>Path);<br>- if (CachedIt != CompilationDatabases.end())<br>- return CachedIt->second.get();<br>- std::string Error;<br>- auto CDB = tooling::CompilationDatabase::<wbr>loadFromDirectory(Path, Error);<br>- if (!CDB) {<br>- if (!Error.empty()) {<br>- Output.log("Error when trying to load compilation database from " +<br>- Twine(Path) + ": " + Twine(Error) + "\n");<br>- }<br>- continue;<br>- }<br>-<br>- // TODO(ibiryukov): Invalidate cached compilation databases on changes<br>- auto result = CDB.get();<br>- CompilationDatabases.<wbr>insert(std::make_pair(Path, std::move(CDB)));<br>- return result;<br>- }<br>-<br>- Output.log("Failed to find compilation database for " + Twine(File) + "\n");<br>- return nullptr;<br>-}<br>-<br>-std::unique_ptr<clang::<wbr>ASTUnit><br>-ASTManager::<wbr>createASTUnitForFile(StringRef File, const DocumentStore &Docs) {<br>- tooling::CompilationDatabase *CDB =<br>- <wbr>getOrCreateCompilationDatabase<wbr>ForFile(File);<br>-<br>- std::vector<tooling::<wbr>CompileCommand> Commands;<br>-<br>- if (CDB) {<br>- Commands = CDB->getCompileCommands(File);<br>- // chdir. This is thread hostile.<br>- if (!Commands.empty())<br>- llvm::sys::fs::set_<wbr>current_path(Commands.front().<wbr>Directory);<br>- }<br>- if (Commands.empty()) {<br>- // Add a fake command line if we know nothing.<br>- Commands.push_back(tooling:<wbr>:CompileCommand(<br>- llvm::sys::path::<wbr>parent_path(File), llvm::sys::path::filename(<wbr>File),<br>- {"clang", "-fsyntax-only", File.str()}, ""));<br>- }<br>-<br>- // Inject the resource dir.<br>- // FIXME: Don't overwrite it if it's already there.<br>- static int Dummy; // Just an address in this process.<br>- std::string ResourceDir =<br>- CompilerInvocation::<wbr>GetResourcesPath("clangd", (void *)&Dummy);<br>- Commands.front().CommandLine.<wbr>push_back("-resource-dir=" + ResourceDir);<br>-<br>- IntrusiveRefCntPtr<<wbr>DiagnosticsEngine> Diags =<br>- CompilerInstance::<wbr>createDiagnostics(new DiagnosticOptions);<br>-<br>- std::vector<const char *> ArgStrs;<br>- for (const auto &S : Commands.front().CommandLine)<br>- ArgStrs.push_back(S.c_str()<wbr>);<br>-<br>- auto ArgP = &*ArgStrs.begin();<br>- return std::unique_ptr<clang::<wbr>ASTUnit>(ASTUnit::<wbr>LoadFromCommandLine(<br>- ArgP, ArgP + ArgStrs.size(), PCHs, Diags, ResourceDir,<br>- /*OnlyLocalDecls=*/false, /*CaptureDiagnostics=*/true,<br>- getRemappedFiles(Docs),<br>- /*<wbr>RemappedFilesKeepOriginalName=<wbr>*/true,<br>- /*<wbr>PrecompilePreambleAfterNParses<wbr>=*/1, /*TUKind=*/TU_Complete,<br>- /*<wbr>CacheCodeCompletionResults=*/<wbr>true,<br>- /*<wbr>IncludeBriefCommentsInCodeComp<wbr>letion=*/true,<br>- /*<wbr>AllowPCHWithCompilerErrors=*/<wbr>true));<br>-}<br>-<br>-std::vector<clang::tooling::<wbr>Replacement><br>-ASTManager::getFixIts(<wbr>StringRef File, const clangd::Diagnostic &D) {<br>- // TODO(ibiryukov): the FixIts should be available immediately<br>- // even when parsing is being run on a worker thread<br>- std::lock_guard<std::mutex> Guard(ClangObjectLock);<br>- return DocDatas[File].getFixIts(D);<br>-}<br>-<br>-namespace {<br>-class CompletionItemsCollector : public CodeCompleteConsumer {<br>- std::vector<CompletionItem> *Items;<br>- std::shared_ptr<clang::<wbr>GlobalCodeCompletionAllocator> Allocator;<br>- CodeCompletionTUInfo CCTUInfo;<br>-<br>-public:<br>- CompletionItemsCollector(std:<wbr>:vector<CompletionItem> *Items,<br>- <wbr>const CodeCompleteOptions &CodeCompleteOpts)<br>- : CodeCompleteConsumer(<wbr>CodeCompleteOpts, /*OutputIsBinary=*/false),<br>- Items(Items),<br>- Allocator(std::make_<wbr>shared<clang::<wbr>GlobalCodeCompletionAllocator><wbr>()),<br>- CCTUInfo(Allocator) {}<br>-<br>- void ProcessCodeCompleteResults(<wbr>Sema &S, CodeCompletionContext Context,<br>- <wbr> CodeCompletionResult *Results,<br>- <wbr> unsigned NumResults) override {<br>- for (unsigned I = 0; I != NumResults; ++I) {<br>- CodeCompletionResult &Result = Results[I];<br>- CodeCompletionString *CCS = Result.<wbr>CreateCodeCompletionString(<br>- S, Context, *Allocator, CCTUInfo,<br>- CodeCompleteOpts.<wbr>IncludeBriefComments);<br>- if (CCS) {<br>- CompletionItem Item;<br>- assert(CCS-><wbr>getTypedText());<br>- Item.label = CCS->getTypedText();<br>- Item.kind = getKind(Result.CursorKind);<br>- if (CCS->getBriefComment())<br>- Item.documentation = CCS->getBriefComment();<br>- Items->push_back(std::<wbr>move(Item));<br>- }<br>- }<br>- }<br>-<br>- GlobalCodeCompletionAllocator &getAllocator() override { return *Allocator; }<br>-<br>- CodeCompletionTUInfo &getCodeCompletionTUInfo() override { return CCTUInfo; }<br>-};<br>-<br>-} // namespace<br>-<br>-std::vector<CompletionItem><br>-ASTManager::codeComplete(<wbr>StringRef File, unsigned Line, unsigned Column) {<br>- CodeCompleteOptions CCO;<br>- CCO.IncludeBriefComments = 1;<br>- // This is where code completion stores dirty buffers. Need to free after<br>- // completion.<br>- SmallVector<const llvm::MemoryBuffer *, 4> OwnedBuffers;<br>- SmallVector<StoredDiagnostic, 4> StoredDiagnostics;<br>- IntrusiveRefCntPtr<<wbr>DiagnosticsEngine> DiagEngine(<br>- new DiagnosticsEngine(new DiagnosticIDs, new DiagnosticOptions));<br>- std::vector<CompletionItem> Items;<br>- CompletionItemsCollector Collector(&Items, CCO);<br>-<br>- std::lock_guard<std::mutex> Guard(ClangObjectLock);<br>- auto &DocData = DocDatas[File];<br>- auto Unit = DocData.getAST();<br>- if (!Unit) {<br>- auto newAST = createASTUnitForFile(File, this->Store);<br>- Unit = newAST.get();<br>- DocData.setAST(std::move(<wbr>newAST));<br>- }<br>- if (!Unit)<br>- return {};<br>- IntrusiveRefCntPtr<<wbr>SourceManager> SourceMgr(<br>- new SourceManager(*DiagEngine, Unit->getFileManager()));<br>- // CodeComplete seems to require fresh LangOptions.<br>- LangOptions LangOpts = Unit->getLangOpts();<br>- // The language server protocol uses zero-based line and column numbers.<br>- // The clang code completion uses one-based numbers.<br>- Unit->CodeComplete(File, Line + 1, Column + 1, getRemappedFiles(this->Store),<br>- CCO.<wbr>IncludeMacros, CCO.IncludeCodePatterns,<br>- CCO.<wbr>IncludeBriefComments, Collector, PCHs, *DiagEngine,<br>- LangOpts, *SourceMgr, Unit->getFileManager(),<br>- <wbr>StoredDiagnostics, OwnedBuffers);<br>- for (const llvm::MemoryBuffer *Buffer : OwnedBuffers)<br>- delete Buffer;<br>- return Items;<br>-}<br><br>Removed: clang-tools-extra/trunk/<wbr>clangd/ASTManager.h<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ASTManager.h?rev=303066&view=auto" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/clang-tools-extra/<wbr>trunk/clangd/ASTManager.h?rev=<wbr>303066&view=auto</a><br>==============================<wbr>==============================<wbr>==================<br>--- clang-tools-extra/trunk/<wbr>clangd/ASTManager.h (original)<br>+++ clang-tools-extra/trunk/<wbr>clangd/ASTManager.h (removed)<br>@@ -1,162 +0,0 @@<br>-//===--- ASTManager.h - Clang AST manager -----------------------*- C++ -*-===//<br>-//<br>-// The LLVM Compiler Infrastructure<br>-//<br>-// This file is distributed under the University of Illinois Open Source<br>-// License. See LICENSE.TXT for details.<br>-//<br>-//===------------------------<wbr>------------------------------<wbr>----------------===//<br>-<br>-#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_<wbr>ASTMANAGER_H<br>-#define LLVM_CLANG_TOOLS_EXTRA_CLANGD_<wbr>ASTMANAGER_H<br>-<br>-#include "DocumentStore.h"<br>-#include "JSONRPCDispatcher.h"<br>-#include "Protocol.h"<br>-#include "clang/Tooling/Core/<wbr>Replacement.h"<br>-#include "llvm/ADT/IntrusiveRefCntPtr.<wbr>h"<br>-#include <condition_variable><br>-#include <deque><br>-#include <thread><br>-<br>-namespace clang {<br>-class ASTUnit;<br>-class DiagnosticsEngine;<br>-class PCHContainerOperations;<br>-namespace tooling {<br>-class CompilationDatabase;<br>-} // namespace tooling<br>-<br>-namespace clangd {<br>-<br>-/// Using 'unsigned' here to avoid undefined behaviour on overflow.<br>-typedef unsigned DocVersion;<br>-<br>-/// Stores ASTUnit and FixIts map for an opened document.<br>-class DocData {<br>-public:<br>- typedef std::map<clangd::Diagnostic, std::vector<clang::tooling::<wbr>Replacement>><br>- <wbr>DiagnosticToReplacementMap;<br>-<br>-public:<br>- void setAST(std::unique_ptr<<wbr>ASTUnit> AST);<br>- ASTUnit *getAST() const;<br>-<br>- void cacheFixIts(<wbr>DiagnosticToReplacementMap FixIts);<br>- std::vector<clang::tooling::<wbr>Replacement><br>- getFixIts(const clangd::Diagnostic &D) const;<br>-<br>-private:<br>- std::unique_ptr<ASTUnit> AST;<br>- DiagnosticToReplacementMap FixIts;<br>-};<br>-<br>-enum class ASTManagerRequestType { ParseAndPublishDiagnostics, RemoveDocData };<br>-<br>-/// A request to the worker thread<br>-class ASTManagerRequest {<br>-public:<br>- ASTManagerRequest() = default;<br>- ASTManagerRequest(<wbr>ASTManagerRequestType Type, std::string File,<br>- DocVersion Version);<br>-<br>- ASTManagerRequestType Type;<br>- std::string File;<br>- DocVersion Version;<br>-};<br>-<br>-class ASTManager : public DocumentStoreListener {<br>-public:<br>- ASTManager(JSONOutput &Output, DocumentStore &Store, bool RunSynchronously);<br>- ~ASTManager() override;<br>-<br>- void onDocumentAdd(StringRef File) override;<br>- void onDocumentRemove(StringRef File) override;<br>-<br>- /// Get code completions at a specified \p Line and \p Column in \p File.<br>- ///<br>- /// This function is thread-safe and returns completion items that own the<br>- /// data they contain.<br>- std::vector<CompletionItem> codeComplete(StringRef File, unsigned Line,<br>- <wbr> unsigned Column);<br>-<br>- /// Get the fixes associated with a certain diagnostic in a specified file as<br>- /// replacements.<br>- ///<br>- /// This function is thread-safe. It returns a copy to avoid handing out<br>- /// references to unguarded data.<br>- std::vector<clang::tooling::<wbr>Replacement><br>- getFixIts(StringRef File, const clangd::Diagnostic &D);<br>-<br>- DocumentStore &getStore() const { return Store; }<br>-<br>-private:<br>- JSONOutput &Output;<br>- DocumentStore &Store;<br>-<br>- // Set to true if requests should block instead of being processed<br>- // asynchronously.<br>- bool RunSynchronously;<br>-<br>- /// Loads a compilation database for File. May return nullptr if it fails. The<br>- /// database is cached for subsequent accesses.<br>- clang::tooling::<wbr>CompilationDatabase *<br>- <wbr>getOrCreateCompilationDatabase<wbr>ForFile(StringRef File);<br>- // Creates a new ASTUnit for the document at File.<br>- // FIXME: This calls chdir internally, which is thread unsafe.<br>- std::unique_ptr<clang::<wbr>ASTUnit><br>- createASTUnitForFile(<wbr>StringRef File, const DocumentStore &Docs);<br>-<br>- /// If RunSynchronously is false, queues the request to be run on the worker<br>- /// thread.<br>- /// If RunSynchronously is true, runs the request handler immediately on the<br>- /// main thread.<br>- void queueOrRun(<wbr>ASTManagerRequestType RequestType, StringRef File);<br>-<br>- void runWorker();<br>- void handleRequest(<wbr>ASTManagerRequestType RequestType, StringRef File);<br>-<br>- /// Parses files and publishes diagnostics.<br>- /// This function is called on the worker thread in asynchronous mode and<br>- /// on the main thread in synchronous mode.<br>- void parseFileAndPublishDiagnostics<wbr>(StringRef File);<br>-<br>- /// Caches compilation databases loaded from directories(keys are directories).<br>- llvm::StringMap<std::unique_<wbr>ptr<clang::tooling::<wbr>CompilationDatabase>><br>- CompilationDatabases;<br>-<br>- /// Clang objects.<br>- /// A map from filenames to DocData structures that store ASTUnit and Fixits for<br>- /// the files. The ASTUnits are used for generating diagnostics and fix-it-s<br>- /// asynchronously by the worker thread and synchronously for code completion.<br>- llvm::StringMap<DocData> DocDatas;<br>- std::shared_ptr<clang::<wbr>PCHContainerOperations> PCHs;<br>- /// A lock for access to the DocDatas, CompilationDatabases and PCHs.<br>- std::mutex ClangObjectLock;<br>-<br>- /// Stores latest versions of the tracked documents to discard outdated requests.<br>- /// Guarded by RequestLock.<br>- /// TODO(ibiryukov): the entries are neved deleted from this map.<br>- llvm::StringMap<DocVersion> DocVersions;<br>-<br>- /// A LIFO queue of requests. Note that requests are discarded if the `version`<br>- /// field is not equal to the one stored inside DocVersions.<br>- /// TODO(krasimir): code completion should always have priority over parsing<br>- /// for diagnostics.<br>- std::deque<ASTManagerRequest> RequestQueue;<br>- /// Setting Done to true will make the worker thread terminate.<br>- bool Done = false;<br>- /// Condition variable to wake up the worker thread.<br>- std::condition_variable ClangRequestCV;<br>- /// Lock for accesses to RequestQueue, DocVersions and Done.<br>- std::mutex RequestLock;<br>-<br>- /// We run parsing on a separate thread. This thread looks into RequestQueue to<br>- /// find requests to handle and terminates when Done is set to true.<br>- std::thread ClangWorker;<br>-};<br>-<br>-} // namespace clangd<br>-} // namespace clang<br>-<br>-#endif<br><br>Modified: clang-tools-extra/trunk/<wbr>clangd/CMakeLists.txt<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/CMakeLists.txt?rev=303067&r1=303066&r2=303067&view=diff" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/clang-tools-extra/<wbr>trunk/clangd/CMakeLists.txt?<wbr>rev=303067&r1=303066&r2=<wbr>303067&view=diff</a><br>==============================<wbr>==============================<wbr>==================<br>--- clang-tools-extra/trunk/<wbr>clangd/CMakeLists.txt (original)<br>+++ clang-tools-extra/trunk/<wbr>clangd/CMakeLists.txt Mon May 15 09:17:35 2017<br>@@ -1,6 +1,11 @@<br> add_clang_executable(clangd<br>- ASTManager.cpp<br>+ ClangdLSPServer.cpp<br> ClangdMain.cpp<br>+ ClangdServer.cpp<br>+ ClangdUnit.cpp<br>+ ClangdUnitStore.cpp<br>+ DraftStore.cpp<br>+ GlobalCompilationDatabase.cpp<br> JSONRPCDispatcher.cpp<br> Protocol.cpp<br> ProtocolHandlers.cpp<br><br>Added: clang-tools-extra/trunk/<wbr>clangd/ClangdLSPServer.cpp<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp?rev=303067&view=auto" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/clang-tools-extra/<wbr>trunk/clangd/ClangdLSPServer.<wbr>cpp?rev=303067&view=auto</a><br>==============================<wbr>==============================<wbr>==================<br>--- clang-tools-extra/trunk/<wbr>clangd/ClangdLSPServer.cpp (added)<br>+++ clang-tools-extra/trunk/<wbr>clangd/ClangdLSPServer.cpp Mon May 15 09:17:35 2017<br>@@ -0,0 +1,100 @@<br>+//===--- ClangdLSPServer.cpp - LSP server ------------------------*- C++-*-===//<br>+//<br>+// The LLVM Compiler Infrastructure<br>+//<br>+// This file is distributed under the University of Illinois Open Source<br>+// License. See LICENSE.TXT for details.<br>+//<br>+//===------------------------<wbr>------------------------------<wbr>---------------===//<br>+<br>+#include "ClangdLSPServer.h"<br>+#include "JSONRPCDispatcher.h"<br>+<br>+using namespace clang::clangd;<br>+using namespace clang;<br>+<br>+class ClangdLSPServer::<wbr>LSPDiagnosticsConsumer : public DiagnosticsConsumer {<br>+public:<br>+ LSPDiagnosticsConsumer(<wbr>ClangdLSPServer &Server) : Server(Server) {}<br>+<br>+ virtual void onDiagnosticsReady(PathRef File,<br>+ <wbr> std::vector<DiagWithFixIts> Diagnostics) {<br>+ Server.consumeDiagnostics(<wbr>File, Diagnostics);<br>+ }<br>+<br>+private:<br>+ ClangdLSPServer &Server;<br>+};<br>+<br>+ClangdLSPServer::<wbr>ClangdLSPServer(JSONOutput &Out, bool RunSynchronously)<br>+ : Out(Out),<br>+ Server(llvm::make_unique<<wbr>DirectoryBasedGlobalCompilatio<wbr>nDatabase>(),<br>+ llvm::make_unique<<wbr>LSPDiagnosticsConsumer>(*this)<wbr>,<br>+ RunSynchronously) {}<br>+<br>+void ClangdLSPServer::openDocument(<wbr>StringRef File, StringRef Contents) {<br>+ Server.addDocument(File, Contents);<br>+}<br>+<br>+void ClangdLSPServer::<wbr>closeDocument(StringRef File) {<br>+ Server.removeDocument(File);<br>+}<br>+<br>+std::vector<CompletionItem> ClangdLSPServer::codeComplete(<wbr>PathRef File,<br>+ <wbr> <wbr>Position Pos) {<br>+ return Server.codeComplete(File, Pos);<br>+}<br>+<br>+std::vector<clang::tooling::<wbr>Replacement><br>+ClangdLSPServer::getFixIts(<wbr>StringRef File, const clangd::Diagnostic &D) {<br>+ std::lock_guard<std::mutex> Lock(FixItsMutex);<br>+ auto DiagToFixItsIter = FixItsMap.find(File);<br>+ if (DiagToFixItsIter == FixItsMap.end())<br>+ return {};<br>+<br>+ const auto &DiagToFixItsMap = DiagToFixItsIter->second;<br>+ auto FixItsIter = DiagToFixItsMap.find(D);<br>+ if (FixItsIter == DiagToFixItsMap.end())<br>+ return {};<br>+<br>+ return FixItsIter->second;<br>+}<br>+<br>+std::string ClangdLSPServer::getDocument(<wbr>PathRef File) {<br>+ return Server.getDocument(File);<br>+}<br>+<br>+void ClangdLSPServer::<wbr>consumeDiagnostics(<br>+ PathRef File, std::vector<DiagWithFixIts> Diagnostics) {<br>+ std::string DiagnosticsJSON;<br>+<br>+ DiagnosticToReplacementMap LocalFixIts; // Temporary storage<br>+ for (auto &DiagWithFixes : Diagnostics) {<br>+ auto Diag = DiagWithFixes.Diag;<br>+ DiagnosticsJSON +=<br>+ R"({"range":)" + Range::unparse(Diag.range) +<br>+ R"(,"severity":)" + std::to_string(Diag.severity) +<br>+ R"(,"message":")" + llvm::yaml::escape(Diag.<wbr>message) +<br>+ R"("},)";<br>+<br>+ // We convert to Replacements to become independent of the SourceManager.<br>+ auto &FixItsForDiagnostic = LocalFixIts[Diag];<br>+ std::copy(DiagWithFixes.<wbr>FixIts.begin(), DiagWithFixes.FixIts.end(),<br>+ std::back_<wbr>inserter(FixItsForDiagnostic))<wbr>;<br>+ }<br>+<br>+ // Cache FixIts<br>+ {<br>+ // FIXME(ibiryukov): should be deleted when documents are removed<br>+ std::lock_guard<std::mutex> Lock(FixItsMutex);<br>+ FixItsMap[File] = LocalFixIts;<br>+ }<br>+<br>+ // Publish diagnostics.<br>+ if (!DiagnosticsJSON.empty())<br>+ DiagnosticsJSON.pop_back(); // Drop trailing comma.<br>+ Out.writeMessage(<br>+ R"({"jsonrpc":"2.0","<wbr>method":"textDocument/<wbr>publishDiagnostics","params":{<wbr>"uri":")" +<br>+ URI::fromFile(File).uri + R"(","diagnostics":[)" + DiagnosticsJSON +<br>+ R"(]}})");<br>+}<br><br>Added: clang-tools-extra/trunk/<wbr>clangd/ClangdLSPServer.h<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdLSPServer.h?rev=303067&view=auto" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/clang-tools-extra/<wbr>trunk/clangd/ClangdLSPServer.<wbr>h?rev=303067&view=auto</a><br>==============================<wbr>==============================<wbr>==================<br>--- clang-tools-extra/trunk/<wbr>clangd/ClangdLSPServer.h (added)<br>+++ clang-tools-extra/trunk/<wbr>clangd/ClangdLSPServer.h Mon May 15 09:17:35 2017<br>@@ -0,0 +1,79 @@<br>+//===--- ClangdLSPServer.h - LSP server --------------------------*- C++-*-===//<br>+//<br>+// The LLVM Compiler Infrastructure<br>+//<br>+// This file is distributed under the University of Illinois Open Source<br>+// License. See LICENSE.TXT for details.<br>+//<br>+//===------------------------<wbr>------------------------------<wbr>---------------===//<br>+<br>+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_<wbr>CLANGDLSPSERVER_H<br>+#define LLVM_CLANG_TOOLS_EXTRA_CLANGD_<wbr>CLANGDLSPSERVER_H<br>+<br>+#include "ClangdServer.h"<br>+#include "Path.h"<br>+#include "Protocol.h"<br>+#include "clang/Tooling/Core/<wbr>Replacement.h"<br>+<br>+namespace clang {<br>+namespace clangd {<br>+<br>+class JSONOutput;<br>+<br>+/// This class serves as an intermediate layer of LSP server implementation,<br>+/// glueing the JSON LSP protocol layer and ClangdServer together. It doesn't<br>+/// directly handle input from LSP client.<br>+/// Most methods are synchronous and return their result directly, but<br>+/// diagnostics are provided asynchronously when ready via<br>+/// JSONOutput::writeMessage.<br>+class ClangdLSPServer {<br>+public:<br>+ ClangdLSPServer(JSONOutput &Out, bool RunSynchronously);<br>+<br>+ /// Update the document text for \p File with \p Contents, schedule update of<br>+ /// diagnostics. Out.writeMessage will called to push diagnostics to LSP<br>+ /// client asynchronously when they are ready.<br>+ void openDocument(PathRef File, StringRef Contents);<br>+ /// Stop tracking the document for \p File.<br>+ void closeDocument(PathRef File);<br>+<br>+ /// Run code completion synchronously.<br>+ std::vector<CompletionItem> codeComplete(PathRef File, Position Pos);<br>+<br>+ /// Get the fixes associated with a certain diagnostic in a specified file as<br>+ /// replacements.<br>+ ///<br>+ /// This function is thread-safe. It returns a copy to avoid handing out<br>+ /// references to unguarded data.<br>+ std::vector<clang::tooling::<wbr>Replacement><br>+ getFixIts(StringRef File, const clangd::Diagnostic &D);<br>+<br>+ /// Get the current document contents stored for \p File.<br>+ /// FIXME(ibiryukov): This function is here to allow implementation of<br>+ /// formatCode from ProtocolHandlers.cpp. We should move formatCode to<br>+ /// ClangdServer class and remove this function from public interface.<br>+ std::string getDocument(PathRef File);<br>+<br>+private:<br>+ class LSPDiagnosticsConsumer;<br>+<br>+ /// Function that will be called on a separate thread when diagnostics are<br>+ /// ready. Sends the Dianostics to LSP client via Out.writeMessage and caches<br>+ /// corresponding fixits in the FixItsMap.<br>+ void consumeDiagnostics(PathRef File,<br>+ std::<wbr>vector<DiagWithFixIts> Diagnostics);<br>+<br>+ JSONOutput &Out;<br>+ ClangdServer Server;<br>+<br>+ std::mutex FixItsMutex;<br>+ typedef std::map<clangd::Diagnostic, std::vector<clang::tooling::<wbr>Replacement>><br>+ <wbr>DiagnosticToReplacementMap;<br>+ /// Caches FixIts per file and diagnostics<br>+ llvm::StringMap<<wbr>DiagnosticToReplacementMap> FixItsMap;<br>+};<br>+<br>+} // namespace clangd<br>+} // namespace clang<br>+<br>+#endif<br><br>Modified: clang-tools-extra/trunk/<wbr>clangd/ClangdMain.cpp<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdMain.cpp?rev=303067&r1=303066&r2=303067&view=diff" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/clang-tools-extra/<wbr>trunk/clangd/ClangdMain.cpp?<wbr>rev=303067&r1=303066&r2=<wbr>303067&view=diff</a><br>==============================<wbr>==============================<wbr>==================<br>--- clang-tools-extra/trunk/<wbr>clangd/ClangdMain.cpp (original)<br>+++ clang-tools-extra/trunk/<wbr>clangd/ClangdMain.cpp Mon May 15 09:17:35 2017<br>@@ -7,15 +7,19 @@<br> //<br> //===-------------------------<wbr>------------------------------<wbr>---------------===//<br><br>-#include "ASTManager.h"<br>-#include "DocumentStore.h"<br> #include "JSONRPCDispatcher.h"<br>+#include "ClangdLSPServer.h"<br>+#include "Protocol.h"<br> #include "ProtocolHandlers.h"<br> #include "llvm/Support/CommandLine.h"<br> #include "llvm/Support/FileSystem.h"<br> #include "llvm/Support/Program.h"<br>+<br> #include <iostream><br>+#include <memory><br> #include <string><br>+<br>+using namespace clang;<br> using namespace clang::clangd;<br><br> static llvm::cl::opt<bool><br>@@ -34,9 +38,7 @@ int main(int argc, char *argv[]) {<br><br> // Set up a document store and intialize all the method handlers for JSONRPC<br> // dispatching.<br>- DocumentStore Store;<br>- ASTManager AST(Out, Store, RunSynchronously);<br>- Store.addListener(&AST);<br>+ ClangdLSPServer LSPServer(Out, RunSynchronously);<br> JSONRPCDispatcher Dispatcher(llvm::make_unique<<wbr>Handler>(Out));<br> Dispatcher.registerHandler("<wbr>initialize",<br> <wbr>llvm::make_unique<<wbr>InitializeHandler>(Out));<br>@@ -45,26 +47,26 @@ int main(int argc, char *argv[]) {<br> Dispatcher.registerHandler("<wbr>shutdown", std::move(ShutdownPtr));<br> Dispatcher.registerHandler(<br> "textDocument/didOpen",<br>- llvm::make_unique<<wbr>TextDocumentDidOpenHandler>(<wbr>Out, Store));<br>+ llvm::make_unique<<wbr>TextDocumentDidOpenHandler>(<wbr>Out, LSPServer));<br> Dispatcher.registerHandler(<br> "textDocument/didClose",<br>- llvm::make_unique<<wbr>TextDocumentDidCloseHandler>(<wbr>Out, Store));<br>+ llvm::make_unique<<wbr>TextDocumentDidCloseHandler>(<wbr>Out, LSPServer));<br> Dispatcher.registerHandler(<br> "textDocument/didChange"<wbr>,<br>- llvm::make_unique<<wbr>TextDocumentDidChangeHandler>(<wbr>Out, Store));<br>+ llvm::make_unique<<wbr>TextDocumentDidChangeHandler>(<wbr>Out, LSPServer));<br> Dispatcher.registerHandler(<br> "textDocument/<wbr>rangeFormatting",<br>- llvm::make_unique<<wbr>TextDocumentRangeFormattingHan<wbr>dler>(Out, Store));<br>+ llvm::make_unique<<wbr>TextDocumentRangeFormattingHan<wbr>dler>(Out, LSPServer));<br> Dispatcher.registerHandler(<br> "textDocument/<wbr>onTypeFormatting",<br>- llvm::make_unique<<wbr>TextDocumentOnTypeFormattingHa<wbr>ndler>(Out, Store));<br>+ llvm::make_unique<<wbr>TextDocumentOnTypeFormattingHa<wbr>ndler>(Out, LSPServer));<br> Dispatcher.registerHandler(<br> "textDocument/<wbr>formatting",<br>- llvm::make_unique<<wbr>TextDocumentFormattingHandler><wbr>(Out, Store));<br>+ llvm::make_unique<<wbr>TextDocumentFormattingHandler><wbr>(Out, LSPServer));<br> Dispatcher.registerHandler("<wbr>textDocument/codeAction",<br>- <wbr>llvm::make_unique<<wbr>CodeActionHandler>(Out, AST));<br>+ <wbr>llvm::make_unique<<wbr>CodeActionHandler>(Out, LSPServer));<br> Dispatcher.registerHandler("<wbr>textDocument/completion",<br>- <wbr>llvm::make_unique<<wbr>CompletionHandler>(Out, AST));<br>+ <wbr>llvm::make_unique<<wbr>CompletionHandler>(Out, LSPServer));<br><br> while (std::cin.good()) {<br> // A Language Server Protocol message starts with a HTTP header, delimited<br><br>Copied: clang-tools-extra/trunk/<wbr>clangd/ClangdServer.cpp (from r303060, clang-tools-extra/trunk/<wbr>clangd/ASTManager.cpp)<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.cpp?p2=clang-tools-extra/trunk/clangd/ClangdServer.cpp&p1=clang-tools-extra/trunk/clangd/ASTManager.cpp&r1=303060&r2=303067&rev=303067&view=diff" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/clang-tools-extra/<wbr>trunk/clangd/ClangdServer.cpp?<wbr>p2=clang-tools-extra/trunk/<wbr>clangd/ClangdServer.cpp&p1=<wbr>clang-tools-extra/trunk/<wbr>clangd/ASTManager.cpp&r1=<wbr>303060&r2=303067&rev=303067&<wbr>view=diff</a><br>==============================<wbr>==============================<wbr>==================<br>--- clang-tools-extra/trunk/<wbr>clangd/ASTManager.cpp (original)<br>+++ clang-tools-extra/trunk/<wbr>clangd/ClangdServer.cpp Mon May 15 09:17:35 2017<br>@@ -1,440 +1,149 @@<br>-//===--- ASTManager.cpp - Clang AST manager ------------------------------<wbr>-===//<br>+//===--- ClangdServer.cpp - Main clangd server code --------------*- C++-*-===//<br> //<br> // The LLVM Compiler Infrastructure<br> //<br> // This file is distributed under the University of Illinois Open Source<br> // License. See LICENSE.TXT for details.<br> //<br>-//===------------------------<wbr>------------------------------<wbr>----------------===//<br>+//===------------------------<wbr>------------------------------<wbr>-------------===//<br><br>-#include "ASTManager.h"<br>-#include "JSONRPCDispatcher.h"<br>-#include "Protocol.h"<br>+#include "ClangdServer.h"<br> #include "clang/Frontend/ASTUnit.h"<br> #include "clang/Frontend/<wbr>CompilerInstance.h"<br>+#include "clang/Frontend/<wbr>CompilerInvocation.h"<br> #include "clang/Tooling/<wbr>CompilationDatabase.h"<br>-#include "llvm/Support/Format.h"<br>-#include "llvm/Support/Path.h"<br>-#include <mutex><br>-#include <thread><br>-using namespace clang;<br>-using namespace clangd;<br>+#include "llvm/Support/FileSystem.h"<br><br>-void DocData::setAST(std::unique_<wbr>ptr<ASTUnit> AST) {<br>- this->AST = std::move(AST);<br>-}<br>-<br>-ASTUnit *DocData::getAST() const { return AST.get(); }<br>-<br>-void DocData::cacheFixIts(<wbr>DiagnosticToReplacementMap FixIts) {<br>- this->FixIts = std::move(FixIts);<br>-}<br>+using namespace clang::clangd;<br><br>-std::vector<clang::tooling::<wbr>Replacement><br>-DocData::getFixIts(const clangd::Diagnostic &D) const {<br>- auto it = FixIts.find(D);<br>- if (it != FixIts.end())<br>- return it->second;<br>- return {};<br>-}<br>-<br>-ASTManagerRequest::<wbr>ASTManagerRequest(<wbr>ASTManagerRequestType Type,<br>- <wbr> std::string File,<br>- <wbr> DocVersion Version)<br>- : Type(Type), File(File), Version(Version) {}<br>-<br>-/// Retrieve a copy of the contents of every file in the store, for feeding into<br>-/// ASTUnit.<br>-static std::vector<ASTUnit::<wbr>RemappedFile><br>-getRemappedFiles(const DocumentStore &Docs) {<br>- // FIXME: Use VFS instead. This would allow us to get rid of the chdir below.<br>- std::vector<ASTUnit::<wbr>RemappedFile> RemappedFiles;<br>- for (const auto &P : Docs.getAllDocuments()) {<br>- StringRef FileName = P.first;<br>- RemappedFiles.push_back(<wbr>ASTUnit::RemappedFile(<br>- FileName,<br>- llvm::MemoryBuffer::<wbr>getMemBufferCopy(P.second, FileName).release()));<br>- }<br>- return RemappedFiles;<br>-}<br>+WorkerRequest::WorkerRequest(<wbr>WorkerRequestKind Kind, Path File,<br>+ <wbr>DocVersion Version)<br>+ : Kind(Kind), File(File), Version(Version) {}<br><br>-/// Convert from clang diagnostic level to LSP severity.<br>-static int getSeverity(DiagnosticsEngine:<wbr>:Level L) {<br>- switch (L) {<br>- case DiagnosticsEngine::Remark:<br>- return 4;<br>- case DiagnosticsEngine::Note:<br>- return 3;<br>- case DiagnosticsEngine::Warning:<br>- return 2;<br>- case DiagnosticsEngine::Fatal:<br>- case DiagnosticsEngine::Error:<br>- return 1;<br>- case DiagnosticsEngine::Ignored:<br>- return 0;<br>- }<br>- llvm_unreachable("Unknown diagnostic level!");<br>-}<br>-<br>-static CompletionItemKind getKind(CXCursorKind K) {<br>- switch (K) {<br>- case CXCursor_MacroInstantiation:<br>- case CXCursor_MacroDefinition:<br>- return CompletionItemKind::Text;<br>- case CXCursor_CXXMethod:<br>- return CompletionItemKind::Method;<br>- case CXCursor_FunctionDecl:<br>- case CXCursor_FunctionTemplate:<br>- return CompletionItemKind::Function;<br>- case CXCursor_Constructor:<br>- case CXCursor_Destructor:<br>- return CompletionItemKind::<wbr>Constructor;<br>- case CXCursor_FieldDecl:<br>- return CompletionItemKind::Field;<br>- case CXCursor_VarDecl:<br>- case CXCursor_ParmDecl:<br>- return CompletionItemKind::Variable;<br>- case CXCursor_ClassDecl:<br>- case CXCursor_StructDecl:<br>- case CXCursor_UnionDecl:<br>- case CXCursor_ClassTemplate:<br>- case CXCursor_<wbr>ClassTemplatePartialSpecializa<wbr>tion:<br>- return CompletionItemKind::Class;<br>- case CXCursor_Namespace:<br>- case CXCursor_NamespaceAlias:<br>- case CXCursor_NamespaceRef:<br>- return CompletionItemKind::Module;<br>- case CXCursor_EnumConstantDecl:<br>- return CompletionItemKind::Value;<br>- case CXCursor_EnumDecl:<br>- return CompletionItemKind::Enum;<br>- case CXCursor_TypeAliasDecl:<br>- case CXCursor_<wbr>TypeAliasTemplateDecl:<br>- case CXCursor_TypedefDecl:<br>- case CXCursor_MemberRef:<br>- case CXCursor_TypeRef:<br>- return CompletionItemKind::Reference;<br>- default:<br>- return CompletionItemKind::Missing;<br>- }<br>-}<br>-<br>-ASTManager::ASTManager(<wbr>JSONOutput &Output, DocumentStore &Store,<br>- bool RunSynchronously)<br>- : Output(Output), Store(Store), RunSynchronously(<wbr>RunSynchronously),<br>- PCHs(std::make_shared<<wbr>PCHContainerOperations>()),<br>- ClangWorker([this]() { runWorker(); }) {}<br>-<br>-void ASTManager::runWorker() {<br>- while (true) {<br>- ASTManagerRequest Request;<br>-<br>- // Pick request from the queue<br>- {<br>- std::unique_lock<std::<wbr>mutex> Lock(RequestLock);<br>- // Wait for more requests.<br>- ClangRequestCV.wait(Lock,<br>- [<wbr>this] { return !RequestQueue.empty() || Done; });<br>- if (Done)<br>- return;<br>- assert(!RequestQueue.<wbr>empty() && "RequestQueue was empty");<br>-<br>- Request = std::move(RequestQueue.back())<wbr>;<br>- RequestQueue.pop_back();<br>-<br>- // Skip outdated requests<br>- if (Request.Version != DocVersions.find(Request.File)<wbr>->second) {<br>- Output.log("Version for " + Twine(Request.File) +<br>- " in request is outdated, skipping request\n");<br>- continue;<br>- }<br>- } // unlock RequestLock<br>-<br>- handleRequest(Request.Type, Request.File);<br>- }<br>-}<br>-<br>-void ASTManager::queueOrRun(<wbr>ASTManagerRequestType RequestType, StringRef File) {<br>+ClangdScheduler::<wbr>ClangdScheduler(ClangdServer &Server, bool RunSynchronously)<br>+ : RunSynchronously(<wbr>RunSynchronously) {<br> if (RunSynchronously) {<br>- handleRequest(RequestType, File);<br>+ // Don't start the worker thread if we're running synchronously<br> return;<br> }<br><br>- std::lock_guard<std::mutex> Guard(RequestLock);<br>- // We increment the version of the added document immediately and schedule<br>- // the requested operation to be run on a worker thread<br>- DocVersion version = ++DocVersions[File];<br>- RequestQueue.push_back(<wbr>ASTManagerRequest(RequestType, File, version));<br>- ClangRequestCV.notify_one();<br>-}<br>-<br>-void ASTManager::handleRequest(<wbr>ASTManagerRequestType RequestType,<br>- <wbr>StringRef File) {<br>- switch (RequestType) {<br>- case ASTManagerRequestType::<wbr>ParseAndPublishDiagnostics:<br>- <wbr>parseFileAndPublishDiagnostics<wbr>(File);<br>- break;<br>- case ASTManagerRequestType::<wbr>RemoveDocData: {<br>- std::lock_guard<std::mutex> Lock(ClangObjectLock);<br>- auto DocDataIt = DocDatas.find(File);<br>- // We could get the remove request before parsing for the document is<br>- // started, just do nothing in that case, parsing request will be discarded<br>- // because it has a lower version value<br>- if (DocDataIt == DocDatas.end())<br>- return;<br>- DocDatas.erase(DocDataIt);<br>- break;<br>- } // unlock ClangObjectLock<br>- }<br>-}<br>-<br>-void ASTManager::<wbr>parseFileAndPublishDiagnostics<wbr>(StringRef File) {<br>- std::unique_lock<std::mutex> ClangObjectLockGuard(<wbr>ClangObjectLock);<br>-<br>- auto &DocData = DocDatas[File];<br>- ASTUnit *Unit = DocData.getAST();<br>- if (!Unit) {<br>- auto newAST = createASTUnitForFile(File, this->Store);<br>- Unit = newAST.get();<br>-<br>- DocData.setAST(std::move(<wbr>newAST));<br>- } else {<br>- // Do a reparse if this wasn't the first parse.<br>- // FIXME: This might have the wrong working directory if it changed in the<br>- // meantime.<br>- Unit->Reparse(PCHs, getRemappedFiles(this->Store))<wbr>;<br>- }<br>+ // Initialize Worker in ctor body, rather than init list to avoid potentially<br>+ // using not-yet-initialized members<br>+ Worker = std::thread([&Server, this]() {<br>+ while (true) {<br>+ WorkerRequest Request;<br>+<br>+ // Pick request from the queue<br>+ {<br>+ std::unique_lock<std::<wbr>mutex> Lock(Mutex);<br>+ // Wait for more requests.<br>+ RequestCV.wait(Lock, [this] { return !RequestQueue.empty() || Done; });<br>+ if (Done)<br>+ return;<br>+<br>+ assert(!RequestQueue.<wbr>empty() && "RequestQueue was empty");<br>+<br>+ Request = std::move(RequestQueue.back())<wbr>;<br>+ RequestQueue.pop_back()<wbr>;<br>+<br>+ // Skip outdated requests<br>+ if (Request.Version != Server.DraftMgr.getVersion(<wbr>Request.File)) {<br>+ // FIXME(ibiryukov): Logging<br>+ // Output.log("Version for " + Twine(Request.File) +<br>+ // " in request is outdated, skipping request\n");<br>+ continue;<br>+ }<br>+ } // unlock Mutex<br><br>- if (!Unit)<br>- return;<br>-<br>- // Send the diagnotics to the editor.<br>- // FIXME: If the diagnostic comes from a different file, do we want to<br>- // show them all? Right now we drop everything not coming from the<br>- // main file.<br>- std::string Diagnostics;<br>- DocData::<wbr>DiagnosticToReplacementMap LocalFixIts; // Temporary storage<br>- for (ASTUnit::stored_diag_iterator D = Unit->stored_diag_begin(),<br>- <wbr> DEnd = Unit->stored_diag_end();<br>- D != DEnd; ++D) {<br>- if (!D->getLocation().isValid() ||<br>- !D->getLocation().<wbr>getManager().isInMainFile(D-><wbr>getLocation()))<br>- continue;<br>- Position P;<br>- P.line = D->getLocation().<wbr>getSpellingLineNumber() - 1;<br>- P.character = D->getLocation().<wbr>getSpellingColumnNumber();<br>- Range R = {P, P};<br>- Diagnostics +=<br>- R"({"range":)" + Range::unparse(R) +<br>- R"(,"severity":)" + std::to_string(getSeverity(D-><wbr>getLevel())) +<br>- R"(,"message":")" + llvm::yaml::escape(D-><wbr>getMessage()) +<br>- R"("},)";<br>-<br>- // We convert to Replacements to become independent of the SourceManager.<br>- clangd::Diagnostic Diag = {R, getSeverity(D->getLevel()), D->getMessage()};<br>- auto &FixItsForDiagnostic = LocalFixIts[Diag];<br>- for (const FixItHint &Fix : D->getFixIts()) {<br>- FixItsForDiagnostic.push_<wbr>back(clang::tooling::<wbr>Replacement(<br>- Unit-><wbr>getSourceManager(), Fix.RemoveRange, Fix.CodeToInsert));<br>+ Server.handleRequest(std:<wbr>:move(Request));<br> }<br>- }<br>-<br>- // Put FixIts into place.<br>- DocData.cacheFixIts(std::<wbr>move(LocalFixIts));<br>-<br>- ClangObjectLockGuard.unlock()<wbr>;<br>- // No accesses to clang objects are allowed after this point.<br>-<br>- // Publish diagnostics.<br>- if (!Diagnostics.empty())<br>- Diagnostics.pop_back(); // Drop trailing comma.<br>- Output.writeMessage(<br>- R"({"jsonrpc":"2.0","<wbr>method":"textDocument/<wbr>publishDiagnostics","params":{<wbr>"uri":")" +<br>- URI::fromFile(File).uri + R"(","diagnostics":[)" + Diagnostics + R"(]}})");<br>+ });<br> }<br><br>-ASTManager::~ASTManager() {<br>+ClangdScheduler::~<wbr>ClangdScheduler() {<br>+ if (RunSynchronously)<br>+ return; // no worker thread is running in that case<br>+<br> {<br>- std::lock_guard<std::mutex> Guard(RequestLock);<br>- // Wake up the clang worker thread, then exit.<br>+ std::lock_guard<std::mutex> Lock(Mutex);<br>+ // Wake up the worker thread<br> Done = true;<br>- ClangRequestCV.notify_one()<wbr>;<br>- } // unlock DocDataLock<br>- ClangWorker.join();<br>-}<br>-<br>-void ASTManager::onDocumentAdd(<wbr>StringRef File) {<br>- queueOrRun(<wbr>ASTManagerRequestType::<wbr>ParseAndPublishDiagnostics, File);<br>-}<br>-<br>-void ASTManager::onDocumentRemove(<wbr>StringRef File) {<br>- queueOrRun(<wbr>ASTManagerRequestType::<wbr>RemoveDocData, File);<br>-}<br>-<br>-tooling::CompilationDatabase *<br>-ASTManager::<wbr>getOrCreateCompilationDatabase<wbr>ForFile(StringRef File) {<br>- namespace path = llvm::sys::path;<br>-<br>- assert((path::is_absolute(<wbr>File, path::Style::posix) ||<br>- path::is_absolute(<wbr>File, path::Style::windows)) &&<br>- "path must be absolute");<br>-<br>- for (auto Path = path::parent_path(File); !Path.empty();<br>- Path = path::parent_path(Path)) {<br>-<br>- auto CachedIt = CompilationDatabases.find(<wbr>Path);<br>- if (CachedIt != CompilationDatabases.end())<br>- return CachedIt->second.get();<br>- std::string Error;<br>- auto CDB = tooling::CompilationDatabase::<wbr>loadFromDirectory(Path, Error);<br>- if (!CDB) {<br>- if (!Error.empty()) {<br>- Output.log("Error when trying to load compilation database from " +<br>- Twine(Path) + ": " + Twine(Error) + "\n");<br>- }<br>- continue;<br>- }<br>-<br>- // TODO(ibiryukov): Invalidate cached compilation databases on changes<br>- auto result = CDB.get();<br>- CompilationDatabases.<wbr>insert(std::make_pair(Path, std::move(CDB)));<br>- return result;<br>- }<br>-<br>- Output.log("Failed to find compilation database for " + Twine(File) + "\n");<br>- return nullptr;<br>+ RequestCV.notify_one();<br>+ } // unlock Mutex<br>+ Worker.join();<br> }<br><br>-std::unique_ptr<clang::<wbr>ASTUnit><br>-ASTManager::<wbr>createASTUnitForFile(StringRef File, const DocumentStore &Docs) {<br>- tooling::CompilationDatabase *CDB =<br>- <wbr>getOrCreateCompilationDatabase<wbr>ForFile(File);<br>-<br>- std::vector<tooling::<wbr>CompileCommand> Commands;<br>-<br>- if (CDB) {<br>- Commands = CDB->getCompileCommands(File);<br>- // chdir. This is thread hostile.<br>- if (!Commands.empty())<br>- llvm::sys::fs::set_<wbr>current_path(Commands.front().<wbr>Directory);<br>- }<br>- if (Commands.empty()) {<br>- // Add a fake command line if we know nothing.<br>- Commands.push_back(tooling:<wbr>:CompileCommand(<br>- llvm::sys::path::<wbr>parent_path(File), llvm::sys::path::filename(<wbr>File),<br>- {"clang", "-fsyntax-only", File.str()}, ""));<br>+void ClangdScheduler::enqueue(<wbr>ClangdServer &Server, WorkerRequest Request) {<br>+ if (RunSynchronously) {<br>+ Server.handleRequest(<wbr>Request);<br>+ return;<br> }<br><br>- // Inject the resource dir.<br>- // FIXME: Don't overwrite it if it's already there.<br>- static int Dummy; // Just an address in this process.<br>- std::string ResourceDir =<br>- CompilerInvocation::<wbr>GetResourcesPath("clangd", (void *)&Dummy);<br>- Commands.front().CommandLine.<wbr>push_back("-resource-dir=" + ResourceDir);<br>-<br>- IntrusiveRefCntPtr<<wbr>DiagnosticsEngine> Diags =<br>- CompilerInstance::<wbr>createDiagnostics(new DiagnosticOptions);<br>-<br>- std::vector<const char *> ArgStrs;<br>- for (const auto &S : Commands.front().CommandLine)<br>- ArgStrs.push_back(S.c_str()<wbr>);<br>-<br>- auto ArgP = &*ArgStrs.begin();<br>- return std::unique_ptr<clang::<wbr>ASTUnit>(ASTUnit::<wbr>LoadFromCommandLine(<br>- ArgP, ArgP + ArgStrs.size(), PCHs, Diags, ResourceDir,<br>- /*OnlyLocalDecls=*/false, /*CaptureDiagnostics=*/true,<br>- getRemappedFiles(Docs),<br>- /*<wbr>RemappedFilesKeepOriginalName=<wbr>*/true,<br>- /*<wbr>PrecompilePreambleAfterNParses<wbr>=*/1, /*TUKind=*/TU_Complete,<br>- /*<wbr>CacheCodeCompletionResults=*/<wbr>true,<br>- /*<wbr>IncludeBriefCommentsInCodeComp<wbr>letion=*/true,<br>- /*<wbr>AllowPCHWithCompilerErrors=*/<wbr>true));<br>+ std::lock_guard<std::mutex> Lock(Mutex);<br>+ RequestQueue.push_back(<wbr>Request);<br>+ RequestCV.notify_one();<br> }<br><br>-std::vector<clang::tooling::<wbr>Replacement><br>-ASTManager::getFixIts(<wbr>StringRef File, const clangd::Diagnostic &D) {<br>- // TODO(ibiryukov): the FixIts should be available immediately<br>- // even when parsing is being run on a worker thread<br>- std::lock_guard<std::mutex> Guard(ClangObjectLock);<br>- return DocDatas[File].getFixIts(D);<br>-}<br>+ClangdServer::ClangdServer(<wbr>std::unique_ptr<<wbr>GlobalCompilationDatabase> CDB,<br>+ std:<wbr>:unique_ptr<<wbr>DiagnosticsConsumer> DiagConsumer,<br>+ bool RunSynchronously)<br>+ : CDB(std::move(CDB)), DiagConsumer(std::move(<wbr>DiagConsumer)),<br>+ PCHs(std::make_shared<<wbr>PCHContainerOperations>()),<br>+ WorkScheduler(*this, RunSynchronously) {}<br><br>-namespace {<br>-class CompletionItemsCollector : public CodeCompleteConsumer {<br>- std::vector<CompletionItem> *Items;<br>- std::shared_ptr<clang::<wbr>GlobalCodeCompletionAllocator> Allocator;<br>- CodeCompletionTUInfo CCTUInfo;<br>-<br>-public:<br>- CompletionItemsCollector(std:<wbr>:vector<CompletionItem> *Items,<br>- <wbr>const CodeCompleteOptions &CodeCompleteOpts)<br>- : CodeCompleteConsumer(<wbr>CodeCompleteOpts, /*OutputIsBinary=*/false),<br>- Items(Items),<br>- Allocator(std::make_<wbr>shared<clang::<wbr>GlobalCodeCompletionAllocator><wbr>()),<br>- CCTUInfo(Allocator) {}<br>-<br>- void ProcessCodeCompleteResults(<wbr>Sema &S, CodeCompletionContext Context,<br>- <wbr> CodeCompletionResult *Results,<br>- <wbr> unsigned NumResults) override {<br>- for (unsigned I = 0; I != NumResults; ++I) {<br>- CodeCompletionResult &Result = Results[I];<br>- CodeCompletionString *CCS = Result.<wbr>CreateCodeCompletionString(<br>- S, Context, *Allocator, CCTUInfo,<br>- CodeCompleteOpts.<wbr>IncludeBriefComments);<br>- if (CCS) {<br>- CompletionItem Item;<br>- assert(CCS-><wbr>getTypedText());<br>- Item.label = CCS->getTypedText();<br>- Item.kind = getKind(Result.CursorKind);<br>- if (CCS->getBriefComment())<br>- Item.documentation = CCS->getBriefComment();<br>- Items->push_back(std::<wbr>move(Item));<br>- }<br>- }<br>+void ClangdServer::addDocument(<wbr>PathRef File, StringRef Contents) {<br>+ DocVersion NewVersion = DraftMgr.updateDraft(File, Contents);<br>+ WorkScheduler.enqueue(<br>+ *this, WorkerRequest(<wbr>WorkerRequestKind::<wbr>ParseAndPublishDiagnostics, File,<br>+ <wbr>NewVersion));<br>+}<br>+<br>+void ClangdServer::removeDocument(<wbr>PathRef File) {<br>+ auto NewVersion = DraftMgr.removeDraft(File);<br>+ WorkScheduler.enqueue(<br>+ *this, WorkerRequest(<wbr>WorkerRequestKind::<wbr>RemoveDocData, File, NewVersion));<br>+}<br>+<br>+std::vector<CompletionItem> ClangdServer::codeComplete(<wbr>PathRef File,<br>+ <wbr> <wbr>Position Pos) {<br>+ auto FileContents = DraftMgr.getDraft(File);<br>+ assert(FileContents.Draft && "codeComplete is called for non-added document");<br>+<br>+ std::vector<CompletionItem> Result;<br>+ Units.<wbr>runOnUnitWithoutReparse(<br>+ File, *FileContents.Draft, *CDB, PCHs, [&](ClangdUnit &Unit) {<br>+ Result = Unit.codeComplete(*<wbr>FileContents.Draft, Pos);<br>+ });<br>+ return Result;<br>+}<br>+<br>+std::string ClangdServer::getDocument(<wbr>PathRef File) {<br>+ auto draft = DraftMgr.getDraft(File);<br>+ assert(draft.Draft && "File is not tracked, cannot get contents");<br>+ return *draft.Draft;<br>+}<br>+<br>+void ClangdServer::handleRequest(<wbr>WorkerRequest Request) {<br>+ switch (Request.Kind) {<br>+ case WorkerRequestKind::<wbr>ParseAndPublishDiagnostics: {<br>+ auto FileContents = DraftMgr.getDraft(Request.<wbr>File);<br>+ if (FileContents.Version != Request.Version)<br>+ return; // This request is outdated, do nothing<br>+<br>+ assert(FileContents.Draft &&<br>+ "No contents inside a file that was scheduled for reparse");<br>+ Units.runOnUnit(Request.<wbr>File, *FileContents.Draft, *CDB, PCHs,<br>+ [&](<wbr>ClangdUnit const &Unit) {<br>+ <wbr>DiagConsumer-><wbr>onDiagnosticsReady(<br>+ <wbr>Request.File, Unit.getLocalDiagnostics());<br>+ });<br>+ break;<br> }<br>+ case WorkerRequestKind::<wbr>RemoveDocData:<br>+ if (Request.Version != DraftMgr.getVersion(Request.<wbr>File))<br>+ return; // This request is outdated, do nothing<br><br>- GlobalCodeCompletionAllocator &getAllocator() override { return *Allocator; }<br>-<br>- CodeCompletionTUInfo &getCodeCompletionTUInfo() override { return CCTUInfo; }<br>-};<br>-<br>-} // namespace<br>-<br>-std::vector<CompletionItem><br>-ASTManager::codeComplete(<wbr>StringRef File, unsigned Line, unsigned Column) {<br>- CodeCompleteOptions CCO;<br>- CCO.IncludeBriefComments = 1;<br>- // This is where code completion stores dirty buffers. Need to free after<br>- // completion.<br>- SmallVector<const llvm::MemoryBuffer *, 4> OwnedBuffers;<br>- SmallVector<StoredDiagnostic, 4> StoredDiagnostics;<br>- IntrusiveRefCntPtr<<wbr>DiagnosticsEngine> DiagEngine(<br>- new DiagnosticsEngine(new DiagnosticIDs, new DiagnosticOptions));<br>- std::vector<CompletionItem> Items;<br>- CompletionItemsCollector Collector(&Items, CCO);<br>-<br>- std::lock_guard<std::mutex> Guard(ClangObjectLock);<br>- auto &DocData = DocDatas[File];<br>- auto Unit = DocData.getAST();<br>- if (!Unit) {<br>- auto newAST = createASTUnitForFile(File, this->Store);<br>- Unit = newAST.get();<br>- DocData.setAST(std::move(<wbr>newAST));<br>+ Units.removeUnitIfPresent(<wbr>Request.File);<br>+ break;<br> }<br>- if (!Unit)<br>- return {};<br>- IntrusiveRefCntPtr<<wbr>SourceManager> SourceMgr(<br>- new SourceManager(*DiagEngine, Unit->getFileManager()));<br>- // CodeComplete seems to require fresh LangOptions.<br>- LangOptions LangOpts = Unit->getLangOpts();<br>- // The language server protocol uses zero-based line and column numbers.<br>- // The clang code completion uses one-based numbers.<br>- Unit->CodeComplete(File, Line + 1, Column + 1, getRemappedFiles(this->Store),<br>- CCO.<wbr>IncludeMacros, CCO.IncludeCodePatterns,<br>- CCO.<wbr>IncludeBriefComments, Collector, PCHs, *DiagEngine,<br>- LangOpts, *SourceMgr, Unit->getFileManager(),<br>- <wbr>StoredDiagnostics, OwnedBuffers);<br>- for (const llvm::MemoryBuffer *Buffer : OwnedBuffers)<br>- delete Buffer;<br>- return Items;<br> }<br><br>Copied: clang-tools-extra/trunk/<wbr>clangd/ClangdServer.h (from r303063, clang-tools-extra/trunk/<wbr>clangd/ASTManager.h)<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.h?p2=clang-tools-extra/trunk/clangd/ClangdServer.h&p1=clang-tools-extra/trunk/clangd/ASTManager.h&r1=303063&r2=303067&rev=303067&view=diff" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/clang-tools-extra/<wbr>trunk/clangd/ClangdServer.h?<wbr>p2=clang-tools-extra/trunk/<wbr>clangd/ClangdServer.h&p1=<wbr>clang-tools-extra/trunk/<wbr>clangd/ASTManager.h&r1=303063&<wbr>r2=303067&rev=303067&view=diff</a><br>==============================<wbr>==============================<wbr>==================<br>--- clang-tools-extra/trunk/<wbr>clangd/ASTManager.h (original)<br>+++ clang-tools-extra/trunk/<wbr>clangd/ClangdServer.h Mon May 15 09:17:35 2017<br>@@ -1,4 +1,4 @@<br>-//===--- ASTManager.h - Clang AST manager -----------------------*- C++ -*-===//<br>+//===--- ClangdServer.h - Main clangd server code ----------------*- C++-*-===//<br> //<br> // The LLVM Compiler Infrastructure<br> //<br>@@ -7,153 +7,129 @@<br> //<br> //===-------------------------<wbr>------------------------------<wbr>---------------===//<br><br>-#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_<wbr>ASTMANAGER_H<br>-#define LLVM_CLANG_TOOLS_EXTRA_CLANGD_<wbr>ASTMANAGER_H<br>+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_<wbr>CLANGDSERVER_H<br>+#define LLVM_CLANG_TOOLS_EXTRA_CLANGD_<wbr>CLANGDSERVER_H<br><br>-#include "DocumentStore.h"<br>-#include "JSONRPCDispatcher.h"<br>-#include "Protocol.h"<br>+#include "ClangdUnitStore.h"<br>+#include "DraftStore.h"<br>+#include "GlobalCompilationDatabase.h"<br>+#include "clang/Frontend/ASTUnit.h"<br>+#include "clang/Tooling/<wbr>CompilationDatabase.h"<br> #include "clang/Tooling/Core/<wbr>Replacement.h"<br> #include "llvm/ADT/IntrusiveRefCntPtr.<wbr>h"<br>+#include "llvm/ADT/Optional.h"<br>+#include "llvm/ADT/StringRef.h"<br>+<br>+#include "ClangdUnit.h"<br>+#include "Protocol.h"<br>+<br> #include <condition_variable><br>-#include <deque><br>+#include <mutex><br>+#include <string><br> #include <thread><br>+#include <utility><br><br> namespace clang {<br>-class ASTUnit;<br>-class DiagnosticsEngine;<br> class PCHContainerOperations;<br>-namespace tooling {<br>-class CompilationDatabase;<br>-} // namespace tooling<br><br> namespace clangd {<br><br>-/// Using 'unsigned' here to avoid undefined behaviour on overflow.<br>-typedef unsigned DocVersion;<br>-<br>-/// Stores ASTUnit and FixIts map for an opened document.<br>-class DocData {<br>-public:<br>- typedef std::map<clangd::Diagnostic, std::vector<clang::tooling::<wbr>Replacement>><br>- <wbr>DiagnosticToReplacementMap;<br>-<br>+class DiagnosticsConsumer {<br> public:<br>- void setAST(std::unique_ptr<<wbr>ASTUnit> AST);<br>- ASTUnit *getAST() const;<br>-<br>- void cacheFixIts(<wbr>DiagnosticToReplacementMap FixIts);<br>- std::vector<clang::tooling::<wbr>Replacement><br>- getFixIts(const clangd::Diagnostic &D) const;<br>+ virtual ~DiagnosticsConsumer() = default;<br><br>-private:<br>- std::unique_ptr<ASTUnit> AST;<br>- DiagnosticToReplacementMap FixIts;<br>+ /// Called by ClangdServer when \p Diagnostics for \p File are ready.<br>+ virtual void onDiagnosticsReady(PathRef File,<br>+ <wbr> std::vector<DiagWithFixIts> Diagnostics) = 0;<br> };<br><br>-enum class ASTManagerRequestType { ParseAndPublishDiagnostics, RemoveDocData };<br>+enum class WorkerRequestKind { ParseAndPublishDiagnostics, RemoveDocData };<br><br> /// A request to the worker thread<br>-class ASTManagerRequest {<br>+class WorkerRequest {<br> public:<br>- ASTManagerRequest() = default;<br>- ASTManagerRequest(<wbr>ASTManagerRequestType Type, std::string File,<br>- DocVersion Version);<br>+ WorkerRequest() = default;<br>+ WorkerRequest(<wbr>WorkerRequestKind Kind, Path File, DocVersion Version);<br><br>- ASTManagerRequestType Type;<br>- std::string File;<br>+ WorkerRequestKind Kind;<br>+ Path File;<br> DocVersion Version;<br> };<br><br>-class ASTManager : public DocumentStoreListener {<br>-public:<br>- ASTManager(JSONOutput &Output, DocumentStore &Store, bool RunSynchronously);<br>- ~ASTManager() override;<br>-<br>- void onDocumentAdd(StringRef File) override;<br>- void onDocumentRemove(StringRef File) override;<br>+class ClangdServer;<br><br>- /// Get code completions at a specified \p Line and \p Column in \p File.<br>- ///<br>- /// This function is thread-safe and returns completion items that own the<br>- /// data they contain.<br>- std::vector<CompletionItem> codeComplete(StringRef File, unsigned Line,<br>- <wbr> unsigned Column);<br>-<br>- /// Get the fixes associated with a certain diagnostic in a specified file as<br>- /// replacements.<br>- ///<br>- /// This function is thread-safe. It returns a copy to avoid handing out<br>- /// references to unguarded data.<br>- std::vector<clang::tooling::<wbr>Replacement><br>- getFixIts(StringRef File, const clangd::Diagnostic &D);<br>+/// Handles running WorkerRequests of ClangdServer on a separate threads.<br>+/// Currently runs only one worker thread.<br>+class ClangdScheduler {<br>+public:<br>+ ClangdScheduler(ClangdServer &Server, bool RunSynchronously);<br>+ ~ClangdScheduler();<br><br>- DocumentStore &getStore() const { return Store; }<br>+ /// Enqueue WorkerRequest to be run on a worker thread<br>+ void enqueue(ClangdServer &Server, WorkerRequest Request);<br><br> private:<br>- JSONOutput &Output;<br>- DocumentStore &Store;<br>-<br>- // Set to true if requests should block instead of being processed<br>- // asynchronously.<br> bool RunSynchronously;<br>-<br>- /// Loads a compilation database for File. May return nullptr if it fails. The<br>- /// database is cached for subsequent accesses.<br>- clang::tooling::<wbr>CompilationDatabase *<br>- <wbr>getOrCreateCompilationDatabase<wbr>ForFile(StringRef File);<br>- // Creates a new ASTUnit for the document at File.<br>- // FIXME: This calls chdir internally, which is thread unsafe.<br>- std::unique_ptr<clang::<wbr>ASTUnit><br>- createASTUnitForFile(<wbr>StringRef File, const DocumentStore &Docs);<br>-<br>- /// If RunSynchronously is false, queues the request to be run on the worker<br>- /// thread.<br>- /// If RunSynchronously is true, runs the request handler immediately on the<br>- /// main thread.<br>- void queueOrRun(<wbr>ASTManagerRequestType RequestType, StringRef File);<br>-<br>- void runWorker();<br>- void handleRequest(<wbr>ASTManagerRequestType RequestType, StringRef File);<br>-<br>- /// Parses files and publishes diagnostics.<br>- /// This function is called on the worker thread in asynchronous mode and<br>- /// on the main thread in synchronous mode.<br>- void parseFileAndPublishDiagnostics<wbr>(StringRef File);<br>-<br>- /// Caches compilation databases loaded from directories(keys are directories).<br>- llvm::StringMap<std::unique_<wbr>ptr<clang::tooling::<wbr>CompilationDatabase>><br>- CompilationDatabases;<br>-<br>- /// Clang objects.<br>- /// A map from filenames to DocData structures that store ASTUnit and Fixits for<br>- /// the files. The ASTUnits are used for generating diagnostics and fix-it-s<br>- /// asynchronously by the worker thread and synchronously for code completion.<br>- llvm::StringMap<DocData> DocDatas;<br>- std::shared_ptr<clang::<wbr>PCHContainerOperations> PCHs;<br>- /// A lock for access to the DocDatas, CompilationDatabases and PCHs.<br>- std::mutex ClangObjectLock;<br>-<br>- /// Stores latest versions of the tracked documents to discard outdated requests.<br>- /// Guarded by RequestLock.<br>- /// TODO(ibiryukov): the entries are neved deleted from this map.<br>- llvm::StringMap<DocVersion> DocVersions;<br>-<br>- /// A LIFO queue of requests. Note that requests are discarded if the `version`<br>- /// field is not equal to the one stored inside DocVersions.<br>- /// TODO(krasimir): code completion should always have priority over parsing<br>- /// for diagnostics.<br>- std::deque<ASTManagerRequest> RequestQueue;<br>+ std::mutex Mutex;<br>+ /// We run some tasks on a separate thread(parsing, ClangdUnit cleanup).<br>+ /// This thread looks into RequestQueue to find requests to handle and<br>+ /// terminates when Done is set to true.<br>+ std::thread Worker;<br> /// Setting Done to true will make the worker thread terminate.<br> bool Done = false;<br>+ /// A LIFO queue of requests. Note that requests are discarded if the<br>+ /// `version` field is not equal to the one stored inside DraftStore.<br>+ /// FIXME(krasimir): code completion should always have priority over parsing<br>+ /// for diagnostics.<br>+ std::deque<WorkerRequest> RequestQueue;<br> /// Condition variable to wake up the worker thread.<br>- std::condition_variable ClangRequestCV;<br>- /// Lock for accesses to RequestQueue, DocVersions and Done.<br>- std::mutex RequestLock;<br>-<br>- /// We run parsing on a separate thread. This thread looks into RequestQueue to<br>- /// find requests to handle and terminates when Done is set to true.<br>- std::thread ClangWorker;<br>+ std::condition_variable RequestCV;<br>+};<br>+<br>+/// Provides API to manage ASTs for a collection of C++ files and request<br>+/// various language features(currently, only codeCompletion and async<br>+/// diagnostics for tracked files).<br>+class ClangdServer {<br>+public:<br>+ ClangdServer(std::unique_ptr<<wbr>GlobalCompilationDatabase> CDB,<br>+ std::unique_ptr<<wbr>DiagnosticsConsumer> DiagConsumer,<br>+ bool RunSynchronously);<br>+<br>+ /// Add a \p File to the list of tracked C++ files or update the contents if<br>+ /// \p File is already tracked. Also schedules parsing of the AST for it on a<br>+ /// separate thread. When the parsing is complete, DiagConsumer passed in<br>+ /// constructor will receive onDiagnosticsReady callback.<br>+ void addDocument(PathRef File, StringRef Contents);<br>+<br>+ /// Remove \p File from list of tracked files, schedule a request to free<br>+ /// resources associated with it.<br>+ void removeDocument(PathRef File);<br>+<br>+ /// Run code completion for \p File at \p Pos.<br>+ std::vector<CompletionItem> codeComplete(PathRef File, Position Pos);<br>+<br>+ /// Gets current document contents for \p File. \p File must point to a<br>+ /// currently tracked file.<br>+ /// FIXME(ibiryukov): This function is here to allow implementation of<br>+ /// formatCode from ProtocolHandlers.cpp. We should move formatCode to this<br>+ /// class and remove this function from public interface.<br>+ std::string getDocument(PathRef File);<br>+<br>+private:<br>+ friend class ClangdScheduler;<br>+<br>+ /// This function is called on a worker thread.<br>+ void handleRequest(WorkerRequest Request);<br>+<br>+ std::unique_ptr<<wbr>GlobalCompilationDatabase> CDB;<br>+ std::unique_ptr<<wbr>DiagnosticsConsumer> DiagConsumer;<br>+ DraftStore DraftMgr;<br>+ ClangdUnitStore Units;<br>+ std::shared_ptr<<wbr>PCHContainerOperations> PCHs;<br>+ // WorkScheduler has to be the last member, because its destructor has to be<br>+ // called before all other members to stop the worker thread that references<br>+ // ClangdServer<br>+ ClangdScheduler WorkScheduler;<br> };<br><br> } // namespace clangd<br><br>Added: clang-tools-extra/trunk/<wbr>clangd/ClangdUnit.cpp<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdUnit.cpp?rev=303067&view=auto" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/clang-tools-extra/<wbr>trunk/clangd/ClangdUnit.cpp?<wbr>rev=303067&view=auto</a><br>==============================<wbr>==============================<wbr>==================<br>--- clang-tools-extra/trunk/<wbr>clangd/ClangdUnit.cpp (added)<br>+++ clang-tools-extra/trunk/<wbr>clangd/ClangdUnit.cpp Mon May 15 09:17:35 2017<br>@@ -0,0 +1,224 @@<br>+//===--- ClangdUnit.cpp ------------------------------<wbr>-----------*- C++-*-===//<br>+//<br>+// The LLVM Compiler Infrastructure<br>+//<br>+// This file is distributed under the University of Illinois Open Source<br>+// License. See LICENSE.TXT for details.<br>+//<br>+//===------------------------<wbr>------------------------------<wbr>---------------===//<br>+<br>+#include "ClangdUnit.h"<br>+#include "clang/Frontend/ASTUnit.h"<br>+#include "clang/Frontend/<wbr>CompilerInstance.h"<br>+#include "clang/Frontend/<wbr>CompilerInvocation.h"<br>+#include "clang/Tooling/<wbr>CompilationDatabase.h"<br>+<br>+using namespace clang::clangd;<br>+using namespace clang;<br>+<br>+ClangdUnit::ClangdUnit(<wbr>PathRef FileName, StringRef Contents,<br>+ std::<wbr>shared_ptr<<wbr>PCHContainerOperations> PCHs,<br>+ std::<wbr>vector<tooling::<wbr>CompileCommand> Commands)<br>+ : FileName(FileName), PCHs(PCHs) {<br>+ assert(!Commands.empty() && "No compile commands provided");<br>+<br>+ // Inject the resource dir.<br>+ // FIXME: Don't overwrite it if it's already there.<br>+ static int Dummy; // Just an address in this process.<br>+ std::string ResourceDir =<br>+ CompilerInvocation::<wbr>GetResourcesPath("clangd", (void *)&Dummy);<br>+ Commands.front().CommandLine.<wbr>push_back("-resource-dir=" + ResourceDir);<br>+<br>+ IntrusiveRefCntPtr<<wbr>DiagnosticsEngine> Diags =<br>+ CompilerInstance::<wbr>createDiagnostics(new DiagnosticOptions);<br>+<br>+ std::vector<const char *> ArgStrs;<br>+ for (const auto &S : Commands.front().CommandLine)<br>+ ArgStrs.push_back(S.c_str()<wbr>);<br>+<br>+ ASTUnit::RemappedFile RemappedSource(<br>+ FileName,<br>+ llvm::MemoryBuffer::<wbr>getMemBufferCopy(Contents, FileName).release());<br>+<br>+ auto ArgP = &*ArgStrs.begin();<br>+ Unit = std::unique_ptr<ASTUnit>(<wbr>ASTUnit::LoadFromCommandLine(<br>+ ArgP, ArgP + ArgStrs.size(), PCHs, Diags, ResourceDir,<br>+ /*OnlyLocalDecls=*/false, /*CaptureDiagnostics=*/true, RemappedSource,<br>+ /*<wbr>RemappedFilesKeepOriginalName=<wbr>*/true,<br>+ /*<wbr>PrecompilePreambleAfterNParses<wbr>=*/1, /*TUKind=*/TU_Complete,<br>+ /*<wbr>CacheCodeCompletionResults=*/<wbr>true,<br>+ /*<wbr>IncludeBriefCommentsInCodeComp<wbr>letion=*/true,<br>+ /*<wbr>AllowPCHWithCompilerErrors=*/<wbr>true));<br>+}<br>+<br>+void ClangdUnit::reparse(StringRef Contents) {<br>+ // Do a reparse if this wasn't the first parse.<br>+ // FIXME: This might have the wrong working directory if it changed in the<br>+ // meantime.<br>+ ASTUnit::RemappedFile RemappedSource(<br>+ FileName,<br>+ llvm::MemoryBuffer::<wbr>getMemBufferCopy(Contents, FileName).release());<br>+<br>+ Unit->Reparse(PCHs, RemappedSource);<br>+}<br>+<br>+namespace {<br>+<br>+CompletionItemKind getKind(CXCursorKind K) {<br>+ switch (K) {<br>+ case CXCursor_MacroInstantiation:<br>+ case CXCursor_MacroDefinition:<br>+ return CompletionItemKind::Text;<br>+ case CXCursor_CXXMethod:<br>+ return CompletionItemKind::Method;<br>+ case CXCursor_FunctionDecl:<br>+ case CXCursor_FunctionTemplate:<br>+ return CompletionItemKind::Function;<br>+ case CXCursor_Constructor:<br>+ case CXCursor_Destructor:<br>+ return CompletionItemKind::<wbr>Constructor;<br>+ case CXCursor_FieldDecl:<br>+ return CompletionItemKind::Field;<br>+ case CXCursor_VarDecl:<br>+ case CXCursor_ParmDecl:<br>+ return CompletionItemKind::Variable;<br>+ case CXCursor_ClassDecl:<br>+ case CXCursor_StructDecl:<br>+ case CXCursor_UnionDecl:<br>+ case CXCursor_ClassTemplate:<br>+ case CXCursor_<wbr>ClassTemplatePartialSpecializa<wbr>tion:<br>+ return CompletionItemKind::Class;<br>+ case CXCursor_Namespace:<br>+ case CXCursor_NamespaceAlias:<br>+ case CXCursor_NamespaceRef:<br>+ return CompletionItemKind::Module;<br>+ case CXCursor_EnumConstantDecl:<br>+ return CompletionItemKind::Value;<br>+ case CXCursor_EnumDecl:<br>+ return CompletionItemKind::Enum;<br>+ case CXCursor_TypeAliasDecl:<br>+ case CXCursor_<wbr>TypeAliasTemplateDecl:<br>+ case CXCursor_TypedefDecl:<br>+ case CXCursor_MemberRef:<br>+ case CXCursor_TypeRef:<br>+ return CompletionItemKind::Reference;<br>+ default:<br>+ return CompletionItemKind::Missing;<br>+ }<br>+}<br>+<br>+class CompletionItemsCollector : public CodeCompleteConsumer {<br>+ std::vector<CompletionItem> *Items;<br>+ std::shared_ptr<clang::<wbr>GlobalCodeCompletionAllocator> Allocator;<br>+ CodeCompletionTUInfo CCTUInfo;<br>+<br>+public:<br>+ CompletionItemsCollector(std:<wbr>:vector<CompletionItem> *Items,<br>+ <wbr>const CodeCompleteOptions &CodeCompleteOpts)<br>+ : CodeCompleteConsumer(<wbr>CodeCompleteOpts, /*OutputIsBinary=*/false),<br>+ Items(Items),<br>+ Allocator(std::make_<wbr>shared<clang::<wbr>GlobalCodeCompletionAllocator><wbr>()),<br>+ CCTUInfo(Allocator) {}<br>+<br>+ void ProcessCodeCompleteResults(<wbr>Sema &S, CodeCompletionContext Context,<br>+ <wbr> CodeCompletionResult *Results,<br>+ <wbr> unsigned NumResults) override {<br>+ for (unsigned I = 0; I != NumResults; ++I) {<br>+ CodeCompletionResult &Result = Results[I];<br>+ CodeCompletionString *CCS = Result.<wbr>CreateCodeCompletionString(<br>+ S, Context, *Allocator, CCTUInfo,<br>+ CodeCompleteOpts.<wbr>IncludeBriefComments);<br>+ if (CCS) {<br>+ CompletionItem Item;<br>+ assert(CCS-><wbr>getTypedText());<br>+ Item.label = CCS->getTypedText();<br>+ Item.kind = getKind(Result.CursorKind);<br>+ if (CCS->getBriefComment())<br>+ Item.documentation = CCS->getBriefComment();<br>+ Items->push_back(std::<wbr>move(Item));<br>+ }<br>+ }<br>+ }<br>+<br>+ GlobalCodeCompletionAllocator &getAllocator() override { return *Allocator; }<br>+<br>+ CodeCompletionTUInfo &getCodeCompletionTUInfo() override { return CCTUInfo; }<br>+};<br>+} // namespace<br>+<br>+std::vector<CompletionItem> ClangdUnit::codeComplete(<wbr>StringRef Contents,<br>+ <wbr> Position Pos) {<br>+ CodeCompleteOptions CCO;<br>+ CCO.IncludeBriefComments = 1;<br>+ // This is where code completion stores dirty buffers. Need to free after<br>+ // completion.<br>+ SmallVector<const llvm::MemoryBuffer *, 4> OwnedBuffers;<br>+ SmallVector<StoredDiagnostic, 4> StoredDiagnostics;<br>+ IntrusiveRefCntPtr<<wbr>DiagnosticsEngine> DiagEngine(<br>+ new DiagnosticsEngine(new DiagnosticIDs, new DiagnosticOptions));<br>+ std::vector<CompletionItem> Items;<br>+ CompletionItemsCollector Collector(&Items, CCO);<br>+<br>+ ASTUnit::RemappedFile RemappedSource(<br>+ FileName,<br>+ llvm::MemoryBuffer::<wbr>getMemBufferCopy(Contents, FileName).release());<br>+<br>+ IntrusiveRefCntPtr<<wbr>SourceManager> SourceMgr(<br>+ new SourceManager(*DiagEngine, Unit->getFileManager()));<br>+ // CodeComplete seems to require fresh LangOptions.<br>+ LangOptions LangOpts = Unit->getLangOpts();<br>+ // The language server protocol uses zero-based line and column numbers.<br>+ // The clang code completion uses one-based numbers.<br>+ Unit->CodeComplete(FileName, Pos.line + 1, Pos.character + 1, RemappedSource,<br>+ CCO.<wbr>IncludeMacros, CCO.IncludeCodePatterns,<br>+ CCO.<wbr>IncludeBriefComments, Collector, PCHs, *DiagEngine,<br>+ LangOpts, *SourceMgr, Unit->getFileManager(),<br>+ <wbr>StoredDiagnostics, OwnedBuffers);<br>+ for (const llvm::MemoryBuffer *Buffer : OwnedBuffers)<br>+ delete Buffer;<br>+ return Items;<br>+}<br>+<br>+namespace {<br>+/// Convert from clang diagnostic level to LSP severity.<br>+static int getSeverity(DiagnosticsEngine:<wbr>:Level L) {<br>+ switch (L) {<br>+ case DiagnosticsEngine::Remark:<br>+ return 4;<br>+ case DiagnosticsEngine::Note:<br>+ return 3;<br>+ case DiagnosticsEngine::Warning:<br>+ return 2;<br>+ case DiagnosticsEngine::Fatal:<br>+ case DiagnosticsEngine::Error:<br>+ return 1;<br>+ case DiagnosticsEngine::Ignored:<br>+ return 0;<br>+ }<br>+ llvm_unreachable("Unknown diagnostic level!");<br>+}<br>+} // namespace<br>+<br>+std::vector<DiagWithFixIts> ClangdUnit::<wbr>getLocalDiagnostics() const {<br>+ std::vector<DiagWithFixIts> Result;<br>+ for (ASTUnit::stored_diag_iterator D = Unit->stored_diag_begin(),<br>+ <wbr> DEnd = Unit->stored_diag_end();<br>+ D != DEnd; ++D) {<br>+ if (!D->getLocation().isValid() ||<br>+ !D->getLocation().<wbr>getManager().isInMainFile(D-><wbr>getLocation()))<br>+ continue;<br>+ Position P;<br>+ P.line = D->getLocation().<wbr>getSpellingLineNumber() - 1;<br>+ P.character = D->getLocation().<wbr>getSpellingColumnNumber();<br>+ Range R = {P, P};<br>+ clangd::Diagnostic Diag = {R, getSeverity(D->getLevel()), D->getMessage()};<br>+<br>+ llvm::SmallVector<tooling::<wbr>Replacement, 1> FixItsForDiagnostic;<br>+ for (const FixItHint &Fix : D->getFixIts()) {<br>+ FixItsForDiagnostic.push_<wbr>back(clang::tooling::<wbr>Replacement(<br>+ Unit-><wbr>getSourceManager(), Fix.RemoveRange, Fix.CodeToInsert));<br>+ }<br>+ Result.push_back({Diag, std::move(FixItsForDiagnostic)<wbr>});<br>+ }<br>+ return Result;<br>+}<br><br>Added: clang-tools-extra/trunk/<wbr>clangd/ClangdUnit.h<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdUnit.h?rev=303067&view=auto" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/clang-tools-extra/<wbr>trunk/clangd/ClangdUnit.h?rev=<wbr>303067&view=auto</a><br>==============================<wbr>==============================<wbr>==================<br>--- clang-tools-extra/trunk/<wbr>clangd/ClangdUnit.h (added)<br>+++ clang-tools-extra/trunk/<wbr>clangd/ClangdUnit.h Mon May 15 09:17:35 2017<br>@@ -0,0 +1,63 @@<br>+//===--- ClangdUnit.h ------------------------------<wbr>-------------*- C++-*-===//<br>+//<br>+// The LLVM Compiler Infrastructure<br>+//<br>+// This file is distributed under the University of Illinois Open Source<br>+// License. See LICENSE.TXT for details.<br>+//<br>+//===------------------------<wbr>------------------------------<wbr>---------------===//<br>+<br>+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_<wbr>CLANGDUNIT_H<br>+#define LLVM_CLANG_TOOLS_EXTRA_CLANGD_<wbr>CLANGDUNIT_H<br>+<br>+#include "Protocol.h"<br>+#include "Path.h"<br>+#include "clang/Frontend/ASTUnit.h"<br>+#include "clang/Tooling/Core/<wbr>Replacement.h"<br>+#include <memory><br>+<br>+namespace clang {<br>+class ASTUnit;<br>+class PCHContainerOperations;<br>+<br>+namespace tooling {<br>+struct CompileCommand;<br>+}<br>+<br>+namespace clangd {<br>+<br>+/// A diagnostic with its FixIts.<br>+struct DiagWithFixIts {<br>+ clangd::Diagnostic Diag;<br>+ llvm::SmallVector<tooling::<wbr>Replacement, 1> FixIts;<br>+};<br>+<br>+/// Stores parsed C++ AST and provides implementations of all operations clangd<br>+/// would want to perform on parsed C++ files.<br>+class ClangdUnit {<br>+public:<br>+ ClangdUnit(PathRef FileName, StringRef Contents,<br>+ std::shared_ptr<<wbr>PCHContainerOperations> PCHs,<br>+ std::vector<<wbr>tooling::CompileCommand> Commands);<br>+<br>+ /// Reparse with new contents.<br>+ void reparse(StringRef Contents);<br>+<br>+ /// Get code completions at a specified \p Line and \p Column in \p File.<br>+ ///<br>+ /// This function is thread-safe and returns completion items that own the<br>+ /// data they contain.<br>+ std::vector<CompletionItem> codeComplete(StringRef Contents, Position Pos);<br>+ /// Returns diagnostics and corresponding FixIts for each diagnostic that are<br>+ /// located in the current file.<br>+ std::vector<DiagWithFixIts> getLocalDiagnostics() const;<br>+<br>+private:<br>+ Path FileName;<br>+ std::unique_ptr<ASTUnit> Unit;<br>+ std::shared_ptr<<wbr>PCHContainerOperations> PCHs;<br>+};<br>+<br>+} // namespace clangd<br>+} // namespace clang<br>+#endif<br><br>Added: clang-tools-extra/trunk/<wbr>clangd/ClangdUnitStore.cpp<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdUnitStore.cpp?rev=303067&view=auto" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/clang-tools-extra/<wbr>trunk/clangd/ClangdUnitStore.<wbr>cpp?rev=303067&view=auto</a><br>==============================<wbr>==============================<wbr>==================<br>--- clang-tools-extra/trunk/<wbr>clangd/ClangdUnitStore.cpp (added)<br>+++ clang-tools-extra/trunk/<wbr>clangd/ClangdUnitStore.cpp Mon May 15 09:17:35 2017<br>@@ -0,0 +1,34 @@<br>+//===--- ClangdUnitStore.cpp - A ClangdUnits container -----------*-C++-*-===//<br>+//<br>+// The LLVM Compiler Infrastructure<br>+//<br>+// This file is distributed under the University of Illinois Open Source<br>+// License. See LICENSE.TXT for details.<br>+//<br>+//===------------------------<wbr>------------------------------<wbr>----------------===//<br>+<br>+#include "ClangdUnitStore.h"<br>+#include "llvm/Support/Path.h"<br>+<br>+using namespace clang::clangd;<br>+using namespace clang;<br>+<br>+void ClangdUnitStore::<wbr>removeUnitIfPresent(PathRef File) {<br>+ std::lock_guard<std::mutex> Lock(Mutex);<br>+<br>+ auto It = OpenedFiles.find(File);<br>+ if (It == OpenedFiles.end())<br>+ return;<br>+ OpenedFiles.erase(It);<br>+}<br>+<br>+std::vector<tooling::<wbr>CompileCommand> ClangdUnitStore::<wbr>getCompileCommands(<wbr>GlobalCompilationDatabase &CDB, PathRef File) {<br>+ std::vector<tooling::<wbr>CompileCommand> Commands = CDB.getCompileCommands(File);<br>+ if (Commands.empty()) {<br>+ // Add a fake command line if we know nothing.<br>+ Commands.push_back(tooling:<wbr>:CompileCommand(<br>+ llvm::sys::path::<wbr>parent_path(File), llvm::sys::path::filename(<wbr>File),<br>+ {"clang", "-fsyntax-only", File.str()}, ""));<br>+ }<br>+ return Commands;<br>+}<br><br>Added: clang-tools-extra/trunk/<wbr>clangd/ClangdUnitStore.h<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdUnitStore.h?rev=303067&view=auto" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/clang-tools-extra/<wbr>trunk/clangd/ClangdUnitStore.<wbr>h?rev=303067&view=auto</a><br>==============================<wbr>==============================<wbr>==================<br>--- clang-tools-extra/trunk/<wbr>clangd/ClangdUnitStore.h (added)<br>+++ clang-tools-extra/trunk/<wbr>clangd/ClangdUnitStore.h Mon May 15 09:17:35 2017<br>@@ -0,0 +1,93 @@<br>+//===--- ClangdUnitStore.h - A ClangdUnits container -------------*-C++-*-===//<br>+//<br>+// The LLVM Compiler Infrastructure<br>+//<br>+// This file is distributed under the University of Illinois Open Source<br>+// License. See LICENSE.TXT for details.<br>+//<br>+//===------------------------<wbr>------------------------------<wbr>---------------===//<br>+<br>+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_<wbr>CLANGDUNITSTORE_H<br>+#define LLVM_CLANG_TOOLS_EXTRA_CLANGD_<wbr>CLANGDUNITSTORE_H<br>+<br>+#include <mutex><br>+<br>+#include "ClangdUnit.h"<br>+#include "GlobalCompilationDatabase.h"<br>+#include "Path.h"<br>+#include "clang/Tooling/<wbr>CompilationDatabase.h"<br>+<br>+namespace clang {<br>+namespace clangd {<br>+<br>+/// Thread-safe collection of ASTs built for specific files. Provides<br>+/// synchronized access to ASTs.<br>+class ClangdUnitStore {<br>+public:<br>+ /// Run specified \p Action on the ClangdUnit for \p File.<br>+ /// If the file is not present in ClangdUnitStore, a new ClangdUnit will be<br>+ /// created from the \p FileContents. If the file is already present in the<br>+ /// store, ClangdUnit::reparse will be called with the new contents before<br>+ /// running \p Action.<br>+ template <class Func><br>+ void runOnUnit(PathRef File, StringRef FileContents,<br>+ <wbr>GlobalCompilationDatabase &CDB,<br>+ std::shared_<wbr>ptr<PCHContainerOperations> PCHs, Func Action) {<br>+ runOnUnitImpl(File, FileContents, CDB, PCHs, /*ReparseBeforeAction=*/true,<br>+ std::forward<<wbr>Func>(Action));<br>+ }<br>+<br>+ /// Run specified \p Action on the ClangdUnit for \p File.<br>+ /// If the file is not present in ClangdUnitStore, a new ClangdUnit will be<br>+ /// created from the \p FileContents. If the file is already present in the<br>+ /// store, the \p Action will be run directly on it.<br>+ template <class Func><br>+ void runOnUnitWithoutReparse(<wbr>PathRef File, StringRef FileContents,<br>+ <wbr>GlobalCompilationDatabase &CDB,<br>+ <wbr>std::shared_ptr<<wbr>PCHContainerOperations> PCHs,<br>+ <wbr>Func Action) {<br>+ runOnUnitImpl(File, FileContents, CDB, PCHs, /*ReparseBeforeAction=*/false,<br>+ std::forward<<wbr>Func>(Action));<br>+ }<br>+<br>+ /// Remove ClangdUnit for \p File, if any<br>+ void removeUnitIfPresent(PathRef File);<br>+<br>+private:<br>+ /// Run specified \p Action on the ClangdUnit for \p File.<br>+ template <class Func><br>+ void runOnUnitImpl(PathRef File, StringRef FileContents,<br>+ <wbr>GlobalCompilationDatabase &CDB,<br>+ std::<wbr>shared_ptr<<wbr>PCHContainerOperations> PCHs,<br>+ bool ReparseBeforeAction, Func Action) {<br>+ std::lock_guard<std::mutex> Lock(Mutex);<br>+<br>+ auto Commands = getCompileCommands(CDB, File);<br>+ assert(!Commands.empty() &&<br>+ "getCompileCommands should add default command");<br>+ // chdir. This is thread hostile.<br>+ // FIXME(ibiryukov): get rid of this<br>+ llvm::sys::fs::set_current_<wbr>path(Commands.front().<wbr>Directory);<br>+<br>+ auto It = OpenedFiles.find(File);<br>+ if (It == OpenedFiles.end()) {<br>+ It = OpenedFiles<br>+ .insert(std::<wbr>make_pair(<br>+ File, ClangdUnit(File, FileContents, PCHs, Commands)))<br>+ .first;<br>+ } else if (ReparseBeforeAction) {<br>+ It->second.reparse(<wbr>FileContents);<br>+ }<br>+ return Action(It->second);<br>+ }<br>+<br>+ std::vector<tooling::<wbr>CompileCommand><br>+ getCompileCommands(<wbr>GlobalCompilationDatabase &CDB, PathRef File);<br>+<br>+ std::mutex Mutex;<br>+ llvm::StringMap<ClangdUnit> OpenedFiles;<br>+};<br>+} // namespace clangd<br>+} // namespace clang<br>+<br>+#endif<br><br>Removed: clang-tools-extra/trunk/<wbr>clangd/DocumentStore.h<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/DocumentStore.h?rev=303066&view=auto" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/clang-tools-extra/<wbr>trunk/clangd/DocumentStore.h?<wbr>rev=303066&view=auto</a><br>==============================<wbr>==============================<wbr>==================<br>--- clang-tools-extra/trunk/<wbr>clangd/DocumentStore.h (original)<br>+++ clang-tools-extra/trunk/<wbr>clangd/DocumentStore.h (removed)<br>@@ -1,86 +0,0 @@<br>-//===--- DocumentStore.h - File contents container --------------*- C++ -*-===//<br>-//<br>-// The LLVM Compiler Infrastructure<br>-//<br>-// This file is distributed under the University of Illinois Open Source<br>-// License. See LICENSE.TXT for details.<br>-//<br>-//===------------------------<wbr>------------------------------<wbr>----------------===//<br>-<br>-#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_<wbr>DOCUMENTSTORE_H<br>-#define LLVM_CLANG_TOOLS_EXTRA_CLANGD_<wbr>DOCUMENTSTORE_H<br>-<br>-#include "clang/Basic/LLVM.h"<br>-#include "llvm/ADT/StringMap.h"<br>-#include <mutex><br>-#include <string><br>-#include <vector><br>-<br>-namespace clang {<br>-namespace clangd {<br>-class DocumentStore;<br>-<br>-struct DocumentStoreListener {<br>- virtual ~DocumentStoreListener() = default;<br>- virtual void onDocumentAdd(StringRef File) {}<br>- virtual void onDocumentRemove(StringRef File) {}<br>-};<br>-<br>-/// A container for files opened in a workspace, addressed by File. The contents<br>-/// are owned by the DocumentStore.<br>-class DocumentStore {<br>-public:<br>- /// Add a document to the store. Overwrites existing contents.<br>- void addDocument(StringRef File, StringRef Text) {<br>- {<br>- std::lock_guard<std::<wbr>mutex> Guard(DocsMutex);<br>- Docs[File] = Text;<br>- }<br>- for (const auto &Listener : Listeners)<br>- Listener->onDocumentAdd(<wbr>File);<br>- }<br>- /// Delete a document from the store.<br>- void removeDocument(StringRef File) {<br>- {<br>- std::lock_guard<std::<wbr>mutex> Guard(DocsMutex);<br>- Docs.erase(File);<br>- }<br>- for (const auto &Listener : Listeners)<br>- Listener-><wbr>onDocumentRemove(File);<br>- }<br>- /// Retrieve a document from the store. Empty string if it's unknown.<br>- ///<br>- /// This function is thread-safe. It returns a copy to avoid handing out<br>- /// references to unguarded data.<br>- std::string getDocument(StringRef File) const {<br>- // FIXME: This could be a reader lock.<br>- std::lock_guard<std::mutex> Guard(DocsMutex);<br>- return Docs.lookup(File);<br>- }<br>-<br>- /// Add a listener. Does not take ownership.<br>- void addListener(<wbr>DocumentStoreListener *DSL) { Listeners.push_back(DSL); }<br>-<br>- /// Get name and constents of all documents in this store.<br>- ///<br>- /// This function is thread-safe. It returns a copies to avoid handing out<br>- /// references to unguarded data.<br>- std::vector<std::pair<std::<wbr>string, std::string>> getAllDocuments() const {<br>- std::vector<std::pair<std::<wbr>string, std::string>> AllDocs;<br>- std::lock_guard<std::mutex> Guard(DocsMutex);<br>- for (const auto &P : Docs)<br>- AllDocs.emplace_back(P.<wbr>first(), P.second);<br>- return AllDocs;<br>- }<br>-<br>-private:<br>- llvm::StringMap<std::string> Docs;<br>- std::vector<<wbr>DocumentStoreListener *> Listeners;<br>-<br>- mutable std::mutex DocsMutex;<br>-};<br>-<br>-} // namespace clangd<br>-} // namespace clang<br>-<br>-#endif<br><br>Added: clang-tools-extra/trunk/<wbr>clangd/DraftStore.cpp<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/DraftStore.cpp?rev=303067&view=auto" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/clang-tools-extra/<wbr>trunk/clangd/DraftStore.cpp?<wbr>rev=303067&view=auto</a><br>==============================<wbr>==============================<wbr>==================<br>--- clang-tools-extra/trunk/<wbr>clangd/DraftStore.cpp (added)<br>+++ clang-tools-extra/trunk/<wbr>clangd/DraftStore.cpp Mon May 15 09:17:35 2017<br>@@ -0,0 +1,48 @@<br>+//===--- DraftStore.cpp - File contents container ---------------*- C++ -*-===//<br>+//<br>+// The LLVM Compiler Infrastructure<br>+//<br>+// This file is distributed under the University of Illinois Open Source<br>+// License. See LICENSE.TXT for details.<br>+//<br>+//===------------------------<wbr>------------------------------<wbr>----------------===//<br>+<br>+#include "DraftStore.h"<br>+<br>+using namespace clang::clangd;<br>+<br>+VersionedDraft DraftStore::getDraft(PathRef File) const {<br>+ std::lock_guard<std::mutex> Lock(Mutex);<br>+<br>+ auto It = Drafts.find(File);<br>+ if (It == Drafts.end())<br>+ return {0, llvm::None};<br>+ return It->second;<br>+}<br>+<br>+DocVersion DraftStore::getVersion(PathRef File) const {<br>+ std::lock_guard<std::mutex> Lock(Mutex);<br>+<br>+ auto It = Drafts.find(File);<br>+ if (It == Drafts.end())<br>+ return 0;<br>+ return It->second.Version;<br>+}<br>+<br>+DocVersion DraftStore::updateDraft(<wbr>PathRef File, StringRef Contents) {<br>+ std::lock_guard<std::mutex> Lock(Mutex);<br>+<br>+ auto &Entry = Drafts[File];<br>+ DocVersion NewVersion = ++Entry.Version;<br>+ Entry.Draft = Contents;<br>+ return NewVersion;<br>+}<br>+<br>+DocVersion DraftStore::removeDraft(<wbr>PathRef File) {<br>+ std::lock_guard<std::mutex> Lock(Mutex);<br>+<br>+ auto &Entry = Drafts[File];<br>+ DocVersion NewVersion = ++Entry.Version;<br>+ Entry.Draft = llvm::None;<br>+ return NewVersion;<br>+}<br><br>Copied: clang-tools-extra/trunk/<wbr>clangd/DraftStore.h (from r303060, clang-tools-extra/trunk/<wbr>clangd/DocumentStore.h)<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/DraftStore.h?p2=clang-tools-extra/trunk/clangd/DraftStore.h&p1=clang-tools-extra/trunk/clangd/DocumentStore.h&r1=303060&r2=303067&rev=303067&view=diff" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/clang-tools-extra/<wbr>trunk/clangd/DraftStore.h?p2=<wbr>clang-tools-extra/trunk/<wbr>clangd/DraftStore.h&p1=clang-<wbr>tools-extra/trunk/clangd/<wbr>DocumentStore.h&r1=303060&r2=<wbr>303067&rev=303067&view=diff</a><br>==============================<wbr>==============================<wbr>==================<br>--- clang-tools-extra/trunk/<wbr>clangd/DocumentStore.h (original)<br>+++ clang-tools-extra/trunk/<wbr>clangd/DraftStore.h Mon May 15 09:17:35 2017<br>@@ -1,4 +1,4 @@<br>-//===--- DocumentStore.h - File contents container --------------*- C++ -*-===//<br>+//===--- DraftStore.h - File contents container -----------------*- C++ -*-===//<br> //<br> // The LLVM Compiler Infrastructure<br> //<br>@@ -7,9 +7,10 @@<br> //<br> //===-------------------------<wbr>------------------------------<wbr>---------------===//<br><br>-#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_<wbr>DOCUMENTSTORE_H<br>-#define LLVM_CLANG_TOOLS_EXTRA_CLANGD_<wbr>DOCUMENTSTORE_H<br>+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_<wbr>DRAFTSTORE_H<br>+#define LLVM_CLANG_TOOLS_EXTRA_CLANGD_<wbr>DRAFTSTORE_H<br><br>+#include "Path.h"<br> #include "clang/Basic/LLVM.h"<br> #include "llvm/ADT/StringMap.h"<br> #include <mutex><br>@@ -18,66 +19,40 @@<br><br> namespace clang {<br> namespace clangd {<br>-class DocumentStore;<br><br>-struct DocumentStoreListener {<br>- virtual ~DocumentStoreListener() = default;<br>- virtual void onDocumentAdd(StringRef File) {}<br>- virtual void onDocumentRemove(StringRef File) {}<br>+/// Using 'unsigned' here to avoid undefined behaviour on overflow.<br>+typedef unsigned DocVersion;<br>+<br>+/// Document draft with a version of this draft.<br>+struct VersionedDraft {<br>+ DocVersion Version;<br>+ /// If the value of the field is None, draft is now deleted<br>+ llvm::Optional<std::string> Draft;<br> };<br><br>-/// A container for files opened in a workspace, addressed by File. The contents<br>-/// are owned by the DocumentStore.<br>-class DocumentStore {<br>+/// A thread-safe container for files opened in a workspace, addressed by<br>+/// filenames. The contents are owned by the DraftStore. Versions are mantained<br>+/// for the all added documents, including removed ones. The document version is<br>+/// incremented on each update and removal of the document.<br>+class DraftStore {<br> public:<br>- /// Add a document to the store. Overwrites existing contents.<br>- void addDocument(StringRef File, StringRef Text) {<br>- {<br>- std::lock_guard<std::<wbr>mutex> Guard(DocsMutex);<br>- Docs[File] = Text;<br>- }<br>- for (const auto &Listener : Listeners)<br>- Listener->onDocumentAdd(<wbr>File);<br>- }<br>- /// Delete a document from the store.<br>- void removeDocument(StringRef File) {<br>- {<br>- std::lock_guard<std::<wbr>mutex> Guard(DocsMutex);<br>- Docs.erase(File);<br>- }<br>- for (const auto &Listener : Listeners)<br>- Listener-><wbr>onDocumentRemove(File);<br>- }<br>- /// Retrieve a document from the store. Empty string if it's unknown.<br>- ///<br>- /// This function is thread-safe. It returns a copy to avoid handing out<br>- /// references to unguarded data.<br>- std::string getDocument(StringRef File) const {<br>- // FIXME: This could be a reader lock.<br>- std::lock_guard<std::mutex> Guard(DocsMutex);<br>- return Docs.lookup(File);<br>- }<br>-<br>- /// Add a listener. Does not take ownership.<br>- void addListener(<wbr>DocumentStoreListener *DSL) { Listeners.push_back(DSL); }<br>-<br>- /// Get name and constents of all documents in this store.<br>- ///<br>- /// This function is thread-safe. It returns a copies to avoid handing out<br>- /// references to unguarded data.<br>- std::vector<std::pair<std::<wbr>string, std::string>> getAllDocuments() const {<br>- std::vector<std::pair<std::<wbr>string, std::string>> AllDocs;<br>- std::lock_guard<std::mutex> Guard(DocsMutex);<br>- for (const auto &P : Docs)<br>- AllDocs.emplace_back(P.<wbr>first(), P.second);<br>- return AllDocs;<br>- }<br>+ /// \return version and contents of the stored document.<br>+ /// For untracked files, a (0, None) pair is returned.<br>+ VersionedDraft getDraft(PathRef File) const;<br>+ /// \return version of the tracked document.<br>+ /// For untracked files, 0 is returned.<br>+ DocVersion getVersion(PathRef File) const;<br>+<br>+ /// Replace contents of the draft for \p File with \p Contents.<br>+ /// \return The new version of the draft for \p File.<br>+ DocVersion updateDraft(PathRef File, StringRef Contents);<br>+ /// Remove the contents of the draft<br>+ /// \return The new version of the draft for \p File.<br>+ DocVersion removeDraft(PathRef File);<br><br> private:<br>- llvm::StringMap<std::string> Docs;<br>- std::vector<<wbr>DocumentStoreListener *> Listeners;<br>-<br>- mutable std::mutex DocsMutex;<br>+ mutable std::mutex Mutex;<br>+ llvm::StringMap<<wbr>VersionedDraft> Drafts;<br> };<br><br> } // namespace clangd<br><br>Added: clang-tools-extra/trunk/<wbr>clangd/<wbr>GlobalCompilationDatabase.cpp<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/GlobalCompilationDatabase.cpp?rev=303067&view=auto" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/clang-tools-extra/<wbr>trunk/clangd/<wbr>GlobalCompilationDatabase.cpp?<wbr>rev=303067&view=auto</a><br>==============================<wbr>==============================<wbr>==================<br>--- clang-tools-extra/trunk/<wbr>clangd/<wbr>GlobalCompilationDatabase.cpp (added)<br>+++ clang-tools-extra/trunk/<wbr>clangd/<wbr>GlobalCompilationDatabase.cpp Mon May 15 09:17:35 2017<br>@@ -0,0 +1,65 @@<br>+//===--- GlobalCompilationDatabase.cpp --------------------------*- C++-*-===//<br>+//<br>+// The LLVM Compiler Infrastructure<br>+//<br>+// This file is distributed under the University of Illinois Open Source<br>+// License. See LICENSE.TXT for details.<br>+//<br>+//===------------------------<wbr>------------------------------<wbr>---------------===//<br>+<br>+#include "GlobalCompilationDatabase.h"<br>+#include "clang/Tooling/<wbr>CompilationDatabase.h"<br>+#include "llvm/Support/FileSystem.h"<br>+#include "llvm/Support/Path.h"<br>+<br>+using namespace clang::clangd;<br>+using namespace clang;<br>+<br>+std::vector<tooling::<wbr>CompileCommand><br>+<wbr>DirectoryBasedGlobalCompilatio<wbr>nDatabase::getCompileCommands(<wbr>PathRef File) {<br>+ std::vector<tooling::<wbr>CompileCommand> Commands;<br>+<br>+ auto CDB = getCompilationDatabase(File);<br>+ if (!CDB)<br>+ return {};<br>+ return CDB->getCompileCommands(File);<br>+}<br>+<br>+tooling::CompilationDatabase *<br>+<wbr>DirectoryBasedGlobalCompilatio<wbr>nDatabase::<wbr>getCompilationDatabase(PathRef File) {<br>+ std::lock_guard<std::mutex> Lock(Mutex);<br>+<br>+ namespace path = llvm::sys::path;<br>+<br>+ assert((path::is_absolute(<wbr>File, path::Style::posix) ||<br>+ path::is_absolute(<wbr>File, path::Style::windows)) &&<br>+ "path must be absolute");<br>+<br>+ for (auto Path = path::parent_path(File); !Path.empty();<br>+ Path = path::parent_path(Path)) {<br>+<br>+ auto CachedIt = CompilationDatabases.find(<wbr>Path);<br>+ if (CachedIt != CompilationDatabases.end())<br>+ return CachedIt->second.get();<br>+ std::string Error;<br>+ auto CDB = tooling::CompilationDatabase::<wbr>loadFromDirectory(Path, Error);<br>+ if (!CDB) {<br>+ if (!Error.empty()) {<br>+ // FIXME(ibiryukov): logging<br>+ // Output.log("Error when trying to load compilation database from " +<br>+ // Twine(Path) + ": " + Twine(Error) + "\n");<br>+ }<br>+ continue;<br>+ }<br>+<br>+ // FIXME(ibiryukov): Invalidate cached compilation databases on changes<br>+ auto result = CDB.get();<br>+ CompilationDatabases.<wbr>insert(std::make_pair(Path, std::move(CDB)));<br>+ return result;<br>+ }<br>+<br>+ // FIXME(ibiryukov): logging<br>+ // Output.log("Failed to find compilation database for " + Twine(File) +<br>+ // "\n");<br>+ return nullptr;<br>+}<br><br>Added: clang-tools-extra/trunk/<wbr>clangd/<wbr>GlobalCompilationDatabase.h<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/GlobalCompilationDatabase.h?rev=303067&view=auto" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/clang-tools-extra/<wbr>trunk/clangd/<wbr>GlobalCompilationDatabase.h?<wbr>rev=303067&view=auto</a><br>==============================<wbr>==============================<wbr>==================<br>--- clang-tools-extra/trunk/<wbr>clangd/<wbr>GlobalCompilationDatabase.h (added)<br>+++ clang-tools-extra/trunk/<wbr>clangd/<wbr>GlobalCompilationDatabase.h Mon May 15 09:17:35 2017<br>@@ -0,0 +1,59 @@<br>+//===--- GlobalCompilationDatabase.h ----------------------------*- C++-*-===//<br>+//<br>+// The LLVM Compiler Infrastructure<br>+//<br>+// This file is distributed under the University of Illinois Open Source<br>+// License. See LICENSE.TXT for details.<br>+//<br>+//===------------------------<wbr>------------------------------<wbr>---------------===//<br>+<br>+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_<wbr>GLOBALCOMPILATIONDATABASE_H<br>+#define LLVM_CLANG_TOOLS_EXTRA_CLANGD_<wbr>GLOBALCOMPILATIONDATABASE_H<br>+<br>+#include "Path.h"<br>+#include "llvm/ADT/StringMap.h"<br>+#include <memory><br>+#include <mutex><br>+<br>+namespace clang {<br>+<br>+namespace tooling {<br>+class CompilationDatabase;<br>+struct CompileCommand;<br>+} // namespace tooling<br>+<br>+namespace clangd {<br>+<br>+/// Provides compilation arguments used for building ClangdUnit.<br>+class GlobalCompilationDatabase {<br>+public:<br>+ virtual ~GlobalCompilationDatabase() = default;<br>+<br>+ virtual std::vector<tooling::<wbr>CompileCommand><br>+ getCompileCommands(PathRef File) = 0;<br>+<br>+ /// FIXME(ibiryukov): add facilities to track changes to compilation flags of<br>+ /// existing targets.<br>+};<br>+<br>+/// Gets compile args from tooling::CompilationDatabases built for parent<br>+/// directories.<br>+class DirectoryBasedGlobalCompilatio<wbr>nDatabase<br>+ : public GlobalCompilationDatabase {<br>+public:<br>+ std::vector<tooling::<wbr>CompileCommand><br>+ getCompileCommands(PathRef File) override;<br>+<br>+private:<br>+ tooling::CompilationDatabase *getCompilationDatabase(<wbr>PathRef File);<br>+<br>+ std::mutex Mutex;<br>+ /// Caches compilation databases loaded from directories(keys are<br>+ /// directories).<br>+ llvm::StringMap<std::unique_<wbr>ptr<clang::tooling::<wbr>CompilationDatabase>><br>+ CompilationDatabases;<br>+};<br>+} // namespace clangd<br>+} // namespace clang<br>+<br>+#endif<br><br>Added: clang-tools-extra/trunk/<wbr>clangd/Path.h<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/Path.h?rev=303067&view=auto" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/clang-tools-extra/<wbr>trunk/clangd/Path.h?rev=<wbr>303067&view=auto</a><br>==============================<wbr>==============================<wbr>==================<br>--- clang-tools-extra/trunk/<wbr>clangd/Path.h (added)<br>+++ clang-tools-extra/trunk/<wbr>clangd/Path.h Mon May 15 09:17:35 2017<br>@@ -0,0 +1,29 @@<br>+//===--- Path.h - Helper typedefs ------------------------------<wbr>--*- C++-*-===//<br>+//<br>+// The LLVM Compiler Infrastructure<br>+//<br>+// This file is distributed under the University of Illinois Open Source<br>+// License. See LICENSE.TXT for details.<br>+//<br>+//===------------------------<wbr>------------------------------<wbr>----------------===//<br>+<br>+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_<wbr>PATH_H<br>+#define LLVM_CLANG_TOOLS_EXTRA_CLANGD_<wbr>PATH_H<br>+<br>+#include "llvm/ADT/StringRef.h"<br>+#include <string><br>+<br>+namespace clang {<br>+namespace clangd {<br>+<br>+/// A typedef to represent a file path. Used solely for more descriptive<br>+/// signatures.<br>+using Path = std::string;<br>+/// A typedef to represent a ref to file path. Used solely for more descriptive<br>+/// signatures.<br>+using PathRef = llvm::StringRef;<br>+<br>+} // namespace clangd<br>+} // namespace clang<br>+<br>+#endif<br><br>Modified: clang-tools-extra/trunk/<wbr>clangd/ProtocolHandlers.cpp<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ProtocolHandlers.cpp?rev=303067&r1=303066&r2=303067&view=diff" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/clang-tools-extra/<wbr>trunk/clangd/ProtocolHandlers.<wbr>cpp?rev=303067&r1=303066&r2=<wbr>303067&view=diff</a><br>==============================<wbr>==============================<wbr>==================<br>--- clang-tools-extra/trunk/<wbr>clangd/ProtocolHandlers.cpp (original)<br>+++ clang-tools-extra/trunk/<wbr>clangd/ProtocolHandlers.cpp Mon May 15 09:17:35 2017<br>@@ -8,9 +8,10 @@<br> //===-------------------------<wbr>------------------------------<wbr>---------------===//<br><br> #include "ProtocolHandlers.h"<br>-#include "ASTManager.h"<br>-#include "DocumentStore.h"<br>+#include "ClangdServer.h"<br>+#include "DraftStore.h"<br> #include "clang/Format/Format.h"<br>+#include "ClangdLSPServer.h"<br> using namespace clang;<br> using namespace clangd;<br><br>@@ -21,7 +22,7 @@ void TextDocumentDidOpenHandler::<wbr>handleN<br> Output.log("Failed to decode DidOpenTextDocumentParams!\n")<wbr>;<br> return;<br> }<br>- Store.addDocument(DOTDP-><wbr>textDocument.uri.file, DOTDP->textDocument.text);<br>+ AST.openDocument(DOTDP-><wbr>textDocument.uri.file, DOTDP->textDocument.text);<br> }<br><br> void TextDocumentDidCloseHandler::<wbr>handleNotification(<br>@@ -32,7 +33,7 @@ void TextDocumentDidCloseHandler::<wbr>handle<br> return;<br> }<br><br>- Store.removeDocument(DCTDP-><wbr>textDocument.uri.file);<br>+ AST.closeDocument(DCTDP-><wbr>textDocument.uri.file);<br> }<br><br> void TextDocumentDidChangeHandler::<wbr>handleNotification(<br>@@ -43,7 +44,7 @@ void TextDocumentDidChangeHandler::<wbr>handl<br> return;<br> }<br> // We only support full syncing right now.<br>- Store.addDocument(DCTDP-><wbr>textDocument.uri.file, DCTDP->contentChanges[0].text)<wbr>;<br>+ AST.openDocument(DCTDP-><wbr>textDocument.uri.file, DCTDP->contentChanges[0].text)<wbr>;<br> }<br><br> /// Turn a [line, column] pair into an offset in Code.<br>@@ -110,7 +111,7 @@ void TextDocumentRangeFormattingHan<wbr>dler:<br> return;<br> }<br><br>- std::string Code = Store.getDocument(DRFP-><wbr>textDocument.uri.file);<br>+ std::string Code = AST.getDocument(DRFP-><wbr>textDocument.uri.file);<br><br> size_t Begin = positionToOffset(Code, DRFP->range.start);<br> size_t Len = positionToOffset(Code, DRFP->range.end) - Begin;<br>@@ -129,7 +130,7 @@ void TextDocumentOnTypeFormattingHa<wbr>ndler<br><br> // Look for the previous opening brace from the character position and format<br> // starting from there.<br>- std::string Code = Store.getDocument(DOTFP-><wbr>textDocument.uri.file);<br>+ std::string Code = AST.getDocument(DOTFP-><wbr>textDocument.uri.file);<br> size_t CursorPos = positionToOffset(Code, DOTFP->position);<br> size_t PreviousLBracePos = StringRef(Code).find_last_of('<wbr>{', CursorPos);<br> if (PreviousLBracePos == StringRef::npos)<br>@@ -149,7 +150,7 @@ void TextDocumentFormattingHandler:<wbr>:hand<br> }<br><br> // Format everything.<br>- std::string Code = Store.getDocument(DFP-><wbr>textDocument.uri.file);<br>+ std::string Code = AST.getDocument(DFP-><wbr>textDocument.uri.file);<br> writeMessage(formatCode(<wbr>Code, DFP->textDocument.uri.file,<br> {<wbr>clang::tooling::Range(0, Code.size())}, ID));<br> }<br>@@ -164,7 +165,7 @@ void CodeActionHandler::<wbr>handleMethod(llv<br><br> // We provide a code action for each diagnostic at the requested location<br> // which has FixIts available.<br>- std::string Code = AST.getStore().getDocument(<wbr>CAP->textDocument.uri.file);<br>+ std::string Code = AST.getDocument(CAP-><wbr>textDocument.uri.file);<br> std::string Commands;<br> for (Diagnostic &D : CAP->context.diagnostics) {<br> std::vector<clang::<wbr>tooling::Replacement> Fixes = AST.getFixIts(CAP-><wbr>textDocument.uri.file, D);<br>@@ -195,8 +196,8 @@ void CompletionHandler::<wbr>handleMethod(llv<br> return;<br> }<br><br>- auto Items = AST.codeComplete(TDPP-><wbr>textDocument.uri.file, TDPP->position.line,<br>- <wbr> TDPP->position.character);<br>+ auto Items = AST.codeComplete(TDPP-><wbr>textDocument.uri.file, Position{TDPP->position.line,<br>+ TDPP->position.<wbr>character});<br> std::string Completions;<br> for (const auto &Item : Items) {<br> Completions += CompletionItem::unparse(Item);<br><br>Modified: clang-tools-extra/trunk/<wbr>clangd/ProtocolHandlers.h<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ProtocolHandlers.h?rev=303067&r1=303066&r2=303067&view=diff" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/clang-tools-extra/<wbr>trunk/clangd/ProtocolHandlers.<wbr>h?rev=303067&r1=303066&r2=<wbr>303067&view=diff</a><br>==============================<wbr>==============================<wbr>==================<br>--- clang-tools-extra/trunk/<wbr>clangd/ProtocolHandlers.h (original)<br>+++ clang-tools-extra/trunk/<wbr>clangd/ProtocolHandlers.h Mon May 15 09:17:35 2017<br>@@ -22,8 +22,8 @@<br><br> namespace clang {<br> namespace clangd {<br>-class ASTManager;<br>-class DocumentStore;<br>+class ClangdLSPServer;<br>+class ClangdLSPServer;<br><br> struct InitializeHandler : Handler {<br> InitializeHandler(JSONOutput &Output) : Handler(Output) {}<br>@@ -56,83 +56,83 @@ private:<br> };<br><br> struct TextDocumentDidOpenHandler : Handler {<br>- TextDocumentDidOpenHandler(<wbr>JSONOutput &Output, DocumentStore &Store)<br>- : Handler(Output), Store(Store) {}<br>+ TextDocumentDidOpenHandler(<wbr>JSONOutput &Output, ClangdLSPServer &AST)<br>+ : Handler(Output), AST(AST) {}<br><br> void handleNotification(llvm::yaml:<wbr>:MappingNode *Params) override;<br><br> private:<br>- DocumentStore &Store;<br>+ ClangdLSPServer &AST;<br> };<br><br> struct TextDocumentDidChangeHandler : Handler {<br>- TextDocumentDidChangeHandler(<wbr>JSONOutput &Output, DocumentStore &Store)<br>- : Handler(Output), Store(Store) {}<br>+ TextDocumentDidChangeHandler(<wbr>JSONOutput &Output, ClangdLSPServer &AST)<br>+ : Handler(Output), AST(AST) {}<br><br> void handleNotification(llvm::yaml:<wbr>:MappingNode *Params) override;<br><br> private:<br>- DocumentStore &Store;<br>+ ClangdLSPServer &AST;<br> };<br><br> struct TextDocumentDidCloseHandler : Handler {<br>- TextDocumentDidCloseHandler(<wbr>JSONOutput &Output, DocumentStore &Store)<br>- : Handler(Output), Store(Store) {}<br>+ TextDocumentDidCloseHandler(<wbr>JSONOutput &Output, ClangdLSPServer &AST)<br>+ : Handler(Output), AST(AST) {}<br><br> void handleNotification(llvm::yaml:<wbr>:MappingNode *Params) override;<br><br> private:<br>- DocumentStore &Store;<br>+ ClangdLSPServer &AST;<br> };<br><br> struct TextDocumentOnTypeFormattingHa<wbr>ndler : Handler {<br>- <wbr>TextDocumentOnTypeFormattingHa<wbr>ndler(JSONOutput &Output, DocumentStore &Store)<br>- : Handler(Output), Store(Store) {}<br>+ <wbr>TextDocumentOnTypeFormattingHa<wbr>ndler(JSONOutput &Output, ClangdLSPServer &AST)<br>+ : Handler(Output), AST(AST) {}<br><br> void handleMethod(llvm::yaml::<wbr>MappingNode *Params, StringRef ID) override;<br><br> private:<br>- DocumentStore &Store;<br>+ ClangdLSPServer &AST;<br> };<br><br> struct TextDocumentRangeFormattingHan<wbr>dler : Handler {<br>- <wbr>TextDocumentRangeFormattingHan<wbr>dler(JSONOutput &Output, DocumentStore &Store)<br>- : Handler(Output), Store(Store) {}<br>+ <wbr>TextDocumentRangeFormattingHan<wbr>dler(JSONOutput &Output, ClangdLSPServer &AST)<br>+ : Handler(Output), AST(AST) {}<br><br> void handleMethod(llvm::yaml::<wbr>MappingNode *Params, StringRef ID) override;<br><br> private:<br>- DocumentStore &Store;<br>+ ClangdLSPServer &AST;<br> };<br><br> struct TextDocumentFormattingHandler : Handler {<br>- <wbr>TextDocumentFormattingHandler(<wbr>JSONOutput &Output, DocumentStore &Store)<br>- : Handler(Output), Store(Store) {}<br>+ <wbr>TextDocumentFormattingHandler(<wbr>JSONOutput &Output, ClangdLSPServer &AST)<br>+ : Handler(Output), AST(AST) {}<br><br> void handleMethod(llvm::yaml::<wbr>MappingNode *Params, StringRef ID) override;<br><br> private:<br>- DocumentStore &Store;<br>+ ClangdLSPServer &AST;<br> };<br><br> struct CodeActionHandler : Handler {<br>- CodeActionHandler(JSONOutput &Output, ASTManager &AST)<br>+ CodeActionHandler(JSONOutput &Output, ClangdLSPServer &AST)<br> : Handler(Output), AST(AST) {}<br><br> void handleMethod(llvm::yaml::<wbr>MappingNode *Params, StringRef ID) override;<br><br> private:<br>- ASTManager &AST;<br>+ ClangdLSPServer &AST;<br> };<br><br> struct CompletionHandler : Handler {<br>- CompletionHandler(JSONOutput &Output, ASTManager &AST)<br>+ CompletionHandler(JSONOutput &Output, ClangdLSPServer &AST)<br> : Handler(Output), AST(AST) {}<br><br> void handleMethod(llvm::yaml::<wbr>MappingNode *Params, StringRef ID) override;<br><br> private:<br>- ASTManager &AST;<br>+ ClangdLSPServer &AST;<br> };<br><br> } // namespace clangd<br><br><br>______________________________<wbr>_________________<br>cfe-commits mailing list<br><a href="mailto:cfe-commits@lists.llvm.org" target="_blank">cfe-commits@lists.llvm.org</a><br><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/cfe-commits</a><br></div></div></blockquote></div><br></div></div></div></blockquote></div><br></div></div></div></blockquote></div><br></div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div>Regards,</div><div>Ilya Biryukov</div></div></div></div></div>
</div>