<div dir="ltr">Hopefully fixed in r304067.<div><br></div><div>See also; <a href="https://reviews.llvm.org/D33416#766533">https://reviews.llvm.org/D33416#766533</a><br></div></div><br><div class="gmail_quote"><div dir="ltr">On Sat, May 27, 2017 at 6:12 AM Vedant Kumar via cfe-commits <<a href="mailto:cfe-commits@lists.llvm.org">cfe-commits@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word">Hi Ilya,<div><br></div><div>The unit test introduced by this commit hits an assertion failure on our bots. Could you please take a look at the issue?</div><div><br></div><div> <a href="http://lab.llvm.org:8080/green/job/clang-stage1-cmake-RA-expensive/6733" target="_blank">lab.llvm.org:8080/green/job/clang-stage1-cmake-RA-expensive/6733</a></div><div><br></div><div>I see:</div><div><br></div><span><div>[ RUN ] ClangdVFSTest.Reparse</div><br class="m_-4995522196484167973Apple-interchange-newline"><div>Assertion failed: (Unit && "Unit wasn't created"), function ClangdUnit, file /Users/buildslave/jenkins/sharedspace/clang-stage1-cmake-RA_workspace/llvm/tools/clang/tools/extra/clangd/ClangdUnit.cpp, line 58.</div><br class="m_-4995522196484167973Apple-interchange-newline"><div>0 ClangdTests 0x0000000102fec2f8 llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 40</div><div>1 ClangdTests 0x0000000102fec9e6 SignalHandler(int) + 454</div><div>2 libsystem_platform.dylib 0x00007fff97ed352a _sigtramp + 26</div><div>3 libsystem_platform.dylib 000000000000000000 _sigtramp + 1746062064</div><div>4 libsystem_c.dylib 0x00007fff97d776df abort + 129</div><div>5 libsystem_c.dylib 0x00007fff97d3edd8 basename + 0</div><div>6 ClangdTests 0x00000001030c95dd clang::clangd::ClangdUnit::ClangdUnit(llvm::StringRef, llvm::StringRef, std::__1::shared_ptr<clang::PCHContainerOperations>, std::__1::vector<clang::tooling::CompileCommand, std::__1::allocator<clang::tooling::CompileCommand> >, llvm::IntrusiveRefCntPtr<clang::vfs::FileSystem>) + 1837</div><div>7 ClangdTests 0x00000001030c824f std::__1::__function::__func<clang::clangd::ClangdServer::addDocument(llvm::StringRef, llvm::StringRef)::$_1, std::__1::allocator<clang::clangd::ClangdServer::addDocument(llvm::StringRef, llvm::StringRef)::$_1>, void ()>::operator()() + 975</div><div>8 ClangdTests 0x00000001030c6e03 void* std::__1::__thread_proxy<std::__1::tuple<clang::clangd::ClangdScheduler::ClangdScheduler(bool)::$_0> >(void*) + 579</div><div>9 libsystem_pthread.dylib 0x00007fff8e26099d _pthread_body + 131</div><div>10 libsystem_pthread.dylib 0x00007fff8e26091a _pthread_body + 0</div><div>11 libsystem_pthread.dylib 0x00007fff8e25e351 thread_start + 13</div><div><br></div><div>********************</div><div>FAIL: Extra Tools Unit Tests :: clangd/ClangdTests/ClangdVFSTest.ReparseOnHeaderChange (13778 of 37955)</div><div>******************** TEST 'Extra Tools Unit Tests :: clangd/ClangdTests/ClangdVFSTest.ReparseOnHeaderChange' FAILED ********************</div><div>Note: Google Test filter = ClangdVFSTest.ReparseOnHeaderChange</div><div>[==========] Running 1 test from 1 test case.</div><div>[----------] Global test environment set-up.</div><div>[----------] 1 test from ClangdVFSTest</div><div>[ RUN ] ClangdVFSTest.ReparseOnHeaderChange</div><br class="m_-4995522196484167973Apple-interchange-newline"><div>Assertion failed: (Unit && "Unit wasn't created"), function ClangdUnit, file /Users/buildslave/jenkins/sharedspace/clang-stage1-cmake-RA_workspace/llvm/tools/clang/tools/extra/clangd/ClangdUnit.cpp, line 58.</div><br class="m_-4995522196484167973Apple-interchange-newline"><div>0 ClangdTests 0x00000001005b72f8 llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 40</div><div>1 ClangdTests 0x00000001005b79e6 SignalHandler(int) + 454</div><div>2 libsystem_platform.dylib 0x00007fff97ed352a _sigtramp + 26</div><div>3 libsystem_platform.dylib 000000000000000000 _sigtramp + 1746062064</div><div>4 libsystem_c.dylib 0x00007fff97d776df abort + 129</div><div>5 libsystem_c.dylib 0x00007fff97d3edd8 basename + 0</div><div>6 ClangdTests 0x00000001006945dd clang::clangd::ClangdUnit::ClangdUnit(llvm::StringRef, llvm::StringRef, std::__1::shared_ptr<clang::PCHContainerOperations>, std::__1::vector<clang::tooling::CompileCommand, std::__1::allocator<clang::tooling::CompileCommand> >, llvm::IntrusiveRefCntPtr<clang::vfs::FileSystem>) + 1837</div><div>7 ClangdTests 0x000000010069324f std::__1::__function::__func<clang::clangd::ClangdServer::addDocument(llvm::StringRef, llvm::StringRef)::$_1, std::__1::allocator<clang::clangd::ClangdServer::addDocument(llvm::StringRef, llvm::StringRef)::$_1>, void ()>::operator()() + 975</div><div>8 ClangdTests 0x0000000100691e03 void* std::__1::__thread_proxy<std::__1::tuple<clang::clangd::ClangdScheduler::ClangdScheduler(bool)::$_0> >(void*) + 579</div><div>9 libsystem_pthread.dylib 0x00007fff8e26099d _pthread_body + 131</div><div>10 libsystem_pthread.dylib 0x00007fff8e26091a _pthread_body + 0</div><div>11 libsystem_pthread.dylib 0x00007fff8e25e351 thread_start + 13</div></span><span><br></span><div>thanks,</div><div>vedant</div></div><div style="word-wrap:break-word"><div><br><div><blockquote type="cite"><div>On May 26, 2017, at 5:26 AM, Ilya Biryukov via cfe-commits <<a href="mailto:cfe-commits@lists.llvm.org" target="_blank">cfe-commits@lists.llvm.org</a>> wrote:</div><br class="m_-4995522196484167973Apple-interchange-newline"><div><div>Author: ibiryukov<br>Date: Fri May 26 07:26:51 2017<br>New Revision: 303977<br><br>URL: <a href="http://llvm.org/viewvc/llvm-project?rev=303977&view=rev" target="_blank">http://llvm.org/viewvc/llvm-project?rev=303977&view=rev</a><br>Log:<br>[clangd] Allow to use vfs::FileSystem for file accesses.<br><br>Summary:<br>Custom vfs::FileSystem is currently used for unit tests.<br>This revision depends on <a href="https://reviews.llvm.org/D33397" target="_blank">https://reviews.llvm.org/D33397</a>.<br><br>Reviewers: bkramer, krasimir<br><br>Reviewed By: bkramer, krasimir<br><br>Subscribers: klimek, cfe-commits, mgorny<br><br>Differential Revision: <a href="https://reviews.llvm.org/D33416" target="_blank">https://reviews.llvm.org/D33416</a><br><br>Added:<br> clang-tools-extra/trunk/unittests/clangd/<br> clang-tools-extra/trunk/unittests/clangd/CMakeLists.txt<br> clang-tools-extra/trunk/unittests/clangd/ClangdTests.cpp<br>Modified:<br> clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp<br> clang-tools-extra/trunk/clangd/ClangdServer.cpp<br> clang-tools-extra/trunk/clangd/ClangdServer.h<br> clang-tools-extra/trunk/clangd/ClangdUnit.cpp<br> clang-tools-extra/trunk/clangd/ClangdUnit.h<br> clang-tools-extra/trunk/clangd/ClangdUnitStore.h<br> clang-tools-extra/trunk/unittests/CMakeLists.txt<br><br>Modified: clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp?rev=303977&r1=303976&r2=303977&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp?rev=303977&r1=303976&r2=303977&view=diff</a><br>==============================================================================<br>--- clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp (original)<br>+++ clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp Fri May 26 07:26:51 2017<br>@@ -199,6 +199,7 @@ ClangdLSPServer::ClangdLSPServer(JSONOut<br> : Out(Out),<br> Server(llvm::make_unique<DirectoryBasedGlobalCompilationDatabase>(),<br> llvm::make_unique<LSPDiagnosticsConsumer>(*this),<br>+ llvm::make_unique<RealFileSystemProvider>(),<br> RunSynchronously) {}<br><br> void ClangdLSPServer::run(std::istream &In) {<br><br>Modified: clang-tools-extra/trunk/clangd/ClangdServer.cpp<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.cpp?rev=303977&r1=303976&r2=303977&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.cpp?rev=303977&r1=303976&r2=303977&view=diff</a><br>==============================================================================<br>--- clang-tools-extra/trunk/clangd/ClangdServer.cpp (original)<br>+++ clang-tools-extra/trunk/clangd/ClangdServer.cpp Fri May 26 07:26:51 2017<br>@@ -58,6 +58,10 @@ Position clangd::offsetToPosition(String<br> return {Lines, Cols};<br> }<br><br>+IntrusiveRefCntPtr<vfs::FileSystem> RealFileSystemProvider::getFileSystem() {<br>+ return vfs::getRealFileSystem();<br>+}<br>+<br> ClangdScheduler::ClangdScheduler(bool RunSynchronously)<br> : RunSynchronously(RunSynchronously) {<br> if (RunSynchronously) {<br>@@ -135,8 +139,10 @@ void ClangdScheduler::addToEnd(std::func<br><br> ClangdServer::ClangdServer(std::unique_ptr<GlobalCompilationDatabase> CDB,<br> std::unique_ptr<DiagnosticsConsumer> DiagConsumer,<br>+ std::unique_ptr<FileSystemProvider> FSProvider,<br> bool RunSynchronously)<br> : CDB(std::move(CDB)), DiagConsumer(std::move(DiagConsumer)),<br>+ FSProvider(std::move(FSProvider)),<br> PCHs(std::make_shared<PCHContainerOperations>()),<br> WorkScheduler(RunSynchronously) {}<br><br>@@ -150,10 +156,11 @@ void ClangdServer::addDocument(PathRef F<br><br> assert(FileContents.Draft &&<br> "No contents inside a file that was scheduled for reparse");<br>- Units.runOnUnit(<br>- FileStr, *FileContents.Draft, *CDB, PCHs, [&](ClangdUnit const &Unit) {<br>- DiagConsumer->onDiagnosticsReady(FileStr, Unit.getLocalDiagnostics());<br>- });<br>+ Units.runOnUnit(FileStr, *FileContents.Draft, *CDB, PCHs,<br>+ FSProvider->getFileSystem(), [&](ClangdUnit const &Unit) {<br>+ DiagConsumer->onDiagnosticsReady(<br>+ FileStr, Unit.getLocalDiagnostics());<br>+ });<br> });<br> }<br><br>@@ -168,15 +175,22 @@ void ClangdServer::removeDocument(PathRe<br> });<br> }<br><br>+void ClangdServer::forceReparse(PathRef File) {<br>+ // The addDocument schedules the reparse even if the contents of the file<br>+ // never changed, so we just call it here.<br>+ addDocument(File, getDocument(File));<br>+}<br>+<br> std::vector<CompletionItem> ClangdServer::codeComplete(PathRef File,<br> Position Pos) {<br> auto FileContents = DraftMgr.getDraft(File);<br> assert(FileContents.Draft && "codeComplete is called for non-added document");<br><br> std::vector<CompletionItem> Result;<br>+ auto VFS = FSProvider->getFileSystem();<br> Units.runOnUnitWithoutReparse(<br>- File, *FileContents.Draft, *CDB, PCHs, [&](ClangdUnit &Unit) {<br>- Result = Unit.codeComplete(*FileContents.Draft, Pos);<br>+ File, *FileContents.Draft, *CDB, PCHs, VFS, [&](ClangdUnit &Unit) {<br>+ Result = Unit.codeComplete(*FileContents.Draft, Pos, VFS);<br> });<br> return Result;<br> }<br><br>Modified: clang-tools-extra/trunk/clangd/ClangdServer.h<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.h?rev=303977&r1=303976&r2=303977&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.h?rev=303977&r1=303976&r2=303977&view=diff</a><br>==============================================================================<br>--- clang-tools-extra/trunk/clangd/ClangdServer.h (original)<br>+++ clang-tools-extra/trunk/clangd/ClangdServer.h Fri May 26 07:26:51 2017<br>@@ -50,6 +50,17 @@ public:<br> std::vector<DiagWithFixIts> Diagnostics) = 0;<br> };<br><br>+class FileSystemProvider {<br>+public:<br>+ virtual ~FileSystemProvider() = default;<br>+ virtual IntrusiveRefCntPtr<vfs::FileSystem> getFileSystem() = 0;<br>+};<br>+<br>+class RealFileSystemProvider : public FileSystemProvider {<br>+public:<br>+ IntrusiveRefCntPtr<vfs::FileSystem> getFileSystem() override;<br>+};<br>+<br> class ClangdServer;<br><br> /// Handles running WorkerRequests of ClangdServer on a separate threads.<br>@@ -94,6 +105,7 @@ class ClangdServer {<br> public:<br> ClangdServer(std::unique_ptr<GlobalCompilationDatabase> CDB,<br> std::unique_ptr<DiagnosticsConsumer> DiagConsumer,<br>+ std::unique_ptr<FileSystemProvider> FSProvider,<br> bool RunSynchronously);<br><br> /// Add a \p File to the list of tracked C++ files or update the contents if<br>@@ -104,6 +116,8 @@ public:<br> /// Remove \p File from list of tracked files, schedule a request to free<br> /// resources associated with it.<br> void removeDocument(PathRef File);<br>+ /// Force \p File to be reparsed using the latest contents.<br>+ void forceReparse(PathRef File);<br><br> /// Run code completion for \p File at \p Pos.<br> std::vector<CompletionItem> codeComplete(PathRef File, Position Pos);<br>@@ -129,6 +143,7 @@ public:<br> private:<br> std::unique_ptr<GlobalCompilationDatabase> CDB;<br> std::unique_ptr<DiagnosticsConsumer> DiagConsumer;<br>+ std::unique_ptr<FileSystemProvider> FSProvider;<br> DraftStore DraftMgr;<br> ClangdUnitStore Units;<br> std::shared_ptr<PCHContainerOperations> PCHs;<br><br>Modified: clang-tools-extra/trunk/clangd/ClangdUnit.cpp<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdUnit.cpp?rev=303977&r1=303976&r2=303977&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdUnit.cpp?rev=303977&r1=303976&r2=303977&view=diff</a><br>==============================================================================<br>--- clang-tools-extra/trunk/clangd/ClangdUnit.cpp (original)<br>+++ clang-tools-extra/trunk/clangd/ClangdUnit.cpp Fri May 26 07:26:51 2017<br>@@ -11,6 +11,7 @@<br> #include "clang/Frontend/ASTUnit.h"<br> #include "clang/Frontend/CompilerInstance.h"<br> #include "clang/Frontend/CompilerInvocation.h"<br>+#include "clang/Frontend/Utils.h"<br> #include "clang/Tooling/CompilationDatabase.h"<br><br> using namespace clang::clangd;<br>@@ -18,7 +19,8 @@ using namespace clang;<br><br> ClangdUnit::ClangdUnit(PathRef FileName, StringRef Contents,<br> std::shared_ptr<PCHContainerOperations> PCHs,<br>- std::vector<tooling::CompileCommand> Commands)<br>+ std::vector<tooling::CompileCommand> Commands,<br>+ IntrusiveRefCntPtr<vfs::FileSystem> VFS)<br> : FileName(FileName), PCHs(PCHs) {<br> assert(!Commands.empty() && "No compile commands provided");<br><br>@@ -48,10 +50,16 @@ ClangdUnit::ClangdUnit(PathRef FileName,<br> /*PrecompilePreambleAfterNParses=*/1, /*TUKind=*/TU_Prefix,<br> /*CacheCodeCompletionResults=*/true,<br> /*IncludeBriefCommentsInCodeCompletion=*/true,<br>- /*AllowPCHWithCompilerErrors=*/true));<br>+ /*AllowPCHWithCompilerErrors=*/true,<br>+ /*SkipFunctionBodies=*/false,<br>+ /*UserFilesAreVolatile=*/false, /*ForSerialization=*/false,<br>+ /*ModuleFormat=*/llvm::None,<br>+ /*ErrAST=*/nullptr, VFS));<br>+ assert(Unit && "Unit wasn't created");<br> }<br><br>-void ClangdUnit::reparse(StringRef Contents) {<br>+void ClangdUnit::reparse(StringRef Contents,<br>+ IntrusiveRefCntPtr<vfs::FileSystem> VFS) {<br> // Do a reparse if this wasn't the first parse.<br> // FIXME: This might have the wrong working directory if it changed in the<br> // meantime.<br>@@ -59,7 +67,7 @@ void ClangdUnit::reparse(StringRef Conte<br> FileName,<br> llvm::MemoryBuffer::getMemBufferCopy(Contents, FileName).release());<br><br>- Unit->Reparse(PCHs, RemappedSource);<br>+ Unit->Reparse(PCHs, RemappedSource, VFS);<br> }<br><br> namespace {<br>@@ -146,8 +154,9 @@ public:<br> };<br> } // namespace<br><br>-std::vector<CompletionItem> ClangdUnit::codeComplete(StringRef Contents,<br>- Position Pos) {<br>+std::vector<CompletionItem><br>+ClangdUnit::codeComplete(StringRef Contents, Position Pos,<br>+ IntrusiveRefCntPtr<vfs::FileSystem> VFS) {<br> CodeCompleteOptions CCO;<br> CCO.IncludeBriefComments = 1;<br> // This is where code completion stores dirty buffers. Need to free after<br>@@ -163,8 +172,10 @@ std::vector<CompletionItem> ClangdUnit::<br> FileName,<br> llvm::MemoryBuffer::getMemBufferCopy(Contents, FileName).release());<br><br>+ IntrusiveRefCntPtr<FileManager> FileMgr(<br>+ new FileManager(Unit->getFileSystemOpts(), VFS));<br> IntrusiveRefCntPtr<SourceManager> SourceMgr(<br>- new SourceManager(*DiagEngine, Unit->getFileManager()));<br>+ new SourceManager(*DiagEngine, *FileMgr));<br> // CodeComplete seems to require fresh LangOptions.<br> LangOptions LangOpts = Unit->getLangOpts();<br> // The language server protocol uses zero-based line and column numbers.<br>@@ -172,8 +183,8 @@ std::vector<CompletionItem> ClangdUnit::<br> Unit->CodeComplete(FileName, Pos.line + 1, Pos.character + 1, RemappedSource,<br> CCO.IncludeMacros, CCO.IncludeCodePatterns,<br> CCO.IncludeBriefComments, Collector, PCHs, *DiagEngine,<br>- LangOpts, *SourceMgr, Unit->getFileManager(),<br>- StoredDiagnostics, OwnedBuffers);<br>+ LangOpts, *SourceMgr, *FileMgr, StoredDiagnostics,<br>+ OwnedBuffers);<br> for (const llvm::MemoryBuffer *Buffer : OwnedBuffers)<br> delete Buffer;<br> return Items;<br><br>Modified: clang-tools-extra/trunk/clangd/ClangdUnit.h<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdUnit.h?rev=303977&r1=303976&r2=303977&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdUnit.h?rev=303977&r1=303976&r2=303977&view=diff</a><br>==============================================================================<br>--- clang-tools-extra/trunk/clangd/ClangdUnit.h (original)<br>+++ clang-tools-extra/trunk/clangd/ClangdUnit.h Fri May 26 07:26:51 2017<br>@@ -10,8 +10,8 @@<br> #ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_CLANGDUNIT_H<br> #define LLVM_CLANG_TOOLS_EXTRA_CLANGD_CLANGDUNIT_H<br><br>-#include "Protocol.h"<br> #include "Path.h"<br>+#include "Protocol.h"<br> #include "clang/Frontend/ASTUnit.h"<br> #include "clang/Tooling/Core/Replacement.h"<br> #include <memory><br>@@ -24,6 +24,10 @@ namespace clang {<br> class ASTUnit;<br> class PCHContainerOperations;<br><br>+namespace vfs {<br>+class FileSystem;<br>+}<br>+<br> namespace tooling {<br> struct CompileCommand;<br> }<br>@@ -42,16 +46,19 @@ class ClangdUnit {<br> public:<br> ClangdUnit(PathRef FileName, StringRef Contents,<br> std::shared_ptr<PCHContainerOperations> PCHs,<br>- std::vector<tooling::CompileCommand> Commands);<br>+ std::vector<tooling::CompileCommand> Commands,<br>+ IntrusiveRefCntPtr<vfs::FileSystem> VFS);<br><br> /// Reparse with new contents.<br>- void reparse(StringRef Contents);<br>+ void reparse(StringRef Contents, IntrusiveRefCntPtr<vfs::FileSystem> VFS);<br><br> /// Get code completions at a specified \p Line and \p Column in \p File.<br> ///<br> /// This function is thread-safe and returns completion items that own the<br> /// data they contain.<br>- std::vector<CompletionItem> codeComplete(StringRef Contents, Position Pos);<br>+ std::vector<CompletionItem><br>+ codeComplete(StringRef Contents, Position Pos,<br>+ IntrusiveRefCntPtr<vfs::FileSystem> VFS);<br> /// Returns diagnostics and corresponding FixIts for each diagnostic that are<br> /// located in the current file.<br> std::vector<DiagWithFixIts> getLocalDiagnostics() const;<br><br>Modified: clang-tools-extra/trunk/clangd/ClangdUnitStore.h<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdUnitStore.h?rev=303977&r1=303976&r2=303977&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdUnitStore.h?rev=303977&r1=303976&r2=303977&view=diff</a><br>==============================================================================<br>--- clang-tools-extra/trunk/clangd/ClangdUnitStore.h (original)<br>+++ clang-tools-extra/trunk/clangd/ClangdUnitStore.h Fri May 26 07:26:51 2017<br>@@ -32,9 +32,10 @@ public:<br> template <class Func><br> void runOnUnit(PathRef File, StringRef FileContents,<br> GlobalCompilationDatabase &CDB,<br>- std::shared_ptr<PCHContainerOperations> PCHs, Func Action) {<br>+ std::shared_ptr<PCHContainerOperations> PCHs,<br>+ IntrusiveRefCntPtr<vfs::FileSystem> VFS, Func Action) {<br> runOnUnitImpl(File, FileContents, CDB, PCHs, /*ReparseBeforeAction=*/true,<br>- std::forward<Func>(Action));<br>+ VFS, std::forward<Func>(Action));<br> }<br><br> /// Run specified \p Action on the ClangdUnit for \p File.<br>@@ -45,9 +46,10 @@ public:<br> void runOnUnitWithoutReparse(PathRef File, StringRef FileContents,<br> GlobalCompilationDatabase &CDB,<br> std::shared_ptr<PCHContainerOperations> PCHs,<br>+ IntrusiveRefCntPtr<vfs::FileSystem> VFS,<br> Func Action) {<br> runOnUnitImpl(File, FileContents, CDB, PCHs, /*ReparseBeforeAction=*/false,<br>- std::forward<Func>(Action));<br>+ VFS, std::forward<Func>(Action));<br> }<br><br> /// Run the specified \p Action on the ClangdUnit for \p File.<br>@@ -71,24 +73,23 @@ private:<br> void runOnUnitImpl(PathRef File, StringRef FileContents,<br> GlobalCompilationDatabase &CDB,<br> std::shared_ptr<PCHContainerOperations> PCHs,<br>- bool ReparseBeforeAction, Func Action) {<br>+ bool ReparseBeforeAction,<br>+ IntrusiveRefCntPtr<vfs::FileSystem> VFS, Func Action) {<br> std::lock_guard<std::mutex> Lock(Mutex);<br><br> auto Commands = getCompileCommands(CDB, File);<br> assert(!Commands.empty() &&<br> "getCompileCommands should add default command");<br>- // chdir. This is thread hostile.<br>- // FIXME(ibiryukov): get rid of this<br>- llvm::sys::fs::set_current_path(Commands.front().Directory);<br>+ VFS->setCurrentWorkingDirectory(Commands.front().Directory);<br><br> auto It = OpenedFiles.find(File);<br> if (It == OpenedFiles.end()) {<br> It = OpenedFiles<br> .insert(std::make_pair(<br>- File, ClangdUnit(File, FileContents, PCHs, Commands)))<br>+ File, ClangdUnit(File, FileContents, PCHs, Commands, VFS)))<br> .first;<br> } else if (ReparseBeforeAction) {<br>- It->second.reparse(FileContents);<br>+ It->second.reparse(FileContents, VFS);<br> }<br> return Action(It->second);<br> }<br><br>Modified: clang-tools-extra/trunk/unittests/CMakeLists.txt<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/CMakeLists.txt?rev=303977&r1=303976&r2=303977&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/CMakeLists.txt?rev=303977&r1=303976&r2=303977&view=diff</a><br>==============================================================================<br>--- clang-tools-extra/trunk/unittests/CMakeLists.txt (original)<br>+++ clang-tools-extra/trunk/unittests/CMakeLists.txt Fri May 26 07:26:51 2017<br>@@ -11,4 +11,5 @@ add_subdirectory(clang-move)<br> add_subdirectory(clang-query)<br> add_subdirectory(clang-tidy)<br> add_subdirectory(clang-rename)<br>+add_subdirectory(clangd)<br> add_subdirectory(include-fixer)<br><br>Added: clang-tools-extra/trunk/unittests/clangd/CMakeLists.txt<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/clangd/CMakeLists.txt?rev=303977&view=auto" target="_blank">http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/clangd/CMakeLists.txt?rev=303977&view=auto</a><br>==============================================================================<br>--- clang-tools-extra/trunk/unittests/clangd/CMakeLists.txt (added)<br>+++ clang-tools-extra/trunk/unittests/clangd/CMakeLists.txt Fri May 26 07:26:51 2017<br>@@ -0,0 +1,24 @@<br>+set(LLVM_LINK_COMPONENTS<br>+ support<br>+ )<br>+<br>+get_filename_component(CLANGD_SOURCE_DIR<br>+ ${CMAKE_CURRENT_SOURCE_DIR}/../../clangd REALPATH)<br>+include_directories(<br>+ ${CLANGD_SOURCE_DIR}<br>+ )<br>+<br>+add_extra_unittest(ClangdTests<br>+ ClangdTests.cpp<br>+ )<br>+<br>+target_link_libraries(ClangdTests<br>+ clangBasic<br>+ clangDaemon<br>+ clangFormat<br>+ clangFrontend<br>+ clangSema<br>+ clangTooling<br>+ clangToolingCore<br>+ LLVMSupport<br>+ )<br><br>Added: clang-tools-extra/trunk/unittests/clangd/ClangdTests.cpp<br>URL: <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/clangd/ClangdTests.cpp?rev=303977&view=auto" target="_blank">http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/clangd/ClangdTests.cpp?rev=303977&view=auto</a><br>==============================================================================<br>--- clang-tools-extra/trunk/unittests/clangd/ClangdTests.cpp (added)<br>+++ clang-tools-extra/trunk/unittests/clangd/ClangdTests.cpp Fri May 26 07:26:51 2017<br>@@ -0,0 +1,364 @@<br>+//===-- ClangdTests.cpp - Clangd unit tests ---------------------*- C++ -*-===//<br>+//<br>+// The LLVM Compiler Infrastructure<br>+//<br>+// This file is distributed under the University of Illinois Open Source<br>+// License. See LICENSE.TXT for details.<br>+//<br>+//===----------------------------------------------------------------------===//<br>+<br>+#include "ClangdServer.h"<br>+#include "clang/Basic/VirtualFileSystem.h"<br>+#include "llvm/ADT/SmallVector.h"<br>+#include "llvm/ADT/StringMap.h"<br>+#include "llvm/Config/config.h"<br>+#include "llvm/Support/Errc.h"<br>+#include "llvm/Support/Path.h"<br>+#include "llvm/Support/Regex.h"<br>+#include "gtest/gtest.h"<br>+#include <algorithm><br>+#include <iostream><br>+#include <string><br>+#include <vector><br>+<br>+namespace clang {<br>+namespace vfs {<br>+<br>+/// An implementation of vfs::FileSystem that only allows access to<br>+/// files and folders inside a set of whitelisted directories.<br>+///<br>+/// FIXME(ibiryukov): should it also emulate access to parents of whitelisted<br>+/// directories with only whitelisted contents?<br>+class FilteredFileSystem : public vfs::FileSystem {<br>+public:<br>+ /// The paths inside \p WhitelistedDirs should be absolute<br>+ FilteredFileSystem(std::vector<std::string> WhitelistedDirs,<br>+ IntrusiveRefCntPtr<vfs::FileSystem> InnerFS)<br>+ : WhitelistedDirs(std::move(WhitelistedDirs)), InnerFS(InnerFS) {<br>+ assert(std::all_of(WhitelistedDirs.begin(), WhitelistedDirs.end(),<br>+ [](const std::string &Path) -> bool {<br>+ return llvm::sys::path::is_absolute(Path);<br>+ }) &&<br>+ "Not all WhitelistedDirs are absolute");<br>+ }<br>+<br>+ virtual llvm::ErrorOr<Status> status(const Twine &Path) {<br>+ if (!isInsideWhitelistedDir(Path))<br>+ return llvm::errc::no_such_file_or_directory;<br>+ return InnerFS->status(Path);<br>+ }<br>+<br>+ virtual llvm::ErrorOr<std::unique_ptr<File>><br>+ openFileForRead(const Twine &Path) {<br>+ if (!isInsideWhitelistedDir(Path))<br>+ return llvm::errc::no_such_file_or_directory;<br>+ return InnerFS->openFileForRead(Path);<br>+ }<br>+<br>+ llvm::ErrorOr<std::unique_ptr<llvm::MemoryBuffer>><br>+ getBufferForFile(const Twine &Name, int64_t FileSize = -1,<br>+ bool RequiresNullTerminator = true,<br>+ bool IsVolatile = false) {<br>+ if (!isInsideWhitelistedDir(Name))<br>+ return llvm::errc::no_such_file_or_directory;<br>+ return InnerFS->getBufferForFile(Name, FileSize, RequiresNullTerminator,<br>+ IsVolatile);<br>+ }<br>+<br>+ virtual directory_iterator dir_begin(const Twine &Dir, std::error_code &EC) {<br>+ if (!isInsideWhitelistedDir(Dir)) {<br>+ EC = llvm::errc::no_such_file_or_directory;<br>+ return directory_iterator();<br>+ }<br>+ return InnerFS->dir_begin(Dir, EC);<br>+ }<br>+<br>+ virtual std::error_code setCurrentWorkingDirectory(const Twine &Path) {<br>+ return InnerFS->setCurrentWorkingDirectory(Path);<br>+ }<br>+<br>+ virtual llvm::ErrorOr<std::string> getCurrentWorkingDirectory() const {<br>+ return InnerFS->getCurrentWorkingDirectory();<br>+ }<br>+<br>+ bool exists(const Twine &Path) {<br>+ if (!isInsideWhitelistedDir(Path))<br>+ return false;<br>+ return InnerFS->exists(Path);<br>+ }<br>+<br>+ std::error_code makeAbsolute(SmallVectorImpl<char> &Path) const {<br>+ return InnerFS->makeAbsolute(Path);<br>+ }<br>+<br>+private:<br>+ bool isInsideWhitelistedDir(const Twine &InputPath) const {<br>+ SmallString<128> Path;<br>+ InputPath.toVector(Path);<br>+<br>+ if (makeAbsolute(Path))<br>+ return false;<br>+<br>+ for (const auto &Dir : WhitelistedDirs) {<br>+ if (Path.startswith(Dir))<br>+ return true;<br>+ }<br>+ return false;<br>+ }<br>+<br>+ std::vector<std::string> WhitelistedDirs;<br>+ IntrusiveRefCntPtr<vfs::FileSystem> InnerFS;<br>+};<br>+<br>+/// Create a vfs::FileSystem that has access only to temporary directories<br>+/// (obtained by calling system_temp_directory).<br>+IntrusiveRefCntPtr<vfs::FileSystem> getTempOnlyFS() {<br>+ llvm::SmallString<128> TmpDir1;<br>+ llvm::sys::path::system_temp_directory(/*erasedOnReboot=*/false, TmpDir1);<br>+ llvm::SmallString<128> TmpDir2;<br>+ llvm::sys::path::system_temp_directory(/*erasedOnReboot=*/true, TmpDir2);<br>+<br>+ std::vector<std::string> TmpDirs;<br>+ TmpDirs.push_back(TmpDir1.str());<br>+ if (TmpDir2 != TmpDir2)<br>+ TmpDirs.push_back(TmpDir2.str());<br>+ return new vfs::FilteredFileSystem(std::move(TmpDirs),<br>+ vfs::getRealFileSystem());<br>+}<br>+} // namespace vfs<br>+<br>+namespace clangd {<br>+namespace {<br>+<br>+class ErrorCheckingDiagConsumer : public DiagnosticsConsumer {<br>+public:<br>+ void onDiagnosticsReady(PathRef File,<br>+ std::vector<DiagWithFixIts> Diagnostics) override {<br>+ bool HadError = false;<br>+ for (const auto &DiagAndFixIts : Diagnostics) {<br>+ // FIXME: severities returned by clangd should have a descriptive<br>+ // diagnostic severity enum<br>+ const int ErrorSeverity = 1;<br>+ HadError = DiagAndFixIts.Diag.severity == ErrorSeverity;<br>+ }<br>+<br>+ std::lock_guard<std::mutex> Lock(Mutex);<br>+ HadErrorInLastDiags = HadError;<br>+ }<br>+<br>+ bool hadErrorInLastDiags() {<br>+ std::lock_guard<std::mutex> Lock(Mutex);<br>+ return HadErrorInLastDiags;<br>+ }<br>+<br>+private:<br>+ std::mutex Mutex;<br>+ bool HadErrorInLastDiags = false;<br>+};<br>+<br>+class MockCompilationDatabase : public GlobalCompilationDatabase {<br>+public:<br>+ std::vector<tooling::CompileCommand><br>+ getCompileCommands(PathRef File) override {<br>+ return {};<br>+ }<br>+};<br>+<br>+class MockFSProvider : public FileSystemProvider {<br>+public:<br>+ IntrusiveRefCntPtr<vfs::FileSystem> getFileSystem() override {<br>+ IntrusiveRefCntPtr<vfs::InMemoryFileSystem> MemFS(<br>+ new vfs::InMemoryFileSystem);<br>+ for (auto &FileAndContents : Files)<br>+ MemFS->addFile(FileAndContents.first(), time_t(),<br>+ llvm::MemoryBuffer::getMemBuffer(FileAndContents.second,<br>+ FileAndContents.first()));<br>+<br>+ auto OverlayFS = IntrusiveRefCntPtr<vfs::OverlayFileSystem>(<br>+ new vfs::OverlayFileSystem(vfs::getTempOnlyFS()));<br>+ OverlayFS->pushOverlay(std::move(MemFS));<br>+ return OverlayFS;<br>+ }<br>+<br>+ llvm::StringMap<std::string> Files;<br>+};<br>+<br>+/// Replaces all patterns of the form 0x123abc with spaces<br>+void replacePtrsInDump(std::string &Dump) {<br>+ llvm::Regex RE("0x[0-9a-fA-F]+");<br>+ llvm::SmallVector<StringRef, 1> Matches;<br>+ while (RE.match(Dump, &Matches)) {<br>+ assert(Matches.size() == 1 && "Exactly one match expected");<br>+ auto MatchPos = Matches[0].data() - Dump.data();<br>+ std::fill(Dump.begin() + MatchPos,<br>+ Dump.begin() + MatchPos + Matches[0].size(), ' ');<br>+ }<br>+}<br>+<br>+std::string dumpASTWithoutMemoryLocs(ClangdServer &Server, PathRef File) {<br>+ auto Dump = Server.dumpAST(File);<br>+ replacePtrsInDump(Dump);<br>+ return Dump;<br>+}<br>+<br>+template <class T><br>+std::unique_ptr<T> getAndMove(std::unique_ptr<T> Ptr, T *&Output) {<br>+ Output = Ptr.get();<br>+ return Ptr;<br>+}<br>+} // namespace<br>+<br>+class ClangdVFSTest : public ::testing::Test {<br>+protected:<br>+ SmallString<16> getVirtualTestRoot() {<br>+#ifdef LLVM_ON_WIN32<br>+ return SmallString<16>("C:\\clangd-test");<br>+#else<br>+ return SmallString<16>("/clangd-test");<br>+#endif<br>+ }<br>+<br>+ llvm::SmallString<32> getVirtualTestFilePath(PathRef File) {<br>+ assert(llvm::sys::path::is_relative(File) && "FileName should be relative");<br>+<br>+ llvm::SmallString<32> Path;<br>+ llvm::sys::path::append(Path, getVirtualTestRoot(), File);<br>+ return Path;<br>+ }<br>+<br>+ std::string parseSourceAndDumpAST(<br>+ PathRef SourceFileRelPath, StringRef SourceContents,<br>+ std::vector<std::pair<PathRef, StringRef>> ExtraFiles = {},<br>+ bool ExpectErrors = false) {<br>+ MockFSProvider *FS;<br>+ ErrorCheckingDiagConsumer *DiagConsumer;<br>+ ClangdServer Server(<br>+ llvm::make_unique<MockCompilationDatabase>(),<br>+ getAndMove(llvm::make_unique<ErrorCheckingDiagConsumer>(),<br>+ DiagConsumer),<br>+ getAndMove(llvm::make_unique<MockFSProvider>(), FS),<br>+ /*RunSynchronously=*/false);<br>+ for (const auto &FileWithContents : ExtraFiles)<br>+ FS->Files[getVirtualTestFilePath(FileWithContents.first)] =<br>+ FileWithContents.second;<br>+<br>+ auto SourceFilename = getVirtualTestFilePath(SourceFileRelPath);<br>+ Server.addDocument(SourceFilename, SourceContents);<br>+ auto Result = dumpASTWithoutMemoryLocs(Server, SourceFilename);<br>+ EXPECT_EQ(ExpectErrors, DiagConsumer->hadErrorInLastDiags());<br>+ return Result;<br>+ }<br>+};<br>+<br>+TEST_F(ClangdVFSTest, Parse) {<br>+ // FIXME: figure out a stable format for AST dumps, so that we can check the<br>+ // output of the dump itself is equal to the expected one, not just that it's<br>+ // different.<br>+ auto Empty = parseSourceAndDumpAST("foo.cpp", "", {});<br>+ auto OneDecl = parseSourceAndDumpAST("foo.cpp", "int a;", {});<br>+ auto SomeDecls = parseSourceAndDumpAST("foo.cpp", "int a; int b; int c;", {});<br>+ EXPECT_NE(Empty, OneDecl);<br>+ EXPECT_NE(Empty, SomeDecls);<br>+ EXPECT_NE(SomeDecls, OneDecl);<br>+<br>+ auto Empty2 = parseSourceAndDumpAST("foo.cpp", "");<br>+ auto OneDecl2 = parseSourceAndDumpAST("foo.cpp", "int a;");<br>+ auto SomeDecls2 = parseSourceAndDumpAST("foo.cpp", "int a; int b; int c;");<br>+ EXPECT_EQ(Empty, Empty2);<br>+ EXPECT_EQ(OneDecl, OneDecl2);<br>+ EXPECT_EQ(SomeDecls, SomeDecls2);<br>+}<br>+<br>+TEST_F(ClangdVFSTest, ParseWithHeader) {<br>+ parseSourceAndDumpAST("foo.cpp", "#include \"foo.h\"", {},<br>+ /*ExpectErrors=*/true);<br>+ parseSourceAndDumpAST("foo.cpp", "#include \"foo.h\"", {{"foo.h", ""}},<br>+ /*ExpectErrors=*/false);<br>+<br>+ const auto SourceContents = R"cpp(<br>+#include "foo.h"<br>+int b = a;<br>+)cpp";<br>+ parseSourceAndDumpAST("foo.cpp", SourceContents, {{"foo.h", ""}},<br>+ /*ExpectErrors=*/true);<br>+ parseSourceAndDumpAST("foo.cpp", SourceContents, {{"foo.h", "int a;"}},<br>+ /*ExpectErrors=*/false);<br>+}<br>+<br>+TEST_F(ClangdVFSTest, Reparse) {<br>+ MockFSProvider *FS;<br>+ ErrorCheckingDiagConsumer *DiagConsumer;<br>+ ClangdServer Server(<br>+ llvm::make_unique<MockCompilationDatabase>(),<br>+ getAndMove(llvm::make_unique<ErrorCheckingDiagConsumer>(), DiagConsumer),<br>+ getAndMove(llvm::make_unique<MockFSProvider>(), FS),<br>+ /*RunSynchronously=*/false);<br>+<br>+ const auto SourceContents = R"cpp(<br>+#include "foo.h"<br>+int b = a;<br>+)cpp";<br>+<br>+ auto FooCpp = getVirtualTestFilePath("foo.cpp");<br>+ auto FooH = getVirtualTestFilePath("foo.h");<br>+<br>+ FS->Files[FooH] = "int a;";<br>+ FS->Files[FooCpp] = SourceContents;<br>+<br>+ Server.addDocument(FooCpp, SourceContents);<br>+ auto DumpParse1 = dumpASTWithoutMemoryLocs(Server, FooCpp);<br>+ EXPECT_FALSE(DiagConsumer->hadErrorInLastDiags());<br>+<br>+ Server.addDocument(FooCpp, "");<br>+ auto DumpParseEmpty = dumpASTWithoutMemoryLocs(Server, FooCpp);<br>+ EXPECT_FALSE(DiagConsumer->hadErrorInLastDiags());<br>+<br>+ Server.addDocument(FooCpp, SourceContents);<br>+ auto DumpParse2 = dumpASTWithoutMemoryLocs(Server, FooCpp);<br>+ EXPECT_FALSE(DiagConsumer->hadErrorInLastDiags());<br>+<br>+ EXPECT_EQ(DumpParse1, DumpParse2);<br>+ EXPECT_NE(DumpParse1, DumpParseEmpty);<br>+}<br>+<br>+TEST_F(ClangdVFSTest, ReparseOnHeaderChange) {<br>+ MockFSProvider *FS;<br>+ ErrorCheckingDiagConsumer *DiagConsumer;<br>+<br>+ ClangdServer Server(<br>+ llvm::make_unique<MockCompilationDatabase>(),<br>+ getAndMove(llvm::make_unique<ErrorCheckingDiagConsumer>(), DiagConsumer),<br>+ getAndMove(llvm::make_unique<MockFSProvider>(), FS),<br>+ /*RunSynchronously=*/false);<br>+<br>+ const auto SourceContents = R"cpp(<br>+#include "foo.h"<br>+int b = a;<br>+)cpp";<br>+<br>+ auto FooCpp = getVirtualTestFilePath("foo.cpp");<br>+ auto FooH = getVirtualTestFilePath("foo.h");<br>+<br>+ FS->Files[FooH] = "int a;";<br>+ FS->Files[FooCpp] = SourceContents;<br>+<br>+ Server.addDocument(FooCpp, SourceContents);<br>+ auto DumpParse1 = dumpASTWithoutMemoryLocs(Server, FooCpp);<br>+ EXPECT_FALSE(DiagConsumer->hadErrorInLastDiags());<br>+<br>+ FS->Files[FooH] = "";<br>+ Server.forceReparse(FooCpp);<br>+ auto DumpParseDifferent = dumpASTWithoutMemoryLocs(Server, FooCpp);<br>+ EXPECT_TRUE(DiagConsumer->hadErrorInLastDiags());<br>+<br>+ FS->Files[FooH] = "int a;";<br>+ Server.forceReparse(FooCpp);<br>+ auto DumpParse2 = dumpASTWithoutMemoryLocs(Server, FooCpp);<br>+ EXPECT_FALSE(DiagConsumer->hadErrorInLastDiags());<br>+<br>+ EXPECT_EQ(DumpParse1, DumpParse2);<br>+ EXPECT_NE(DumpParse1, DumpParseDifferent);<br>+}<br>+<br>+} // namespace clangd<br>+} // namespace clang<br><br><br>_______________________________________________<br>cfe-commits mailing list<br><a href="mailto:cfe-commits@lists.llvm.org" target="_blank">cfe-commits@lists.llvm.org</a><br><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits</a><br></div></div></blockquote></div><br></div></div>_______________________________________________<br>
cfe-commits mailing list<br>
<a href="mailto:cfe-commits@lists.llvm.org" target="_blank">cfe-commits@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits</a><br>
</blockquote></div>