[cfe-dev] ClangD

Thu Jan 26 06:40:30 PST 2017

On Thu, Jan 26, 2017 at 3:33 PM Nikolai Kosjar <nikolai.kosjar at qt.io> wrote:

> On 01/25/2017 02:11 PM, Manuel Klimek via cfe-dev wrote:
> > Hi fellow clang devs,
> >
> > we wanted to let you know that we're (finally) starting up work on
> > ClangD, which you might know from email threads such as [1] (June 2012!).
> >
> > In the meantime, YCM had done a good enough job at packaging up a
> > libclang interface to our favorite editors, and other priorities (like
> > modules) have eaten up a lot of folks priority lunches.
> >
> > What has changed?
> > 1. YCM is starting to develop more and more into a language multiplexer,
> > with other languages (Go, Typescript, etc) providing their own servers
> > to talk to
> > 2. MS has created a language server protocol [2], which already has both
> > a bunch of client and server implementations
> > 3. Debugging through python into libclang crashers is a pain and eating
> > our support resources
> > 4. While libclang is a good abstraction if you want to have something
> > run in your process with close coupling, a standard protocol like the
> > language server protocol seems like a better way to allow fast
> > iterations on the server implementation without the need to keep
> > backward-compatibility hacks through a restrictive C API.
> >
> > One of the cool things about the MS language server protocol is that it
> > seems to not actually do any networking, which means that we do not need
> > to introduce any new dependencies into clang-tools-extra, which is where
> > we want to start developing this.
> >
> > If you have any thoughts / concerns let me know; otherwise look forward
> > to code reviews on initial ClangD dropping by :D
>
> [Huch, my first association with "ClangD" was support for the D language]
>
> Hi!
>
> We from the Qt Creator IDE team are also interested in this. :)
>
> Some notes:
>
> 1. You've indirectly referenced the initial design document at [1]. Will
> this get an update or is it already updated somewhere else? An updated
> document clarifying the scope and goals would be probably useful to all
> interested parties, also in regard to contribution and how to align own
> current development that uses libclang/libtooling/clang tools.
>

So far the scope is replacing the C++ parts of YCM (that is, code
completion, code browsing, "fast" syntax checking and fixits)

> E.g. Qt Creator also already launches a separate process
> ("clangbackend") that makes use of libclang to provide the basic
> functionality like updated diagnostics, highlighting information and
> results for completion requests.
>
>
> 2. How is clangd related to "clang-refactor" [2]?
>
> If it's not related at all, what's the state of clang-refactor?
>

clang-refactor is still in the design stage. I think at some point we'll
probably want to integrate clang-refactor, but we first want a prototype
and see how well that goes.

>
> 3. I take this opportunity to share our biggest problems with libclang
> from the Qt Creator perspective since clangd users will probably face
> them also (as far as I understand, clangd will be based upon libclang -
> please correct me if I'm wrong). I share this in hope that they will be
> addressed or taken into consideration during the development of clangd
> for the benefit of all interested parties now that the development of
> clangd is announced. Please keep in mind that the actual problems might
> be in other components that libclang makes use of.
>
> The problems we are facing with libclang, in this order, are:
>
> 3.1 Windows-specific issues
>
> The whole performance of parse/reparse in general is worser than on
> Mac/Linux (probably also because of slow IO on Windows). At the end of
> the day, this makes it less usable on Windows for real-world
> projects/files. There are also other Windows-specific issues.
>
> My gut feeling is that there aren't that many libclang users on Windows.
>
> @Manual: To which extend do you or your team use and develop on Windows?
>

We are not Windows developers :) We hope that by integrating with VS Code /
MS language protocol we can get some interested parties on Windows, but so
far we rely on help from Windows wizards.

> Are any Windows developers working on clangd?
>

Well, it'll be an open source project ;)

> Concrete issues are:
>
>   * Preamble is also generated for the second reparse on Windows
>     https://llvm.org/bugs/show_bug.cgi?id=28535
>
>   * Problems with FileManager caches leading to leaking file descriptors
>     https://reviews.llvm.org/D27810
>
>   * Very slow reparsing for big header and even slower with
>     preamble-generation enabled (might be related to the one above)
>     https://llvm.org/bugs/show_bug.cgi?id=31353
>
>   * Windows: Crash with absolute path with forward slashes
>     https://llvm.org/bugs/show_bug.cgi?id=28381
>
> 3.2 Synchronous/blocking API
> Low-latency completion results at all times are impossible with a single
> CXTranslationUnit that is just being parsed or reparsed. This gets worse
> the longer the parse/reparse takes (hello Windows).
>
> One can mitigate this by triggering the parse/reparse after a
> significant timeout of user-typing-inactivity, but this leads to a
> sluggish experience regarding new diagnostics and highlighting
> informations (you don't want to wait more than 2s for new diagnostics to
> see if your code is valid now).
>
> Another approach is to use two CXTranslationUnits per file, so at least
> there is always a non-blocking CXTranslationUnits ready for completion
> requests. Downside is doubled memory consumption and potentially
> threading issues (@Manual: do they still exist or are they already
> resolved?).
>

I work on a very large codebase and don't have problems with code
completion. I think all YCM integrations I know take care to not block the
UI for completion requests (or in general).

> 3.3 libclang requires unsaved files to exist on disk.
>
> If the unsaved file does not exist a preamble generation will happen for
> every reparse which simply takes to long.
>

I would be surprised if that was true - I often work on unsaved files
without seeing these problems.

>
> For the concrete use case, refer to
>
>         https://llvm.org/bugs/show_bug.cgi?id=31112
>
> This one might be also a problem if you do version control checkouts and
> files suddenly vanish.
>
>
>
> Best regards,
>     Nikolai
>
>
>
> [1] https://github.com/chandlerc/llvm-designs/blob/master/ClangService.rst
> [2] http://lists.llvm.org/pipermail/cfe-dev/2016-September/050641.html
>
> >
> > Cheers,
> > /Manuel
> >
> > [1] http://lists.llvm.org/pipermail/cfe-dev/2012-June/022028.html
> > [2] https://github.com/Microsoft/language-server-protocol
> >
> >
> >
> > _______________________________________________
> > cfe-dev mailing list
> > cfe-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20170126/dde3a0fe/attachment.html>