[llvm-dev] Network RPCs in LLVM projects

[Sam McCall via llvm-dev]

Short version: clangd would like to be able to build a client+server that
can make RPCs across the internet. An RPC system isn't a trivial dependency
and rolling our own from scratch isn't appealing.
Have other projects had a need for this? Any advice on how to approach such
dependencies?

--

Longer: clangd (a language server, like an IDE backend) builds an index of
the project you're working on in order to answer queries (go to definition,
code completion...). This takes *lots* of CPU-time to build, and RAM to
serve.
For large codebases with many developers, sharing an index across users
<https://llvm.discourse.group/t/sharing-indexes-for-multiple-users/202> is
a better approach - you spend the CPU in one place, you spend the RAM in a
few places, and an RPC is fast enough even for code completion. We have
experience with this approach inside Google.

We'd like to build this index server upstream (just a shell around clangd's
current index code) and put the client in clangd. For open-source projects,
I imagine the server being publicly accessible over the internet.
This means we care about
 - latency (this is interactive, every 10ms counts)
 - security
 - proxy traversal, probably
 - sensible behavior under load
 - auth is probably nice-to-have

I don't think this is something we want to build from scratch, I hear
portable networking is hard :-)
The most obvious thing is to depend on something like Thrift, grpc, etc,
but these aren't trivial dependencies to take on. They could probably be
structured as an optional CMake dependency, which we'd want to ask
distributors to enable.

Have other projects had anything like these requirements? Any solutions, or
desire to use such infrastructure? I saw some RPC layer in ORC, but it
seems mostly abstract/FD-based IPC.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20191212/800ffbc8/attachment.html>