[llvm-dev] Subprojects, GitHub, and the Monorepo

Sat Oct 20 15:16:18 PDT 2018

I think these issues you raise are all basically fixable, without
fragmenting the community.

# Bugtracker

Yes, everyone has a github account -- but I think everyone could also use
bugzilla easily enough, if we add a "Login with github" button. I would
like someone to volunteer to work on getting bugzilla patched with that
functionality. (Someone, please volunteer!)

# Website, and raw HTML vs markdown.

Clangd doesn't have a website now, in the same sense as clang, llvm and
other llvm projects do. It just has a little bit of sphinx documentation
hidden deep within clang's docs. I think this is a large part of the
frustration you raise, but it's not a necessary part of living under the
llvm project umbrella. I think you probably want to make clangd have its
own actual landing page.

As for the authoring format, IMO it's definitely a good idea to start
migrating to using markdown for the website, and migrate to github pages
autogeneration and hosting. (But I think that should be done project-wide,
not one-off.)

# Exposing LLVM community identity

Clangd is part of the LLVM community, and having a community identity can
be a good thing. But, indeed, the projects should also have their own
identities within that.

You've expressed "Users don't want to know that clangd is part of the llvm
community". But I think that's exactly wrong. We _should_ be exposing that
fact and pushing that identity, and I don't think that, by itself, bothers
any uesrs.

However, I think I see a different underlying issue hiding in that stated
concern: finding clangd in the current website is too confusing. Having
clangd be part of the LLVM community -- or part of the LLVM website --
isn't the issue problem. Having clangd not be _easily accessible_ on the
LLVM website is a problem.

It could have higher visibility, even while being "part of the LLVM
project", and represented and structured as such.

# Path forward

Instead of starting up a parallel infrastructure, I'd suggest that the way
forward should be:
1. Keep (and make minor enhancement to) bugzilla. Add a clangd component to
bugzilla.
2. Add a more user-facing, top-level, clangd landing page within the llvm
website, like we have for other projects, to give the project higher
visibility. With links to code, bugtracker (e.g. link directly to the
search/enter-bug urls in the clangd component), as appropriate.
3. Start converting the LLVM website to a github pages site.

Note that github pages allows a mixture of html, markdown, and jekyll site
generator features, and automatically updates upon commits to the
repository. I think a fairly extensive but still relatively simple example
of a bunch of functionality is www.mono-project.org, autogenerated from <
https://github.com/mono/website>.

(I don't think the restructured-text "docs" directories necessarily need be
migrated; for the most part those are somewhat of a different kind of thing
than the "website-proper", and could be left as is, at least for now.)

Website migration to git/github-pages does not need to wait for the entire
svn->git repository migration process to finish! We could pretty easily
decide to make the "www" git repository canonical first, before the code.

On Sat, Oct 20, 2018 at 10:39 AM Sam McCall via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> I work on clangd, the language server/IDE backend in clang/tools/extra.
> Clangd is at a stage where the core functionality is stable and useful
> enough that we want to put it front of more users. I've been spending time
> recently thinking about user-facing things: packaging, mailing lists, docs,
> bugtracking.
>
> And I think we should do much of this on GitHub, rather than *.llvm.org.
> And not in the upcoming monorepo, but in a separate repository. (e.g.
> github.com/llvm/clang)
>
> I expect this to be controversial. It's definitely community
> fragmentation. I think the reasons to do it for clangd are strong, but they
> won't apply equally to all projects. And I'd like to know what people
> think. So here's my reasoning.
>
> *Point 1: It's what people expect.*
> Everyone knows how to use the Github bug tracker, and has a Github account.
> Everyone knows markdown, how to edit-and-preview, and how to send a doc
> pull request.
> Everyone has these workflows in their muscle memory when a github project
> is the top websearch result.
> (Current LLVM developers *also* know the LLVM equivalents, but that's a
> small group).
> This is largely why we're moving the code to Github, too.
>
> *Point 2: exposing the LLVM monolith is bad for users.*
> Clangd's customers don't care about the structure of the LLVM umbrella
> project, or even that it exists.
> If they search for clangd on the web, they want to find a tree that looks
> like this:
> clangd
> - features
> - installation
> - bugs
> - code
> Not like this:
> llvm.org
> - docs
> -- lldb, etc
> -- clang
> --- features, etc
> --- tools
> ---- clang-tidy, etc
> ---- clangd
> ----- features
> ----- installation
> - bugs
> -- lldb, etc
> -- clang
> --- tools
> ---- clang-tidy, etc
> ---- clangd
> - code
> -- lldb, etc
> ...
> LLVM's source repository is monolithic for technical reasons (versioning),
> but we that's not a strong reason that the bug trackers, documentation etc
> should be monolithic. Spraying hyperlinks around won't fix the fact that
> the website is the wrong shape.
>
> *Point 3: the tools are just better.*
> I have nothing but respect and gratitude for the people that admin
> bugzilla, wrangle CMake and sphinx to generate docs, and keep mailman
> running. But unsurprisingly the state of the art has moved on, and the
> equivalents are in my experience easier to use, faster, and more reliable.
> Symptoms of this are people routing around the tools: LLDB doesn't use
> sphinx for docs, sanitizers don't use bugzilla.
> I'm sure there's going to be some agreement and disagreement on this point
> :-)
>
> *Point 4: but the tools are designed for smaller, focused repositories*
> The "github-native" community is mostly using fairly narrowly scoped
> repositories, and the tools work better this way. For example, labels are
> enough to organize issues in a project the size of clangd, but too
> lightweight if the scope is LLVM and all subprojects.
>
> *What does the logical conclusion of this look like?*
> I don't know. I suspect other subprojects in a similar boat may
> independently come to the same conclusion. Projects that have e.g. lots of
> bug history will need a migration story.
> None of this mitigates the need for a source monorepo, so we'd be stuck
> with all the code in llvm/llvm and just issues/docs in llvm/clangd. Not
> ideal, but manageable.
> Clangd is a pretty easy case, so I don't know if this makes it a good
> trial or a bad one.
>
> <*dons flame-retardant suit*>
> What do you all think?
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20181020/56e900e1/attachment.html>