[llvm-dev] [RFC] One or many git repositories?

Thu Jul 28 00:36:07 PDT 2016

Hi James,

I don’t agree that it holds none of the advantages.  LLVM components are, broadly speaking, in two categories:

- Those that are tightly coupled to others
- Those that are not

Pulling the LLVM Support library out of the LLVM repository, for example, would not make sense because (even though it can be built stand-alone and doesn’t depend on the rest of LLVM) it is tightly coupled to other things and someone who changes it may need to atomically update the other LLVM libraries and possibly projects such as clang to conform to the new APIs.

Similarly, lldb, lld, and clang are all dependent on unstable APIs in LLVM (and lldb on unstable APIs in clang).  We therefore need to either retain the ability to update these consistently.  This gives us three choices:

1) Put LLVM, lldb, lld and clang all into the same repository.

2) Make LLVM a submodule of clang and lld, and clang a submodule of lld so that any API-breaking change doesn’t get propagated into the projects that depend on them until the consumers of the APIs have also been updated

3) Provide stronger API stability guarantees and deprecation mechanisms so that (for example) LLVM can change an API, then clang can adopt it.

Of the three, option 1 is the closest to what we have now and probably the simplest (and also has other advantages with respect to bisecting and so on).

This does not apply to libc++.  We support building the entire LLVM suite with other C++ standard library implementations (at least libstdc++, and I think also with Visual Studio’s implementation), so there is no dependency of anything on libc++.  Similarly, we support building libc++ with other compilers (in FreeBSD, we currently build it with gcc 6.1 for RISC-V, for example, where the LLVM toolchain is not quite useable).

The same applies to libunwind, to an even greater degree (where libc++ implements a standard API, libunwind implements a standard ABI).

David

> On 27 Jul 2016, at 19:32, James Molloy <james at jamesmolloy.co.uk> wrote:
> 
> David,
> 
> While I understand the rationale here, I vote for all or nothing. A middle ground seems to me to hold none of the advantages and all of the disadvantages!
> 
> James
> On Wed, 27 Jul 2016 at 19:30, David Chisnall via llvm-dev <llvm-dev at lists.llvm.org> wrote:
> On 27 Jul 2016, at 19:03, Bruce Hoult via llvm-dev <llvm-dev at lists.llvm.org> wrote:
> >
> > What do you believe is this increased burden?
> >
> > The entire commit history of all llvm projects in a mono-repository is a 449 MB .git directory. It can be downloaded in about two minutes on a typical domestic internet connection (50 Mbps).
> >
> > If you download only a snapshot of the current HEAD commit then the .git repository is 88 MB and takes under a minute. Any other individual commit should be similar.
> >
> > This doesn't seem like a big burden to me.
> >
> > The checked out llvm source directory -- which you say is all that many people want -- is 202 MB. That's without even building it.
> >
> > Why is this burden unacceptable? It seems rather small to me.
> >
> > For comparison, using svn to checkout llvm using ...
> >
> > svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm
> >
> > ... took me 1 minute 28 seconds and gives a 222 MB .svn directory, 428 MB total (so 206 MB for the source files checked out).
> 
> The burden, as he pointed out, is for people who contribute to llvm-umbrella projects that are not llvm.
> 
> I think that this is a very good reason for not including libc++ in the monorepo.  Currently, it takes me about 15 seconds to do an checkout of libc++ and the result is about 100MB.  A clone from the git mirror takes just under 6 seconds, downloads 17.15MB, and produces a 72MB directory.  The build directory is about 7MB.
> 
> There is no dependency from libc++ to anything else in LLVM.  In FreeBSD, we use libc++ with both clang and gcc and people hacking on libc++ do not typically need to build a new toolchain to do so.  Pulling in all of LLVM, clang, lldb, lld, and so on just to hack on libc++ seems a bit excessive.
> 
> libunwind is similar.  It exposes a public interface that’s ABI compatible with two other libraries and is tiny.  We currently have people working on it in the process of replacing the libunwind in FreeBSD with the LLVM version.  Some of them also hack on other bits of LLVM, but getting the entire LLVM repo just to hack on a library that’s 1.6MB of source (1.5 seconds to do an svn checkout currently) seems a bit excessive.
> 
> I don’t think it’s a problem for the larger projects (clang, lldb, lld) to be in a combined repo, because the size of any of them is fairly small in comparison to the size of LLVM and tiny in comparison to the build directory.
> 
> David
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3719 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160728/5bfbab85/attachment.bin>