[llvm-dev] [RFC] One or many git repositories?

[Chandler Carruth via llvm-dev]

I wanted to present some of the particular reasons why I'm pretty strongly
opposed to a purely flat layout of projects the way the current github
'llvm-project' repository looks, as that hasn't happened on the list yet.
I'm replying to myself as I don't see a much better place to hang that
conversation.

On Wed, Jul 20, 2016 at 7:38 PM Chandler Carruth <chandlerc at google.com>
wrote:

> On Wed, Jul 20, 2016 at 7:08 PM James Y Knight via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Should the layout in the merged repository be:
>> 1) Like the "llvm-project" git repository is now:
>>
>> <root>/llvm/
>> <root>/clang/
>> <root>/compiler-rt
>> ...
>>
>> 2) Like the "ideal merged checkout" is now:
>> llvm/
>> llvm/tools/clang
>> llvm/projects/compiler-rt
>> ...
>>
>>
>> I don't much care which of those is chosen. I have a slight preference
>> for #1, for ease of doing things like grep/log/etc on llvm by itself,
>> excluding all the other projects. But either way seems probably fine, and
>> an improvement over multiple repositories.
>>
>
> FWIW, I strongly prefer #2, but I think the high order bit is the
> repository question.
>

So, a reasonable question might be, why do I prefer #2?
I have a lot of not terribly connected reasons.

First, I want to consider what happens if we go with #1. Today, LLVM
subprojects have been formed essentially any time it was conducive to do
so. This worked around the subversion sparse checkout challenges (arguably
also solved by newer subversion features, but that's neither here nor
there) and didn't cause any problem because we could lay out the tree any
way that made sense and we always had a global revision number. A classic
example: clang-tools-extra. At the time it was added, it was perceived as
very useful to segregate. These days, I'm not sure the risk is interesting
any more, and the cost is probably higher than the benefit. But it probably
doesn't make sense to have a "cfe" directory and "clang-tools-extra"
directory as peers. If we're moving to a monolithic repo, the
clang-tools-extra stuff should almost *certainly* move under the 'tools'
directory in the clang repo, where ever that ends up.

So, if we go with #1 above and just use the existing subversion repos as
the top level directories, how would we rationally make a decision in the
future about "should X new directory be a top-level directory, or a just
fit it into the existing hierarchy?". I don't think we will ever have a
good and principled response. We will constantly have oddball warts where
things happen to be top-level because at one point we wanted the ability to
not check out those Subversion repos, and now that has been enshrined.


I'm not actually arguing that #2 is a *good* layout. But I think it is a
(slightly) less arbitrary layout than #1. And by breaking this weird mold
of "all Subversion projects are top level", I think we'll be in a better
place to make reasonably and considered decisions about re-structuring the
layout long-term to reflect a useful and rational layout based on some set
of reasonable technical principles.

It also has the advantage of being the layout which, if people's existing
scripts and systems are set up around the defaults in the CMake build, will
be the simplest to migrate to. I certainly know that all of my habits and
patterns are geared around this layout and it will be dramatically easier
for me to migrate to a single repo if it preserves this layout.


Long term, I want to see us use a layout that reasonably connotes the
logical and practical structure of the code and project as a whole. I also
long-term want to see the layout effectively address the pragmatic needs of
tools and systems developers rely on such as "git log". On the whole, I
think #2 is (slightly) closer to that than #1 so I strongly prefer it, but
it clearly isn't perfect here. I just think we can incrementally fix and
improve the layout over time. I don't think we're stuck in a single layout
forever.

Hope this helps motivate why I would very much prefer to retain the default
layout suggested in our docs and build system for now, and phrase any
re-organization as follow-on changes once we had a single repo that made
such changes straightforward and easily history-preserving.

-Chandler
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160722/8b17ecdc/attachment.html>