[llvm-dev] [RFC] One or many git repositories?

Fri Jul 22 01:15:32 PDT 2016

Ok, throwing in my opinion, so we do not under-represent folks working far
down the chain (I mainly have branches on clang-tools-extra I juggle with).
Also with the disclaimer that a move to any git model will be super welcome
from my side, and huge props & thanks to folks who have put so much time
and effort into that!

If you work on clang-tools-extra, a single repo would be helpful. Version
skew is annoying, and I have already tried (and I think so far failed to)
understand git-submodules. Working on phabricator, which uses git
submodules extensively, I've multiple times run into a problem where I was
in a state that had the wrong git submodule version, and I was basically
lost on how to solve that aside from completely re-initializing the
submodules, which only works if you at least are backwards compatible (that
is, the code works with a much newer submodule package version).

Regarding the specific point of being modular: without a stable code
interface, we're not really cross-version-modular anyway. And if we want to
encourage modularity within the libraries, there are other things we can do
than splitting up the repos: for example, use the clang module maps to
ensure #includes match dependencies, etc.

I do agree that it would be great to have a good solution to rebase forks
onto the new repo, though.

Cheers,
/Manuel

On Fri, Jul 22, 2016 at 9:51 AM Chandler Carruth via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> I wanted to present some of the particular reasons why I'm pretty strongly
> opposed to a purely flat layout of projects the way the current github
> 'llvm-project' repository looks, as that hasn't happened on the list yet.
> I'm replying to myself as I don't see a much better place to hang that
> conversation.
>
> On Wed, Jul 20, 2016 at 7:38 PM Chandler Carruth <chandlerc at google.com>
> wrote:
>
>> On Wed, Jul 20, 2016 at 7:08 PM James Y Knight via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
> Should the layout in the merged repository be:
>>> 1) Like the "llvm-project" git repository is now:
>>>
>>> <root>/llvm/
>>> <root>/clang/
>>> <root>/compiler-rt
>>> ...
>>>
>>> 2) Like the "ideal merged checkout" is now:
>>> llvm/
>>> llvm/tools/clang
>>> llvm/projects/compiler-rt
>>> ...
>>>
>>>
>>> I don't much care which of those is chosen. I have a slight preference
>>> for #1, for ease of doing things like grep/log/etc on llvm by itself,
>>> excluding all the other projects. But either way seems probably fine, and
>>> an improvement over multiple repositories.
>>>
>>
>> FWIW, I strongly prefer #2, but I think the high order bit is the
>> repository question.
>>
>
> So, a reasonable question might be, why do I prefer #2?
> I have a lot of not terribly connected reasons.
>
> First, I want to consider what happens if we go with #1. Today, LLVM
> subprojects have been formed essentially any time it was conducive to do
> so. This worked around the subversion sparse checkout challenges (arguably
> also solved by newer subversion features, but that's neither here nor
> there) and didn't cause any problem because we could lay out the tree any
> way that made sense and we always had a global revision number. A classic
> example: clang-tools-extra. At the time it was added, it was perceived as
> very useful to segregate. These days, I'm not sure the risk is interesting
> any more, and the cost is probably higher than the benefit. But it probably
> doesn't make sense to have a "cfe" directory and "clang-tools-extra"
> directory as peers. If we're moving to a monolithic repo, the
> clang-tools-extra stuff should almost *certainly* move under the 'tools'
> directory in the clang repo, where ever that ends up.
>
> So, if we go with #1 above and just use the existing subversion repos as
> the top level directories, how would we rationally make a decision in the
> future about "should X new directory be a top-level directory, or a just
> fit it into the existing hierarchy?". I don't think we will ever have a
> good and principled response. We will constantly have oddball warts where
> things happen to be top-level because at one point we wanted the ability to
> not check out those Subversion repos, and now that has been enshrined.
>
>
> I'm not actually arguing that #2 is a *good* layout. But I think it is a
> (slightly) less arbitrary layout than #1. And by breaking this weird mold
> of "all Subversion projects are top level", I think we'll be in a better
> place to make reasonably and considered decisions about re-structuring the
> layout long-term to reflect a useful and rational layout based on some set
> of reasonable technical principles.
>
> It also has the advantage of being the layout which, if people's existing
> scripts and systems are set up around the defaults in the CMake build, will
> be the simplest to migrate to. I certainly know that all of my habits and
> patterns are geared around this layout and it will be dramatically easier
> for me to migrate to a single repo if it preserves this layout.
>
>
> Long term, I want to see us use a layout that reasonably connotes the
> logical and practical structure of the code and project as a whole. I also
> long-term want to see the layout effectively address the pragmatic needs of
> tools and systems developers rely on such as "git log". On the whole, I
> think #2 is (slightly) closer to that than #1 so I strongly prefer it, but
> it clearly isn't perfect here. I just think we can incrementally fix and
> improve the layout over time. I don't think we're stuck in a single layout
> forever.
>
> Hope this helps motivate why I would very much prefer to retain the
> default layout suggested in our docs and build system for now, and phrase
> any re-organization as follow-on changes once we had a single repo that
> made such changes straightforward and easily history-preserving.
>
> -Chandler
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160722/5ded50ec/attachment.html>