[llvm-dev] [RFC] One or many git repositories?

Sun Jul 24 15:04:53 PDT 2016

On Sun, Jul 24, 2016 at 10:46 AM, Mehdi Amini via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

>
> On Jul 22, 2016, at 12:51 AM, Chandler Carruth via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
> I wanted to present some of the particular reasons why I'm pretty strongly
> opposed to a purely flat layout of projects the way the current github
> 'llvm-project' repository looks, as that hasn't happened on the list yet.
> I'm replying to myself as I don't see a much better place to hang that
> conversation.
>
> On Wed, Jul 20, 2016 at 7:38 PM Chandler Carruth <chandlerc at google.com>
> wrote:
>
>> On Wed, Jul 20, 2016 at 7:08 PM James Y Knight via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>> Should the layout in the merged repository be:
>>> 1) Like the "llvm-project" git repository is now:
>>>
>>> <root>/llvm/
>>> <root>/clang/
>>> <root>/compiler-rt
>>> ...
>>>
>>> 2) Like the "ideal merged checkout" is now:
>>> llvm/
>>> llvm/tools/clang
>>> llvm/projects/compiler-rt
>>> ...
>>>
>>>
>>> I don't much care which of those is chosen. I have a slight preference
>>> for #1, for ease of doing things like grep/log/etc on llvm by itself,
>>> excluding all the other projects. But either way seems probably fine, and
>>> an improvement over multiple repositories.
>>>
>>
>> FWIW, I strongly prefer #2, but I think the high order bit is the
>> repository question.
>>
>
> So, a reasonable question might be, why do I prefer #2?
> I have a lot of not terribly connected reasons.
>
> First, I want to consider what happens if we go with #1. Today, LLVM
> subprojects have been formed essentially any time it was conducive to do
> so. This worked around the subversion sparse checkout challenges (arguably
> also solved by newer subversion features, but that's neither here nor
> there) and didn't cause any problem because we could lay out the tree any
> way that made sense and we always had a global revision number. A classic
> example: clang-tools-extra. At the time it was added, it was perceived as
> very useful to segregate. These days, I'm not sure the risk is interesting
> any more, and the cost is probably higher than the benefit. But it probably
> doesn't make sense to have a "cfe" directory and "clang-tools-extra"
> directory as peers. If we're moving to a monolithic repo, the
> clang-tools-extra stuff should almost *certainly* move under the 'tools'
> directory in the clang repo, where ever that ends up.
>
> So, if we go with #1 above and just use the existing subversion repos as
> the top level directories, how would we rationally make a decision in the
> future about "should X new directory be a top-level directory, or a just
> fit it into the existing hierarchy?". I don't think we will ever have a
> good and principled response. We will constantly have oddball warts where
> things happen to be top-level because at one point we wanted the ability to
> not check out those Subversion repos, and now that has been enshrined.
>
>
> I'm not actually arguing that #2 is a *good* layout. But I think it is a
> (slightly) less arbitrary layout than #1.
>
>
> I’m not sold on this: the existing build system layout could be seen even
> more arbitrary than the SVN repo separations.
>

+1. I don't think we should expect to find a "principled" solution to the
layout we should choose. We just need to build a table of pros and cons and
then stare at it.

>
>
> And by breaking this weird mold of "all Subversion projects are top
> level", I think we'll be in a better place to make reasonably and
> considered decisions about re-structuring the layout long-term to reflect a
> useful and rational layout based on some set of reasonable technical
> principles.
>
> It also has the advantage of being the layout which, if people's existing
> scripts and systems are set up around the defaults in the CMake build, will
> be the simplest to migrate to. I certainly know that all of my habits and
> patterns are geared around this layout and it will be dramatically easier
> for me to migrate to a single repo if it preserves this layout.
>
>
> Again, I’m not convinced: right now the default is that “cmake
> path/to/llvm” is building *only* LLVM. The flat layout preserves naturally
> this behavior, while unifying everything using the build-system layout
> makes it difficult to get the existing behavior.
>
>
> Long term, I want to see us use a layout that reasonably connotes the
> logical and practical structure of the code and project as a whole. I also
> long-term want to see the layout effectively address the pragmatic needs of
> tools and systems developers rely on such as "git log". On the whole, I
> think #2 is (slightly) closer to that than #1 so I strongly prefer it, but
> it clearly isn't perfect here. I just think we can incrementally fix and
> improve the layout over time. I don't think we're stuck in a single layout
> forever.
>
>
> `git log` is suddenly harder i.e. instead of getting llvm, I get libcxx +
> libcxx-abi + compiler-rt + clang + clang-tools-extras + lld + …
>

This is the reason for my preference as well.

-- Sean Silva

> Also it is not clear to me why the layout can’t be fixed gradually over
> time (`git mv`) starting from the flat one.
>
> Since the existing build system (and none of the tooling) is currently
> able to handle #2 (but could be fine with #1), I’m not convinced about
> "starting with #2 and then improving the situation” because  the disruption
> seems to important this way.
>
> —
> Mehdi
>
>
> Hope this helps motivate why I would very much prefer to retain the
> default layout suggested in our docs and build system for now, and phrase
> any re-organization as follow-on changes once we had a single repo that
> made such changes straightforward and easily history-preserving.
>
> -Chandler
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160724/e6f7c150/attachment.html>