[llvm-dev] [RFC] One or many git repositories?

[Robinson, Paul via llvm-dev]

Can you please clarify your use of “cost” (bandwidth, disk space, extra command to type initially?),

Developer time, barrier to entry for new contributors.  Getting the sparse-checkout business right looks like it is actually non-trivial and not recommended for the git novice.  *Changing* the sparse-checkout configuration later appears to be fraught with peril (easy to get wrong).

The claim is to keep the existing history (I.e. not hash changes) that is currently at http://llvm.org/git/llvm.git and continue to accumulate there any new commit that would touch the llvm subdirectory of the unified repo.
This would be a read-only view of course, but just like it is now.

Hmmm so there's still a per-old-project view?  Missed that aspect, sorry…  it would let us preserve our processes in terms of integrating the flow from upstream, although being able to get a correctly linearized flow of commits from the unified repo would be preferable and we would *want* to change over.  Still not clear how to make that work with a sparse checkout.
--paulr

From: mehdi.amini at apple.com [mailto:mehdi.amini at apple.com]
Sent: Thursday, July 21, 2016 4:52 PM
To: Robinson, Paul
Cc: Renato Golin; Justin Lebar; llvm-dev at lists.llvm.org
Subject: Re: [llvm-dev] [RFC] One or many git repositories?


On Jul 21, 2016, at 4:39 PM, Robinson, Paul <paul.robinson at sony.com<mailto:paul.robinson at sony.com>> wrote:




-----Original Message-----
From: mehdi.amini at apple.com<mailto:mehdi.amini at apple.com> [mailto:mehdi.amini at apple.com]
Sent: Thursday, July 21, 2016 3:16 PM
To: Robinson, Paul
Cc: Renato Golin; Justin Lebar; llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
Subject: Re: [llvm-dev] [RFC] One or many git repositories?



On Jul 21, 2016, at 2:33 PM, Robinson, Paul via llvm-dev <llvm-
dev at lists.llvm.org<mailto:dev at lists.llvm.org>> wrote:



On 21 July 2016 at 18:12, Justin Lebar <jlebar at google.com<mailto:jlebar at google.com>> wrote:

llvm, clang, clang-tools-extra, lld, polly, lldb, llgo, compiler-rt,
openmp, and parallel-libs.

I really, *really* would like to see libc++ / abi / unwind. :)

My reason is that, when building toolchains, the C++ ABI and unwinding
are fundamental parts of the run-time library, of which RT is only
part of.

When building *your* toolchain...

My toolchain uses clang but not libc++/abi/unwind, we have our own, and
we don't currently include them in our tree.  We do include compiler-rt.

If we should change our minds later we can opt-in to anything else we
want (libcxx etc, lld? lldb? who knows) but in the meantime they are
unnecessary baggage for my purposes.

As a developer, you can checkout part of the repo with sparse-checkout.

I'm not clear why imposing this cost on everybody who wants less-than-all
(which I'd think would be most people)

Can you please clarify your use of “cost” (bandwidth, disk space, extra command to type initially?), otherwise it is hard for me to address you concerns (for instance I’m actually sensitive to the one you spelled out clearly in another email about a commit in lld requiring a rebase in llvm).

is superior to the submodule thing
which can be maintained centrally by people who actually understand how to
do it.

While I see some good principled way to have a submodule umbrella repo in git, I don’t see any *without  server-side hooks* that does not have any flaw*. Unfortunately this is not addressed by Renato’s proposal, and github does not allow server-side hooks, and another git hosting service is spelled out-of-discussion for Renato’s proposal.

* we may consider the flaws acceptable, but they need to be understood and accepted, and I don’t think it has been spelled out clearly in Renato’s proposal.

As a downstream integrator, you can filter out the repo history as you
want before merging into your repo.

Hmmm maybe, maybe not.  It sounds like the claim is: you can do a sparse
checkout of upstream, then merge it to a different branch, and get only
the history of the stuff that was sparsely checked out.

No that’s not the claim (sparse checkout are totally unrelated to this part of my claim).

The claim is to keep the existing history (I.e. not hash changes) that is currently at http://llvm.org/git/llvm.git and continue to accumulate there any new commit that would touch the llvm subdirectory of the unified repo.
This would be a read-only view of course, but just like it is now.

I.e. if you’re using the existing git repo, we can keep maintaining your workflow *as-is* forever. It means *no* migration would be forced on any CI/integration system (other than those relying on SVN).
(We’d need some creativity around the git-svn-id in the commit messages for the new commits though).


—
Mehdi



Does this work
with subtree merges?  Our branches are not rooted at the 'llvm' directory,
and I am suspicious about what the sparse checkout config would do to the
local branch.  (I know, I should do the experiment myself, but right now
I'm in the middle of a release-prep circus and really shouldn't be
spending the time to write this email:-).)

If all of this magic *does* work, then mainly it's a matter of scripting
the sparse-checkout config and deploying that internally.  Not free, but
maybe not horrible either.
--paulr



—
Mehdi

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160722/506e82c3/attachment.html>