[llvm-dev] [RFC] One or many git repositories?

Fri Jul 29 11:51:57 PDT 2016

> On Jul 29, 2016, at 11:50 AM, Robinson, Paul <paul.robinson at sony.com> wrote:
> 
> I don’t know what you mean by dealing with the merging, I don’t expect any
> difficulties, you need to elaborate.
>  
> What I don’t see you addressing here is why this should be more of a problem in the monorepo case (as it was implied in the email I was answering to).
>  
> Your answer made it sound like you thought the monorepo would solve all downstream problems ("I don't know what you mean… I don't expect any difficulties"), which it clearly does not.  Sorry for the misunderstanding.

My bad, I wrongly expressed myself, I should have written “I don’t expect any *extra* difficulties over what we have now or what we'd have with submodules”.

— 
Mehdi

> --paulr
>   <>
> From: mehdi.amini at apple.com [mailto:mehdi.amini at apple.com] 
> Sent: Friday, July 29, 2016 11:17 AM
> To: Robinson, Paul
> Cc: David Chisnall; Bruce Hoult; llvm-dev at lists.llvm.org
> Subject: Re: [llvm-dev] [RFC] One or many git repositories?
>  
>  
> On Jul 29, 2016, at 11:07 AM, Robinson, Paul <paul.robinson at sony.com <mailto:paul.robinson at sony.com>> wrote:
>  
> 
> 
> 
> -----Original Message-----
> From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org <mailto:llvm-dev-bounces at lists.llvm.org>] On Behalf Of Mehdi
> Amini via llvm-dev
> Sent: Friday, July 29, 2016 10:02 AM
> To: David Chisnall
> Cc: llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>; Bruce Hoult
> Subject: Re: [llvm-dev] [RFC] One or many git repositories?
> 
> 
> 
> On Jul 29, 2016, at 2:19 AM, David Chisnall
> <david.chisnall at cl.cam.ac.uk <mailto:david.chisnall at cl.cam.ac.uk>> wrote:
> 
> 
> On 29 Jul 2016, at 05:11, Mehdi Amini via llvm-dev <llvm-
> dev at lists.llvm.org <mailto:dev at lists.llvm.org>> wrote:
> 
> 
> What I meant by “different problem" is that “downstream users” for
> instance don’t need to commit, that makes their problem/workflow quite
> different from an upstream developer (for instance it is fairly easy to
> maintain a read-only view of the existing individual git repo currently on
> llvm.org <http://llvm.org/>).
> 
> 
> I’m not convinced by this distinction.  A lot of downstream developers
> need to patch LLVM and we benefit when they upstream their changes.
> 
> I made a difference between downstream users and developers. I.e. someone
> that just need to get and build compiler-rt vs someone that want to
> *commit* to LLVM. Note that even by getting a single repo you can still
> send a patch to the mailing list and someone can commit it for you
> (including correct author attribution contrary to SVN).
> 
> 
> We should not make it harder for them to do this.  To give a couple of
> example downstream projects, both FreeBSD and Swift have patches on LLVM /
> Clang in their versions that they gradually filter upstream.  Both
> projects have LLVM committers among their members.  If the workflow that
> we recommend for them makes upstreaming easy then they benefit
> (maintaining a fork is effort) and LLVM benefits (having people provide
> bug fixes makes our code better).
> 
> 
> The workflow that we want to recommend to these people is:
> 
> - Fork the repo that you’re interested in from the LLVM GitHub
> organisation
> 
> - Make your changes
> - Send pull requests for anything that you think is of interest to
> upstream
> 
> 
> Note that the workflow you describe above still requires to export their
> patch and import it in this clone before pushing.
> (Note also that we accept patches on the mailing list, so one does not
> even need to clone the official repo).
> 
> 
> This makes the barrier to entry for sending code back upstream *much*
> lower than it currently is,
> 
> I don’t understand this statement. As of today you can send a diff to the
> mailing list, I don’t see how lower the bar can be.
> 
> 
> 
> to the benefit of all.  If the alternative is:
> 
> - Fork a read-only repo that you’re interested in from the LLVM GitHub
> organisation
> 
> - Make your changes
> 
> Why? If you know you want to *push* commits upstream, fork the only useful
> repo for that in the first place.
> 
> 
> - Fork a different repo from the LLVM GitHub organisation
> - Run a script to filter some of your changes into that one
> 
> I don’t know why you think there is a need for a script, or why it is
> different from today.
> Let say I’m working on a fork of the compiler-rt read-only repo and I want
> to upstream a patch at some point:
> 
> Today:
> 
> - cd /path/to/compiler_rt-forked
> - git format-patch …
> - cd /path/to/compiler_rt-upstream
> - git am  /path/to/compiler_rt-forked/0001-My-awesome-changes.patch
> - git svn dcommit
> - done
> 
> Tomorrow with a monorepo:
> 
> - cd /path/to/compiler_rt-forked
> - git format-patch …
> - cd /path/to/unifiedrepo-upstream
> - git am  /path/to/compiler_rt-forked/0001-My-awesome-changes.patch —
> directory=compiler-rt
> - git push
> - done
> 
> Alternatively, if I’m upstream a patch once a year, I don’t really need to
> push it myself.
> 
> - cd /path/to/compiler_rt-forked
> - git format-patch …
> - email the patch.
> 
> 
> 
> - Send a pull request from that
> 
> Note that I think we deferred any change to the workflow for future
> discussions (pull-request are not part of our workflow today).
> 
> 
> - Deal with merging between the two yourself
> 
> I don’t know what you mean by dealing with the merging, I don’t expect any
> difficulties, you need to elaborate.
> 
> Almost no upstream patch survives contact with the reviewers unscathed
> (and quite often it doesn't look exactly like your local patch anyway,
> for reasons ranging from expediency to local coding standards).
> Therefore the patch that you merge back in to your local repo from 
> upstream differs from the one you already have locally.  If you are *very*
> lucky, the differences can be handled with accept-theirs; in my experience
> this is not normal, and in many cases you need to manually revert your
> original local patch before merging the patch as received from upstream.
> That's what "dealing with merging" means.  I invite you to review
> "Living Downstream Without Drowning" if you still don't understand;
>  
> Don’t worry, I’m living downstream, and perfectly aware of the situation.
> (And I was in the room at the dev meeting).
>  
> 
> 
> it describes a number of tactics we've developed over the years to
> try to simplify the dealing-with-merging problem.
> 
> I had two entire dev-meeting sessions full of people who seemed to be
> having this problem.  Maybe you don't; if so, you're lucky, but it
> doesn't mean the problem fails to exist.
>  
> The “problem” exists, it exists today, and it will tomorrow whatever solution we go with (monorepo or split ones).
>  
> What I don’t see you addressing here is why this should be more of a problem in the monorepo case (as it was implied in the email I was answering to).
>  
> — 
> Mehdi
>  
> 
> 
> --paulr
> 
> 
> 
> 
> 
> I strongly suspect that we’ll get a lot fewer useful contributions from
> downstream.  Or downstream people will just work on the monorepo and eat
> the cost.
> 
> 
> If someone is working on a downstream LLVM project and becoming familiar
> with our codebase, then we want them to be subtly nudging their workflow
> so that they eventually become LLVM contributors without noticing!
> 
> Sure. The distinction between “downstream users” and “developers” was made
> in response to “there exists many user that just download and build a
> subproject”. These are not people that are *developing* on a downstream
> fork.
> 
> —
> Mehdi
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160729/ad52dc8b/attachment.html>