[llvm-dev] RFC: Adding a staging branch (temporarily) to facilitate upstreaming

Tue Jun 30 10:48:19 PDT 2020

On Tue, Jun 30, 2020 at 10:11 AM Nicolai Hähnle <nhaehnle at gmail.com> wrote:

> On Tue, Jun 30, 2020 at 7:15 AM Chandler Carruth via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
> >
> > On Mon, Jun 29, 2020 at 9:43 PM Mehdi AMINI via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
> >>
> >> Hey Duncan,
> >>
> >> On Mon, Jun 29, 2020 at 8:28 PM Duncan Exon Smith via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
> >>>
> >>> To facilitate collaboration on an upstreaming effort (see "More
> context" below), we'd like to push a branch (with history) called
> "staging/apple" to github.com/llvm/llvm-project to serve as an official
> contribution to the LLVM project. This enables motivated parties to work
> with us to craft incremental patches for review on Phabricator. This branch
> would live during the effort and then be deleted after. It would not be
> merged.
> >>>
> >>> Does this seem fine? If you have a strong objection, please share your
> concern.
> >>>
> >>> For reference, I ran some experiments:
> >>>
> >>> A `--bare` clone (just the Git database) I have of
> github.com/llvm/llvm-project was around ~1GB. Fetching this branch from
> github.com/apple/llvm-project increased it to ~1.2GB. Running `git gc
> --aggressive` brought it down to ~850MB.
> >>> The worktree of the "master" branch is ~1GB. Adding the Git database
> gives ~2GB, ~2.2GB, and ~1.9GB.
> >>> The diff of the proposed staging/apple branch is 3.1MB at `-U0`, 4.1MB
> at `-U3`, and 32MB at `-U999999` (Phabricator settings).
> >>>
> >>>
> >>>
> >>> More context
> >>>
> >>> We're making a major push over the next few months to upstream changes
> that have accumulated over time in the branch called "apple/master" at
> github.com/apple/llvm-project. It has always been a non-goal for us to
> have changes, but over the years we've accumulated a non-trivial diff vs.
> github.com/llvm/llvm-project. This includes (but is not limited to)
> tweaks/features related to tuple hashing, modules hashing, source
> attributes, API notes, pointer authentication, indexing-while-building, and
> local refactoring.
> >>>
> >>> Our goal is to eliminate this difference. Besides paying off some
> debt, this upstreaming effort unblocks the Swift compiler (
> github.com/apple/swift) from building directly against an upstream
> checkout of LLVM. That's why non-Apple contributors are motivated to help
> craft incremental patches.
> >>>
> >>>
> >>> Alternatives considered
> >>>
> >>> As an alternative, we could post a GitHub pull request and close it
> without merging. From our perspective this would serve the same purpose.
> However, pull requests are contentious in LLVM.
> >>>
> >>> Another alternative is to post a bulk Phabricator review and then
> "abandon" it. However, this has the disadvantage of not contributing the
> history (~30k commits).
> >>
> >>
> >> Seeing the alternatives, if a closed pull-request would fit your
> use-case, then I'm not sure why you need the branch to actually live in the
> monorepo instead of in any fork (github.com/apple/llvm-project or
> another)? It seems publicly accessible the same way?
> >
> >
> > As I understand it, a key need is to explicitly contribute this to the
> LLVM project to make it unambiguous that it has been contributed and is
> completely available for folks not at Apple to iterate on the code and turn
> it into code-reviewable chunks.
>
> How do you envision this working? Has anybody outside of Apple
> _actually_ expressed interest in working on it in this way?
>

Yes, as far as I know, folks approached those at Apple to figure out how to
collaborate on upstreaming things.

FWIW, I don't think having this branch in the upstream LLVM repository
would make much sense without that -- the only purpose it solves is to
allow a group of interested people across the LLVM community to collaborate
effectively.

> I'm mostly asking because we also have a fork of LLVM that we
> continuously keep aligned with github.com/llvm/llvm-project/master
> with the intention of deltas being broken out to be moved back
> upstream. That branch is currently at
> https://github.com/GPUOpen-Drivers/llvm-project, but if the LLVM
> community decides that branches whose goal is to be contributed
> upstream can (and should?) live in github.com/llvm/llvm-project, we'd
> probably be interested in doing that and having a branch such as
> amd/gfx-dev in the github.com/llvm/llvm-project repository.
>

See above -- I personally think this is an effective strategy when it
solves a practical problem for people in the LLVM community at different
organizations who want to collaborate on upstreaming something. I'd hope
that anything like that was only done contingent on genuine interest in the
community and with a pretty clear "it goes away" plan when merging stuff is
complete.

>
> Cheers,
> Nicolai
>
>
> > So whatever happens needs to be quite explicit in its nature as a
> contribution. IMO, a branch of the repository definitely qualifies.
> >
> > IMO, a pull request isn't as clear given that they haven't been used for
> contributions before. This is not a time to be innovative IMO. A branch as
> a staging location has been used many times over the history of the project
> though and seems nicely unambiguous in that regard.
> >
> >>
> >> Your initial description mentions "collaboration on an upstreaming
> effort", maybe you can elaborate a bit on what this collaboration would
> look like and how these patches would end up in master?
> >>
> >> Thanks,
> >>
> >> --
> >> Mehdi
> >>
> >>
> >> _______________________________________________
> >> LLVM Developers mailing list
> >> llvm-dev at lists.llvm.org
> >> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>
> --
> Lerne, wie die Welt wirklich ist,
> aber vergiss niemals, wie sie sein sollte.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200630/f0823704/attachment.html>