[llvm-dev] [cfe-dev] GitHub anyone?

Thu Jun 2 10:53:35 PDT 2016

My personal 2 cents is that I’d love to see us move to github and a git-native workflow. I’m already on a mostly git workflow and the only thing that tears me out of it is git-svn, which corrupts itself way too often.

As a late-comer on the thread I do have a few thoughts I want to share based on the discussion.

(1) We really don’t know how many users are using git vs svn. What if we encouraged users on git to setup a commit-msg hook that appended text to the end of their commit messages to identify the commits as being from git?

Something like this -> http://reviews.llvm.org/P4788

Then we could figure out easily a rough approximation of how many of our developers are on git vs svn. Having that information would make a lot of these discussions more clear, and it would give us some level of empirical data.

(2) I strongly think merging repositories is a bad idea. In particular we support building clang without having an LLVM checkout by building against an installed LLVM. Merging the repositories would break this, and that may be an important feature. Similarly compiler-rt is used with GCC, so we need to support it not being in the LLVM repo. All of this gets messy.

(3) Submodules can be messy. I’d prefer if we had a solution where they were optional or something like the github llvm-project-submodule setup where the submodules are all part of a separate repository. We could even make it so that repository is updated by a bot only after a successful compile. That would be kinda cool.

-Chris

> On Jun 2, 2016, at 9:28 AM, Scott Warren via cfe-dev <cfe-dev at lists.llvm.org> wrote:
> 
> My two cents worth: our group tries very hard to avoid Git because we find it immature, hard to use, and unreliable. I know others feel differently but our vote is to stay with SVN. If distributed repositories are a priority we’d be much happier with Mercurial.
> 
> skw
> 
> 
>> On Jun 2, 2016, at 11:21 AM, Tanya Lattner via cfe-dev <cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>> wrote:
>> 
>> I personally find this email thread very hard to follow and read (this isn’t anyones fault.. its just a lot of replies). I am sure others do as well. I think it would be good to have a form/survey of some sort that can get feedback from users such as: who they are, how they use LLVM/contributions/etc,  if they are pro-github move, how it impacts them, etc. People could then submit their feedback in an organized way and we could get a better idea of how the community feels on the topic. 
>> 
>> I am happy to try to set something like this up.
>> 
>> -Tanya
>> 
>>> On Jun 2, 2016, at 8:48 AM, Renato Golin via llvm-dev <llvm-dev at lists.llvm.org> wrote:
>>> 
>>> A little summary...
>>> 
>>> After a lot of discussion, I think we converged to a few issues that
>>> we need to solved before we finally decide to move.
>>> 
>>> Firstly, the responses were overwhelmingly positive (I counted 20 of
>>> the ~25 people strongly supporting and another 2~3 weakly supporting).
>>> This is a good indication that the move could be very beneficial to
>>> the community as a whole, including downstream infrastructure, not
>>> just the reduction in upstream infrastructure admin costs.
>>> 
>>> But that doesn't mean we have cleared up all the issues...
>>> 
>>> 
>>>  The benefits I gathered from the thread:
>>> 
>>> * Infrastructure admin (not just server costs) is too expensive.
>>> We're not sysadmins and maintaining all the tools is a full time job.
>>> Volunteering works for odd problems, not for production services.
>>> Furthermore, most of the infrastructure we need is covered by
>>> GitHub/Lab/BB for free, on a scale that we would not have, even with a
>>> full time sysadmin. Gratis.
>>> 
>>> * Having one official repository instead of two is beneficial to most
>>> developers. A lot of people (most people replying on this thread), use
>>> Git in addition to SVN. Git also seems to be used more on validation
>>> infrastructure than SVN (no example was put forward on this thread, at
>>> least), due to the simplicity of controlling the repository and the
>>> tools available. Reports of how teams decided to script Git to have
>>> linear behaviour instead of falling back to SVN are enlightening.
>>> 
>>> * Git developer tooling is a growing trend, while SVN tooling is
>>> dying. This is not just about GUIs, but repository management (GitHub,
>>> GitLab, BitBucket, etc versus SourceForge), bisects, branches,
>>> remotes, hooks, workdir, submodules and all the new development seem
>>> to be done on Git nowadays, not SVN. Windows may be an odd one related
>>> to GUIs, but Visual Studio has Git integration and I hear it's similar
>>> to the other MSVC VCSs. GitHub's desktop interface seems pretty cool,
>>> too.
>>> 
>>> * Web repositories make it *a lot* easier to create add-hoc pull
>>> requests by non-developers, which could boost the number of
>>> contributions and future contributors, as well as external projects
>>> using LLVM components.
>>> 
>>> * GitHub's SVN RW interface has been reported to work well for
>>> simpler projects, but we need a more thorough examination before
>>> declare it good enough for our purposes.
>>> 
>>> * All reports on the thread pointed that downstream infrastructure is
>>> already using Git, so that's one less problem to worry about if we do
>>> move.
>>> 
>>> 
>>>  The issues that were raised:
>>> 
>>> * Co-dependent patches already break buildbots, but the sequential ID
>>> helps us identify and ignore. They will continue to break, even if we
>>> use git sub-modules, so that doesn't change much, but it will be
>>> harder to spot the issue. Server side hooks may help, as well as
>>> sub-modules.
>>> 
>>> * Windows tooling may be an issue. There's a separate thread handling
>>> that part, so I won't cover it here. But I have to say it wasn't by a
>>> long shot a resonant problem. It may also have some problem with
>>> symlinks and in-tree checkouts (when interacting with llvm-projects
>>> and sub-modules).
>>> 
>>> * Sub-modules may help with a lot of the current relationship we have
>>> inside the SVN repo, but it also has some problems. Namely they:
>>> - require a modern version of git (1.7/1.9), but that's 2013 onward.
>>> - may need additional server side scripting, but we can keep that
>>> in another repo to control it.
>>> - won't replace SVN's monotonic IDs, but do we *really* need them?
>>> Sub-modules have a bad fame, I gather, but people in the thread
>>> reported success on using it to build validation and release
>>> infrastructure as well as doing bisects, checking out code, etc. We
>>> probably need some documentation on how to do these things, as well as
>>> some scripts to help people work out the dependencies (or use them).
>>> 
>>> * GitHub/Lab/BB are not perfect. They have some interface issues, but
>>> nothing more serious than we already have on our current
>>> infrastructure. We'll probably have to keep Bugzilla (as GitHub's own
>>> is really poor), but we can replace all our repos (SVN, Git),
>>> visualisation tools (ViewVC, Klaus) and Phabricator.
>>> 
>>> Of all those issues, Windows tooling is a minor problem that shouldn't
>>> impact decision that much and sub-modules need a lot of ironing out to
>>> be considered good enough. My *personal* take away is that sub-modules
>>> (or an alternative server side solution) is the only strong technical
>>> issue we need to solve before we decide.
>>> 
>>> 
>>> How does a move look like?
>>> 
>>> If we decide to move, the proposed schedule is something like this:
>>> 
>>> STEP #1 : Pre Move
>>> 
>>> 0. Update docs to mention the move, so people are aware the it's going on.
>>> 1. Register an official GitHub project with the LLVM foundation.
>>> 2. Setup another (read-only) mirror of llvm.org/git at this GitHub project
>>> 3. Make sure we have a la llvm-project-submodules setup in the
>>> official account. (Optional or necessary for the buildbots?)
>>> 4. Make sure bisecting with llvm-project-submodules is a good experience
>>> 5. Make sure no one has any blocker
>>> 
>>> STEP #2 : Git Move
>>> 
>>> 6. Update the buildbots to pick up updates and commits from the
>>> official git repository
>>> 7. Update Phabricator to pick up commits from the official git repository
>>> 8. Tell people living downstream to pick up commits from the official
>>> git repository
>>> 9. Give things time to settle. We could play some games like disabling
>>> the svn repository for a few hours on purpose so that people can test
>>> that their infrastructure has really become independent of the svn
>>> repository.
>>> 
>>> ... Until this point nothing has changed for developers, it will just
>>> boil down to a lot of work for buildbot and other infrastructure
>>> owners ...
>>> 
>>> STEP #3: Write Access Move
>>> 
>>> 10. Collect peoples GitHub account information, give them push access.
>>> Ideally while still locking the GitHub repository somehow...
>>> 11. Switch SVN repository to read-only and allow pushes to the GitHub
>>> repository.
>>> 12. Mirror Git to SVN.
>>> 
>>> STEP #4 : Post Move
>>> 
>>> 13. Archive the SVN repository, if GitHub's SVN is good enough.
>>> 14. Review and update *all* LLVM documentation.
>>> 15. Review website links pointing to viewvc/klaus/phab etc. to point
>>> to GitHub instead.
>>> 
>>> This is an adapted version of Matthias' and Mehdi's proposal, and it's
>>> not a final version in any way, but these are the basic things we need
>>> to worry about.
>>> 
>>> 
>>>  Steps from here...
>>> 
>>> Aaron has started the Windows tooling thread, and if you have any
>>> comments, please follow from there. I suggest sub-modules supporters
>>> to start another thread to iron that out separately.
>>> 
>>> Once those issues are resolved, we shall start another thread to
>>> finally take a decision to move or not.
>>> 
>>> Thanks everyone!
>>> 
>>> cheers,
>>> --renato
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> 
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev>
> 
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160602/528d1fc0/attachment.html>