[llvm-dev] New LLVM git repository conversion prototype

Fri Nov 16 13:10:33 PST 2018

Yes, I believe all the known issues are resolved and that we're ready to
go, whenever the other thread (on whether to abandon this monorepo, and
zipper-commit the existing gitrepositories) is concluded.

On Thu, Nov 15, 2018 at 4:09 PM Tom Stellard <tstellar at redhat.com> wrote:

> On 10/11/2018 03:27 PM, James Y Knight via llvm-dev wrote:
> > TLDR: https://github.com/llvm-git-prototype/ exists as a read-only
> mirror of SVN, and is being updated continuously with a script running on
> an llvm-project AWS VM.
> >
>
> Hi James,
>
> What is the current status of the monorepo, have you resolved all the
> known issues with the history?  Are there any other changes that need
> to be made before it can be finalized?
>
> -Tom
>
> > Let me know what you think.
> >
> > I had meant to get this prototype finalized 6 months ago, and I must
> apologize for the delay. I hope this is close to final for what we want our
> git repository to look like, and that we can move forward with the
> remainder of the work to convert to git.
> >
> > At this point, there's no guarantee that the repository won't be rebuilt
> from scratch with new hashes, if some problem is discovered which requires
> changing something way back in history. But I hope we're now close to being
> able to declare a conversion final -- and let people start depending on the
> hashes being stable.
> >
> > This conversion uses the "flat monorepo" layout, like the previous
> existing git monorepo, and as discussed previously. The process generating
> it is different, which allows a more faithful conversion, including
> branches. I've also converted a bunch of the auxiliary repositories.
> >
> > I would request that other people help take charge of the remainder of
> the work. Most importantly -- making a plan for implementing the *rest* of
> the migration. We have https://llvm.org/docs/Proposals/GitHubMove.html,
> but I think it'll need significant fleshing out and updating. I'm happy to
> assist with the rest of the migration, but I'd like to _not_ be primarily
> responsible for other parts beyond svn->git repository conversion.
> >
> > Some things that could be discussed in such a plan:
> >   * Verifying that this conversion is good, what we want, and declaring
> it final (at which point the hashes can be relied upon not to change).
> >     * Any particular steps wanted here?
> >   * Converting buildbots to use git.
> >   * Phabricator changes?
> >   * How do email notifications get sent for commits?
> >   * Gathering github accounts for all committers, adding them to a
> github team.
> >   * Deciding upon and announcing a timeline for switching over.
> >   * Proposing, implementing, and testing new workflows for direct git
> usage:
> >     * Github pull requests instead of (or in addition to?) phabricator?
> >     * Github Protected Branch configuration options?
> >       * E.g. -- direct pushing to git without any restriction, or,
> require that pull requests be created first?
> >       * Automated Pre-commit testing? Do we setup CI (e.g. travis-ci.org
> <http://travis-ci.org>) to do some testing on pull requests, to reduce
> avoidable tree breakages?
> >       * Any other github configuration options that need to be decided
> upon?
> >   * ....other things I forgot about at the moment...
> >   * Timeline for switchover.
> >
> >
> >
> > Anyways, what's been done _so far_ is a full SVN->Git repository
> conversion. This conversion:
> >   * Places the SVN revision number into the commit message, as
> "llvm-svn=1234"
> >
> >   * Automatically preserves all branches from the SVN repository (it
> merges the branches named /$project/branches/$name into a single "$name"
> branch, attempting, as much as possible, to make the branch-creation
> commits not look insane).
> >
> >   * Attempts to convert the svn branches in the "tags" subdir into
> annotated git tags pointing to the proper commit on the parent branch,
> where feasible. Sometimes this is impossible, since the "tags" have had
> modifications after their creation. (They're just branches in SVN, so you
> can do that, although you shouldn't). If so, they're preserved as a branch
> named "svntag/$name", instead.
> >
> >   * Preserves the svn id -> email mapping that was in-use at the time of
> each SVN commit, as far as is known.
> >
> >   * Fixes a bunch of -- but not all -- the CVS->SVN conversion errors
> (due, e.g., to files being renamed directly in the CVS repository).
> >
> >
> >
> > Most of the SVN directories are migrated into sub-directories inside the
> main "llvm" mono-repository:
> >   * cfe (renamed to clang in the conversion)
> >   * clang-tools-extra
> >   * compiler-rt
> >   * debuginfo-tests
> >   * dragonegg (also "gcc-plugin", the original name)
> >   * libclc
> >   * libcxx
> >   * libcxxabi
> >   * libunwind
> >   * lld
> >   * lldb
> >   * llgo
> >   * llvm
> >   * openmp
> >   * parallel-libs
> >   * polly
> >   * pstl
> >   * stacker (deleted after r40406)
> > (Additionally, files added to the "monorepo-root/trunk" directory in SVN
> end up at the root of this repository).
> >
> > Some SVN projects are still active, but not part of the LLVM codebase.
> These get migrated to their own separate git repositories:
> >   * lnt
> >   * test-suite
> >   * www
> >   * www-pubs
> >   * www-releases ## TODO. Not done yet as it requires the use of
> git-lfs, due to large files.
> >   * zorg
> >
> > A couple inactive projects which are somewhat related to the LLVM
> codebase, migrated to separate repos:
> >   * poolalloc
> >   * safecode
> >
> > Legacy projects that are not particularly interesting, migrated to a
> single separate git repository named "archive":
> >   * clang-tests # Copy of GCC 4.2 testsuite, modified to work with clang
> >   * clang-tests-external # Copy of GDB testsuite
> >   * llvm-gcc-4.0 # GCC 4.0, modified for llvm
> >   * llvm-gcc-4.2 # GCC 4.2, modified for llvm
> >   * llvm-gcc-4-2 # (merge with above)
> >   * java
> >   * vmkit
> >   * nightly-test-server
> >   * llbrowse # An LLVM bitcode GUI browser
> >   * television # A different LLVM GUI browser; shows effects of
> transforms, etc
> >   * website # 2007-era snapshot of website, not actually maintained here.
> >   * core, llvm-top, sample, support, hlvm # from the "HLVM" refactoring
> attempt.
> >
> > Projects _not_ migrated from SVN in this conversion, since they're
> elsewhere already:
> >   * giri # Never actually developed here; actually
> https://github.com/liuml07/giri
> >   * klee # Already migrated to github with history;
> https://github.com/klee/klee
> >
> >
> >
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20181116/17f5c05e/attachment.html>