[llvm-dev] New LLVM git repository conversion prototype

Mon Oct 15 13:55:28 PDT 2018

Thanks for setting up the session.

I'll be attending the Dev meeting, and will be at the 4:30 Wednesday round
table.

On Oct 15, 2018 10:58 AM, "Tom Stellard" <tstellar at redhat.com> wrote:

On 10/11/2018 03:27 PM, James Y Knight via llvm-dev wrote:
> TLDR: https://github.com/llvm-git-prototype/ exists as a read-only mirror
of SVN, and is being updated continuously with a script running on an
llvm-project AWS VM.
>
> Let me know what you think.
>
> I had meant to get this prototype finalized 6 months ago, and I must
apologize for the delay. I hope this is close to final for what we want our
git repository to look like, and that we can move forward with the
remainder of the work to convert to git.
>
> At this point, there's no guarantee that the repository won't be rebuilt
from scratch with new hashes, if some problem is discovered which requires
changing something way back in history. But I hope we're now close to being
able to declare a conversion final -- and let people start depending on the
hashes being stable.
>
> This conversion uses the "flat monorepo" layout, like the previous
existing git monorepo, and as discussed previously. The process generating
it is different, which allows a more faithful conversion, including
branches. I've also converted a bunch of the auxiliary repositories.
>
> I would request that other people help take charge of the remainder of
the work. Most importantly -- making a plan for implementing the *rest* of
the migration. We have https://llvm.org/docs/Proposals/GitHubMove.html, but
I think it'll need significant fleshing out and updating. I'm happy to
assist with the rest of the migration, but I'd like to _not_ be primarily
responsible for other parts beyond svn->git repository conversion.
>

Hi James,

Mike Edwards and I were planning to do a round-table during this week's
developer meeting to work on the migration plan.  I have scheduled it
for 4:30PM during the Wednesday session.  It would be great if you or
anyone else who is interested could make it.

The goal of this round-table is to come away with a concrete set of
actions with assigned owners that we can begin implementing.  We
don't want to use up the meeting time with long debates about undecided
issues, so any longer discussions will likely need to be turned into action
items and taken to the mailing list.

Thanks,
Tom

> Some things that could be discussed in such a plan:
>   * Verifying that this conversion is good, what we want, and declaring
it final (at which point the hashes can be relied upon not to change).
>     * Any particular steps wanted here?
>   * Converting buildbots to use git.
>   * Phabricator changes?
>   * How do email notifications get sent for commits?
>   * Gathering github accounts for all committers, adding them to a github
team.
>   * Deciding upon and announcing a timeline for switching over.
>   * Proposing, implementing, and testing new workflows for direct git
usage:
>     * Github pull requests instead of (or in addition to?) phabricator?
>     * Github Protected Branch configuration options?
>       * E.g. -- direct pushing to git without any restriction, or,
require that pull requests be created first?
>       * Automated Pre-commit testing? Do we setup CI (e.g. travis-ci.org <
http://travis-ci.org>) to do some testing on pull requests, to reduce
avoidable tree breakages?

>       * Any other github configuration options that need to be decided
upon?
>   * ....other things I forgot about at the moment...
>   * Timeline for switchover.
>
>
>
> Anyways, what's been done _so far_ is a full SVN->Git repository
conversion. This conversion:
>   * Places the SVN revision number into the commit message, as
"llvm-svn=1234"
>
>   * Automatically preserves all branches from the SVN repository (it
merges the branches named /$project/branches/$name into a single "$name"
branch, attempting, as much as possible, to make the branch-creation
commits not look insane).
>
>   * Attempts to convert the svn branches in the "tags" subdir into
annotated git tags pointing to the proper commit on the parent branch,
where feasible. Sometimes this is impossible, since the "tags" have had
modifications after their creation. (They're just branches in SVN, so you
can do that, although you shouldn't). If so, they're preserved as a branch
named "svntag/$name", instead.
>
>   * Preserves the svn id -> email mapping that was in-use at the time of
each SVN commit, as far as is known.
>
>   * Fixes a bunch of -- but not all -- the CVS->SVN conversion errors
(due, e.g., to files being renamed directly in the CVS repository).
>
>
>
> Most of the SVN directories are migrated into sub-directories inside the
main "llvm" mono-repository:
>   * cfe (renamed to clang in the conversion)
>   * clang-tools-extra
>   * compiler-rt
>   * debuginfo-tests
>   * dragonegg (also "gcc-plugin", the original name)
>   * libclc
>   * libcxx
>   * libcxxabi
>   * libunwind
>   * lld
>   * lldb
>   * llgo
>   * llvm
>   * openmp
>   * parallel-libs
>   * polly
>   * pstl
>   * stacker (deleted after r40406)
> (Additionally, files added to the "monorepo-root/trunk" directory in SVN
end up at the root of this repository).
>
> Some SVN projects are still active, but not part of the LLVM codebase.
These get migrated to their own separate git repositories:
>   * lnt
>   * test-suite
>   * www
>   * www-pubs
>   * www-releases ## TODO. Not done yet as it requires the use of git-lfs,
due to large files.
>   * zorg
>
> A couple inactive projects which are somewhat related to the LLVM
codebase, migrated to separate repos:
>   * poolalloc
>   * safecode
>
> Legacy projects that are not particularly interesting, migrated to a
single separate git repository named "archive":
>   * clang-tests # Copy of GCC 4.2 testsuite, modified to work with clang
>   * clang-tests-external # Copy of GDB testsuite
>   * llvm-gcc-4.0 # GCC 4.0, modified for llvm
>   * llvm-gcc-4.2 # GCC 4.2, modified for llvm
>   * llvm-gcc-4-2 # (merge with above)
>   * java
>   * vmkit
>   * nightly-test-server
>   * llbrowse # An LLVM bitcode GUI browser
>   * television # A different LLVM GUI browser; shows effects of
transforms, etc
>   * website # 2007-era snapshot of website, not actually maintained here.
>   * core, llvm-top, sample, support, hlvm # from the "HLVM" refactoring
attempt.
>
> Projects _not_ migrated from SVN in this conversion, since they're
elsewhere already:
>   * giri # Never actually developed here; actually
https://github.com/liuml07/giri
>   * klee # Already migrated to github with history;
https://github.com/klee/klee
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20181015/2e140662/attachment.html>