[llvm-dev] New LLVM git repository conversion prototype

Fri Oct 12 08:17:53 PDT 2018

Yes, files in the toplevel are sourced from
http://llvm.org/svn/llvm-project/monorepo-root/

(Please don't commit anything in there which has the same name as a project
subdir, it'll break migration!)

On Fri, Oct 12, 2018 at 11:04 AM Zachary Turner <zturner at google.com> wrote:

> I notice there’s a README.md in the top level llvm folder. With the
> current monorepo it seems impossible to put files at that level. It seems
> that will no longer be the case?
>
> I’d like for our top level CMakeLists.txt to eventually be at this level
> as it makes more sense. It sounds like this will eventually be possible?
> On Thu, Oct 11, 2018 at 9:52 PM Renato Golin <renato.golin at linaro.org>
> wrote:
>
>> One thing at a time... :-)
>>
>> James' list is already very large indeed as it is. I'd even be happy with
>> core move before less active / dead projects.
>>
>> On Fri, 12 Oct 2018, 03:27 Zachary Turner via llvm-dev, <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>> It brings a tear to my eye seeing this actually make real progress.
>>>
>>> "Let's not get ahead of ourselves" is the correct answer here, but I do
>>> wonder if we should consider moving our issue tracker to Github as well,
>>> since it integrates nicely with the pull request workflow and you can
>>> reference bugs in commit messages for cross-referencing.
>>>
>>> On Thu, Oct 11, 2018 at 3:28 PM James Y Knight via llvm-dev <
>>> llvm-dev at lists.llvm.org> wrote:
>>>
>>>> TLDR: https://github.com/llvm-git-prototype/ exists as a read-only
>>>> mirror of SVN, and is being updated continuously with a script running on
>>>> an llvm-project AWS VM.
>>>>
>>>> Let me know what you think.
>>>>
>>>> I had meant to get this prototype finalized 6 months ago, and I must
>>>> apologize for the delay. I hope this is close to final for what we want our
>>>> git repository to look like, and that we can move forward with the
>>>> remainder of the work to convert to git.
>>>>
>>>> At this point, there's no guarantee that the repository won't be
>>>> rebuilt from scratch with new hashes, if some problem is discovered which
>>>> requires changing something way back in history. But I hope we're now close
>>>> to being able to declare a conversion final -- and let people start
>>>> depending on the hashes being stable.
>>>>
>>>> This conversion uses the "flat monorepo" layout, like the previous
>>>> existing git monorepo, and as discussed previously. The process generating
>>>> it is different, which allows a more faithful conversion, including
>>>> branches. I've also converted a bunch of the auxiliary repositories.
>>>>
>>>> I would request that other people help take charge of the remainder of
>>>> the work. Most importantly -- making a plan for implementing the *rest* of
>>>> the migration. We have https://llvm.org/docs/Proposals/GitHubMove.html,
>>>> but I think it'll need significant fleshing out and updating. I'm happy to
>>>> assist with the rest of the migration, but I'd like to _not_ be primarily
>>>> responsible for other parts beyond svn->git repository conversion.
>>>>
>>>> Some things that could be discussed in such a plan:
>>>>   * Verifying that this conversion is good, what we want, and declaring
>>>> it final (at which point the hashes can be relied upon not to change).
>>>>     * Any particular steps wanted here?
>>>>   * Converting buildbots to use git.
>>>>   * Phabricator changes?
>>>>   * How do email notifications get sent for commits?
>>>>   * Gathering github accounts for all committers, adding them to a
>>>> github team.
>>>>   * Deciding upon and announcing a timeline for switching over.
>>>>   * Proposing, implementing, and testing new workflows for direct git
>>>> usage:
>>>>     * Github pull requests instead of (or in addition to?) phabricator?
>>>>     * Github Protected Branch configuration options?
>>>>       * E.g. -- direct pushing to git without any restriction, or,
>>>> require that pull requests be created first?
>>>>       * Automated Pre-commit testing? Do we setup CI (e.g.
>>>> travis-ci.org) to do some testing on pull requests, to reduce
>>>> avoidable tree breakages?
>>>>       * Any other github configuration options that need to be decided
>>>> upon?
>>>>   * ....other things I forgot about at the moment...
>>>>   * Timeline for switchover.
>>>>
>>>>
>>>>
>>>> Anyways, what's been done _so far_ is a full SVN->Git repository
>>>> conversion. This conversion:
>>>>   * Places the SVN revision number into the commit message, as
>>>> "llvm-svn=1234"
>>>>
>>>>   * Automatically preserves all branches from the SVN repository (it
>>>> merges the branches named /$project/branches/$name into a single "$name"
>>>> branch, attempting, as much as possible, to make the branch-creation
>>>> commits not look insane).
>>>>
>>>>   * Attempts to convert the svn branches in the "tags" subdir into
>>>> annotated git tags pointing to the proper commit on the parent branch,
>>>> where feasible. Sometimes this is impossible, since the "tags" have had
>>>> modifications after their creation. (They're just branches in SVN, so you
>>>> can do that, although you shouldn't). If so, they're preserved as a branch
>>>> named "svntag/$name", instead.
>>>>
>>>>   * Preserves the svn id -> email mapping that was in-use at the time
>>>> of each SVN commit, as far as is known.
>>>>
>>>>   * Fixes a bunch of -- but not all -- the CVS->SVN conversion errors
>>>> (due, e.g., to files being renamed directly in the CVS repository).
>>>>
>>>>
>>>>
>>>> Most of the SVN directories are migrated into sub-directories inside
>>>> the main "llvm" mono-repository:
>>>>   * cfe (renamed to clang in the conversion)
>>>>   * clang-tools-extra
>>>>   * compiler-rt
>>>>   * debuginfo-tests
>>>>   * dragonegg (also "gcc-plugin", the original name)
>>>>   * libclc
>>>>   * libcxx
>>>>   * libcxxabi
>>>>   * libunwind
>>>>   * lld
>>>>   * lldb
>>>>   * llgo
>>>>   * llvm
>>>>   * openmp
>>>>   * parallel-libs
>>>>   * polly
>>>>   * pstl
>>>>   * stacker (deleted after r40406)
>>>> (Additionally, files added to the "monorepo-root/trunk" directory in
>>>> SVN end up at the root of this repository).
>>>>
>>>> Some SVN projects are still active, but not part of the LLVM codebase.
>>>> These get migrated to their own separate git repositories:
>>>>   * lnt
>>>>   * test-suite
>>>>   * www
>>>>   * www-pubs
>>>>   * www-releases ## TODO. Not done yet as it requires the use of
>>>> git-lfs, due to large files.
>>>>   * zorg
>>>>
>>>> A couple inactive projects which are somewhat related to the LLVM
>>>> codebase, migrated to separate repos:
>>>>   * poolalloc
>>>>   * safecode
>>>>
>>>> Legacy projects that are not particularly interesting, migrated to a
>>>> single separate git repository named "archive":
>>>>   * clang-tests # Copy of GCC 4.2 testsuite, modified to work with clang
>>>>   * clang-tests-external # Copy of GDB testsuite
>>>>   * llvm-gcc-4.0 # GCC 4.0, modified for llvm
>>>>   * llvm-gcc-4.2 # GCC 4.2, modified for llvm
>>>>   * llvm-gcc-4-2 # (merge with above)
>>>>   * java
>>>>   * vmkit
>>>>   * nightly-test-server
>>>>   * llbrowse # An LLVM bitcode GUI browser
>>>>   * television # A different LLVM GUI browser; shows effects of
>>>> transforms, etc
>>>>   * website # 2007-era snapshot of website, not actually maintained
>>>> here.
>>>>   * core, llvm-top, sample, support, hlvm # from the "HLVM" refactoring
>>>> attempt.
>>>>
>>>> Projects _not_ migrated from SVN in this conversion, since they're
>>>> elsewhere already:
>>>>   * giri # Never actually developed here; actually
>>>> https://github.com/liuml07/giri
>>>>   * klee # Already migrated to github with history;
>>>> https://github.com/klee/klee
>>>>
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20181012/dbf16752/attachment.html>