[LLVMdev] Mirroring of LLVM repository

[Andreas Bolka]

On Mon Aug 03 21:09:59 +0200 2009, Anton Korobeynikov wrote:
> Currently the load of llvm.org is too high. This influences the whole
> project services like buildbots, bugzilla, etc. It was found that this
> workload is possible caused by massive mirroring of LLVM SVN
> repository into git/hg/whatever. Please don't do that :)

Let me add to what Anton said:

I can only speak for git-svn, but the main burden it imposes on LLVM's
SVN repository are attempts of mirroring the _complete_ history.
Unfortunatly, a naive `git svn clone` will do just that -- so _please_
avoid this at all cost.

If you want to use git-svn to work with LLVM's SVN repository, please
only clone the latest SVN revision using git-svn's `--revision <n>`
option (combined with a huge log window: `--log-window-size 999999`).
You may find the following script helpful:

    #!/bin/bash
    if [ $# -ne 2 ]; then echo "Usage: $0 <svn_url> <dir>" >&2; exit 64; fi
    SVN_URL=$1
    SVN_REV=`svn info $SVN_URL | awk '/Rev:/ {print $4}'`
    git svn clone --log-window-size 999999 --revision $SVN_REV $SVN_URL $2

This makes the initial clone roughly as demanding as a `svn checkout`.

> [..] There are some git mirrors already [..]

Next to the mirrors mentioned (and kept up-to-date) by Anton, I've been
maintaining a mirror of LLVM's trunk (only llvm, no llvm-gcc or clang)
on Github for the last few months:

- http://github.com/earl/llvm-mirror/

This mirror is automatically updated once an hour and includes the
complete history of LLVM, back to the initial revision [1].

As Anton mentioned, you can use those public Git mirrors to bootstrap
your own git-svn repository. Here's how to do this using my Github
mirror:

    git clone git://github.com/earl/llvm-mirror.git llvm
    cd llvm
    git config --add remote.origin.fetch '+refs/remotes/*:refs/remotes/*'
    git fetch
    git svn init https://llvm.org/svn/llvm-project/llvm/trunk
    git svn rebase --local

Once that's done, you can work with git-svn just as you're used to.

Alternatively, you can keep up-to-date purely via the Git mirror by
using `git fetch; git svn rebase --local`. With that, the only time you
need to hit SVN is for committing. This is most effective with a recent
Git (1.6.1+) where you can freely intermix fetching via `git fetch` from
a Git mirror with `git svn fetch` directly from SVN (git-svn will
incrementally update it's revision map).


[1]
http://github.com/earl/llvm-mirror/tree/455694886d7a341a76164cc67d3fcc2741c73c52

-- 
Regards,
Andreas