[cfe-dev] RFC: Bugzilla migration plan

[David Blaikie via cfe-dev]

If I recall correctly, the previous discussion had a fair bit of
pushback on having two numbering systems. In part because the old bug
numbers are littered throughout the codebase, commit messages, etc,
and aren't always prefixed with "PR", sometimes just "bug XXXX" -
having two numberings is likely to mean some amount of
confusiong/friction/ongoing cost to looking in multiple places for
bugs.

(but I'm not personally holding this process up on that issue - I
think I can live with it, if that's what those with the
time/inclination to make this migration happen decide is the right
tradeoff)

On Fri, Jul 10, 2020 at 1:11 AM Anton Korobeynikov via cfe-dev
<cfe-dev at lists.llvm.org> wrote:
>
> Dear all,
>
> Over the last few weeks with the help of GH folks I've been exploring
> the options of Bugzilla migration. I believe finally we came to the
> viable solution which is detailed below.
>
> It turned out that GitHub has an internal project rehydration tool
> that could be used to populate the empty repo contents from the simple
> serialized format. There is a big advantage of this approach as
> compared to using GH API as we are not bound to various thresholds and
> throttling limits (remember, that we need to import 35k+ bz issues).
> The downside is that such rehydration requires the empty repo and we
> cannot delete the current llvm-project: this way we will lose
> releases, fork connections, stars and watches. Unfortunately, there is
> no way to recreate releases while keeping the origins dates, so this
> is a no-go for us. Losing forks connections would strongly affect
> downstream users as well. This allowed to formulate the following
> scheme:
>
> 1. Migrate Bugzilla to a new repo, say, llvm-bugzilla-import using the
> internal storage format.
> 2. Install redirects llvm.org/PR1234 => gh/llvm/llvm-bugzilla-import/issues/1234
> 3. Wipe existing issues and pull requests
> 4. Migrate all issues from llvm-bugzilla-import to llvm-project using
> GH API. Github will take about llvm-bugzilla-import/issues/1234 =>
> llvm-project/issues/5678 redirects
>
> The only downside of this approach is that we will be seeing 30k
> events like "llvm-bugzilla-import/issues/1234 migrated to
> llvm-project/issues/5678".
>
> Here is the tentative timeline / list of action points:
>
> 1. Collect the mapping email (used by bugzilla) => GH account name
> (used by issues). We are going to collect using different sources:
>   - Auto-populating the mapping from the list of known committers
>   - Asking GH API (works only if a person made their email public and
> only when allowed by local law)
>   - Emailing everyone who submitted to Bugzilla over last year or
> maybe two asking to fill in the form with the GH username
>   - We would likely allow a month or so to let everyone respond.
> 2. While 1. is in progress, we will work on various format issues for
> migration. For this we will use probable first 1k issues or so. It
> would be nice to include some meta-bugs here to ensure we could
> re-recreate issues. Things to consider:
>   - Comment migration (GH uses markdown everywhere, so we'd need to
> carefully escape bugzilla contents)
>   - Components => labels mapping and migration
>   - Linking between the issues. Maybe automatically replace PR1234 in
> the text with #1234 to enable auto linking.
>   - Authorship: reporter / commenter
>   - Attaches
> 3. After we are sure everyone is ready, we will do the test migration
> of the whole bugzilla.
>   - Estimate the necessary time it would be required to make such a transition.
>   - Fix remaining issues, if any
> 4. Put bugzilla into read-only mode and perform the final migration to
> llvm-bugzilla-archive
> 5. Wipe issues / PRs in llvm-project repo and perform migration from
> llvm-bugzilla-archive to llvm-project
> 6. Migration done. Probably bugzilla will be kept in read-only mode
> for some time just for the sake of consistency and should any issues
> be found.
>
> Any comments & ideas?
> --
> With best regards, Anton Korobeynikov
> Department of Statistical Modelling, Saint Petersburg State University
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev