[cfe-dev] RFC: Bugzilla migration plan

[Tom Stellard via cfe-dev]

On 7/10/20 1:12 PM, David Blaikie via cfe-dev wrote:
> If I recall correctly, the previous discussion had a fair bit of
> pushback on having two numbering systems. In part because the old bug
> numbers are littered throughout the codebase, commit messages, etc,
> and aren't always prefixed with "PR", sometimes just "bug XXXX" -
> having two numberings is likely to mean some amount of
> confusiong/friction/ongoing cost to looking in multiple places for
> bugs.
> 

This was my take away from the discussions too.  I think it's important
that we try to preserve the single numbering system.

-Tom

> (but I'm not personally holding this process up on that issue - I
> think I can live with it, if that's what those with the
> time/inclination to make this migration happen decide is the right
> tradeoff)
> 
> On Fri, Jul 10, 2020 at 1:11 AM Anton Korobeynikov via cfe-dev
> <cfe-dev at lists.llvm.org> wrote:
>>
>> Dear all,
>>
>> Over the last few weeks with the help of GH folks I've been exploring
>> the options of Bugzilla migration. I believe finally we came to the
>> viable solution which is detailed below.
>>
>> It turned out that GitHub has an internal project rehydration tool
>> that could be used to populate the empty repo contents from the simple
>> serialized format. There is a big advantage of this approach as
>> compared to using GH API as we are not bound to various thresholds and
>> throttling limits (remember, that we need to import 35k+ bz issues).
>> The downside is that such rehydration requires the empty repo and we
>> cannot delete the current llvm-project: this way we will lose
>> releases, fork connections, stars and watches. Unfortunately, there is
>> no way to recreate releases while keeping the origins dates, so this
>> is a no-go for us. Losing forks connections would strongly affect
>> downstream users as well. This allowed to formulate the following
>> scheme:
>>
>> 1. Migrate Bugzilla to a new repo, say, llvm-bugzilla-import using the
>> internal storage format.
>> 2. Install redirects llvm.org/PR1234 => gh/llvm/llvm-bugzilla-import/issues/1234
>> 3. Wipe existing issues and pull requests
>> 4. Migrate all issues from llvm-bugzilla-import to llvm-project using
>> GH API. Github will take about llvm-bugzilla-import/issues/1234 =>
>> llvm-project/issues/5678 redirects
>>
>> The only downside of this approach is that we will be seeing 30k
>> events like "llvm-bugzilla-import/issues/1234 migrated to
>> llvm-project/issues/5678".
>>
>> Here is the tentative timeline / list of action points:
>>
>> 1. Collect the mapping email (used by bugzilla) => GH account name
>> (used by issues). We are going to collect using different sources:
>>    - Auto-populating the mapping from the list of known committers
>>    - Asking GH API (works only if a person made their email public and
>> only when allowed by local law)
>>    - Emailing everyone who submitted to Bugzilla over last year or
>> maybe two asking to fill in the form with the GH username
>>    - We would likely allow a month or so to let everyone respond.
>> 2. While 1. is in progress, we will work on various format issues for
>> migration. For this we will use probable first 1k issues or so. It
>> would be nice to include some meta-bugs here to ensure we could
>> re-recreate issues. Things to consider:
>>    - Comment migration (GH uses markdown everywhere, so we'd need to
>> carefully escape bugzilla contents)
>>    - Components => labels mapping and migration
>>    - Linking between the issues. Maybe automatically replace PR1234 in
>> the text with #1234 to enable auto linking.
>>    - Authorship: reporter / commenter
>>    - Attaches
>> 3. After we are sure everyone is ready, we will do the test migration
>> of the whole bugzilla.
>>    - Estimate the necessary time it would be required to make such a transition.
>>    - Fix remaining issues, if any
>> 4. Put bugzilla into read-only mode and perform the final migration to
>> llvm-bugzilla-archive
>> 5. Wipe issues / PRs in llvm-project repo and perform migration from
>> llvm-bugzilla-archive to llvm-project
>> 6. Migration done. Probably bugzilla will be kept in read-only mode
>> for some time just for the sake of consistency and should any issues
>> be found.
>>
>> Any comments & ideas?
>> --
>> With best regards, Anton Korobeynikov
>> Department of Statistical Modelling, Saint Petersburg State University
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>