[cfe-dev] RFC: Bugzilla migration plan

[Anton Korobeynikov via cfe-dev]

Thanks, Mehdi

We are eagerly waiting for GitHub for responses on several important
questions / issues.

On Thu, Jun 24, 2021 at 9:01 AM Mehdi AMINI <joker.eph at gmail.com> wrote:
>
> Hi,
>
> I don't where we're at with the migration from Bugzilla to Github issues, but there are improvements from GitHub in this area these days: https://github.blog/changelog/2021-06-23-issues-forms-beta-for-public-repositories/
>
> This seems interesting!
>
> --
> Mehdi
>
>
> On Fri, Jul 10, 2020 at 1:11 AM Anton Korobeynikov via cfe-dev <cfe-dev at lists.llvm.org> wrote:
>>
>> Dear all,
>>
>> Over the last few weeks with the help of GH folks I've been exploring
>> the options of Bugzilla migration. I believe finally we came to the
>> viable solution which is detailed below.
>>
>> It turned out that GitHub has an internal project rehydration tool
>> that could be used to populate the empty repo contents from the simple
>> serialized format. There is a big advantage of this approach as
>> compared to using GH API as we are not bound to various thresholds and
>> throttling limits (remember, that we need to import 35k+ bz issues).
>> The downside is that such rehydration requires the empty repo and we
>> cannot delete the current llvm-project: this way we will lose
>> releases, fork connections, stars and watches. Unfortunately, there is
>> no way to recreate releases while keeping the origins dates, so this
>> is a no-go for us. Losing forks connections would strongly affect
>> downstream users as well. This allowed to formulate the following
>> scheme:
>>
>> 1. Migrate Bugzilla to a new repo, say, llvm-bugzilla-import using the
>> internal storage format.
>> 2. Install redirects llvm.org/PR1234 => gh/llvm/llvm-bugzilla-import/issues/1234
>> 3. Wipe existing issues and pull requests
>> 4. Migrate all issues from llvm-bugzilla-import to llvm-project using
>> GH API. Github will take about llvm-bugzilla-import/issues/1234 =>
>> llvm-project/issues/5678 redirects
>>
>> The only downside of this approach is that we will be seeing 30k
>> events like "llvm-bugzilla-import/issues/1234 migrated to
>> llvm-project/issues/5678".
>>
>> Here is the tentative timeline / list of action points:
>>
>> 1. Collect the mapping email (used by bugzilla) => GH account name
>> (used by issues). We are going to collect using different sources:
>>   - Auto-populating the mapping from the list of known committers
>>   - Asking GH API (works only if a person made their email public and
>> only when allowed by local law)
>>   - Emailing everyone who submitted to Bugzilla over last year or
>> maybe two asking to fill in the form with the GH username
>>   - We would likely allow a month or so to let everyone respond.
>> 2. While 1. is in progress, we will work on various format issues for
>> migration. For this we will use probable first 1k issues or so. It
>> would be nice to include some meta-bugs here to ensure we could
>> re-recreate issues. Things to consider:
>>   - Comment migration (GH uses markdown everywhere, so we'd need to
>> carefully escape bugzilla contents)
>>   - Components => labels mapping and migration
>>   - Linking between the issues. Maybe automatically replace PR1234 in
>> the text with #1234 to enable auto linking.
>>   - Authorship: reporter / commenter
>>   - Attaches
>> 3. After we are sure everyone is ready, we will do the test migration
>> of the whole bugzilla.
>>   - Estimate the necessary time it would be required to make such a transition.
>>   - Fix remaining issues, if any
>> 4. Put bugzilla into read-only mode and perform the final migration to
>> llvm-bugzilla-archive
>> 5. Wipe issues / PRs in llvm-project repo and perform migration from
>> llvm-bugzilla-archive to llvm-project
>> 6. Migration done. Probably bugzilla will be kept in read-only mode
>> for some time just for the sake of consistency and should any issues
>> be found.
>>
>> Any comments & ideas?
>> --
>> With best regards, Anton Korobeynikov
>> Department of Statistical Modelling, Saint Petersburg State University
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev



-- 
With best regards, Anton Korobeynikov
Department of Statistical Modelling, Saint Petersburg State University