<div dir="ltr"><div dir="ltr">On Fri, Dec 10, 2021 at 7:33 AM Anton Korobeynikov <<a href="mailto:anton@korobeynikov.info">anton@korobeynikov.info</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Thanks for the try!<br>
<br>
>From the quick scan:<br>
<br>
1. There are no labels<br></blockquote><div><br></div><div>There are labels, but only according to the "keywords" field from Bugzilla.</div><div><a href="https://github.com/Quuxplusone/LLVMBugzillaTest/issues?q=is%3Aopen+is%3Aissue+label%3Aaccepts-invalid">https://github.com/Quuxplusone/LLVMBugzillaTest/issues?q=is%3Aopen+is%3Aissue+label%3Aaccepts-invalid</a></div><div>I agree it would make sense to apply more labels in <a href="https://github.com/Quuxplusone/BugzillaToGithub#step-3-process-each-xml-bug-into-githubs-json-schema">Step 3</a> (e.g. according to the "Product" field).</div><div>If you document the mapping somewhere, it would be trivial to add to my script and I could have 10,000 issues regenerated in about 3 hours.</div><div>Also needed: the mapping from Bugzilla usernames to GitHub usernames.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
2. Attachments are not real – they are just links to bugzilla and will<br>
be obsolete if bugzilla is e.g. down<br></blockquote><div><br></div><div>Right. This is part of the "dumbed-down Load step", i.e. "take the actual data and munge it into the closest possible thing that can be loaded using the public API": GitHub's beta Issues Import API doesn't support adding files to issues. (Also, e.g.,</div><div>- forging authorship of comments is impossible using the public API</div><div>- for cross-referencing to other issues, I'm currently using links back into the old Bugzilla's show_bug.cgi; but really these links should go to something like <a href="https://reviews.llvm.org/PR1234">https://reviews.llvm.org/PR1234</a>, which would be under our control and could be HTTP-redirected to their corresponding GitHub issues</div><div>)</div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
3. Each attachment results in 2 comments, one of each is redundant<br></blockquote><div><br></div><div>Ack. I wrote code to fix this for the very simplest "Created attachment 1234" auto-comments, but had not noticed that sometimes the auto-comment is more complicated.</div><div>E.g. <a href="https://github.com/Quuxplusone/LLVMBugzillaTest/issues/10729#issuecomment-990590574">https://github.com/Quuxplusone/LLVMBugzillaTest/issues/10729#issuecomment-990590574</a></div><div>This wouldn't be hard to fix.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
4. CC list is strange, e.g.<br>
<a href="https://github.com/Quuxplusone/LLVMBugzillaTest/issues/12187" rel="noreferrer" target="_blank">https://github.com/Quuxplusone/LLVMBugzillaTest/issues/12187</a> CC's to<br>
"<a href="http://mail.sandbox.de" rel="noreferrer" target="_blank">mail.sandbox.de</a>"<br></blockquote><div><br></div><div>That's partly an artifact of my lack of mapping from Bugzilla usernames to GitHub usernames (<a href="https://github.com/Quuxplusone/BugzillaToGithub/blob/822dbac/xml-to-json.py#L186-L195">the relevant codepath</a> is just a stub), but also something super weird...!</div><div>The email addresses from Bugzilla show up in the XML when viewed in Chrome, but not when fetched in Python or curl.</div><div><a href="https://stackoverflow.com/questions/70307092/fetching-xml-from-bugzilla-gives-different-results-with-curl-versus-browser">https://stackoverflow.com/questions/70307092/fetching-xml-from-bugzilla-gives-different-results-with-curl-versus-browser</a><br></div><div><br></div></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
5. All text is in verbatim boxes (e.g.<br>
<a href="https://github.com/Quuxplusone/LLVMBugzillaTest/issues/12092" rel="noreferrer" target="_blank">https://github.com/Quuxplusone/LLVMBugzillaTest/issues/12092</a>) making<br>
it almost impossible to read due to horizontal scroll<br></blockquote><div><br></div><div>The monospace font is intentional on my part, and important even for <a href="https://bugs.llvm.org/show_bug.cgi?id=12092">https://bugs.llvm.org/show_bug.cgi?id=12092</a> because a big part of the initial comment is indented C++ code. However, I should implement linebreaking: looks like Bugzilla's website layout breaks around 84 characters, and 80 would be perfectly sensible.<br></div><div>Will fix.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
6. There are no "depends on" / "blocks on" references (see<br>
<a href="https://github.com/Quuxplusone/LLVMBugzillaTest/issues/10900" rel="noreferrer" target="_blank">https://github.com/Quuxplusone/LLVMBugzillaTest/issues/10900</a>)<br></blockquote><div><br></div><div>Ack.</div><div>(This is an artifact of my not knowing that the <dependson> element exists. I should have thought to grep and get a list of all the tags that exist in the XML (that is, in the 51567 "xml/*.xml" files produced during <a href="https://github.com/Quuxplusone/BugzillaToGithub#step-1-export-your-bugzilla-bugs-to-xml">Step 1 in the README</a>), to make sure I understood each of them.)</div><div>Will fix, at least for the <dependson> tag.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
7. There are no cross-references in case of duplicates (see<br>
<a href="https://github.com/Quuxplusone/LLVMBugzillaTest/issues/10729" rel="noreferrer" target="_blank">https://github.com/Quuxplusone/LLVMBugzillaTest/issues/10729</a>)<br></blockquote><div><br></div><div>Ack.</div><div>I thought about mangling the duplicate-bug-number into the "Status" line, like Bugzilla does, but decided not to worry about it in the interest of being-done-by-my-self-imposed-EOW-deadline. :)</div><div>There's also a harder issue on <a href="https://github.com/Quuxplusone/LLVMBugzillaTest/issues/10729#issuecomment-990590581">bug 10729's final comment</a>, where it says</div><div><pre class="gmail-bz_comment_text" id="gmail-comment_text_4" style="white-space:pre-wrap;width:50em;color:rgb(0,0,0)"> Yes, apparently I did. Sorry. I'll attach the logs to that issue instead :)
*** This bug has been marked as a duplicate of <a class="gmail-bz_bug_link gmail-bz_status_RESOLVED gmail-bz_closed" title="RESOLVED FIXED - [META][Win64] build and test issues on MinGW-w64" href="https://bugs.llvm.org/show_bug.cgi?id=9072" style="color:rgb(0,51,153);text-decoration-line:line-through">bug 9072</a> ***</pre></div><div>where we want that to be both monospaced <i>and</i> hyperlinked — Markdown can't do hyperlinks inside triple-backticks.</div><div>The obvious solution is for the script to special-case Bugzilla's auto-comment and pull it outside of the triple-backticked section.</div><div>I should grep for all the different Bugzilla auto-comments too. It looks like there are only three possible auto-comments:</div><div><br></div><div>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">$ grep -hor '[*][*][*] .* [*][*][*]' xml/ > out</span></p><p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">$ sed 's/[0-9][0-9]*/9/g' out | sort | uniq -c | sort -rn | eyeballing-by-arthur</span></p><p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">2563 *** Bug 9 has been marked as a duplicate of this bug. ***</span><br></p></div><div>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">2504 *** This bug has been marked as a duplicate of bug 9 ***</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span class="gmail-Apple-converted-space" style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures">76 *** This bug has been marked as a duplicate of 9 ***</span><br></p></div><div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
...<br>
<br>
It's pretty straightforward to come to the present state and there are<br>
tools for this, we've been at this point in 2019 (see e.g.<br>
<a href="https://github.com/asl/llvm-bugzilla/issues" rel="noreferrer" target="_blank">https://github.com/asl/llvm-bugzilla/issues</a> as it was outlined in LLVM<br>
DevMtg 2019 roundtable discussion). The non-trivial part is to<br>
workaround various GitHub issues which are also different depending on<br>
API used.<br></blockquote><div><br></div><div>Nice! Yeah, <a href="https://github.com/Quuxplusone/BugzillaToGithub#step-1-export-your-bugzilla-bugs-to-xml">steps 1, 2, 3</a> (Export and Transform) are possible for literally anyone to do — and also relatively <i>simple</i>, in that I wrote those scripts in a single week of evenings. :) <a href="https://github.com/Quuxplusone/BugzillaToGithub#step-4-import-your-json-bugs-into-github">Step 4</a>, the Load step, is equally <i>simple</i> but requires special magic powers that only a GitHub SRE would have — e.g., forging comment authorship. If I were doing this migration for real, I'd ask what API they plan to use, and ask them to test it out on a blank repo in exactly the same way that you and I have now both done with </div><div><a href="https://github.com/asl/llvm-bugzilla/issues" rel="noreferrer" target="_blank">https://github.com/asl/llvm-bugzilla/issues</a></div><div>and</div><div><a href="https://github.com/Quuxplusone/LLVMBugzillaTest/issues" rel="noreferrer" target="_blank">https://github.com/Quuxplusone/LLVMBugzillaTest/issues</a><br></div><div>That is, write the script that's going to be used, and then test it out, repeatedly, until it works perfectly... and then test once more, just for safety's sake, before doing it live.</div><div>The mantras here are</div><div>- "With enough eyeballs, all bugs are shallow" (we're both identified deficiencies in each other's scripts, and can now fix them!)</div><div>- "Measure twice, cut once" (rehearse the entire deploy plan in blank repos until it's perfect, then do <i>only the perfect version</i> live)</div><div><br></div><div>(Also, ideally, someone involved with LLVM would just get hired at GitHub, to cut down on round-trip time. But I'm not volunteering. ;))</div><div><br></div><div>–Arthur</div></div></div>