[cfe-dev] [llvm-dev] [RFC] LLVM bug lifecycle BoF - triaging

[Tom Stellard via cfe-dev]

On 10/31/2018 06:27 AM, Kristof Beyls wrote:
> 
> 
>> On 26 Oct 2018, at 17:26, Kristof Beyls <Kristof.Beyls at arm.com <mailto:Kristof.Beyls at arm.com>> wrote:
>>
>>
>>
>>> On 26 Oct 2018, at 16:25, Kristof Beyls via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>>>
>>>
>>>
>>>> On 26 Oct 2018, at 04:26, Richard Smith <richard at metafoo.co.uk <mailto:richard at metafoo.co.uk>> wrote:
>>>>
>>>> On Thu, 25 Oct 2018 at 05:10, Kristof Beyls via cfe-dev <cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>> wrote:
>>>>
>>>>>     On 5 Oct 2018, at 07:04, Dean Michael Berris <dean.berris at gmail.com <mailto:dean.berris at gmail.com>> wrote:
>>>>>
>>>>>     Thank you for starting this conversation! I look forward to the results of the BoF discussion summarised as well.
>>>>
>>>>     Dean, all,
>>>>
>>>>     There was a lively discussion at the BoF; we’ve tried to take notes at https://etherpad.net/p/LLVMBugLifeCycleBoF but probably have failed to capture all the points. The slides used to kick start the discussion can be found at https://docs.google.com/presentation/d/1ERB9IQAjSwaNpEnlbchQzd_V9IQlLOojgo9J0_NXvYA/edit
>>>>
>>>>     Both at the BoF and in the mail thread, there have been many suggestions for improvements. So many that if we’d want to introduce all of them at once, we’d probably get stuck and not introduce any. To try and make progress on the ones I myself feel are most useful, I’ve volunteered for 2 actions:
>>>>
>>>>     1. Write up a proposal for documentation on what to do during bug triaging/closing/etc. I’ve just done so and put it up for review at https://reviews.llvm.org/D53691.
>>>>     2. Write an email to the mailing lists to ask for volunteers for being on the “default-cc” list for components, implying you’re willing to triage bugs reported against those components. I’ve decided to first try and get consensus on what is expected when triaging a bug (see point above) before actively searching for volunteers for all components. That being said, both at the dev meeting and in the days after, I already received many requests from people to be added to the default-cc list for specific components. Of course, I’m very happy to add people volunteering to default-cc lists, so if you don’t want to wait to get added to a default-cc list, please email bugs-admin at lists.llvm.org <mailto:bugs-admin at lists.llvm.org> or raise it as a ticket in bugs.llvm.org <http://bugs.llvm.org/> under “Bugzilla Admin”/“Products”.
>>>>
>>>>     Furthermore, since the BoF, I’ve seen a quite a few requests to clean up and introduce new components in Bugzilla. We’ve implemented the changes quickly and will aim to continue to have a quick response time in the future. Please file a ticket in bugs.llvm.org <http://bugs.llvm.org/> under “Bugzilla Admin”/“Products” if you want to request a specific change.
>>>>
>>>>     For most of the other points that were raised: I don’t currently plan on acting on them immediately myself and hope to first see an impact of the above actions.
>>>>
>>>>
>>>> In the original post, there was a suggestion to bring back the "UNCONFIRMED" status. I think that'd be a great idea, as it both makes it easy to search for untriaged bugs and to give feedback to a reporter that their bug is real and acknowledged. Is that planned?
>>>
>>> I hadn’t seen too much feedback on the idea for introducing (reintroducing?) the “UNCONFIRMED” status, so hadn’t planned on making changes to that now.
>>> However, I also think it’s a good idea, so I’ll investigate in more detail now if it wouldn’t be overly hard on the Bugzilla admin side to do so (e.g. I believe I’ll have to give all existing users the rights to be able to confirm bugs). If the “UNCONFIRMED” status can be introduced relatively easily, I now plan to do so, and will adapt the proposed documentation at https://reviews.llvm.org/D53691 accordingly.
>>> Thanks for pointing to this!
>>
>> Just a status update on investigations to enable the UNCONFIRMED/CONFIRMED states: it seems that bugzilla by-default will put bugs in the “CONFIRMED” state when creating new bugs, if the reporter has “can-confirm” rights. I believe our de facto policy is for everyone with an account to be able to confirm bugs. Also, you need an account to be able to report a bug. The result is that all new bugs by default will go to status “CONFIRMED”, unless the reporter carefully remembers to change the default and select “UNCONFIRMED”. (Also see https://bugzilla.mozilla.org/show_bug.cgi?id=915682 which suggests this behaviour is not configurable).
>>
>> Unless that can be solved, I fear that many bugs will be reported with the default “CONFIRMED” status even though they aren’t triaged yet. I believe that is worse than the current default “NEW” state.
>>
>> The only work around for this behaviour where we still get a “to be triaged” state I can think of at the moment is to introduce a custom “CONFIRMED” status, not using bugzilla’s built-in special “UNCONFIRMED” support and have a flow that would allow something like:
>>
>> NEW -> CONFIRMED -> ASSIGNED -> RESOLVED -> …
>>
>> The advantage is we’d have separate states to represent “this bug was raised” (NEW) vs “this bug was triaged & confirmed” (CONFIRMED).
>> The disadvantage is that we’ll start deviating from the default bugzilla workflows.
>>
>> Overall, I think it’s still a win to implement the above idea. I’ll look into that next week and am all ears for better solutions.
>>
>> Thanks,
>>
>> Kristof
> 
> After some further discussion on https://reviews.llvm.org/D53691 and after having looked into bugzilla’s abilities to adapt workflow/statuses, I think we should simplify the workflow to the following:
> 
> Start status will remain “NEW”.
> After triaging happened, an issue moves to status “CONFIRMED”. This is a new status becoming available.
> When an issue is finalised, it moves to status “RESOLVED”.
> Status “REOPENED” remains available to reflect that a resolved issue got reopened.
> 
> When an issue is actively being worked on, that is indicated by the “Assigned to” field pointing to a real person rather than the “unassigned” pseudo account. 
> 
> That means I propose to drop the following currently available statuses:
> 
>   * VERIFIED, CLOSED: we didn’t make much/any distinction between “final” statuses RESOLVED, VERIFIED and CLOSED. Therefore, it just simplifies things to only have 1 final state. Bugzilla treats the RESOLVED status special and requires it to always be present. That makes it easy to decide that RESOLVED is the “final” state to keep.
>   * ASSIGNED: bugzilla requires the “Assigned to” field to always be filled in. As a result, we have an “unassigned” pseudo account to indicate that an issue is not assigned. Conversely, when the “Assigned to” field contains someone else than this “unassigned” pseudo account, it means the issue is assigned to that specific person. A result of that is that whether an issue is assigned or not is already fully represented in the “Assigned to” field. The ASSIGNED status doesn’t add value on top of this, so let’s just remove it.
>     The 64 bugs (see https://bugs.llvm.org/buglist.cgi?bug_status=ASSIGNED&email1=unassigned&emailassigned_to1=1&emailtype1=substring&list_id=148699&query_format=advanced&resolution=---) currently in status ASSIGNED with “Assigned to” pointing to the “unassigned” pseudo account indicates that it has negative value to be able to indicate “assignedness” in two different ways. 
> 
> In an attempt to make forward progress: I intend to make the above changes & commit the documentation for bug life cycle at https://reviews.llvm.org/D53691 next week, unless I hear major opposition by then.
> 

This all sounds good to me.  I really like having a more simple process.
Thanks for working on this.

-Tom

> Thanks,
> 
> Kristof
> 
>