[llvm-dev] [PITCH] Improvements to LLVM Decision Making

Thu Jan 16 23:56:31 PST 2020

On Wed, Jan 15, 2020 at 2:04 AM Doerfert, Johannes via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> Hi Chris,
>
> I experience a fair share of problems when it comes to code reviews and
> change requests so I appreciate any improvement we can make here.
>
>
> On 01/14, Chris Lattner via llvm-dev wrote:
> > Numerous people have been bringing up challenges with consensus driven
> > decision making in the LLVM community.  After considering this and
> > seeing the frustrations it is causing many people, I think we should
> > make a formal process change to help improve decision making going
> > forward.
>
> Improving is always a good idea.
>
>
> > Here is the outline of the draft proposal
> > <https://gist.github.com/lattner/e3679998a7609c99b1243f09d30f0132>.
> > Caveats: this is itself just a draft proposal to inspire discussion
> > not a mandate - I welcome feedback, thoughts and concerns, ideas of
> > how to make this better, or better ideas/approaches overall on how we
> > can level up the community.
> >
> > Please let me know what you think, thanks!
>
> First thought: This should not be a gist on github. Maybe it should be
> on phabricator or at least in an email so it is easier to respond to it.
>
>
> I'll try to inline the gist below so I can respond to the points.
>
> ---
>
> > "It isn't clear how to propose some changes in the first place, and it
> > is often unclear who the decision makers are."
>
> I feel that patches and RFCs are well established *and documented* [1,2]
> ways to propose changes. In addition, the *-dev lists, IRC, etc. do
> provide ways to clear uncertainties. Adding more documentation on this
> can obviously be helpful. Also, we are already improving the existing
> documentation [0].
>
> That said, we need to differentiate in more detail what the problems
> actually are. Do people not find the documentation? Is the documentation
> unclear? Are people waiting for "a decision maker" to step in or afraid
> to ask questions? Are questions not answered? ...
>
>
> > "It isn't clear what mailing lists to send such proposals to: llvm-dev
> > gets a lot of traffic and many people don’t read it."
>
> I argue that people that do participate in the decision making (wording
> took from the previous point) already do, or at least should, follow
> llvm-dev.
> I am not quite sure why it is unclear what mailing list is the right one
> given that we have one *-dev mailing list per subproject and on llvm-dev,
> often the first points-of-contact, people are referred to the right one.
>
>
> > "It is hard to know whether something is a serious proposal that you
> > must take seriously, and so it is easy to miss important changes
> > because you don't have time to review everything. Even though you
> > technically had a chance to participate, you can end up surprised when
> > some change goes into effect."
>
> I'm unsure what kinds of proposal are supposed to be "not serious" and
> who is supposed to be making that decision.
>
>
> > "Sometimes people chime in late with dissent after a decision has been
> > apparently made: this can be frustrating to people who need a decision
> > made, because they aren't sure how to proceed."
>
> It is unclear to me how the proposal helps on this front. Could you
> elaborate?
>
>
> > "Sometimes people express a loud voice on discussion threads even if
> > they aren't active contributors, and they can derail discussions.
> > There is no "moderator" for these discussions."
>
> With the caveat of finding the moderator (as mentioned below), this
> makes sense. We probably/hopefully don't need a moderator (for this
> reason) on many discussions but it might certainly help if people step
> up if a discussion is derailed (for any reason and by anyone).
>
>
> > "The initial discussion phase of a proposal can have lots of back and
> > forth discussions to shape a idea, and the eventual proposal can have
> > significant changes from that initial review. It can be difficult to
> > discern what feedback from the initial review is addressed or ignored
> > in the final rounds of discussions."
>
> Yes. I am not sure how the proposed solution remedies the problem
> though. Could you elaborate?
>
>
> > "Complex changes (e.g. the relicensing project) sometimes take many
> > rounds of iteration, and it can be easy to lose track of where the
> > proposal is and what the history was."
>
> This is certainly true. The proposed solution (with moderators and
> rounds) is probably implementable and reasonable for "complex changes".
>
>
> > "Occasionally, Code Owners may be faced with a technical decision and
> > not be sure what to do, particularly for highly impactful design
> > decisions - e.g. for core LLVM IR changes. It could be useful to have
> > a formal escalation process for these decisions."
>
> TBH, I always thought "code owner" is not a "decision making" but rather
> an "administrative" title. The person can be part of the decision
> making, but the role of "code owner" does not imply special privileges,
> only tasks, e.g., making sure reviews are done by the right people.
>
>
> > I recommend that we add a process similar to (but adapted and changed
> > where it makes sense) the Swift Evolution process. This process was
> > designed to help guide decision making for high impact language and
> > standard library changes to the Swift language. It was inspired by
> > similar processes in other language communities (e.g. the Rust
> > Language RFC process, the Python PEP process, etc) and has worked well
> > in that context - it stands to reason that a variant should work well
> > for LLVM as well.
>
> I tried to determine how Rust RFCs (and Python PEPs) influenced the
> proposed Swift Evolution process but from the history of the linked
> pages (in the gist) that was not clear. Could you elaborate on that?
>
>
> > The solution entails several parts. First, the process should be
> > written down!
>
> Agreed!
>
>
> > This written process should include things like:
> >
> >     An informal "pitch" phase to help collect requirements and shape
> >     the ultimate form of an idea, but which can be ignored by people
> >     who aren't interested in following all of the details of a
> >     discussion.
>
> How is this different form the discussion that happens after an initial
> RFC is send?
>
> People already ignore it if they are not interested in the details.  If
> people chime in late, as mentioned in the problems above, this will not
> help, right? I mean, if the pitch phase is done and then people start to
> chime it starts again.  This can have any reason, they are late, they
> want to see if it is really "a serious proposal", or they just want to
> wait until the first round of discussion changed the proposal to start
> the second round.
>
>
> >     A new mailing list (or Discourse channel) dedicated to formal
> >     proposal review. This would be relatively low volume, allowing
> >     effectively everyone to follow it. This could be named something
> >     like "llvm-proposals".
>
> 1) We have already 30+* Discourse channels. Having so many, and one
>    more, makes it harder to actually monitor them. Additionally, people
>    that do not have Discourse are already excluded from the discussion.
>    (I feel this had/has the opposite effect it was supposed to have.)
>

Are you sure that you are not confusing Discourse
<https://llvm.discourse.group> with Discord here?

Discord is an IRC replacement, Discourse is more of a candidate to replace
the mailing-list (there is a mailing list mode where every post end up in
your mailbox: it should not be a regression over the mailing-list).

-- 
Mehdi

> 2) Arguably you could filter *-dev lists for the tag RFC instead of
>    having a new mailing list^. People sending RFCs send without the tag
>    can be asked to send it again with the tag.
> 3) This would only be low-volume if you do not count the
>    responses/discussion.
>
> * I haven't counted them but I am probably close with my estimate.
> ^ We have a lot already which can be, as implicit mentioned above,
>   confusing.
>
>
> >     A written proposal template, which provides guidelines for how to
> >     write a proposal. This proposal is written in an example template
> >     we can use, and the template can evolve over time.
>
> I'm in favor.
>
>
> >     A new directory on Github that serves as the archive for official
> proposals.
>
> I don't see how that helps so I'd like to ask you to elaborate why this
> is not yet another place one has to look for information.
>
>
> >     A review manager who helps shepherd the discussion in official
> >     rounds of review (which are time bound, e.g. to a week or two).
>
> Could you elaborate on how these review managers are determined?
>
>
> >     A decision making body that evaluates the results, asking for
> >     additional discussions (with advice for how to change the proposal
> >     in the next round), and that ultimately approves or denies a
> >     proposal. This body should aim to get the community to consensus
> >     whenever possible, but can help split decisions in the case of
> >     extreme ambiguity when overwise all hope is lost. Denied proposals
> >     can of course be re-run if a pertinent situation changes or when
> >     revised
>
> Could you elaborate on how these "decision making bodies" are
> determined?
>
>
> Thanks for initiating this,
>   Johannes
>
>
> [0] https://reviews.llvm.org/D71916
> [1] http://llvm.org/docs/DeveloperPolicy.html#code-reviews
> [2] https://www.llvm.org/docs/Contributing.html
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200116/6cac3e2c/attachment-0001.html>