[llvm-dev] [PITCH] Improvements to LLVM Decision Making

[Chris Lattner via llvm-dev]

I’m responding to a bunch of posts all in one email: thank you everyone for the feedback!  I am dropping a few comments, but I did read everything of course :)

On Jan 15, 2020, at 2:04 AM, Doerfert, Johannes <jdoerfert at anl.gov> wrote:
>> Here is the outline of the draft proposal
>> <https://gist.github.com/lattner/e3679998a7609c99b1243f09d30f0132>.
>> Caveats: this is itself just a draft proposal to inspire discussion
>> not a mandate - I welcome feedback, thoughts and concerns, ideas of
>> how to make this better, or better ideas/approaches overall on how we
>> can level up the community.
>> 
>> Please let me know what you think, thanks!
> 
> First thought: This should not be a gist on github. Maybe it should be
> on phabricator or at least in an email so it is easier to respond to it.

Ok, I will move this to a Google doc to make it easier to comment on it easily - I find it easier to work with this flow.  I know that some folks have concerns with Google docs, but we can move to another media in the future as things converge more.

>> "It isn't clear how to propose some changes in the first place, and it
>> is often unclear who the decision makers are."
> 
> I feel that patches and RFCs are well established *and documented* [1,2]
> ways to propose changes. In addition, the *-dev lists, IRC, etc. do
> provide ways to clear uncertainties. Adding more documentation on this
> can obviously be helpful. Also, we are already improving the existing
> documentation [0].

Good point, clarified to "RFC’s are a well established convention in the LLVM community, but it is often unclear who the decision makers are, particularly when there is controversy."

> That said, we need to differentiate in more detail what the problems
> actually are. Do people not find the documentation? Is the documentation
> unclear? Are people waiting for "a decision maker" to step in or afraid
> to ask questions? Are questions not answered? ...

No, I don’t think this is in scope for this sort of process.  This is a heavy-weight process of last resort, not something designed for quick answers.  

My sense of this little project is that it is about solving the “hard” cases where the community is divided.  I’m not interested in making easy cases harder, I’m trying to make nearly-impossible cases "merely hard” and much less ambiguous.

>> "It isn't clear what mailing lists to send such proposals to: llvm-dev
>> gets a lot of traffic and many people don’t read it."
> 
> I argue that people that do participate in the decision making (wording
> took from the previous point) already do, or at least should, follow
> llvm-dev.
> I am not quite sure why it is unclear what mailing list is the right one
> given that we have one *-dev mailing list per subproject and on llvm-dev,
> often the first points-of-contact, people are referred to the right one.

I don’t really agree: some communities have dev lists, but not all, and not everyone reads llvm-dev even if they “should”.    That said, I agree this wasn’t worded the best, I reframed it to "llvm-dev gets a lot of traffic and many people don’t read it - this makes it easy to miss controversial discussions.”  See also David’s point about inclusion to non-llvm-developers below.

>> "Sometimes people chime in late with dissent after a decision has been
>> apparently made: this can be frustrating to people who need a decision
>> made, because they aren't sure how to proceed."
> 
> It is unclear to me how the proposal helps on this front. Could you
> elaborate?

The problem as I see it is that there is no defined process so the “rules of engagement” are not specified.  The solution to this is to _write down_ the process.

>> "The initial discussion phase of a proposal can have lots of back and
>> forth discussions to shape a idea, and the eventual proposal can have
>> significant changes from that initial review. It can be difficult to
>> discern what feedback from the initial review is addressed or ignored
>> in the final rounds of discussions."
> 
> Yes. I am not sure how the proposed solution remedies the problem
> though. Could you elaborate?

The decision making body either calls for another explicit round of reviews (resetting the process) or the proposal is accepted/denied.  This eliminates a big chunk of ambiguity.

>> "Occasionally, Code Owners may be faced with a technical decision and
>> not be sure what to do, particularly for highly impactful design
>> decisions - e.g. for core LLVM IR changes. It could be useful to have
>> a formal escalation process for these decisions."
> 
> TBH, I always thought "code owner" is not a "decision making" but rather
> an "administrative" title. The person can be part of the decision
> making, but the role of "code owner" does not imply special privileges,
> only tasks, e.g., making sure reviews are done by the right people.

The intention is that code owners do own decision making within their defined scope and technical area of expertise.  That said, you’re right that the meta job of a code owner is to make sure things happen, not necessarily to do everything themselves.

>>    A new directory on Github that serves as the archive for official proposals.
> 
> I don't see how that helps so I'd like to ask you to elaborate why this
> is not yet another place one has to look for information.

The purpose is to capture historical discussions and provide a record, that is easier to browse than a mailing list.


On Jan 15, 2020, at 2:42 AM, Renato Golin <rengolin at gmail.com> wrote:
> My main concerns about the current model:
> * You get hit by a bus

Yes, I am also concerned about that.  :-) :-)

> * You get a lot of pressure to take decisions that are guaranteed to
> divide the community (bad for mental health)
> * You defer to "the community" to avoid mental health breakdown, then
> chaos ensue (we have good examples)

I’m actually ok with these points, at least in the immediate future (we’ll see how it goes).  As I’m sure you’ve seen, I’m not afraid to be controversial if I believe in an outcome.  The second point is also sort of what we have because of an absence of a process here.  I don’t think we have active chaos, but we have a lot of inefficiency (some decisions take way to long to make) and a lot of grumpy contributors as a result.

Your points about the tradeoffs of different social models are really useful.  There is no apparently easy answer here… 

> We also need to be scalable. If we have a decision that only affects a
> small part of the code, we can handle in the sub-community (like we
> currently do RFCs), but if some points may affect a wider range (like
> stepvector in SVE), then we need to widen the scope, involve more
> people, and we need those people to be pro active and responsive.

Yes, I agree, I think that we shouldn’t add process or overhead to easy things. I think the problem to be fixed is to address cases that don’t have a good solution yet.  I think this argues for a relatively heavy-weight process "of last resort”.

I also think your "that's why we need to be flexible on all counts. “ point is really important.  The goal here is to get the best outcome, not trick people or tie ourselves in bureaucratic knots.

> On Jan 16, 2020, at 12:47 AM, Christian Kühnel <kuhnel at google.com> wrote:
> How we do that in detail is less important to me. I'm also a big fan of continuous improvement and the Swift approach sounds reasonable. So let's start with that and then iterate/adapt if we see the need to do so.


The ironic thing is that I’ve been hamstrung for a few months now trying to figure out how to get perfection.  I agree with you, that it is better to try a few things, then learn iterate and adapt.


On Jan 16, 2020, at 8:21 AM, David Greene <greened at obbligato.org> wrote:
> My first reaction upon reading Chris' proposal was, "Great!  I'm glad
> we're finally tackling this."  My second reaction was, "There probably
> can't be a single decision-making group."
> 
> It's really hard to find a group of people with the passion, domain
> knowledge, communication skills and time to make decisions on every
> aspect of the project.

Yes, I agree with you.  I think the project is too big of a scope at this point.  Furthermore, it seems impossible to predict what the “future conflicts” will be, so we can’t proactively set up all the committees.  Given that, I think it makes sense to form them on an “on demand” basis.

> Not knowing exactly what the LLVM Foundation board does in its
> day-to-day work I don't know if one or more of these might fall under
> its purview.

I agree, this isn’t a good fit, exactly for the reason you point out above: it is a fixed group of people with a specific set of expertise, not necessarily aligned to adjudicating the problem of the day.


On Jan 17, 2020, at 9:36 AM, David Chisnall <David.Chisnall at cl.cam.ac.uk> wrote:
<many great points trimmed>

> Electing a core team can work very well if the electorate represents the interests of the community.  The LLVM community has a very sharp in-group / out-group divide currently and I struggle to see any way of defining an electorate for our wider community that would work well.  I would love to see more effort made to address this split independently, because I think it would give us a more healthy community and a better path to a democratic decision making process.


I think this is a really important point - we need to be inclusive of downstream users of LLVM (e.g. the rust community when it is something that would affect them) and provide a way for them to contribute useful signal if a decision could impact them.  I think this inclusion is really important, and is even more argument that this cannot be done on llvm-dev as it is today.



*** overall.

I really appreciate this thread and the discussion on it.  I think I have an idea of how to crack this, and I think the meta point of “let’s try something, see how it works, then learn and adapt over time” is a good approach.

While I’m trying to wiggle out of direct responsibility (to reduce the ‘hit by a bus’ factor), I think that it is fine for start with a known-suboptimal-in-that-way process, iterate a bit, and then refine that over time.

I’ll work on refining the proposal a bit and start another thread when it’s ready (probably tomorrow).

Thanks!

-Chris