[cfe-dev] [llvm-dev] RFC: Code Review Process

Wed Oct 6 10:33:46 PDT 2021

(Note: I’m speaking as an individual here, not with any particular organizational hat; FWIW I personally use both Phabricator and GitHub daily).

Various bits of LLVM infra have had organic roots, e.g. the mailing lists originally hosted at UIUC, Phabricator being set up by individuals, etc.  However, this has historically always become a problem: e.g. UIUC prevented us from having a wiki early on because of user-generated content rules, the mailing lists get overrun by spam, Phabricator becomes unmaintained.

The root issue here is that LLVM is a >20 year old project, and many of the technologies that have successfully propelled it to where it is evolve and change, and the people working on LLVM wander into and out of the project over the years.

Meanwhile, LLVM is critical infra for many teams, companies and products.  I consider it to be a serious strategic risk that we depend on infrastructure that is maintained by (well intentioned, and obvious incredibly valued) community volunteers: for example, SVN used to go down when the server filled up.  This simply isn’t a way to run a scalable project, and makes us very susceptible to the “bus factor <https://en.wikipedia.org/wiki/Bus_factor>”.  We as a project need to depend on maintained and hosted infra, not run it ourselves.

Furthermore, while community consensus is an important part of decision making in the LLVM project, it is clear that not every individual will be happy.  Further, from a project management perspective, we don’t want multiple ways to do patch reviews: we need a single standard solution here, this can’t practically be a matter of personal preference.

Coming back to the details at hand, there is a proposal to move to GitHub PR’s for a variety of reasons that other people have made.  It also seems that there are some serious gaps that we need to figure out before it is plausible replacement, which is what the thread is exploring.

This all seems like a good and healthy discussion to be having.  It seems that some folks are surprised by the premise of wanting to get on maintained and hosted infrastructure (e.g. SVN -> GitHub).  I thought this was an extremely clear direction for years now - and that we were problem solving towards the best way to do this.

-Chris

> On Oct 6, 2021, at 1:46 AM, James Henderson via cfe-dev <cfe-dev at lists.llvm.org> wrote:
> 
> Forgive me if I'm wrong, but if the community consensus is that we should continue to use Phabricator, and Phabricator is not being provided/maintained by the LLVM Foundation, isn't it moot what the LLVM Foundation/Infrastructure Working Group recommends/wants to happen? The current maintainers would continue to maintain Phabricator (assuming they wanted to), and people would still be able to review things there. What would happen if the Foundation officially supported PRs, without community consensus (in particular from the Phabricator maintainers), is a potential split in the community, with some continuing in the old way and others using the new way (and presumably some choosing to review on both platforms). This cannot be good.
> 
> I'm all for the discussion to be had, about whether we switch, but as far as I can see, nothing's really changed from the previous conversations on PRs versus Github, apart from the announcement of end of support by the upstream company, but that was quite a while ago now, and even with the stale Arcanist issue, there hasn't been a big push from community members to change: the consensus in the posts discussing this and the moving to PRs seems to still be "there are things that are blocking switching still".
> 
> At the most, from this IWG/Foundation consultation, it should be that the Foundation recommends one or other approach, and is willing to provide X infrastructure required. The community can then choose to agree with whatever approach is recommended or stick with the status quo. There shouldn't be an edict that says we will do one thing or the other.
> 
> Another side-point: whilst the IWG may consist of people who care about LLVM, there are far more people who care as much, but who just don't have the time to participate in such a group. This is particularly important to note, because the community does not elect members to this group. To an extent, the same is also true of the Foundation board itself, since there are plenty of people who may not agree with their decisions, but don't have the time to volunteer for the board. I'm not suggesting that there's any malice in this discussion, and indeed, the fact that it's open to community comments certainly is helpful, but I'd be worried of some kind of echo chamber/unconscious bias within the small groups suggesting there is consensus for one approach, when the wider community thinks otherwise.
> 
> James
> 
> On Tue, 5 Oct 2021 at 20:52, Tanya Lattner via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
> Hello! The purpose of this email is to start a discussion about our code review tools. No decisions have been made about changing tools. The idea behind a timeline is so that information could be gathered in a timely manner. The Infrastructure Working Group was formed to bring together community members who have an experience and/or passion regarding infrastructure. Anyone can participate in this working group and like the LLVM Foundation, the minutes are all made public. 
> 
> The LLVM Foundation’s mission is to support the LLVM project and help ensure the health and productivity of of the community and this is done through numerous ways including infrastructure. I do not think it is a negative thing that the foundation board of directors would be discussing our current tools and gathering information how how well they work and how we can make them better. As the legal entity who bares financial and legal responsibility for a lot of the infrastructure, this would make sense. This also makes sense because of the people involved who care a lot about LLVM and the project. But, the LLVM Foundation does not pay for Phabricator and we are very grateful for Google’s support of this critical piece of our infrastructure. 
> 
> Regarding Phabricator, there are a couple of pieces of information that were provided to the LLVM Foundation by maintainers (maybe previous it sounds like) of this instance and how we may need to look into alternative ways to support it.  In addition, Phacility itself has publicly stated that it is winding down operations. (https://admin.phacility.com/phame/post/view/11/phacility_is_winding_down_operations/ <https://admin.phacility.com/phame/post/view/11/phacility_is_winding_down_operations/>). Lastly, there are questions about why we are not using GitHub pull requests as we are on GitHub and that might be the natural path to take for a number of reasons.
> 
> The above reasons are why the RFC was written. Perhaps it wasn’t written in the best way, but I also feel like it is being read in a negative way which is incredibly disappointing given I don’t feel there is a valid reason for this.
> 
> -Tanya
> 
> 
> 
> 
> 
> 
>> On Oct 5, 2021, at 11:35 AM, Renato Golin via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>> 
>> On Tue, 5 Oct 2021 at 19:16, Tom Stellard <tstellar at redhat.com <mailto:tstellar at redhat.com>> wrote:
>> However, it's not a good position for the Board to be responsible
>> for something that it doesn't have control over.  If Google decided to stop hosting
>> Phabricator for some reason (unlikely, but not impossible), the Board would be
>> responsible for finding a replacement.
>> 
>> Sorry, this is a very weak reason for such a strong worded "RFC".
>> 
>> I _cannot_ imagine "Google" stopping to support something so quickly as to leave the foundation without recourse. And even if they did, *no one* would blame the foundation for that.
>> 
>> Even if you ignore all the effort that hundreds of their engineers have done over the past decade to the project, this would hurt Google more than anyone else. It's a far fetched concern.
>> 
>> And if the foundation wants "control" of a piece of infrastructure that Google has been maintaining for years, then this is a different discussion. Hopefully one that doesn't involve unilateral decisions.
>>  
>> The main risk is that Phabricator is no longer maintained upstream.
>> There was already an issue[1] recently where the arc tool stopped working and won't
>> be fixed upstream. Using unmaintained software is a bigger risk.
>> 
>> I don't like using unmaintained software either, but I think our Phab has had more attention than the upstream project. And no one has to use arc, I certainly never have.
>> 
>> Don't get me wrong, I don't like Phab and I think Github would bring new people to the project, but it's gotta be done the right way, and pushing it isn't it.
>> 
>> We, meaning the LLVM Board of Directors.  And really the problem isn't the self-hosting
>> so much as it's the lack of an enforceable maintenance agreement the Foundation and the
>> maintainers.
>> 
>> The problem isn't self-hosting at all, given that Google is doing that.  (apologies, I assumed otherwise earlier).
>> 
>> Neither is maintenance, given Google is doing that too.
>> 
>> The only thing that's left is control, and I don't really understand why this is important, as I explained above.
>> 
>> cheers,
>> --renato
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20211006/d318784d/attachment.html>