[cfe-dev] [llvm-dev] Phabricator Maintenance

[Zachary Turner via cfe-dev]

I can’t really provide a doc, but i can describe what I believe to be the
biggest problem.

In a GH PR, comments are associated with commit hashes.  If a commit hash
ceases to exist, so do all comments associated with it.  The comments are
quite literally destroyed and irretrievable.

What this means for LLVM is that everyone will have to completely stop
using history rewriting operations.  No more rebase, squash, amend, etc.

A common theme in LLVM development is carefully curating patch histories
for easy reviewability.  For example, someone might tell you to split a
patch up into two patches and have them reviewed separately.  If there are
dependent changes between the two, you have to choices:

1. You can split this into two commits on a single branch.
2. You can make 2 branches A and B, with B based off A.  Request a PR from
A into master and B into A.

If you do 1 and someone requests changes on A, you cannot rebase -i your
fixes so that they still appear in the diff for A, because that destroys
comments.

If you do 2 and someone requests changes on A, you then have to merge (not
rebase) those changes into B.  The problem is compounded the more pieces
you have to split your patch into and it becomes quite a maintenance
headache.

IMO, switching to GH PRs means abandoning LLVMs philosophy on incremental
development for the above reason.

One more interesting quirk of GitHub PR:  it is actually impossible to make
a comment on lines in a PR that aren’t changed as part of the PR (eg
surrounding code).  See
https://github.com/isaacs/github/issues/284 which has been open a very long
time

On Thu, Jun 25, 2020 at 1:13 AM Manuel Klimek via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> Mehdi, Fangrui: are you willing to take on maintenance?
>
> Otherwise, Shoaib, the cost is currently:
> ~$300-350 / mo for sendgrid (300-350k emails / month)
> ~$2k / mo for cloud (we currently run on 1 machine O.O, plus storage &
> backup)
>
> If y'all care about the workflow but don't care about diffs in emails, we
> can switch to stock phab.
> I can ask Phacility for a quote; stock phab comes with significantly
> different emails though.
>
> Re: patch chains - perhaps we can ask github on support for that?
> I think what would help is somebody providing a doc with the features that
> phab provides that github PR doesn't, so we can get some consensus on what
> the diff is.
>
> On Thu, Jun 25, 2020 at 1:21 AM Shoaib Meenai <smeenai at fb.com> wrote:
>
>> I understand that keeping this within one company is easiest from an
>> organization perspective, so if Fangrui and Mehdi (and other Googlers) are
>> able to take this on, that’s great. If not, I can raise this internally at
>> Facebook. An estimate of the total costs incurred would be helpful for
>> that, e.g. you mentioned Sendgrid being a couple of hundred dollars a month.
>>
>>
>>
>> Thanks,
>>
>> Shoaib
>>
>>
>>
>> *From: *llvm-dev <llvm-dev-bounces at lists.llvm.org> on behalf of Manuel
>> Klimek via llvm-dev <llvm-dev at lists.llvm.org>
>> *Reply-To: *Manuel Klimek <klimek at google.com>
>> *Date: *Tuesday, June 23, 2020 at 5:41 AM
>> *To: *Mehdi AMINI <joker.eph at gmail.com>
>> *Cc: *LLVM Dev <llvm-dev at lists.llvm.org>, cfe-dev <cfe-dev at lists.llvm.org
>> >
>> *Subject: *Re: [llvm-dev] [cfe-dev] Phabricator Maintenance
>>
>>
>>
>> Answering multiple questions:
>>
>> 1. the maintenance cost (for our current setup) isn't super high, but
>> it's high enough that I wasn't able to find Googlers really willing to take
>> it on, and it's very bursty - I think overall, it's less than a 10%
>> project, but it can eat up a day or two at unspecific times; for example,
>> we currently would need a source upgrade, as I believe we have some auth
>> methods not working any more.
>>
>> Mehdi & Fangrui, if you want to own this together, that would be the
>> easiest path forward. I am happy to continue to make sure sendgrid is paid
>> for (which is currently a couple hundred $ a month).
>>
>>
>>
>> 2. if I can't find Googlers willing to own this, we need to loop this
>> through somebody else, which, depending on the setup, can be a bit more
>> up-front work. For example, if FB was willing to take this on for the next
>> 10 years ;) and you'd find a couple of folks at FB, that might be fairly
>> simple, while the LLVM foundation is currently afaiu not set up to easily
>> create funding even for the relatively cheap cloud setup we need
>>
>>
>>
>> 3. Mehdi is correct,
>> https://github.com/r4nt/phabricator/tree/llvm-production is what's
>> currently running; one of the next tasks would be to see whether we can get
>> rid of more of the customizations we have (I actually have no idea how far
>> we are, Eric is not on the team any more, and he's afaict been the last
>> person to upgrade phab)
>>
>>
>>
>> High prio work items are:
>>
>> - keep the server up to date & secure
>>
>> - keep upgrading to newer Phab versions, reduce or maintain our diff to
>> mainline
>>
>>
>>
>> Thanks!
>>
>> /Manuel
>>
>>
>>
>>
>>
>> On Tue, Jun 23, 2020 at 7:04 AM Mehdi AMINI <joker.eph at gmail.com> wrote:
>>
>>
>>
>>
>>
>>
>>
>> On Mon, Jun 22, 2020 at 9:25 PM David Blaikie <dblaikie at gmail.com> wrote:
>>
>> On Mon, Jun 22, 2020 at 8:15 PM Mehdi AMINI via cfe-dev
>> <cfe-dev at lists.llvm.org> wrote:
>> >
>> >
>> >
>> > On Mon, Jun 22, 2020 at 2:33 AM Manuel Klimek <klimek at google.com>
>> wrote:
>> >>
>> >> On Fri, Jun 19, 2020 at 10:04 PM Mehdi AMINI via cfe-dev <
>> cfe-dev at lists.llvm.org> wrote:
>> >>>
>> >>>
>> >>>
>> >>> On Fri, Jun 19, 2020 at 9:56 AM Hubert Tong via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>> >>>>
>> >>>> On Fri, Jun 19, 2020 at 12:32 PM Anton Korobeynikov via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>> >>>>>
>> >>>>> Just my 2 cents here: we are working on enabling this as a part of
>> >>>>> bugzilla migration as PRs and issues are very tied inside GitHub.
>> Stay
>> >>>>> tuned for updates!
>> >>>>
>> >>>> I am not aware that the previous long thread about usage of GitHub
>> PRs in place of Phabricator reviews got anywhere near the point where the
>> option of Phabricator reviews was being dropped
>> >>>
>> >>>
>> >>> That's my impression as well, I find GitHub review is frustrating in
>> comparison to phab, in particular the way comments are handled across
>> updates, unless you stick to never rebase and only append commits and
>> merges from master. This is unfortunately not compatible with the LLVM repo
>> history right now.
>> >>>
>> >>> https://www.phacility.com
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.phacility.com&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=lJwxc7o34UKit8nM2fj4O0ASyu1JnOqeXvhr24fX66Y&s=9P-x7k7eVVMR4DlfgeXAq8p-1R_cVMVqOek61HN60Ho&e=>
>> offers hosting for Phabricator, could we look into this instead?
>> >>
>> >>
>> >> Last time I checked, they basically said they didn't want us (no
>> customizations, which LLVM folks still wanted).
>> >
>> >
>> > Well, if this is a deal breaker for us, then we can also exclude GitHub
>> PR as a replacement for now: I doubt they offer these customizations.
>>
>> Not sure it's necessarily quite that clear-cut. LLVM folks might want
>> the customizations, but if Phab with customizations isn't maintained,
>> it might be a choice between Phab without those customizations (what
>> are they?)
>>
>>
>>
>> I believe it is all there:
>> https://github.com/r4nt/phabricator/tree/llvm-production
>>
>>
>>
>> As far as I know it is mostly about the interactions with the email
>> workflow for the *-commits@ mailing-lists, but Manuel knows best here :)
>>
>>
>>
>> --
>>
>> Mehdi
>>
>>
>>
>>
>>
>>
>>
>> or GitHub, email, etc - which might be a different
>> choice/tradeoff.
>>
>> > What are our alternatives? (other than going back to pure email for
>> reviews)
>> >
>> > --
>> > Mehdi
>> >
>> >
>> >>
>> >> Whatever we do, we need some volunteer who's willing to spend multiple
>> days on this (and potentially more going forward).
>> >> I'm currently trying to find a volunteer more than solutions.
>> >>
>> >>>
>> >>>
>> >>> --
>> >>> Mehdi
>> >>>
>> >>>
>> >>>>
>> >>>> . The original post on this thread indicated interest in not
>> maintaining Phabricator. How does that affect the availability of
>> Phabricator? Does this mean that the community is going to move to GitHub
>> PRs because the choice of Phabricator is being taken away?
>> >>>>
>> >>>>>
>> >>>>>
>> >>>>> On Fri, Jun 19, 2020 at 3:45 PM Manuel Klimek via llvm-dev
>> >>>>> <llvm-dev at lists.llvm.org> wrote:
>> >>>>> >
>> >>>>> > -Chris' outdated email, +Chris' correct email :)
>> >>>>> >
>> >>>>> > On Fri, Jun 19, 2020 at 2:01 PM Manuel Klimek <klimek at google.com>
>> wrote:
>> >>>>> >>
>> >>>>> >> Hi folks,
>> >>>>> >>
>> >>>>> >> phabricator maintenance is a topic that has been lying dormant
>> for a while now.
>> >>>>> >>
>> >>>>> >> That subsequently creates a non-optimal user experience.
>> >>>>> >> For me personally, given that github provides a secure PR
>> infrastructure, the additional effort required to keep Phab going is not an
>> investment I'm personally willing to make. I understand that there are
>> unique selling points for Phab in its UI compared to github PRs, but there
>> are also significant downsides in the effort to integrate with Phab that
>> github PRs make easier.
>> >>>>> >>
>> >>>>> >> Thus, I see two options:
>> >>>>> >> 1. somebody volunteers to take on Phabricator maintenance and
>> figures out a way to fund it, either through the LLVM foundation or some
>> other means (I'd love for us at Google to pay for it directly and give
>> folks outside Google access, but that is unfortunately a hard problem for a
>> variety of reasons). I'd be happy to help to provide a DB snapshot for the
>> migration, of course.
>> >>>>> >>
>> >>>>> >> 2. We switch to github PRs
>> >>>>> >>
>> >>>>> >> Thoughts?
>> >>>>> >> /Manuel
>> >>>>> >>
>> >>>>> >>
>> >>>>> >> On Thu, Jun 18, 2020 at 6:42 PM Raphael Isemann <
>> teemperor at gmail.com> wrote:
>> >>>>> >>>
>> >>>>> >>> Friendly ping
>> >>>>> >>>
>> >>>>> >>> Am Do., 9. Apr. 2020 um 16:04 Uhr schrieb Alexandre Ganea
>> >>>>> >>> <alexandre.ganea at ubisoft.com>:
>> >>>>> >>> >
>> >>>>> >>> > cc Paul / MyDeveloperDay
>> >>>>> >>> >
>> >>>>> >>> >
>> >>>>> >>> >
>> >>>>> >>> > De : llvm-dev <llvm-dev-bounces at lists.llvm.org> De la part
>> de David Blaikie via llvm-dev
>> >>>>> >>> > Envoyé : April 8, 2020 10:21 PM
>> >>>>> >>> > À : Raphael “Teemperor” Isemann <teemperor at gmail.com>;
>> Manuel Klimek <klimek at google.com>
>> >>>>> >>> > Cc : llvm-dev <llvm-dev at lists.llvm.org>
>> >>>>> >>> > Objet : Re: [llvm-dev] Outdated Phabricator version on
>> reviews.llvm.org
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__reviews.llvm.org&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=lJwxc7o34UKit8nM2fj4O0ASyu1JnOqeXvhr24fX66Y&s=HAgfBPFfo9oOWhEy4OCUUP2n4iCoxmLk-StyjAbO_oo&e=>
>> breaks Google authentication since today
>> >>>>> >>> >
>> >>>>> >>> >
>> >>>>> >>> >
>> >>>>> >>> > hey Manuel - are you/do you know who's likely to be doing any
>> upkeep on Phabricator these days? Might need an update for this...
>> >>>>> >>> >
>> >>>>> >>> >
>> >>>>> >>> >
>> >>>>> >>> > - Dave
>> >>>>> >>> >
>> >>>>> >>> >
>> >>>>> >>> >
>> >>>>> >>> > On Wed, Apr 8, 2020 at 5:57 AM Raphael “Teemperor” Isemann
>> via llvm-dev <llvm-dev at lists.llvm.org> wrote:
>> >>>>> >>> >
>> >>>>> >>> > Hi all,
>> >>>>> >>> >
>> >>>>> >>> > I’m using my Google account to log into my Phabricator
>> account on reviews.llvm.org
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__reviews.llvm.org&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=lJwxc7o34UKit8nM2fj4O0ASyu1JnOqeXvhr24fX66Y&s=HAgfBPFfo9oOWhEy4OCUUP2n4iCoxmLk-StyjAbO_oo&e=>
>> . Since today that no longer works as I don’t seem to get any reply from
>> reviews.llvm.org
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__reviews.llvm.org&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=lJwxc7o34UKit8nM2fj4O0ASyu1JnOqeXvhr24fX66Y&s=HAgfBPFfo9oOWhEy4OCUUP2n4iCoxmLk-StyjAbO_oo&e=>
>> when I’m logged into my account. It tried logging out which fixes the issue
>> of reviews.llvm.org
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__reviews.llvm.org&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=lJwxc7o34UKit8nM2fj4O0ASyu1JnOqeXvhr24fX66Y&s=HAgfBPFfo9oOWhEy4OCUUP2n4iCoxmLk-StyjAbO_oo&e=>
>> not loading, but when I try to login I just get the following error:
>> >>>>> >>> >
>> >>>>> >>> > > Expected to retrieve an "account" email from Google Plus
>> API call to identify account, but failed.
>> >>>>> >>> >
>> >>>>> >>> > After some searching it seems that this error is due to the
>> Google Plus API being shutdown and the Phabricator folks replaced that
>> logic (including this error message string) a year ago here [1]
>> >>>>> >>> >
>> >>>>> >>> > I assume we haven’t updated reviews.llvm.org
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__reviews.llvm.org&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=lJwxc7o34UKit8nM2fj4O0ASyu1JnOqeXvhr24fX66Y&s=HAgfBPFfo9oOWhEy4OCUUP2n4iCoxmLk-StyjAbO_oo&e=>
>> to whatever latest Phabricator release contains that patch.
>> >>>>> >>> >
>> >>>>> >>> > Not sure who’s currently responsible for updating
>> reviews.llvm.org
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__reviews.llvm.org&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=lJwxc7o34UKit8nM2fj4O0ASyu1JnOqeXvhr24fX66Y&s=HAgfBPFfo9oOWhEy4OCUUP2n4iCoxmLk-StyjAbO_oo&e=>
>> so I thought I’ll just drop a mail to the list (and maybe save someone else
>> from figuring out why their login is suddenly broken).
>> >>>>> >>> >
>> >>>>> >>> > [1] https://secure.phabricator.com/D20030
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__secure.phabricator.com_D20030&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=lJwxc7o34UKit8nM2fj4O0ASyu1JnOqeXvhr24fX66Y&s=ll1jFTtzmn4J2nolNST0I0aK1fLbo9yYSUg7mEvxBGs&e=>
>> >>>>> >>> > _______________________________________________
>> >>>>> >>> > LLVM Developers mailing list
>> >>>>> >>> > llvm-dev at lists.llvm.org
>> >>>>> >>> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.llvm.org_cgi-2Dbin_mailman_listinfo_llvm-2Ddev&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=lJwxc7o34UKit8nM2fj4O0ASyu1JnOqeXvhr24fX66Y&s=ACUAN3fWOUwuQzLVRNc-TsqZ63HmaCGb_SL1dK04SHs&e=>
>> >>>>> >
>> >>>>> > _______________________________________________
>> >>>>> > LLVM Developers mailing list
>> >>>>> > llvm-dev at lists.llvm.org
>> >>>>> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.llvm.org_cgi-2Dbin_mailman_listinfo_llvm-2Ddev&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=lJwxc7o34UKit8nM2fj4O0ASyu1JnOqeXvhr24fX66Y&s=ACUAN3fWOUwuQzLVRNc-TsqZ63HmaCGb_SL1dK04SHs&e=>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> With best regards, Anton Korobeynikov
>> >>>>> Department of Statistical Modelling, Saint Petersburg State
>> University
>> >>>>> _______________________________________________
>> >>>>> LLVM Developers mailing list
>> >>>>> llvm-dev at lists.llvm.org
>> >>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.llvm.org_cgi-2Dbin_mailman_listinfo_llvm-2Ddev&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=lJwxc7o34UKit8nM2fj4O0ASyu1JnOqeXvhr24fX66Y&s=ACUAN3fWOUwuQzLVRNc-TsqZ63HmaCGb_SL1dK04SHs&e=>
>> >>>>
>> >>>> _______________________________________________
>> >>>> LLVM Developers mailing list
>> >>>> llvm-dev at lists.llvm.org
>> >>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.llvm.org_cgi-2Dbin_mailman_listinfo_llvm-2Ddev&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=lJwxc7o34UKit8nM2fj4O0ASyu1JnOqeXvhr24fX66Y&s=ACUAN3fWOUwuQzLVRNc-TsqZ63HmaCGb_SL1dK04SHs&e=>
>> >>>
>> >>> _______________________________________________
>> >>> cfe-dev mailing list
>> >>> cfe-dev at lists.llvm.org
>> >>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.llvm.org_cgi-2Dbin_mailman_listinfo_cfe-2Ddev&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=lJwxc7o34UKit8nM2fj4O0ASyu1JnOqeXvhr24fX66Y&s=f_OxOIUmWlaz5vJyYSMPQbGFoGVcDPI53yN7RhkgzIA&e=>
>> >
>> > _______________________________________________
>> > cfe-dev mailing list
>> > cfe-dev at lists.llvm.org
>> > https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.llvm.org_cgi-2Dbin_mailman_listinfo_cfe-2Ddev&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=lJwxc7o34UKit8nM2fj4O0ASyu1JnOqeXvhr24fX66Y&s=f_OxOIUmWlaz5vJyYSMPQbGFoGVcDPI53yN7RhkgzIA&e=>
>>
>> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20200625/9416d848/attachment-0001.html>