[llvm-dev] Preparing BOLT for LLVM monorepo

[James Y Knight via llvm-dev]

>From a technical repository-health standpoint, this looks reasonable to me.
(I have not reviewed and have no opinion on the actual code contained here.)

Your proposed repository has:
- The expected commit-history-shape: A new unconnected root commit as the
parent of a linear history of all the other bolt-specific commits. Followed
by a single merge commit, with llvm mainline as the 1st parent, the bolt
history in 2nd-parent.
- The expected contents: All the bolt-side commits touch only files in
bolt/. The merge commit causes no content changes: the bolt/ tree is
identical compared to the 2nd parent, and the remainder are identical to
the 1st-parent.
- The size is not unexpectedly large (fetching bolt main and then running
"git gc --aggressive" adds ~3MB to the repository size -- 894M vs 891M).


On Mon, Dec 13, 2021 at 3:57 PM Rafael Auler via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> Should be fixed now.
>
>
>
> *From: *Rafael Auler <rafaelauler at fb.com>
> *Date: *Monday, December 13, 2021 at 3:06 PM
> *To: *Peter Waller <Peter.Waller at arm.com>, Hans Wennborg <
> hans at chromium.org>, Maksim Panchenko <maks at fb.com>
> *Cc: *llvm-dev at lists.llvm.org <llvm-dev at lists.llvm.org>, Chris Lattner <
> clattner at llvm.org>, Andrey Bokhanko <andreybokhanko at gmail.com>,
> vladislav.khmelevskyi at huawei.com <vladislav.khmelevskyi at huawei.com>
> *Subject: *Re: Preparing BOLT for LLVM monorepo
>
> Good catch! Thanks for spotting this, I’ll remove those.
>
>
>
> Best,
>
> Rafael
>
>
>
> *From: *Peter Waller <Peter.Waller at arm.com>
> *Date: *Monday, December 13, 2021 at 1:51 PM
> *To: *Rafael Auler <rafaelauler at fb.com>, Hans Wennborg <hans at chromium.org>,
> Maksim Panchenko <maks at fb.com>
> *Cc: *llvm-dev at lists.llvm.org <llvm-dev at lists.llvm.org>, Chris Lattner <
> clattner at llvm.org>, Andrey Bokhanko <andreybokhanko at gmail.com>,
> vladislav.khmelevskyi at huawei.com <vladislav.khmelevskyi at huawei.com>
> *Subject: *Re: Preparing BOLT for LLVM monorepo
>
> Just as a heads up, this pulls in a few dotfiles outside the bolt tree.
> The `.dockerignore` adding `.git` could conceivably break someone, so you
> might want to commit that separately so that it can be discussed/reverted.
> The other `.github/workflows` presumably aren’t intended for this
> repository either(?).
>
>
>
> To diff the merge commit’s first parent (upstream llvm) against the merge
> commit (assuming bash for brace expansion of `{^1,}`), ignoring the bolt
> directory:
>
>
>
> $ git diff-tree -r 'HEAD^{/Merge BOLT into LLVM}'{^1,} :^bolt
>
>
>
> :000000 100644 0000000000000000000000000000000000000000
> 6b8710a711f3b689885aa5c26c6c06bde348e82b A      .dockerignore
>
> :000000 100644 0000000000000000000000000000000000000000
> 477e9927279e0b6706a48f0d18f834137b92f115 A      .github/workflows/Dockerfile
>
> :000000 100644 0000000000000000000000000000000000000000
> 8deef62d15079ace72ac40db2335acf59d9a7e98 A
> .github/workflows/Dockerfile.aarch64
>
> :000000 100644 0000000000000000000000000000000000000000
> 02ee68ff868f6865825db153ee3aea7d4f76813f A
> .github/workflows/docker-image.yml
>
>
>
> Regards,
>
> Peter
>
>
>
> *From: *Rafael Auler <rafaelauler at fb.com>
> *Date: *Friday, 10 December 2021 at 04:34
> *To: *Peter Waller <Peter.Waller at arm.com>, Hans Wennborg <
> hans at chromium.org>, Maksim Panchenko <maks at fb.com>
> *Cc: *llvm-dev at lists.llvm.org <llvm-dev at lists.llvm.org>, Chris Lattner <
> clattner at llvm.org>, Andrey Bokhanko <andreybokhanko at gmail.com>,
> vladislav.khmelevskyi at huawei.com <vladislav.khmelevskyi at huawei.com>
> *Subject: *Re: Preparing BOLT for LLVM monorepo
>
> I updated our github repo https://github.com/facebookincubator/BOLT
> branch “main” with the strategy that Peter mentioned. I put a merge commit
> with all BOLT commits as parent #2, and these commits have an empty root.
> Because of that, the merge commit itself does not modify LLVM in any way
> other than introducing a new bolt folder with all project files and their
> histories. Then I have a small commit on top of that do change LLVM’s
> CMakeList.txt to add bolt as a project.
>
>
>
> At the moment, I believe the easiest way to review the code remains the
> one we have been using during these past months/year: checking our “main”
> branch and opening concerns in our github issues page.  In case there are
> no further suggestions to BOLT, we are targeting pushing the merge commit
> by the end of next week.
>
>
>
> *From: *llvm-dev <llvm-dev-bounces at lists.llvm.org> on behalf of Rafael
> Auler via llvm-dev <llvm-dev at lists.llvm.org>
> *Date: *Wednesday, December 8, 2021 at 4:13 PM
> *To: *Peter Waller <Peter.Waller at arm.com>, Hans Wennborg <
> hans at chromium.org>, Maksim Panchenko <maks at fb.com>
> *Cc: *llvm-dev at lists.llvm.org <llvm-dev at lists.llvm.org>, Chris Lattner <
> clattner at llvm.org>, Andrey Bokhanko <andreybokhanko at gmail.com>,
> vladislav.khmelevskyi at huawei.com <vladislav.khmelevskyi at huawei.com>
> *Subject: *Re: [llvm-dev] Preparing BOLT for LLVM monorepo
>
> Peter, thanks for your input, that’s an excellent suggestion. Let me try
> playing with that, I’ll try to change our history to use an empty root as
> you suggested.
>
>
>
> Best,
>
> Rafael
>
>
>
> *From: *Peter Waller <Peter.Waller at arm.com>
> *Date: *Wednesday, December 8, 2021 at 2:19 AM
> *To: *Rafael Auler <rafaelauler at fb.com>, Hans Wennborg <hans at chromium.org>,
> Maksim Panchenko <maks at fb.com>
> *Cc: *llvm-dev at lists.llvm.org <llvm-dev at lists.llvm.org>,
> vladislav.khmelevskyi at huawei.com <vladislav.khmelevskyi at huawei.com>,
> Andrey Bokhanko <andreybokhanko at gmail.com>, Chris Lattner <
> clattner at llvm.org>
> *Subject: *Re: Preparing BOLT for LLVM monorepo
>
> Your approach sounds reasonable to me and looks good from a glance. One
> thought to share though.
>
>
>
> My read: You take some point in LLVM’s history, then apply a commit which
> introduces a bolt directory, then have a few thousand bolt commits, then
> land a merge. Correct?
>
>
>
> If so, that means if you check out one of those historic commits, you have
> an llvm-project directory with BOLT at some commit, and LLVM subprojects
> always with the same ‘root’ commit.
>
>
>
> Presumably that ‘root’ commit does not work with any commit of BOLT, and
> anyone wanting to experiment with old commits (if this is a legitimate use
> case) would find that those things are unlikely to work except for maybe
> recent commits.
>
>
>
> Have you considered instead of rebasing onto a recent LLVM commit, taking
> an empty root? That way, when a user checks out a historic commit, now they
> will only get a bolt directory, and they can supply their own folders for
> other llvm subprojects from an appropriate point in time without them
> conflicting on the filesystem. Additionally, there is no implication for a
> user to think that those directories at those historic checkouts are
> meaningfully related to the BOLT commit.
>
>
>
> I mention it because that’s what we did for the flang merge  in case the
> same effect is appropriate for you here.
>
>
>
> Regards,
>
>
>
> Peter
>
>
>
> *From: *llvm-dev <llvm-dev-bounces at lists.llvm.org> on behalf of Rafael
> Auler via llvm-dev <llvm-dev at lists.llvm.org>
> *Date: *Wednesday, 8 December 2021 at 01:36
> *To: *Hans Wennborg <hans at chromium.org>, Maksim Panchenko <maks at fb.com>
> *Cc: *llvm-dev at lists.llvm.org <llvm-dev at lists.llvm.org>,
> vladislav.khmelevskyi at huawei.com <vladislav.khmelevskyi at huawei.com>,
> Andrey Bokhanko <andreybokhanko at gmail.com>, Chris Lattner <
> clattner at llvm.org>
> *Subject: *Re: [llvm-dev] Preparing BOLT for LLVM monorepo
>
> Hi folks,
>
>
>
> https://github.com/facebookincubator/BOLT branch “main” contains a merge
> proposal of BOLT into llvm-project. This is llvm from Nov 30th with 1016
> commits on top of it corresponding to the BOLT project.
>
> These 1016 commits would ideally be committed in a merge commit, merging
> LLVM as the first parent and BOLT as the second, and would be there only
> for the purposes of preserving project history. In this way, they should be
> easily skippable during a bisect of LLVM in the same way as the merge
> commit of flang. These commits represent the linear history of BOLT on top
> of rebased LLVM, so most commits are not buildable (since we can’t build a
> very old version of BOLT on top of a recent LLVM base). That’s why this is
> for history/blame only.
>
>
>
> We have addressed the issues in
> https://github.com/facebookincubator/BOLT/issues/248 and we are happy to
> continue working on any extra suggestions.
>
> Would it be better if we put this branch as a PR into llvm-project as a
> way to make it easier for people to review it? I don’t think we can put
> this into phabricator. However, I guess github’s bot will probably
> auto-close the PR. Also feel free to open new issues against our
> facebookincubator/BOLT project as a way to review it.
>
> Thanks
>
>
>
> *From: *Hans Wennborg <hans at chromium.org>
> *Date: *Thursday, November 4, 2021 at 5:42 AM
> *To: *Maksim Panchenko <maks at fb.com>
> *Cc: *Xinliang David Li <xinliangli at gmail.com>, Fāng-ruì Sòng <
> maskray at google.com>, Sriraman Tallam <tmsriram at google.com>, Chris Lattner
> <clattner at llvm.org>, Andrey Bokhanko <andreybokhanko at gmail.com>, Rafael
> Auler <rafaelauler at fb.com>, vladislav.khmelevskyi at huawei.com <
> vladislav.khmelevskyi at huawei.com>, tstellar at redhat.com <
> tstellar at redhat.com>, joker.eph at gmail.com <joker.eph at gmail.com>,
> echristo at gmail.com <echristo at gmail.com>, Nick Desaulniers <
> ndesaulniers at google.com>, llvm-dev at lists.llvm.org <llvm-dev at lists.llvm.org
> >
> *Subject: *Re: Preparing BOLT for LLVM monorepo
>
> Hi Maksim,
>
> On Tue, Nov 2, 2021 at 9:52 PM Maksim Panchenko <maks at fb.com> wrote:
> > We are still working on finalizing the exact logistics of the merge.
> >
> > However, we expect to follow the Flang project's footsteps and run the
> >
> > "--no-ff" merge to preserve the history of ~1K commits. We would like to
> >
> > ask for help and coordination from the release managers Tom Stellard and
> >
> > Hans Wennborg.
>
> I'm no longer involved in release management, and don't really have
> any opinion on how to merge this.
>
> Thanks,
> Hans
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20211213/3f54836d/attachment-0001.html>