[llvm-dev] [monorepo] Downstream branch zipping tool available
Björn Pettersson A via llvm-dev
llvm-dev at lists.llvm.org
Tue Dec 18 02:23:24 PST 2018
Thanks for sharing your branch zipping migration script.
Unfortunately I think our situation is a little bit more complicated.
We have used llvm as the umbrella repo, so in llvm we have a "master"
branch (from the git single repo version of llvm) and a couple of
downstream branches (let's call them "down0", "down1") containing our
downstream work (with frequent merges from "master").
The downstream branches has tools/clang and runtimes/compiler-rt as
submodules, as well as a couple of downstream submodules.
In our downstream version of clang we have a similar structure.
A "master" branch (mapping to the git single repo version clang),
and a couple of downstream branches. The downstream branches has
tools/extra (i.e. clang-tools-extra) as a submodule.
I can also mention that the clang, compiler-rt and clang-tools-extra
submodules aren't present from the beginning of history. They have
been added later on.
I doubt that zip-downstream-fork.py will work out-of-the-box.
Hopefully I'll be able to patch it for our scenario. Any guidelines
might be helpful. But maybe it isn't even worth trying to adapt
zip-downstream-fork.py to do something useful for our scenario?
If someone else got a similar scenario, let me know. Perhaps we can
do some joint effort in adapting the zipper script.
> -----Original Message-----
> From: llvm-dev <llvm-dev-bounces at lists.llvm.org> On Behalf Of David
> Greene via llvm-dev
> Sent: den 12 november 2018 22:27
> To: llvm-dev at lists.llvm.org; cfe-dev at lists.llvm.org; libcxx-
> dev at lists.llvm.org; lldb-dev at lists.llvm.org; openmp-dev at lists.llvm.org;
> libclc-dev at lists.llvm.org; clangd-dev at lists.llvm.org
> Subject: [llvm-dev] [monorepo] Downstream branch zipping tool available
> Building on the great work that James Knight did on
> migrate-downstream-fork.py (Thanks, James!) , I've created a simple
> tool to take migrated downstream fork branches and zip them into a
> single history given a history containing submodule updates of
> subprojects .
> With migrate-downstream-fork.py, one is left with a set of unrelated
> histories, one per subproject:
> llvm clang compiler-rt
> * V Add my fancy LLVM feature * G Fix my dumb clang bug * Z Merge
> from upstream compiler-rt
> One can do an octopus merge to unify them:
> *-- Merge llvm, clang and compiler-rt
> |\ \
> * \ \ V Add my fancy LLVM feature
> | * | G Fix my dumb clang bug
> | | * Z Merge from upstream compiler-rt
> Unfortunately, that doesn't show the logical history of development,
> where changes were effectively applied to subprojects in a linear
> fashion. This makes it more difficult to do bisects, among other things
> because none of the downstream integration happens until the octopus
> Let's say that downstream you have a local mirror for each LLVM
> subproject you work on. Suppose also that you have an "umbrella"
> repository that holds submodule references to all those local mirrors.
> Various commits in the umbrella update submodule references:
> * Update llvm submodule to V
> * Update clang submodule to G
> * Don't update any submodules, fix scripts or something
> * Update compiler-rt submodule to Z
> zip-downstream-fork.py will take these submodule updates and "inline"
> them into the umbrella history, making it appear that the downstream
> commits were applied against the monorepo in the order implied by the
> umbrella history:
> * A Add my fancy LLVM feature
> * B Fix my dumb clang bug
> * C Merge from upstream compiler-rt
> Parent relationships for merges from upstream are preserved, though as
> top-level comments in zip-downstream-fork.py explain, the history graph
> can look a little strange. Commits that don't update submodules are
> skipped on the assumption that they modify things uninteresting to a
> monorepo history. Such commits could be preserved but doing so has some
> caveats as explained in the comments. Perhaps your umbrella repository
> holds your build scripts. You'd probably want to migrate that to the
> zipped history. If there's strong demand for this I could look into
> doing it.
> There are various other limitations to the tool explained in the
> comments. It was enough to get us going and I'm hopeful it will be
> useful for others. It seems to do the right thing with our repositories
> but YMMV. Feel free to open PRs with bug fixes. :)
> To get this to work, you'll need to apply a PR for
> migrate-downstream-fork.py to fix issues with --revmap-out .
>  https://github.com/jyknight/llvm-git-migration/blob/master/migrate-
>  https://github.com/jyknight/llvm-git-
>  https://github.com/jyknight/llvm-git-migration/pull/1
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
More information about the llvm-dev