[cfe-dev] [llvm-dev] [monorepo] Downstream branch zipping tool available

[David Greene via cfe-dev]

David Greene via llvm-dev <llvm-dev at lists.llvm.org> writes:

> Now that I think about it, what you really want is something that runs
> migrate-downstream-fork.py on the commits in llvm and something that
> runs zip-downstream-fork.py on commits in other projects, but they have
> to ruin simultaneously to keep the commits in the proper order.

After pondering this overnight, I think a better approach might be to do
the enhancement of zip-downstream-fork.py described in the comments:

# - The script requires a history with submodule updates.  It should
#   be fairly straightforward to enhance the script to take a revlist
#   directly, ordering the commits according to the revlist.  Such a
#   revlist could be generated from an umbrella repository or via
#   site-specific mechanisms.  This would be passed to
#   fast_filter_branch.py directly, rather than generating a list via
#   expand_ref_pattern(self.reflist) in Zipper.run as is currently
#   done.  Changes would need to be made to fast_filter_branch.py to
#   accept a revlist to process directly, bypassing its invocation of
#   git rev-list within do_filter.

If that were done, then it should be possible to write a tool to
generate such a revlist from your llvm master project.  The tool would
examine each commit in llvm and if it were a commit to llvm, it would
add its hash to the revlist.  If it were a submodule update it would
traverse the gitlink to find the commit in the corresponding project
(see find_submodules_in_entry in zip-downstream-fork.py).  It would then
add that commit's hash to the revlist.  If a commit updates multiple
submodules then you just have to pick an arbitrary order.

All of the code to do the traversal is already in
zip-downstream-fork.py.  You could enhance it to output a revlist in the
same way fast_filter_branch can output a revmap and have it not actually
rewrite any commits.  You would have to tell it to not skip
non-submodule-update commits as described in my previous message.

This all assumes that each submodule update only adds one new commit
from the project linked by the submodule (zip-downstream-fork.py also
makes this assumption).  If a submodule update represents moving a
submodule up multiple commits, then you'd need something that can walk
that history and add hashes to the revlist.

The more I think about it, the more it seems to me that this is the
easiest way to go.  It's much less work that refactoring two tools and
should require relatively minimal changes to migrate-downstream-fork.py.

                            -David