[llvm-dev] [RFC] One or many git repositories?

Mon Jul 25 13:21:28 PDT 2016

> On Jul 25, 2016, at 9:54 AM, Justin Lebar via llvm-dev <llvm-dev at lists.llvm.org> wrote:
> 
> Hi, all.
> 
> I feel like we've strayed pretty far from the question originally
> posed in this thread.
> 
> One of the pieces of feedback I got before I started this thread was
> that many people felt that, the last time the question of multiple
> repos vs. monorepo was discussed, it was interspersed with other
> topics, making it difficult for some people to weigh in appropriately
> (or even to be aware that the discussion was occurring).  I'm afraid
> that the discussion of github workflows we're having here may cause
> the same problem.
> 
> Maybe we can move the discussion about github workflows into a
> different thread?  Again, I don't mean to stop it, just move it.
> 
> To re-focus this thread on its original topic: It sounds to me like,
> broadly speaking, we have consensus on using a single repository.  But
> there are still some outstanding related questions.  Among these are:
> 
> 1) Should the repository have "unified history"?  (Meaning, should I
> be able to check out a single git revision from before the migration
> and have it contain all of the llvm subprojects?)
> 2) Should the monorepo have a "nested" repository layout (e.g. clang
> goes in /tools/clang) or a "flat" layout (clang goes in /clang)?
> 
> 3) Assuming we want unified history, should the new canonical
> repository's hashes be based on
> https://github.com/llvm-project/llvm-project, or should it start
> afresh?
> 
> FWIW my answers to these are:
> 
> 1) Yes to unified history.  The main advantage of non-unified history
> is that it's easier for people to import old branches -- it's a matter
> of "git merge" instead of running the git filter-branch script I
> wrote.  But this is a relatively small (~20 minute) one-time cost to
> some of us, whereas our repository history is born by all of us
> forever.  Moreover unified history also helps people with long-running
> branches, as it lets them check out old versions of their branch and
> get a compatible version of all of the other llvm subprojects.

I think this is a nice property to have (unified rewritten history).
The fact that the existing hashes in the official git repo won’t exist in this new repository can be efficiently counter-balanced by continuing to update “forever” the existing official read-only repositories. This way clients that are already based off these won’t have to change anything to their workflow (unless they need the git-svn id).

> 
> 2) Yes to nested layout.  I find Chandler and Richard Smith's
> arguments compelling.

The flat layout is less disruptive and I haven't read a compelling argument to me to not adopt it.
As they mentioned: "we will almost certainly restructure the tree to make more sense in a monorepo* and "the result is actually very likely to look much more flat than the current layout, and to also be significantly superior to any of the current layouts.”.
This makes me think that on the contrary, starting with a flat repo and then moving pieces where they make sense while adjusting the build system is more likely to converge with a more “ideal” layout. This is also a less disruptive process.
Finally, starting with a nested layout requires immediate changes to the build system. And I haven’t seen a proposal for a non trivial interface to not disrupt the existing flow (i.e. cmake only builds LLVM by default and nothing else).

> 3) No to basing the new canonical repo on
> https://github.com/llvm-project/llvm-project.  That repo's history is
> missing svn revision numbers, and there are enough emails floating
> around that reference svn revision numbers that I think we need them
> in our canonical repo.  Also llvm-project/llvm-project has a flat
> structure, and if we end up going with a nested layout, it would be
> better to have that layout starting with the first commit.

I don’t see a reason to preserve the history of this repo (but someone may?).
Also this repo has Klee and dragonegg that I don’t see we want to keep (or at least I haven’t see anyone asking for support for these).
At the same time I don’t really care about the git-svn id in the commit message either, it seems this mapping could be archived outside the repo itself, they won’t make much sense as the time goes by, but will stay there forever.
Maybe an argument to keep them is that the commit message will match the existing official git repos?

— 
Mehdi

> 
> -Justin
> 
> On Mon, Jul 25, 2016 at 8:10 AM, Bruce Hoult via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>> git-imerge can run an arbitrary script to decide whether a commit is good or
>> bad. Lack of textual merge conflicts is only the most basic test -- you can
>> check that it compiles, run tests .. whatever you want and have time to
>> execute.
>> 
>> On Tue, Jul 26, 2016 at 2:12 AM, Robinson, Paul via llvm-dev
>> <llvm-dev at lists.llvm.org> wrote:
>>> 
>>> 
>>> 
>>>> -----Original Message-----
>>>> From: Renato Golin [mailto:renato.golin at linaro.org]
>>>> Sent: Monday, July 25, 2016 7:11 AM
>>>> To: Daniel Sanders
>>>> Cc: Robinson, Paul; llvm-dev at lists.llvm.org
>>>> Subject: Re: [llvm-dev] [RFC] One or many git repositories?
>>>> 
>>>> On 25 July 2016 at 14:55, Daniel Sanders <Daniel.Sanders at imgtec.com>
>>>> wrote:
>>>>> I know of a way but it's not very nice. The gist of it is to checkout
>>>> the
>>>>> downstream branch just before the bad merge and then merge the first
>>>>> 100 commits from upstream. If the result is good then merge the next
>>>>> 100, but if it's bad then 'git reset --hard' and merge 10 instead.
>>>> You'll
>>>>> eventually find the commit that made it bad. Essentially, the idea is
>>>>> to
>>>>> make a throwaway branch that merges more frequently. I do something
>>>>> similar to rebase my work to master since gradually rebasing often
>>>>> causes all the conflicts to go away.
>>>> 
>>>> This is essentially what git-imerge does, you only need to define
>>>> "good merge" in the form of a script or CI job.
>>>> 
>>>> cheers,
>>>> -renato
>>> 
>>> Except I understood git-imerge to be looking for physical conflicts,
>>> not "when did this test start failing."  If it does the latter also,
>>> that would be awesome.
>>> --paulr
>>> 
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> 
>> 
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev