[llvm-dev] [RFC] Introduce an LLVM "Incubator" Process

[Serge Guelton via llvm-dev]

> 1) What are best practices for developing 'out-of-tree' projects like
this?

We have the example of Polly here, which recently adopted the « compiler
extension » framework, which makes it possible to write new passes that
work either as plugins or built-in passes, based on configuration options.
I think that's something we should favor when it makes sense.

On Sun, Jun 21, 2020 at 1:13 AM Stephen Neuendorffer via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> +1 from me.  I think this is a great idea.  A few things I've been
> thinking about recently:
>
> 1) What are best practices for developing 'out-of-tree' projects like
> this?  In addition to coding guidelines, code of conduct, etc., these
> projects would probably benefit from common structures on cmake integration
> (e.g LLVM_EXTERNAL_PROJECT vs. live in llvm/projects vs. cmake
> configuration export?), buildbots, and code review.  Of course projects
> should probably be allowed approach these things in new ways if they desire.
>
> 2) Managing dependencies is often tricky, and could arise between several
> incubated projects.  Even the 'simple' dependency on LLVM can be tricky to
> manage.  I've often done this by using git-submodules, but this is far from
> perfect.  Some projects put effort into supporting lots of different LLVM
> versions, with varying levels of success.
>
> 3) Some LLVM projects have nice extensibility interfaces (e.g. "Targets")
> which are relatively easy to live out of tree.  Others LLVM projects lack
> such well-defined interfaces and commonly result in merge conflicts.   Some
> projects could be relatively tightly integrated, whereas others (MLIR)
> might not.
>
> Steve
>
>
> On Sat, Jun 20, 2020 at 3:43 PM Chris Lattner via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Hi all,
>>
>> Today, we maintain a high bar for getting a new subproject into LLVM:
>> first a subproject has to be built far enough along to “prove its worth” to
>> be part of the LLVM monorepo (e.g. demonstrate community, etc).  Once
>> conceptually approved, it needs to follow all of the policies and practices
>> expected by an LLVM subproject.
>>
>> This is problematic for a couple reasons: it implicitly means that
>> projects have to start *somewhere else* but proactively decide to follow
>> LLVM design methodology and principles in the hope of being accepted.  It
>> is sometimes socially difficult to get these projects going because there
>> are many other forces that could encourage other practices.  For example, I
>> personally encountered this at Google with MLIR - “why aren’t you using
>> Google coding standards?”, several of us are currently discussing this in a
>> new skunkworks project in the “compilers for hardware” world, and the Flang
>> and other projects have found this challenging in the past.  Once the
>> project gets to a point of critical mass with the “wrong” approach, it is
>> very difficult and expensive to convert to the LLVM style, and from a
>> social perspective, inertia sometimes leads to forking off to separate
>> projects instead of folding back in to LLVM.
>>
>> A former colleague recently suggested the idea of introducing an
>> incubator process of some sort (e.g. xref the Apache version of this idea
>> <https://incubator.apache.org/>).  I think this is a really interesting
>> idea, and it is much easier now that the majority of the “official” code is
>> in the LLVM monorepo.
>>
>> Here is a sketch of how this could work:
>>
>>  - We maintain the same high bar to get into the LLVM monorepo, LLVM CI
>> etc.  No change here.
>>
>>  - We have a very light-weight proposal process that allows people to
>> create incubator projects in the LLVM organization, with no code up front.
>> The project would be required to have e.g. a charter document and README.
>>
>>  - Such projects are required to follow the LLVM developer policy, coding
>> standards, CoC, etc, but can define their own stability and evolution
>> process, code owners, etc.
>>
>>  - When the project is ready to graduate, it would follow the existing
>> process for becoming a first-class part of the mono repo.
>>
>>  - We have some policy on when to retire/delete projects, which can be
>> ironed out the first time it comes up (e.g. start with a nomination).
>>
>>  - We could even try to help encourage new projects to include a ‘mentor’
>> that has experience with the LLVM project to help nudge things in the right
>> direction and encourage proper development approach.
>>
>> What do you think?  Is anyone interested in helping to write up a more
>> detailed proposal?
>>
>> -Chris
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200622/f7fa4830/attachment-0001.html>