[LLVMdev] RFC: Staging area proposal for new backends

[Tom Stellard]

Hi,

I would like to follow up on the recent discussion on the mailing list
about requirements for new backends[1] by submitting the following
proposal for a staging area for new LLVM backends.  This proposal
incorporates ideas from Owen, Chandler, and others who chimed in on
the original thread, and I hope the LLVM developers will be able to
come to a consensus on this proposal or a modified version, so the
project is able to accept new backends.

The goals of the staging area will be:
  1. Facilitate communication between the LLVM project and backend
     developers
  2. Ensure that new backends meet LLVM standards
  3. Give the backend more exposure to users and prospective developers

++ Staging Area:

Similar to the Linux kernel, the staging area for new backends will
be in the main LLVM tree, with building of the backend being disabled
by default.  There will also be a TODO file in the backend's root
directory, that contains a list of improvements that are required to
promote the backend out of the staging area.  The backend will be
assigned a steward who's role will be to guide the backend through
the staging process and help solicit feedback from other developers.

There are several advantages to having the staging area be in the main
tree as opposed to a separate branch:

1. It will be easier for LLVM developers to become familiar with the
   new backend and identify areas for improvement.

  If the new backend is in the main tree, LLVM developers are more
  likely to encounter it in their day to day development.  Imagine a
  scenario where a developer makes a change to LLVM core that impacts
  several backends.  The developer may grep the code looking for
  backends that make use of the feature that they have added or
  changed.  If the new backend is in tree and uses that feature,
  the developer will see the code and might take a few moments to
  read through.  While doing this the developer may notice an area for
  improvement for the backend and can update the backends's TODO file.
  The end result of this is that the LLVM developer has been able to
  provide some feedback with a minimal time commitment on their part.

  If the backend were staged in a separate tree, this kind of
  simple review would not be possible, and I would be concerned that
  developers would be too busy to ever get around to checking out
  the staging tree.

2. It will allow the backend developers to always develop against TOT.

  Developing against TOT is the recommended development procedure for
  anyone working on LLVM, and this is regularly reiterated on the
  mailing list.  If the new backend is included in the main tree,
  the backend developers will have no choice but to work against TOT.

3. It will make it easier for end users and distributions to test and
   also make it easier for new contributors.

  New backends will be more visible to the public if they are in the
  main tree.  This will mean more users, an expanded testing base, and
  more potential developers which will lead to a higher quality backend.


++ Promotion/Demotion from staging area:

  After a period of time, or when the tasks in the TODO file have been
  completed, the backend developers or the steward can initiate the
  review process.  The review process will be conducted by either the
  steward, a committee, or some select developers, who will decide
  (maybe by vote in the case of a committee) whether the backend
  should be:

  - Promoted = Build of backend will be enabled by default.
  - Extended = Backend remains in the staging area.
  - Demoted  = Removed from the main tree
    (I can't really think of any disadvantages to having a backend be
    in the main tree as long as its not being built by default, so maybe
    demotion would be reserved for cases of long term absence
    of maintainership)

  The Promoted/Extended/Demoted decision will be made using the
  following criteria (These won't necessarily all be absolutely
  required, they merely serve as a way for a backend's progress to
  be measured) :

  - Progress towards completion of TODO tasks
  - Active maintainership
  - Use of incremental development techniques
  - Adherence to LLVM coding style
  - Usage of modern LLVM features
  - Quality and quantity of regression tests
  - Availability of buildbots
  - Size of user base
  - Other criteria deemed important by LLVM developers

  - Contributions to core LLVM
    In the previous mailing list discussions there were differing
    opinions of how important contributing to the core LLVM code is
    for having a backend accepted.  It seems like a good middle ground
    would be that backends should be free of code that works around
    bugs or deficiency in core LLVM and instead fix the problem in
    shared code, and also should make an effort to push optimization
    passes that may be useful to other targets into the shared parts
    of the code.


++ What is needed from the LLVM developers:

  In order to make this staging program successful, the LLVM project
  will need to appoint a "code owner" for the staging process, who
  backend developers can contact when they are interested in getting
  the backend included in the main tree.  An LLVM developer will also
  be needed to act as a steward for the new backend and help guide
  the backend developers through the process.

Looking forward to comments on this proposal.

Thanks,
Tom Stellard

[1] http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20120716/146560.html