[LLVMdev] RFC: Bug fix releases for 3.3 and beyond

[Bill Wendling]

On Apr 2, 2013, at 2:10 PM, Tom Stellard <tom at stellard.net> wrote:

> On Tue, Apr 02, 2013 at 11:52:09AM -0700, Bill Wendling wrote:
>> On Apr 2, 2013, at 9:51 AM, Tom Stellard <tom at stellard.net> wrote:
>> 
>> As Chris said, the only thing preventing this is manpower. But if there are people ready and willing to do this, then I don't see it as a Bad Thing.
>> 
>> My first comment is that these should be strictly bug fix releases. So your (1) above wouldn't include changes other than bug fixes. There would need to be a hierarchy for bugs. The reason is that *any* change to the stable branch inherently carries risk. So for instance, a bug that's a crasher, but would affect only a small number of people may not be worthy of inclusion into a dot-release. Etc.
>> 
> 
> I was thinking for shared code it would be strictly bug fix only, but
> maybe things like additional C API implementations or standalone passes
> might be OK too assuming there is interest.
> 
> However, for target specific code, I think backend code owners should
> have a little more leeway as far as what gets accepted.  For the
> R600 backend I can see a situation where I may want to backport a new
> feature or something else, so it would be nice to have a little more
> flexibility and maybe other backend owners would want to have this
> option as well.

If we are going to go this route, I would be *very* selective about taking non-bug fixes. Target specific enhancements to our non-"tier 1" platforms ("tier 1" being x86 and ARM) would depend upon that back-end owner's judgement. However for the "tier 1" platforms, I will continue to be very skeptical. Adding new C APIs is challenging, because once we release with a C API it's there forever. Of course, there will be a suitable amount of bake time, etc.

Basically, I will be very skeptical about any changes that will go into a dot-release. The two questions, which I would want answered, are "why is this change important enough to warrant adding it to a new release?" and "how safe is this change?".

>> My second comment is that top-of-tree moves very fast. This can make changes hard to back-port to a stable branch that may be months old. You are stuck having to either do it yourself, or begging the original author to back-port the fix. :-)
> 
> I understand.  I think if there is a patch that is too difficult for the
> release maintainer to backport he/she should be able to go to the author
> and request help.  If the author or some other party doesn't care enough
> about the fix to help backport it, then it probably isn't important
> enough to go to stable.
> 
Fair enough. (And that's exactly what should happen. :-) )

>> How frequently do you expect to do a dot-release? Our major release schedule is roughly every 6 months. Do you think bimonthly would be too many? or would you do it only when enough changes require it?
> 
> This is hard to say, but probably as required.
> 
>> Finally, there is testing to consider. Obviously, full testing would be too rough on the community; people simply don't have enough time to spend a full month testing a dot-release. What are you thoughts on how testing would proceed?
> 
> What is it about testing for releases that takes a month to complete?
> Is it just coordinating with all the interested parties and making sure
> they are happy with the state of the code?
> 
Pretty much. Basically, we allow for a two week test and fix sequence. The testers are given one week to test the newest release candidate. We could give them a shorter amount of time, but that may put too much pressure on our volunteers. (In actuality, most testers are finished quickly.) We post the binaries they build so that the community can test on their software. We then give developers a week to address any issues that came up during the testing phase. This cycle is repeated once (twice if something disastrous comes up in the second cycle).

Two weeks is more or less an arbitrary choice, but one which allows us ample time to make sure that the release is as good as can be.

That said, there are a couple of options that we could consider:

1) Shorten the release schedule for a dot-release.

  - Considering that we are working with a known good base, we could reasonably expect fewer, if any, regressions. So we wouldn't have to worry as much about allocating so much time for fixing bugs. Plus, those who created the bad patch should be able to fix it fairly quickly.

2) Don't test quite as much, and tag the dot-release as "beta" or some other weasel wordy thing.

  - I'm not a big fan of this (even though I proposed it). I want external developers to be confident with the releases we send out.

I'm sure there are other choices here. But I think that the greatest hurdle we will come across is finding the resources to do the testing.

-bw