[cfe-dev] [LLVMdev] [3.6 Release] RC3 has been tagged

[Jack Howarth]

On Wed, Feb 18, 2015 at 4:12 PM, Renato Golin <renato.golin at linaro.org> wrote:
> On 18 February 2015 at 20:59, Tom Honermann <thonermann at coverity.com> wrote:
>> My company also does not use ToT.  Being able to associate a product with a
>> well-known release is *very* important to us.  It enables communication with
>> our customers.  It allows us to determine compatibility between our Clang
>> derived product and other Clang derived products.  It allows us to easily
>> discuss groups of feature sets.
>
> It's good to know that more and more people are actually interested in
> the releases. Am I right to assume that you test the release
> candidates as we put them forward with your own products? It'd be good
> to get that extra level of testing during releases.
>
> After all, a release is only as good as the people testing it. If it
> works for you, it's good for you, and so on. If there are enough
> people testing and reporting bugs, we'll end up with a very stable
> product, even if we don't realise at a first glance.
>
>
>> I personally find it rather crazy that developers are expected to use ToT
>> (and then determine stability on their own).  Why have an RC series at all
>> if that is the case?
>
> Core developers tend to use ToT. External users tend to use stable
> commits (you can check the already existing buildbots for that) or
> releases + cherry-pick. No one I know uses pure releases and expect
> them to be perfect.
>
> The RC series is to fix the urgent bugs so we can call the release
> minimally stable. Right now it is, and the only serious problem I know
> is the multiple regressions in performance in that phoronix run. I
> can't vouch for those numbers and not many people cared about that to
> make it a blocker, so all in all, the release is "good".
>
> However, it's not perfect, and that's why we started doing
> dot-releases two years ago and why I still think it's worth.
>
> I can only see two solutions to external users that rely on releases
> being always better than the previous:
>
> 1. Test the full release, report all problems, wait for dot-release 1, re-base.
>
> 2. Be an active part in the release process, fix bugs (not just report
> it), keeping in mind that we normally don't spend more than a month
> during a release cycle.
>
> cheers,
> --renato

Renato,
      Do we have any buildbots that routinely run the commercial SPEC
benchmarks and flag run-time regressions for trunk and the release
branches? I believe FSF gcc has that in their testing scheme.
                 Jack