[llvm-dev] Should I worry about test failures in a release?

[Renato Golin via llvm-dev]

On 12 October 2015 at 16:29, Joachim Durchholz via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> I've been planning to switch to a new release whenever it's out.
> What are the advantages of following trunk?

Some people prefer following releases and having a higher cost when
migrating, other people prefer to spread the cost over many months,
but you'll have to resolve conflicts most of the time you re-base your
project regardless of what you choose.

If you stick to a release...

Pros:
 * ABI stability
 * Known functionality (ie. no experimental stuff)
 * Re-base can be done separately, and merged once stable
Cons:
 * Known problems are less likely to be fixed upstream (1)
 * Larger re-base periods, may invalidate a lot of your bridge code

If you stick to trunk...

Pros:
 * Faster re-base periods
 * Bugs are more likely to be fixed quickly
 * New features, target support will come earlier
 * You can influence/propose/submit changes upstream
Cons:
 * Unpredictable re-base. Sometimes, a re-base can destroy a lot of stuff.
 * You'll have to track buildbots to make sure all targets are green
on the revision you want to pull
 * You won't have the additional validation people do on releases
(larger code-bases, benchmarks, etc).
 * Re-bases are better done as a "stop the press" moment in the
project history, may delay other tasks

Ultimately, it depends on how you and your team work, and how you
expect the software you're building to evolve. The vast majority of
people stick to trunk.

cheers,
--renato

(1) The community is very focused on trunk, so problems fixed on trunk
may not be easily back-ported. You may want to do the back-port
yourself, but that still doesn't mean people will accept it on the
release branch, for example, due to potential instability concerns.
Releases are more "stable points in time", than proper products.