[cfe-dev] LLVM Releases: Upstream vs. Downstream / Distros

[Renato Golin via cfe-dev]

On 12 May 2016 at 01:10, Robinson, Paul <paul.robinson at sony.com> wrote:
> A public commitment to the future release schedule would be that much
> more justification for planning to participate.

Indeed, I think we're all in agreement there.

Not only having people committed to validate the release, or having a
"known" ballpark time, but a public commitment: web pages updates,
emails sent to the appropriate lists, deadlines exposed, etc.

For all those interested, release-testers at lists.llvm.org is the list
we discuss about the validation and plans. Regardless, the web pages
should also be changed as part of the planning and release processes.


> Data point: At Sony we ship our stuff every 6 months and that isn't
> going to change.  Our policy has been to base our releases on upstream
> releases, and given our current lead times, the current upstream release
> schedule is actually not bad, especially if it has stopped drifting.
> Having four releases pretty consistently follow the current schedule is
> extremely positive, thanks!

Excellent, thanks! It seems that the timing is (so far) less of a
problem than I anticipated. This is good news.


> It would help our internal planning to publish the schedule for future
> releases.  You never know, we might even be able to fork an internal
> branch in time to help with the release testing, although that is not
> in any way a lightweight process (so no promises).

I'm betting on the fact that this will get easier the more often we
do, and the more of us that does it.

Filtering false positives is a local cost that cannot be avoided, and
only automated to a point, but filtering thousands of real bugs and
reporting them all should be, in no way, the responsibility of any
downstream release.

The more downstream releases and OS distributions we have doing the
testing (like Bero said), the less each one of us will have to track.
And the more of them that say they're affected by a bug, the higher
the priority is should have to the upstream release.

We've done something similar with the GCC abi tag, but that was a
separate email thread, started by Jeff, and external to our release
process. The number of people involved is staggering, but yet, the
patches are sitting there waiting for review for a very long time.

I think this expresses my point really well. Downstream releases and
OS distros can help us *a lot* with validation, not necessarily
implementing core functionality or even fixing those bugs themselves.
But if we don't fix those bugs or implement the features they need,
they won't have *any* incentive in spending a lot of time and
resources validating the upstream release, and then, all of them will
spend *more* time validating their own.

Increasing the importance of stable releases might get us more work to
do for other people with no real benefit to us, yes. But it'll also
bring us a massive validation network and transform Clang/LLVM into a
production compiler from start (upstream) and that will benefit
everyone, including all users of all downstream and upstream tools.



> There has been some talk about moving toward releasing from upstream
> trunk, or more precisely to streamlining our internal testing process
> in order to allow us to seriously contemplate releasing from upstream
> trunk.  I can't argue with the streamlining part, for sure.  Whether
> the rest of it works out must remain to be seen.

That's what Chromium does, and it's mainly because of the problems
Bero exposed (trunk is often much better than any release).

But also they are using the tool to compile a very small subset of
programs (mainly Chromium/Chrome), so it's *a lot* easier to validate
that.

When you're actually shipping a toolchain, you have to worry not only
with the programs you have, but also customers programs you don't (and
can't) have access to.

If the release process (including minor releases) ends up as frequent
as possible, wouldn't that be similar to merging to trunk every other
month?

In that case, the validation process will be minimal (almost the same
code), but you'd have to spend a bit more time sifting through patches
(which I want to automate) instead.

Hope that makes some sense.

cheers,
--renato