[llvm-dev] LLVM Releases: Upstream vs. Downstream / Distros

[Renato Golin via llvm-dev]

Thanks Bero! Comments inline.


On 11 May 2016 at 23:47, Bernhard Rosenkränzer
<bernhard.rosenkranzer at linaro.org> wrote:
> In the OpenMandriva world, we usually try to have clang (our primary
> compiler) as close as possible to the latest upstream stable release.

It would be good to know what's stopping you from using a fully
upstream process (like patch trunk, back-port, pull).

I'm sure you're not the only one having the same problems, and even if
the only take-out of this thread is to streamline our back-port
process, all downstream releases will already benefit.



> We're currently following the release_38 branch, and expect to jump on trunk
> as soon as our distro release has happened (because we expect 3.9 to be
> ready before we'll make our subsequent release - better to get on the branch
> we'll be using for the next release early than to suddenly face problems
> when updating to the next release).

That's a very good strategy, indeed. And somewhat independent on how
good the release is.

Early help can also increase downstream participation pre-release, and
will eventually considerably reduce the need to back-ports.



> In the AOSP world, we obviously have to (somewhat) follow what Google does,
> which is typically pick a trunk snapshot and work from there - but we have
> some work underway to extract their patches so we can apply them on top of a
> release or snapshot of our choice (current thought is mostly nightly builds
> for testing).

This sounds awful, and is the very thing I'm trying to minimise. I can
certainly understand the need to local patches, but using different
trunks makes the whole thing very messy.

If the releases are good and timely, and if the back-porting process
is efficient, I hope we'll never have the need to do that.


> From both perspectives, it would be great to have a common set of "known
> good" and relevant patches like gcc abi_tag, or fixes for bugs commonly
> encountered.
> Ideally, I'd like to see those patches just backported on the release_38
> branch to keep the number of external patches low.

Indeed, my point exactly. Downstream releases and distros can easily
share sets of known "good" patches for this or that, but I'd very much
prefer to have as much as possible upstream.


> gcc abi_tag is a bit of a headache in the OpenMandriva world, while we build
> just about everything with clang these days, of course it would be good to
> restore binary compatibility with the bigger distributions (almost all of
> which are using current gcc with the new abi enabled).

Ubuntu LTS has just released with LLVM 3.8, which doesn't have the
abi_tag fixes.

If we don't back-port them to 3.8.1 or 3.8.2, Ubuntu will have to do
that on their own, and that's exactly what I'm trying to solve here.

I also feel like they're not the only one with that problem...



> The timing has been quite predictable lately -- but of course the website
> still says "TBD" for both 3.8.1 and 3.9.0, maybe communicating the (likely)
> plan could use some improvement.

Right, Hans was saying how we should improve in that area, too.

I think that's a very easy consensus to reach, but we still need all
require people to commit to a more rigorous schedule.


> What can we (this time being OpenMandriva) do? We don't have any great
> compiler engineers, but we're heavy users - would it help to run a mass
> build of all packages for all supported architectures (at this time: i586,
> x86_64, armv7hnl, aarch64) to detect errors on a prerelease builds?

YES PLEASE!! :)


> We have
> the infrastructure in place, even showing a fairly nice list of failed
> builds along with build logs. (But of course we there will be false
> positives caused by e.g. a library update that happened around the same time
> as the compiler update.)

Can you compare the failures of two different builds?

We don't want to fix *all* the problems during the releases, we just
want to know if the new release breaks more stuff than the previous.

How we deal with the remaining bugs is irrelevant to this discussion...


> That would be a good idea IMO, we've run into "current trunk is much better
> than the last stable release anyway" situations more than once (in both
> projects).

I expect that, if distros chime in during the release process, this
will be a lot less of a problem.

It also seems that the timing is not that bad, so maybe the best
course of action now is to streamline the process and only if the
pressure is still great, we change the timings.


> Backporting some more fixes to the stable branches would be great too (but
> of course I realize that's a daunting and not very interesting task).

I think that's crucial to keeping the releases relevant.


> Sounds good to me, volunteering to participate in both.

Thanks Bero! If you haven't yet, please subscribe to the
release-testers at lists.llvm.org mailing list.


> While interesting from an upstream perspective, I doubt that will happen
> reliably -- there's too many people working on the build scripts who would
> not automatically have write access to the tree etc. and most distro build
> farms rely on having the build scripts in a common place, so duplication
> would be unavoidable.

I was sceptical about the shared scripts, too. It was an idea that
came to my mind in the last minute, but I'm not sure how much that
would help anyway.

cheers,
--renato