[llvm-dev] LLVM Releases: Upstream vs. Downstream / Distros

Wed May 11 15:47:19 PDT 2016

Hi,
for those who don't know me, I'm an AOSP developer at work and an
OpenMandriva developer (including maintainer of its toolchains) outside of
work, so I'm looking at this from (at least) 2 different perspectives and
may jump around between them.
Replies inline...

On 11 May 2016 at 16:08, Renato Golin <renato.golin at linaro.org> wrote:

> There has been enough discussion about keeping development downstream
> and how painful that is. Even more so, I think we all agree, is having
> downstream releases while tracking upstream releases, trunk and other
> branches (ex. Android).
>

In the OpenMandriva world, we usually try to have clang (our primary
compiler) as close as possible to the latest upstream stable release.
We're currently following the release_38 branch, and expect to jump on
trunk as soon as our distro release has happened (because we expect 3.9 to
be ready before we'll make our subsequent release - better to get on the
branch we'll be using for the next release early than to suddenly face
problems when updating to the next release).

In the AOSP world, we obviously have to (somewhat) follow what Google does,
which is typically pick a trunk snapshot and work from there - but we have
some work underway to extract their patches so we can apply them on top of
a release or snapshot of our choice (current thought is mostly nightly
builds for testing).

This work involves a *lot* of premises that are not encoded yet, so
> we'll need a lot of work from all of us. But from the recent problems
> with GCC abi_tag and the arduous job of downstream release managers to
> know which patches to pick, I think there has been a lot of wasted
> effort by everyone, and that generates stress, conflicts, etc.
>

>From both perspectives, it would be great to have a common set of "known
good" and relevant patches like gcc abi_tag, or fixes for bugs commonly
encountered.
Ideally, I'd like to see those patches just backported on the release_38
branch to keep the number of external patches low.

gcc abi_tag is a bit of a headache in the OpenMandriva world, while we
build just about everything with clang these days, of course it would be
good to restore binary compatibility with the bigger distributions (almost
all of which are using current gcc with the new abi enabled).

Many downstream release managers, as well as distro maintainers have
> complained about the timing of our releases,

The timing has been quite predictable lately -- but of course the website
still says "TBD" for both 3.8.1 and 3.9.0, maybe communicating the (likely)
plan could use some improvement.

>  * Do the same average on the projects that are willing to lend a
> serious hand to the upstream release process.

What can we (this time being OpenMandriva) do? We don't have any great
compiler engineers, but we're heavy users - would it help to run a mass
build of all packages for all supported architectures (at this time: i586,
x86_64, armv7hnl, aarch64) to detect errors on a prerelease builds? We have
the infrastructure in place, even showing a fairly nice list of failed
builds along with build logs. (But of course we there will be false
positives caused by e.g. a library update that happened around the same
time as the compiler update.)

 * Try to release more often. The current cost of a release is high,
> but if we managed to lower it (by having more people, more automation,
> shared efforts), than it could be feasible and much fairer than
> weighted averages.
>

That would be a good idea IMO, we've run into "current trunk is much better
than the last stable release anyway" situations more than once (in both
projects).

> For example, we have no defined date to start, or to end. We have no
> assigned people to do the official releases, or test the supported
> targets. We still rely on voluntary work from all parties. That's ok
> when the release is just "a point in time", but if downstream releases
> and OS distributions start relying on our releases, we really should
> get a bit more professional.
>

Backporting some more fixes to the stable branches would be great too (but
of course I realize that's a daunting and not very interesting task).

>  * Downstream managers should be an integral part of the upstream
> release process. Whenever the release manager sends the email, they
> should test on their end and reply with GREEN/RED flags.
>  * Downstream managers should also propose back-ports that are
> important to them in the upstream release. It's likely that a fix is
> important to a number of downstream releases but not many people
> upstream (since we're all using trunk). So, unless they tell us, we
> won't know.
>

Sounds good to me, volunteering to participate in both.

 * We could have the scripts that distros use for building their own
> packages in our tree, so they could maintain them locally and we'd
> know which changes are happening and would be much easier to warn the
> others, common up the interface, etc.
>

While interesting from an upstream perspective, I doubt that will happen
reliably -- there's too many people working on the build scripts who would
not automatically have write access to the tree etc. and most distro build
farms rely on having the build scripts in a common place, so duplication
would be unavoidable.

ttyl
bero
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160512/4504b957/attachment.html>