[cfe-dev] LLVM Releases: Upstream vs. Downstream / Distros

[David Chisnall via cfe-dev]

On 12 May 2016, at 16:27, Bernhard Rosenkränzer via cfe-dev <cfe-dev at lists.llvm.org> wrote:
> 
> On 12 May 2016 at 16:56, Kristof Beyls <Kristof.Beyls at arm.com> wrote:
>> In my opinion, it would be better overall for the LLVM project if top-of-trunk is
>> tested as much as possible, if testing resources are so scarce that a choice
>> has to be made between testing top-of-trunk or testing a release branch.
>> 
> I agree that trunk is more important, with both of my hats on.
> 
> But releases are not completely irrelevant - one thing making them important is the fact that there's other projects out there using the LLVM libraries - and as a distro, we have to make sure they all work (so they agree on the same API), preferably without having to ship multiple versions of LLVM and preferably without having to patch external code too much to adjust to API changes.
> 
> In OpenMandriva, we have to keep Mesa, creduce, emscripten and the LLVMified Qt moc working (list expected to grow -- ultimately we'd also like to use the system LLVM libraries for the swift compiler).
> In AOSP, RenderScript relies on the LLVM API, but there's nothing else using it, so there's currently no need to force a common version of the API between different projects there.

I think that our API stability policy is really hurting us here.  Downstream users of clang have few problems - clang is basically backwards compatible and so you only need to test for regressions.  We periodically build the entire FreeBSD ports collection (around 25,000 open source packages) with the clang-devel port (which is a periodically updated trunk snapshot) and report regressions.  That’s very easy to do.  It’s easy for me to test for clang regressions by just setting CC=clang-devel CXX=clang++-devel, rebuilding my own projects and running their test suites.

In contrast, for projects that I maintain that use LLVM as a library, I generally hop from release to release.  We’ve had several instances of APIs changing multiple times between releases, so updating the code to each one is more effort than just updating to the newest one.  APIs often go in long before documentation (which is often nonexistent or rushed in for the release), so you’re more likely to see useful documentation if you pick a release.  Finally, most distributions package releases so if you depend on LLVM X.Y then your code is easy to package, whereas if you depend on LLVM svn rxzy then it’s basically impossible.  Even between releases, expecting everyone who might use / test your project’s trunk to build the specific LLVM revision that you depend on is too much of a barrier to entry for many potential contributors and so following trunk reduces the number of contributors and the amount of testing.

The end result is that shortly after a release (sometimes every alternate release) is branched a load of downstream projects update to the new APIs, test things, and find a bunch of regressions that have been sitting in the tree for months.  We then have to scrabble to bisect and try to track them down.

TL;DR version: If we want downstream people to test ToT, then we need to make updating LLVM library consumers to ToT far less painful than it is now.

David


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3719 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20160512/4138a442/attachment.bin>