[cfe-dev] [llvm-dev] Fwd: Raising CMake minimum version to 3.4.3

Wed Apr 27 11:44:36 PDT 2016

> On Apr 27, 2016, at 11:17 AM, Renato Golin <renato.golin at linaro.org> wrote:
> 
> On 27 April 2016 at 18:41, Chris Bieneman <beanz at apple.com> wrote:
>> Renato, in your most recent email you comment about differentiating *stable* vs random versions of CMake built from source. I believe as a community our recommendation should be that people download CMake sources from https://CMake.org/download/ instead of git. Then they get the source of a known stable release. If people choose to pull random git hashes or tags, that is their decision, but since the minimum version I'm proposing is widely available I think it will be uncommon.
> 
> I'm guilty of cloning the git repo on my local tree, because it's
> easier to do that than to find the link, download, untar, clean up,
> etc. Then there's a bug, and you need to upgrade, "git pull" is much
> simpler than repeating the whole process again. :)
> 
> Also, recent "stable", as Chuck has shown, is less stable than
> previous stable. I'd like to err in the side of caution.
> 
> 
>> I've been working to try and un-do the horrible Darwin approach we have because I think it has some serious problems, but an alternative solution would be to extend the Darwin approach to every other platform. If you do this you don't need ExternalProject, but you do need to do some really dirty hacks. If you want to understand those hacks, just look at CompilerRTDarwinUtils.cmake in compiler-rt. That is a bit of evil that I’m responsible for, and I’ll be paying down that debt for a long time to come.
> 
> Point taken, it is horrible. (sorry)
> 
> 
>> Apart from this we also have another problem. The build system makes configuration decisions based on the compiler *at configuration time*. The problem is you really want to configure compiler-rt (and the other runtimes) *after* you've already built clang so the runtimes are configured and built with the clang you just built. The only way to do that is with some mechanism similar to ExternalProject.
> 
> <devil's advocate>We could come up with a set of rules that makes it
> possible (not preferable) to do so at config time.</da>
> 
> Is that where the current complication comes from? What makes it so
> horrible and ends up as a home brew version of ExternalProject?

Recent CMakes have a few interesting things for us in ExternalProject.

CMake 3.1 added BUILD_ALWAYS (2a93843) which allows the build target of a sub-project to always be called. This is really important because when you use external projects the top-level build file doesn’t have the dependency information for the sub-projects. So if you change sources in compiler-rt it won’t know to rebuild compiler-rt. You need to force it to always invoke ninja in the sub-project so that it will pick up changes.

Unfortunately even with CMake 3.1 BUILD_ALWAYS doesn’t actually work for Ninja because of a bug that was fixed in CMake 3.4.1 (b6f4717).

I’m suggesting the version 3.4.3 because it is the last 3.4.x build, and the CMake community is really strict about what gets put into patch releases.

> 
> 
>> We could roll our own. At this point I'm going to put on my "non-constructive reality" hat. I'm not going to do that.
> 
> Absolutely not. Ten shots to the head before we start rolling thing
> our own that are perfectly covered by a tool we already use.
> 
> 
>> Replicating ExternalProject would be a lot of work...
> 
> Just to be clear, I didn't mean that. If you're saying that, to get
> the desired functionality we'll either need ExternalProject or
> something identical to it, then the choice is clear. I was asking if
> there was another (good, but less good) solution to the problem.
> 
> Rolling our own is not even a solution, much less a good one.

It isn’t necessarily something identical to it. ExternalProject isn’t actually designed for this use, and it has loads of functionality we don’t need.

> 
> 
>> As a data point, the one place in LLVM where we don't use ExternalProject (and we should be) is the cross-compiling goop, and it is a gigantic mess (see: CrossCompile.cmake and every mention of LLVM_NATIVE_BUILD in TableGen.cmake). It is filled with bugs and in desperate need of disentangling. This is another one of my sins that I need to repent.
> 
> Oh, so ExternalProject would also fix cross compilation!? I'm sold! :)

And hopefully we can use it to make the optimized tablegen option more robust too.

> 
> 
>> I feel I should also point out that ExternalProject isn't a silver bullet. It isn't going to solve all our problems, and since it is one of the more actively changing parts of CMake we may find ourselves re-visiting this conversation in a year or two talking about a newer version of CMake.
> 
> Right, this is an interesting point. I don't want to do this every
> year, so let's be as reasonable as we can.
> 
> Is there anything in 3.5 that can really save us a lot of effort in
> the current work?

I don’t believe so. I’ve read the release notes (https://cmake.org/cmake/help/v3.5/release/3.5.html <https://cmake.org/cmake/help/v3.5/release/3.5.html>), and I don’t think there is anything really important to us in them. I am interested in a few things being discussed on the cmake-developers list though (http://public.kitware.com/pipermail/cmake-developers/2016-January/027370.html <http://public.kitware.com/pipermail/cmake-developers/2016-January/027370.html>). It looks like that might make CMake 3.6, and it could be interesting for us. That said, this is likely a case where we could easily support the old and new versions without much problem and it doesn’t impact build correctness.

> 
> I'm now using CMake 3.2.2 on all buildbots. If I have to move it up to
> 3.4.3, I'm fairly confident that there will be absolutely no problem
> with it.
> 
> But as Chuck said, there were some crashes on 3.5.2 that needed 3.5.3,
> and by using it on so many different platforms, we may uncover bugs,
> and may have to roll back to 3.4 temporarily, and then forwards to
> 3.5, etc.

This is a big part of why I’m suggesting 3.4.3 instead of 3.5.0 or 3.5.1 which are also widely available.

> 
> I really want to avoid that. But I also want to avoid doing this over
> again next year.

While I don’t want to advocate for us updating frequently “just because”, I do think that if there are compelling reasons to update every year or two, we shouldn’t be afraid to.

-Chris

> 
> Finally, Linaro cares more about Linux on ARM (v7+) than bare-metal,
> but we can't just break the rest. I know there are people out there
> that have infrastructure to test that kind of restricted environment,
> not only on ARM, but also MIPS and old PPC32 stuff. I think they
> should voice their concerns before we take any harsh decision.
> 
> cheers,
> --renato

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20160427/45b39cc4/attachment.html>