[llvm-dev] Upgrading LLVM's minimum required CMake version

Thu Apr 9 15:22:14 PDT 2020

Currently, it’s not hard to just invoke CMake correctly. I’m reluctant to have a wrapper script because it encourages people to change this status quo. “I can add this complicated pre-step, I’ll just update setup.sh” Then they do so, but forget to update setup.bat, or update it incorrectly and the whole thing gets out of sync. Additionally, there are issues with corner cases. What does setup.sh do on Cygwin? MSYS? git-bash? Powershell? cmd.exe? Does setup.bat do the same thing on all of these environments? How about on Windows 8? How about on Windows on ARM? Does setup.sh even work on Linux on shells other than bash? Finally, there’s the issue of customization. How much does setup.bat let me customize? Does it just generate a Ninja project for me? What if I want a Visual Studio solution?

The current system makes all of this the user’s problem. This is feasible because it’s not really that hard to build LLVM.

From: Alexandre Ganea <alexandre.ganea at ubisoft.com>
Sent: Thursday, April 9, 2020 2:56 PM
To: Shoaib Meenai <smeenai at fb.com>; Reid Kleckner <rnk at google.com>; James Y Knight <jyknight at google.com>; LLVM Dev <llvm-dev at lists.llvm.org>; Chris Tetreault <ctetreau at quicinc.com>
Subject: [EXT] RE: [llvm-dev] Upgrading LLVM's minimum required CMake version

Sorry if this was discussed,

Why not have a setup.sh/setup.bat/setup.exe which could download/install/setup everything we want, at the version we want, on a vanilla system? Handle the cases described by Chris below? Why leave the burden to the end-user?

Currently, it’s not exactly trivial to setup everything for building & running LLVM on Windows on a cloud VM if you want `ninja check-all` to pass on -DLLVM_ENABLE_PROJECTS=llvm;mlir;clang;lld;clang-tools-extra;compiler-rt;lldb. There are many manuals steps, and often things that you forget (GnuWin32 FTW). On Ubuntu it is a bit easier, but still lots of trial and error.

We use NuGet<https://docs.microsoft.com/en-us/nuget/install-nuget-client-tools> packages in our build system and for our developers, to ensure they always have the right setup. Our games always build and use the toolings from the .nugets, not the default installations on the machine. This guarantees universal determinism everywhere and makes the developer setup a one-click-exe. Setting up a similar thing for LLVM for different OSes could be a bit more tricky, but nothing insurmountable?

De : llvm-dev <llvm-dev-bounces at lists.llvm.org<mailto:llvm-dev-bounces at lists.llvm.org>> De la part de Shoaib Meenai via llvm-dev
Envoyé : April 9, 2020 4:26 PM
À : Reid Kleckner <rnk at google.com<mailto:rnk at google.com>>; James Y Knight <jyknight at google.com<mailto:jyknight at google.com>>
Cc : llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
Objet : Re: [llvm-dev] Upgrading LLVM's minimum required CMake version

I agree that’s valuable, but then it’s also important to pin down exactly what a “modern OS” is, and which ones we should keep in mind when we’re considering e.g. which CMake versions are feasible. (The same applies even more so to toolchain requirements, of course.)

From: llvm-dev <llvm-dev-bounces at lists.llvm.org<mailto:llvm-dev-bounces at lists.llvm.org>> on behalf of Reid Kleckner via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>>
Reply-To: Reid Kleckner <rnk at google.com<mailto:rnk at google.com>>
Date: Thursday, April 9, 2020 at 1:20 PM
To: James Y Knight <jyknight at google.com<mailto:jyknight at google.com>>
Cc: "llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>" <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>>
Subject: Re: [llvm-dev] Upgrading LLVM's minimum required CMake version

I would add my voice to Chris's: building out of the box on standard distros is a valuable feature. I don't have time to really participate in this discussion, but I'd discourage us from adding any more steps at all to the LLVM getting started document.

When I started working on compilers, it was important that I could build LLVM on the system and hardware that I had. That was a meaningful differentiating advantage over GCC, which sent me off on a side quest to check out two unfamiliar arbitrary precision math libraries, and asked me if I wanted to do a two-stage bootstrap. Forget that. From a modern OS, building LLVM should be as simple as:
- Install a standard C++ toolchain
- Clone source
- Paste a standard configuration command
- make

If we get too far away from that, we've lost something.

On Wed, Apr 8, 2020 at 2:49 PM James Y Knight via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
I am strongly in favor of increasing the minimum CMake version.

I think we should NOT be waiting for the next LLVM release to do so, but should do so as soon as practical (e.g. maybe a month from now). A warning message emitted in CMake spam is not likely to help users very much, IMO. An entry in the release notes saying "This version of LLVM now requires CMake X.Y.Z." along with a link to the completely-trivial instructions on how to get, build, and use a new version for building LLVM (WITHOUT having to install it!), should both suffice -- and is more likely to be useful.

I do NOT think we should have a policy tying ourselves to versions supported by certain LTS releases. Users of such LTSes can download a newer cmake. We should, of course, strive to not be unnecessarily annoying -- even though the amount of work to download a new version is small, it's not zero. We should _consider_ the versions that developers are likely to have pre-installed on their machines, and drop support for those versions only when the new features are judged compelling enough to offset the cost (small-pain-per-developer times number-of-deveopers-affected).

I agree with others who say we should only upgrade when would be truly valuable -- not automatically just because a new version exists. We should avoid changing the minimum-requirement too often.

FInally, I think we should put the decision of when such an upgrade is judged valuable into the hands of those who are spending the most time working on the build system. We do not need to redo this discussion every time.

On Wed, Apr 8, 2020 at 4:36 PM Chris Tetreault via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
We should decide who we consider the major distros to be. In my mind this is Ubuntu, Debian, Fedora, and CentOS/RHEL. We should also consider Visual Studio releases and whatever OSX and the major BSD’s have. (I honestly have no idea so I’ll refrain from speculating)

For all of these, we should try very hard to support the most recent LTS. For the previous LTS, it would be nice if we could support it, but we shouldn’t require it. Old LTSs tend to have really out of date packages, especially in times like now with Ubuntu where we’re really close to the current LTS becoming the old LTS.

That said, if CMake version X has a killer feature that we need, we should consider upgrading even if it doesn’t fit this criteria. Similarly, we should not just upgrade because the minimum bound of CMake versions supported by this set of OSs increased. The point is to upgrade only when there’s a compelling reason, and this set of OSs is just a heuristic of “most people probably already have this CMake version.”

From: Shoaib Meenai <smeenai at fb.com<mailto:smeenai at fb.com>>
Sent: Wednesday, April 8, 2020 12:58 PM
To: Chris Tetreault <ctetreau at quicinc.com<mailto:ctetreau at quicinc.com>>; Eric Christopher <echristo at gmail.com<mailto:echristo at gmail.com>>
Cc: llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
Subject: [EXT] Re: [llvm-dev] Upgrading LLVM's minimum required CMake version

Yeah, I don’t anticipate Windows posing problems. Also, it’s pretty common in Windows to just install software yourself, and CMake ships prebuilt binaries and an installer, so it’s pretty easy to get set up with it.

Chris, I’m gonna reiterate a question of mine from an earlier email, since you may have thoughts on it:

* If we want to limit ourselves to CMake versions supported by LTS releases of distros, which distros should we consider, and how far back should we go (i.e. is it just the latest LTS or the last two LTS versions)?

From: llvm-dev <llvm-dev-bounces at lists.llvm.org<mailto:llvm-dev-bounces at lists.llvm.org>> on behalf of Chris Tetreault via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>>
Reply-To: Chris Tetreault <ctetreau at quicinc.com<mailto:ctetreau at quicinc.com>>
Date: Wednesday, April 8, 2020 at 12:51 PM
To: Eric Christopher <echristo at gmail.com<mailto:echristo at gmail.com>>
Cc: "llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>" <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>>
Subject: Re: [llvm-dev] Upgrading LLVM's minimum required CMake version

Visual studio 2019 ships with CMake 3.15.5, which is pretty darn new IMO. From what I can tell, CMake versions are tied to visual studio releases. So assuming we go with “what do recent LTS distros have” as our metric, I think it’s reasonable to say “what do recent visual studio versions have”. It probably makes sense to confirm with MS though before we assume that this is the case.

From: Eric Christopher <echristo at gmail.com<mailto:echristo at gmail.com>>
Sent: Wednesday, April 8, 2020 12:41 PM
To: Chris Tetreault <ctetreau at quicinc.com<mailto:ctetreau at quicinc.com>>
Cc: Mehdi AMINI <joker.eph at gmail.com<mailto:joker.eph at gmail.com>>; Louis Dionne <ldionne at apple.com<mailto:ldionne at apple.com>>; llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
Subject: [EXT] Re: [llvm-dev] Upgrading LLVM's minimum required CMake version

Hi All,

Throwing a couple of comments in:

Chris's position here has a lot of good points and we want to make sure we're not raising the barrier too high. I definitely want to be able to push ahead with our versions of tools; being able to update quickly is one of the hallmarks of the llvm project. That said, binary packages that can be updated are a minimal first step IMO. I'd really like to not build anything from source :) It seems like there are binaries available for cmake for all of our current platforms, but the windows use case that he brings is definitely a significant one. Can we perhaps reach out and find out the likelihood of a reasonably soonish update there? Linux distros are probably less of a problem - while we all can't use ppas we should be able to do something, similarly with osx.

Thoughts?

-eric

On Wed, Apr 8, 2020 at 9:53 AM Chris Tetreault via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:

A line has to be drawn in the sand somewhere. How many “easy” things are we going to require the user to do? Today it’s build a specific CMake from source. What’s next?

Not having to manually track down a bunch of dependencies before building is a feature. Not having to have an internet connection at build time (if we were to script the getting of the custom CMake) is a feature. Being able to just call cmake instead of using some build_llvm.sh that (probably poorly) wraps cmake and downloads the correct version is a feature. Being able to use CMake that is distributed with visual studio so that invoking cmake from the developer powershell just works without fiddling with PATHs is a feature. Not having to install msys so that I can invoke download_cmake.sh is a feature. Not having to have the correct version of python (is it 2 or 3?) be on the path in order to invoke download_cmake.py is a feature. Not having to remember to do --recurse-submodules on the llvm repo if we include it as a git submodule is a feature. The list goes on. Yeah, these are all little things, but a bunch of little things adds up to a huge barrier.

People use Linux distos because by and large they just have all the dependencies that they need. I know I personally hate installing some open source thing on my machines when they have some dependency that’s not in the repos. Sure, it may be easy to build CMake from source. But now I have two CMakes: one that is automatically updated when I do sudo apt-get upgrade, and one that is just randomly in some folder that’s probably not on the PATH. I personally would really appreciate it if we made an attempt to reduce this sort of friction.

Thanks,
   Christopher Tetreault

From: llvm-dev <llvm-dev-bounces at lists.llvm.org<mailto:llvm-dev-bounces at lists.llvm.org>> On Behalf Of Mehdi AMINI via llvm-dev
Sent: Wednesday, April 8, 2020 9:06 AM
To: Louis Dionne <ldionne at apple.com<mailto:ldionne at apple.com>>
Cc: llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
Subject: [EXT] Re: [llvm-dev] Upgrading LLVM's minimum required CMake version

On Wed, Apr 8, 2020 at 9:02 AM Louis Dionne <ldionne at apple.com<mailto:ldionne at apple.com>> wrote:

On Apr 7, 2020, at 22:16, Mehdi AMINI via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:

On Tue, Apr 7, 2020 at 11:27 AM David Blaikie <dblaikie at gmail.com<mailto:dblaikie at gmail.com>> wrote:
I think it does make a difference how many things we ask new developers to do to get up and running - because we've asked them to do one thing doesn't mean it's low-cost to ask them to do another thing.

In this case I see it rather that if we ask them to do one quite big thing already, we should be OK with what seems like a trivial one.

I strongly agree. I think Mehdi's point can be summarized as (Mehdi, feel free to correct me):

    It's incredibly trivial to install CMake, so if a user is *already* required to install a non-default toolchain (which is not so trivial), requiring them to install a non-default CMake is not increasing the barrier by much.

Thanks, this is my point indeed!

I think it is even slightly stronger than what you wrote since you don't even need to *install* CMake as it can be built and used directly from the build directory: it is entirely non-intrusive on the system.

--
Mehdi
_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.llvm.org_cgi-2Dbin_mailman_listinfo_llvm-2Ddev&d=DwMGaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=OUvi60kOTuMyRUJcCtHsN-RK1gHy4sXxEGS0pAunCoE&s=W77RObkJ6AlX4-NZ-OApzF80Y5rSjh4gDzuBG4ScjEQ&e=>
_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.llvm.org_cgi-2Dbin_mailman_listinfo_llvm-2Ddev&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=RvaD9Q_VY7Y2QSH5JAtJlUDjQZKLr1RSWDceq7JxvZE&s=SdeUoPXT_jrLA6F111g6osrDG6MW34YbjgUwfq1YSl0&e=>
_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.llvm.org_cgi-2Dbin_mailman_listinfo_llvm-2Ddev&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=RvaD9Q_VY7Y2QSH5JAtJlUDjQZKLr1RSWDceq7JxvZE&s=SdeUoPXT_jrLA6F111g6osrDG6MW34YbjgUwfq1YSl0&e=>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200409/3770b094/attachment-0001.html>