[llvm-dev] Upgrading LLVM's minimum required CMake version

Tue Apr 7 11:27:17 PDT 2020

I think it does make a difference how many things we ask new developers to
do to get up and running - because we've asked them to do one thing doesn't
mean it's low-cost to ask them to do another thing.

On Tue, Apr 7, 2020 at 11:20 AM Mehdi AMINI via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

>
>
> On Tue, Apr 7, 2020 at 9:16 AM Chris Tetreault <ctetreau at quicinc.com>
> wrote:
>
>> > You're saying "doesn’t mean that we should" while I've been saying in
>> this situation that "we can", there is quite a difference here I believe.
>>
>>
>>
>> Technically “we can” do anything we want. We can always require that the
>> project be built with the current release candidate of CMake. That doesn’t
>> mean that we should.
>>
>
> I don't feel you're really addressing the point, so I'll phrased it in the
> other direction: is there a strong reason we should limit ourself to the
> supported CMake on a given distribution where we already can't build with
> the supported toolchain?
> I was saying I don't see why we should here: when we look at the list of
> distribution we can bootstrap with the supported packages, I don't see why
> make a difference here.
>
> --
> Mehdi
>
>
>
>>
>>
>> *From:* Mehdi AMINI <joker.eph at gmail.com>
>> *Sent:* Tuesday, April 7, 2020 12:01 AM
>> *To:* Chris Tetreault <ctetreau at quicinc.com>
>> *Cc:* Neil Nelson <nnelson at infowest.com>; llvm-dev at lists.llvm.org
>> *Subject:* [EXT] Re: [llvm-dev] Upgrading LLVM's minimum required CMake
>> version
>>
>>
>>
>>
>>
>>
>>
>> On Mon, Apr 6, 2020 at 12:48 PM Chris Tetreault <ctetreau at quicinc.com>
>> wrote:
>>
>> Every additional dependency that we force the user to manually install
>> (either by building from source, or adding some new PPA to their ubuntu
>> system), raises the barrier to entry that much higher.
>>
>>
>>
>> Right: I think it is important to quantify the "that much higher", I
>> believe that the commands I showed earlier (which can be put in a script in
>> the repo) is really as low as one can expect.
>>
>>
>>
>>
>>
>> Just because we may require the user to manually install some newer
>> compiler on their system doesn’t mean that we should also require them to
>> install some newer CMake than what’s on their system.
>>
>>
>>
>> You're saying "doesn’t mean that we should" while I've been saying in
>> this situation that "we can", there is quite a difference here I believe.
>>
>>
>>
>> --
>>
>> Mehdi
>>
>>
>>
>>
>>
>> (unless it is actually necessary, but we’ve beaten that dead horse long
>> enough)
>>
>>
>>
>> I agree with Neil’s interpretation that the definition of “supported”
>> being used here is default system packages and not third party repos that
>> claim to work for that system.
>>
>>
>>
>> *From:* llvm-dev <llvm-dev-bounces at lists.llvm.org> *On Behalf Of *Mehdi
>> AMINI via llvm-dev
>> *Sent:* Saturday, April 4, 2020 2:11 PM
>> *To:* Neil Nelson <nnelson at infowest.com>
>> *Cc:* llvm-dev <llvm-dev at lists.llvm.org>
>> *Subject:* [EXT] Re: [llvm-dev] Upgrading LLVM's minimum required CMake
>> version
>>
>>
>>
>>
>>
>>
>>
>> On Sat, Apr 4, 2020 at 12:48 PM Neil Nelson via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>> 'Supported' means that it comes from the packages available from the
>> distribution that can be seen via this page.
>>
>> https://packages.ubuntu.com/
>>
>> These packages have been processed by the Ubuntu community to obtain a
>> reliability expectation that would not apply, for example, to a PPA.
>>
>>
>>
>> Right, so I'm looking for an answer to my question, I'll try make it more
>> concrete what I mean by "in cases where we already requires a more recent
>> compiler than the default one available".
>>
>> If I take Xenial for instance, the most recent GCC version is 5.4.0 as
>> far as I can tell: https://packages.ubuntu.com/xenial/devel/gcc-5 ;
>> assuming LLVM would move at some point to require a more recent version
>> than 5.4, it would mean that you couldn't build LLVM with the packages
>> available on Xenial. In this situation (which I referred to as "cases where
>> we already requires a more recent compiler than the default one available")
>> we already expect the user to get a toolchain from a non-primary package
>> source on this distribution, and if we do this for the toolchain already I
>> would expect that we should be able to do it as well for CMake (again: for
>> a given distribution/version).
>>
>>
>>
>> The difference between installing or building Clang and LLVM from
>> original sources as against installing versions available from the
>> distribution
>>
>> I don't understand this sentence?
>>
>>
>>
>> when compared to doing the same with cmake, is that the user accepts the
>> inherent risks from Clang and LLVM, but Clang and LLVM can not accept the
>> risks from the cmake group and then expect the user to merely assume that
>> there are no additional risks from installing cmake.
>>
>> Maybe a nit here, but there is no need to *install* CMake: it could be
>> trivially build in the build directory. We are talking about a trivial step
>> here:
>>
>>
>>
>> # Inside llvm-project/
>>
>> $ mkdir build/ && cd build/
>>
>> # bootstrap CMake
>>
>> $ wget
>> https://github.com/Kitware/CMake/releases/download/v3.17.0/cmake-3.17.0.tar.gz
>>
>> $ echo "b74c05b55115eacc4fa2b77a814981dbda05cdc95a53e279fe16b7b272f00847
>>  cmake-3.17.0.tar.gz" | sha256sum -c
>>
>> $ tar -xf cmake-3.17.0.tar.gz && cd cmake-3.17.0 && ./bootstrap && make
>>
>> # Done, cmake is usable, *nothing* is installed on the user system,
>> everything is self-contained *inside* the build directory itself.
>>
>> $ ./cmake-3.17.0/bin/cmake ../llvm/ -D.... # build LLVM as usual.
>>
>>
>>
>>
>>
>>
>>
>> The distributions are not merely just collections of software, they are
>> collections of software that have some guarantee of working well together
>> and without bugs and other issues because they have been used and tested by
>> that use in the distribution community.
>>
>> The importance of this distinction between the quality of software
>> expected in a distribution as against installing directly from source is
>> apparently lost on those who did not live through the pre-distribution
>> days. During that time we had to gather up the dependencies ourselves,
>> trying to get the correct versions, hoping that the software compiled and
>> worked with the other dependencies, and hope we did not install malware and
>> hackware. And quite often it was a futile attempt to gather together
>> software dependencies of any size.
>>
>> Those who lived through that time remember it as the dark-ages of long
>> ago, never to be seen again.
>>
>> Been there, done that... (actually suffered from that).
>>
>>
>>
>> I claim this is just not the same situation here: CMake is a
>> self-contained dependencies and as shown above does not need to escape
>> anywhere outside the build directory.
>>
>>
>>
>> --
>>
>> Mehdi
>>
>>
>>
>>
>>
>> On 4/4/20 11:48 AM, Mehdi AMINI via llvm-dev wrote:
>>
>>
>>
>>
>>
>> On Thu, Apr 2, 2020 at 10:49 AM Shoaib Meenai via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>> I’m in favor of all this. Thanks for volunteering! I’m happy to help out
>> in whatever way.
>>
>>
>>
>> Some things it might be worth figuring out for future upgrades:
>>
>> * If we want to limit ourselves to CMake versions supported by LTS
>> releases of distros, which distros should we consider, and how far back
>> should we go (i.e. is it just the latest LTS or the last two LTS versions)?
>>
>>
>>
>> Can you clarify what "supported" means? Does it include PPA on ubuntu for
>> example?
>>
>> I wouldn't limit ourselves artificially to the version of CMake
>> "natively" available on an OS in cases where we already requires a more
>> recent compiler than the default one available: if we consider OK to
>> require as user to build clang or gcc from source or use a PPA, we should
>> be OK the same way with CMake.
>>
>>
>>
>>
>>
>> * For platforms like Ubuntu where CMake publishes its own packages (that
>> you can install via the platform’s package manager), do those count, or do
>> we only consider the CMake that comes in the OS packages?
>>
>> * Do we have any limitations around how often/when we upgrade? You’re
>> tying the upgrade to after the branch, which is pretty standard, but e.g.
>> if we wanted to upgrade to 3.8.0 now and then upgrade to 3.13.4 after the
>> branch, would people be okay with that, or should we limit upgrades to just
>> shortly after a branch?
>>
>>
>>
>> *From: *<ldionne at apple.com> on behalf of Louis Dionne <ldionne at apple.com>
>> *Date: *Thursday, April 2, 2020 at 7:20 AM
>> *To: *Shoaib Meenai <smeenai at fb.com>
>> *Cc: *"llvm-dev at lists.llvm.org" <llvm-dev at lists.llvm.org>, Chris
>> Bieneman <beanz at apple.com>, Petr Hosek <phosek at chromium.org>, Saleem
>> Abdulrasool <compnerd at compnerd.org>, "tstellar at redhat.com" <
>> tstellar at redhat.com>
>> *Subject: *Re: Upgrading LLVM's minimum required CMake version
>>
>>
>>
>> Okay, so we've had some discussion on this thread, and although some
>> people (including me) would like a more aggressive policy, I believe the
>> following will not get any objection (based on the thread). On April 23rd
>> 2020, Ubuntu 20.04 LTS will ship with CMake 3.16.x. This will make the
>> lower bound for LTS distributions be 3.13.4, and so I suggest we upgrade to
>> that. Here's a proposed process:
>>
>>
>>
>> 1. Immediately add a CMake warning in <root>/llvm/CMakeLists.txt saying
>> that CMake 3.13.4 will be the new minimum version starting with LLVM
>> 12.0.0, and mentioning the versions used in various LTSes.
>>
>> 2. Immediately send a courtesy heads-up email to all build-bot owners
>> telling them about the upcoming change.
>>
>> 3. Right after we branch off the release branch for LLVM 11.0.0 (the next
>> one), make the minimum CMake version required be 3.13.4.
>>
>> 4. Iterate on (3) until all bots are migrated.
>>
>> 5. Send a message to the list saying the bump is complete. At that time,
>> projects are free to start using features from 3.13.4.
>>
>>
>>
>> Unless someone else absolutely wants to bite the bullet, I volunteer to
>> do the above steps.
>>
>>
>>
>> Thoughts?
>>
>>
>>
>> Louis
>>
>>
>>
>>
>>
>> On Mar 26, 2020, at 16:07, Shoaib Meenai <smeenai at fb.com> wrote:
>>
>>
>>
>> We had this discussion a few months ago and it petered out, and it’s
>> recently been revived in the context of upgrading the CMake version
>> specifically for libc++ (at which point people suggested upgrading the
>> CMake version used by all of LLVM), so let’s try to move this forward.
>>
>> Our current required minimum version is CMake 3.4.3, which was released
>> on January 25th 2016. It’s interesting to note that LLVM started requiring
>> 3.4.3 on May 31st 2016, which was just 4 months after its release.
>>
>> Let’s look at the CMake versions available on various distros and
>> operating systems. I’m unfamiliar with many of these, so I apologize if I
>> get something wrong. (I’m using pkgs.org
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__pkgs.org&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=atRWn0u_rLa9ITqRSW-W8QvRJN244hhQmDWillcW3gE&s=S6Qvuq5DqECZFfItkAJOL5xjTSp1psRWYtq_WOnXt_o&e=>
>> for most of this information.)
>> * RHEL 6 (released Nov 10th 2010) : 3.6.1 (via EPEL)
>> * RHEL 7 (released June 10th 2014): 3.14.7 (via EPEL)
>> * RHEL 8 (released May 7th 2019): 3.11.4 (maybe pkgs.org
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__pkgs.org&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=atRWn0u_rLa9ITqRSW-W8QvRJN244hhQmDWillcW3gE&s=S6Qvuq5DqECZFfItkAJOL5xjTSp1psRWYtq_WOnXt_o&e=>
>> is screwy on this one, because it doesn’t make sense that RHEL 7 should
>> have a newer available version than RHEL 8)
>> * Debian 9 (released June 17th 2017): 3.7.2
>> * Debian 10 (released July 6th 2019): 3.13.4
>> * Ubuntu 16.04 LTS (released April 21st 2016): 3.5.1
>> * Ubuntu 18.04 LTS (released April 26th 2018): 3.10.2
>> * FreeBSD 11 (released October 10th 2016): 3.15.5 (presumably upgraded in
>> a point release)
>> * FreeBSD 12 (released December 11th 2018): 3.15.5 (presumably upgraded
>> in a point release)
>> * NetBSD 8.1 (released May 31st 2019): 3.16.1
>> * NetBSD 9.0 (released February 14th 2020): 3.16.1
>> * OpenBSD: couldn’t find the version
>> * macOS: latest version is readily available through Homebrew
>> * Windows: You can install it yourself or use the one bundled with Visual
>> Studio. I don't know what versions are bundled with Visual Studio; some
>> searching suggests Visual Studio 2017 has CMake 3.12 and Visual Studio 2019
>> has 3.15, though I have no confirmation of that.
>>
>> Note that CMake provides prebuilt binaries for Linux, macOS, and Windows,
>> and it’s also straightforward to build from source (it has very
>> conservative compiler requirements). One suggestion that was brought up in
>> the past was for LLVM’s build system to just download a newer version of
>> CMake if you attempted to build it using one that was too old, but there
>> was opposition [1]. There was also a suggestion to have a script in LLVM to
>> download and build CMake for you, but there were mixed opinions on this too
>> [2], particularly since many developers might prefer downloading a binary
>> release to building from source themselves (though of course the script
>> could also download binary releases if applicable). I personally think
>> downloading or building CMake yourself isn’t a high barrier for anyone
>> wanting to build LLVM (and in particular it’s *much* more straightforward
>> than building LLVM itself), but I can understand why people would prefer to
>> stick to versions available in distros.
>>
>> Another suggestion that came up last time was to set a policy for
>> upgrading CMake versions on some regular basis. The opposition to this was
>> that we should upgrade CMake versions only when a newer version has a
>> compelling enough feature to justify upgrading, rather than always
>> upgrading. I can see arguments for both approaches, but we should
>> definitely at least think about the benefits we can get from upgrading
>> versions. I've gone through the CMake release notes and highlighted
>> features which seemed potentially valuable for LLVM. Note that I'm only
>> highlighting features for which our minimum CMake version would have to be
>> bumped up in order for our build system to take advantage of. There are
>> other useful features in newer CMake versions, but you can take advantage
>> of them just by using a newer CMake yourself. For example, 3.9 loosens the
>> dependencies of object compilation, which should result in faster Ninja
>> builds.
>>
>> CMake 3.5 (released March 8th 2016):
>> * install(DIRECTORY) supports generator expressions
>>
>> CMake 3.6 (released July 7th 2016):
>> * install() supports EXCLUDE_FROM_ALL
>> * list() supports FILTER to filter by regular expression
>> * Subninja support, which could theoretically be used for much faster
>> runtimes builds, although in practice we probably want to make
>> ExternalProject support this directly instead of trying to layer our own
>> meta-build system on top
>> * CMAKE_TRY_COMPILE_TARGET_TYPE to tell try_compile to build a static
>> library instead of an executable, which will greatly simplify the
>> compiler-rt build
>>
>> CMake 3.7 (released November 11th 2016):
>> * New if() comparison operators LESS_EQUAL, GREATER_EQUAL, STRLESS_EQUAL,
>> STRGREATER_EQUAL, VERSION_LESS_EQUAL, and VERSION_GREATER_EQUAL
>>
>> CMake 3.8 (released April 10th 2017):
>> * Compile features for C++17, which is required to build libc++ correctly
>> * Support for compile features for specific C++ features instead of only
>> being able to specify standard versions
>> * rpath support via BUILD_RPATH target property and CMAKE_BUILD_RPATH
>> variable
>> * Apple framework support for static libraries
>> * New swig_add_library command in the UseSWIG module
>> * New generator expression $<IF:cond,true-value,false-value>
>>
>> CMake 3.9 (released July 18th 2017):
>> * install(TARGETS) and install(EXPORTS) support for object libraries,
>> which will simplify the compiler-rt build
>> * TARGET_OBJECTS generator expression support in add_custom_command and
>> file(GENERATE)
>> * $<TARGET_BUNDLE_DIR:tgt> and $<TARGET_BUNDLE_CONTENT_DIR:tgt> generator
>> expressions for Apple bundles
>>
>> CMake 3.10 (released November 20th 2017):
>> * include_guard() command for proper guarding against double includes of
>> CMake scripts
>> * An interesting aside is that this is the first verion of CMake to
>> require C++11 to build, which should give a good sense of how conservative
>> they are about compiler requirements
>>
>> CMake 3.11 (released March 28th 2018):
>> * add_library() and add_executable() can be called without sources as
>> long as target_sources() is used later
>> * target_compile_{definitions,features,options},
>> target_include_directories(), target_sources(), and target_link_libraries()
>> can set the corresponding INTERFACE_* properties on imported targets
>> * COMPILE_DEFINITIONS supports generator expressions
>> * COMPILE_OPTIONS source file property added
>> * INCLUDE_DIRECTORIES source file property added
>> * Interface libraries support custom properites
>>
>> CMake 3.12 (released July 17th 2018):
>> * add_compile_definitions() added to add compile definitions for targets
>> (to avoid the global pollution caused by add_definitions())
>> * cmake_minimum_required() supports a version range to indicate tested
>> CMake versions and set policies accordingly
>> * file(TOUCH) and file(TOUCH_NOCREATE) added
>> * list(JOIN), list(SUBLIST) and list(TRANSFORM) added
>> * string(JOIN) added
>> * SHELL: prefix support in target_compile_options to avoid errant
>> deduplication
>> * target_link_libraries() supports object libraries and propagates usage
>> requirements
>> * EXPORT_PROPERTIES target property to control the target properties
>> exported by export() and install(EXPORT)
>> * FindLibXml2 provides imported targets
>> * New FindPython, FindPython2, and FindPython3 modules to ease location
>> Python and selecting a specific version
>> * Modernization of UseSWIG module
>> * New generator expressions $<GENEX_EVAL:...>,
>> $<TARGET_GENEX_EVAL:target,...>, $<IN_LIST:...>, $<TARGET_EXISTS:...> and
>> $<TARGET_NAME_IF_EXISTS:...>
>> * Compile features support for C++20
>>
>> CMake 3.13 (released November 20th 2018):
>> * cmake -E create_symlink supported on Windows
>> * target_link_directories() and target_link_options() commands to set
>> link options instead of awkwardly having to use target_link_libraries() for
>> this purpose
>> * UseSWIG can manage INCLUDE_DIRECTORIES for SWIG compilation
>>
>> CMake 3.14 (released March 14th 2019):
>> * file(CREATE_LINK) to create hard or symbolic links
>> * if(DEFINED CACHE{VAR}) for checking if a cache variable is defined
>> * $<IN_LIST:...> generator expression correctly handles empty argument
>> * Fixes for object library linking propagation
>> * Link options to manage position independent executables added
>> automatically
>>
>> CMake 3.15 (released July 17th 2019):
>> * list(PREPEND), list(POP_FRONT) and list(POP_BACK) added
>> * New message() types NOTICE, VERBOSE, DEBUG and TRACE
>> * string(REPEAT) added
>> * MSVC_RUNTIME_LIBRARY target property and CMAKE_MSVC_RUNTIME_LIBRARY
>> variable to select the runtime library type for MSVC
>> * $<C_COMPILER_ID:...>, $<CXX_COMPILER_ID:...>, $<COMPILE_LANGUAGE:...>,
>> and $<PLATFORM_ID:...> generator expressions support matching one value
>> from a list
>> * $<COMPILE_LANG_AND_ID:...> generator expression added
>> * $<FILTER:list,INCLUDE|EXCLUDE,regex> generator expression added
>> * $<REMOVE_DUPLICATES:list> generator expression added
>> * New $<TARGET_FILE*> generator expressions added:
>> $<TARGET_FILE_PREFIX:...>, $<TARGET_FILE_BASE_NAME:...>,
>> $<TARGET_FILE_SUFFIX:...>, $<TARGET_LINKER_FILE_PREFIX:...>,
>> $<TARGET_LINKER_FILE_BASE_NAME:...>, $<TARGET_LINKER_FILE_SUFFIX:...>,
>> $<TARGET_PDB_FILE_BASE_NAME:...>
>> * $<TARGET_OBJECTS:...> generator expression supports executables and
>> static, shared, and module libraries
>>
>> CMake 3.16 (released November 26th 2019):
>> * Support for generator expressions in BUILD_RPATH and INSTALL_RPATH
>>
>> CMake 3.17 (released March 20th 2020):
>> * Ninja Multi-Config generator, which among other things would greatly
>> simplify LLVM_OPTIMIZED_TABLEGEN
>> * foreach(ZIP_LISTS) added to iterate multiple lists simultaneously
>> * New message() keywords CHECK_START, CHECK_PASS, and CHECK_FAIL
>> * INSTALL_NAME_DIR supports generator expressions
>>
>> Our build system is incredibly complex, and many of these features can be
>> used to clean it up and make it much more maintainable. I would personally
>> like us to at least bump up to CMake 3.12. I also do think it's worth
>> establishing a policy and process around upgrading CMake versions, since
>> newer versions keep on adding useful features (particularly better
>> generator expression support), and we want to be able to keep taking
>> advantage of them.
>>
>> [1] http://lists.llvm.org/pipermail/llvm-dev/2019-November/136485.html
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.llvm.org_pipermail_llvm-2Ddev_2019-2DNovember_136485.html&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=atRWn0u_rLa9ITqRSW-W8QvRJN244hhQmDWillcW3gE&s=HtGj57-MndDqyK71vXRwheQXms3WKx9rT-8WAVyTB3c&e=>
>> [2] http://lists.llvm.org/pipermail/llvm-dev/2019-November/136488.html
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.llvm.org_pipermail_llvm-2Ddev_2019-2DNovember_136488.html&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=atRWn0u_rLa9ITqRSW-W8QvRJN244hhQmDWillcW3gE&s=hHrZYrGk0WStJ3TjjIsXg2NMvVUP-f4woTFtaFlYkG8&e=>
>>
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
>>
>> _______________________________________________
>>
>> LLVM Developers mailing list
>>
>> llvm-dev at lists.llvm.org
>>
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200407/64f044df/attachment.html>