[llvm-dev] RFC: LLVM Build System Future Direction
Mehdi AMINI via llvm-dev
llvm-dev at lists.llvm.org
Tue Oct 29 18:09:24 PDT 2019
On Tue, Oct 29, 2019 at 1:46 PM Martin Storsjö via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Speaking on how to locate/enable projects (like clang/lldb) - even with
> the monorepo, I'm still enabling clang/lldb by symlinking them into
> llvm/tools, for one specific reason: CMake and relative paths.
>
> As long as all the source that is being built is below the toplevel CMake
> directory, and the build directory itself is within the toplevel CMake
> project directory, CMake uses relative paths in all build files.
> (
> https://github.com/Kitware/CMake/blob/master/Source/cmLocalGenerator.h#L338-L346
> contains some mention of this.)
>
> This means, my build layout is that I build in llvm-project/llvm/build. In
> this setup, CMake produces build files with paths e.g.
> ../tools/clang/. If I use LLVM_ENABLE_PROJECTS for enabling clang/lldb,
> all clang/lldb source files are referenced by absolute path (instead of
> ../../clang).
>
> Using relative paths everywhere has the nice (and for me, at this point,
> essential) side effect of making the build output fully independent of the
> path to the source tree, allowing ccache to share build artefacts across
> two source trees at different locations. This allows builds to run a
> couple order of magnitudes faster, if I've built the same code version in
> a different code tree on the same machine.
>
> I guess this could be overcome by adding another top level CMake file at
> the top of the monorepo, allowing that to be the main source root of the
> build.
This seems to be a natural evolution of the monorepo to me to have a
top-level CMake instead of pointing at the one nested in the `llvm/`
directory.
I wasn't at the round table, was this mentioned?
--
Mehdi
> I'm not familiar enough with CMake to know if it's possible to just
> make this a couple-lines CMakeLists.txt that includes llvm/CMakeLists.txt,
> or if something else is needed to make this work.
>
> // Martin
>
> On Tue, 29 Oct 2019, Shoaib Meenai via llvm-dev wrote:
>
> >
> > LLVM_EXTERNAL_*_SOURCE_DIR can be used for specifying paths to external
> > clang, etc., and I agree that with the monorepo, there’s one canonical
> > location for those sub-projects to live, and we don’t need to support it
> for
> > those subprojects. However, LLVM_EXTERNAL_*_SOURCE_DIR can also be used
> in
> > conjunction with LLVM_EXTERNAL_PROJECTS to specify the paths to other
> > projects you want to include in your build but don’t necessarily want to
> > place beside the llvm directory, e.g. Swift. We’ll continue supporting it
> > for the latter use case, correct?
> >
> >
> >
> > From: llvm-dev <llvm-dev-bounces at lists.llvm.org> on behalf of Chris
> Bieneman
> > via llvm-dev <llvm-dev at lists.llvm.org>
> > Reply-To: Chris Bieneman <beanz at apple.com>
> > Date: Tuesday, October 29, 2019 at 10:10 AM
> > To: llvm-dev <llvm-dev at lists.llvm.org>
> > Subject: Re: [llvm-dev] RFC: LLVM Build System Future Direction
> >
> >
> >
> > Sorry for the delay in writing this up and sending it out, but I wanted
> to
> > recap the discussion from the roundtable on October 23rd. The roundtable
> ran
> > for almost two hours and we discussed at most of the main points in my
> RFC.
> > Thank you everyone who participated and contributed to the discussion!
> >
> >
> >
> > TL;DR: We should move to CMake 3.15 (RFC incoming). We should make `all`
> > really `all`. We should strive to reduce complexity and remove options,
> > specifically options that aren't relevant to the monorepo. We should
> work to
> > standardize workflows. We need to keep thinking about how to build
> runtime
> > projects.
> >
> >
> >
> > In my recap I'd like to fully disclose that I was standing up through
> most
> > of the roundtable directing the conversation so I don't have much in the
> way
> > of notes to go off. I may get details or things wrong, so please chime
> in to
> > correct me.
> >
> >
> >
> > We had a brief discussion around raising the minimum required CMake
> version.
> > The general consensus was that since CMake provides binary packages for
> most
> > common OSs, and building CMake from source has lower system requirements
> > than LLVM and is very simple, nobody saw any barrier to adopting new
> > versions. Initially I suggested moving to CMake 3.11 or 3.12, I believe
> it
> > was James Knight who said the actual cost of updating the bots is the
> hard
> > part in raising the version, so maybe we should just take the newest.
> That
> > reasoning made sense to everyone at the roundtable and there were no
> > objections at the roundtable to moving to CMake 3.15. Look for an RFC
> from
> > me shortly that will propose that and lay out a timeline.
> >
> >
> >
> > We spent a lot of time talking about a handful of other topics that all
> > spring out from my RFC. There was a general agreement that the number of
> > options in the build system is unwieldy and the combinatoric effect of
> the
> > options is making the build system difficult to maintain. There was a
> > general agreement at least in principal that we should work to reduce the
> > complexity of the build system.
> >
> >
> >
> > We did discuss specific build system functionality that could be
> simplified
> > or removed, and on some points there was agreement, and on others there
> is
> > need for more discussion. A few examples:
> >
> >
> >
> > There was general agreement on the concept from my RFC that the `all`
> target
> > should always really be `all`. That would mean removing the
> > `LLVM_TOOL_*_BUILD` options.
> >
> >
> >
> > There was also agreement that in the monorepo it no longer makes sense to
> > have an option to specify the source locations of sub-projects. That
> means
> > we can remove the `LLVM_EXTERNAL_*_SOURCE_DIR` variables.
> >
> >
> >
> > There was disagreement over whether or not standalone builds of
> non-runtime
> > sub-projects should remain. This specifically would relate to clang and
> > lldb, and whether or not they can be built against installed or
> separately
> > built copies of LLVM. There were points on both side of this discussion,
> and
> > it will require more discussion to resolve.
> >
> >
> >
> > The roundtable also focused a lot of time on the runtime libraries, and
> how
> > they should be built. It is clear from the discussion that the runtime
> > libraries need to support being built standalone (separately from LLVM)
> in
> > order to fit into the various distribution strategies. Specifically many
> of
> > the runtime libraries are sometimes shipped as OS components rather than
> as
> > part of the toolchain, so you need to be able to build them separately
> from
> > the toolchain. Those same libraries can also be shipped as part of a
> > toolchain, so we need to support that workflow too. The need to support
> > these disparate workflows has led to much of the complexity. The proposal
> > from my RFC of standardizing runtimes builds as "standalone", and using
> the
> > LLVM runtimes directory (which uses CMake's ExternalProject) to configure
> > and build runtimes had pretty wide support in the discussion. I suspect
> this
> > is the direction we should move in, but with some slight changes.
> >
> >
> >
> > One of the points that came up was that building runtimes for multiple
> > platforms at once can be a configuration nightmare requiring hundreds of
> > build settings. It was suggested that it might be easier if you could
> build
> > all of the runtime libraries with a single CMake invocation separately
> from
> > LLVM. There are some dependency complications around the ordering of the
> > builtin libraries build, but otherwise this is largely doable.
> >
> >
> >
> > There are also problems related to cross-runtime dependencies which have
> > come up recently. Some of these problems can be addressed more cleanly
> with
> > modern CMake generator expressions, and other issues may require a larger
> > restructuring of code. For years there have been discussions thrown
> around
> > about breaking the builtins libraries out of compiler-rt. Maybe now is
> the
> > time to consider doing that.
> >
> >
> >
> > The last topic that was brought up at the end of the roundtable was about
> > supporting IDE generators. Saleem Abdulrasool pointed out that Visual
> Studio
> > ships a CMake integration that generates Ninja builds and supports the
> IDE
> > UI interactions as if it were a Visual Studio project without being a
> > project generator. Not supporting IDE build systems would clean up a lot
> of
> > complexity in our build system. As was pointed out during the discussion,
> > Xcode has no such support and is used by members of the community. There
> was
> > discussion of whether or not an Xcode+Ninja generator could be created
> which
> > would use Xcode's external build system mechanism to create an Xcode
> project
> > for browsing, editing, and debugging, while using Ninja to build. The
> > conversation had no real resolution other than "that would be cool" and
> "it
> > would be nice to remove all the `if (XCODE)` blocks".
> >
> >
> >
> > Thank you everyone who participated in the roundtable! If there is
> anything
> > I missed please help fill in the blanks.
> >
> >
> >
> > Thanks,
> >
> > -Chris
> >
> >
> >
> > On Oct 24, 2019, at 10:28 AM, Alex Denisov via llvm-dev
> > <llvm-dev at lists.llvm.org> wrote:
> >
> >
> >
> > That works perfectly.
> >
> >
> >
> > But I’m referring to the case when I want to add LLVM as a sub
> > project, which would allow me to debug/modify LLVM as if it is my
> > code.
> >
> > As I said, this also works, but some parts are not straightforward.
> >
> >
> >
> > On Thu 24. Oct 2019 at 18:15, Louis Dionne <ldionne at apple.com> wrote:
> >
> >
> >
> >
> >
> > On Oct 24, 2019, at 02:17, Alex Denisov via
> > llvm-dev <llvm-dev at lists.llvm.org> wrote:
> >
> >
> >
> > Hi Chris,
> >
> >
> >
> > This is a great initiative and it feels like the right
> > direction.
> >
> >
> >
> > I'd like to add another point to the list: using LLVM as a
> > library, i.e. being able to add it as a CMake subproject.
> >
> > Currently it works pretty good, but some parts can be
> > improved (somehow). E.g.:
> >
> >
> >
> > I believe the CMake-correct way of doing that is to produce LLVM
> > export files when installing LLVM, which would allow
> > find_package to work out of the box. You’d get LLVM targets you
> > can link against and all dependencies would be propagated by
> > CMake properly.
> >
> >
> >
> >
> >
> > Louis
> >
> >
> >
> > - getting LLVM's version: the only way I've found
> > is to parse the CMakeLists.txt with regexes and hope
> > it doesn't break in the future
> >
> > - getting include dirs info: IIRC currently it works
> > transitively when one links against some library, but it
> > feels like a weak point to me
> >
> > - getting other configuration options (for example
> > whether LLVM is built with or without exceptions)
> >
> > - controlling how LLVM is built: the only way I found is
> > to force set a variable (i.e. set (LLVM_BUILD_32_BITS ON
> > CACHE BOOL "" FORCE)) before adding LLVM as a subproject
> >
> >
> >
> > I think my list is not exhaustive since it covers only my
> > use cases, so maybe there are other opinions/requests from
> > others.
> >
> >
> >
> > Cheers,
> >
> > Alex.
> >
> >
> >
> > On Mon, Oct 21, 2019 at 8:26 PM Chris Bieneman via
> > llvm-dev <llvm-dev at lists.llvm.org> wrote:
> >
> > Over the past few years the LLVM CMake build
> > system has gained a lot of new features, which
> > have powered new workflows and capabilities.
> > This development has largely been individual
> > efforts without focus or long-term direction.
> > The build system is of incredible importance
> > to LLVM as it is a primary interface for
> > contributors to build and test their changes.
> >
> > This year, LLVM is making a huge
> > infrastructure shift to GitHub. Along with
> > that shift many of the previously supported
> > build options may not make sense. This is a
> > prefect opportunity to revisit the state of
> > our build infrastructure with an eye toward
> > the future.
> >
> > I would like this RFC, and any discussion it
> > sparks, to serve as a starting point for a
> > conversation we can continue at the LLVM
> > Developer Meeting to set goals and conventions
> > for the development of the build system.
> >
> > Tom Stellard has scheduled a roundtable on
> > CMake from 10:45-11:55 on Wednesday, Oct 23.
> >
> > ## The Problem
> > Lacking clear direction and oversight the
> > build system is evolving in rapidly divergent
> > ways. Further since we don't have a formalized
> > process for unifying workflows and deprecating
> > old behaviors LLVM's build system has a
> > convoluted feature set.
> >
> > This manifests itself in many unfortunate
> > ways. Some examples are:
> >
> > (1) There are three different ways to build
> > compiler-rt as part of LLVM
> > (2) There are lots of incompatible build
> > configurations that need to be accounted for,
> > some that aren't (like
> > -DLLVM_BUILD_LLVM_DYLIB=On
> > -DBUILD_SHARED_LIBS=On, which will explode at
> > runtime)
> >
> > As the build system gains complexity
> > maintaining the build system is getting more
> > expensive to the community.
> >
> > ## Future Directions
> > The following are proposals to enable the
> > build system to better facilitate LLVM
> > development and provide a usable, extensible,
> > and stable build system for LLVM and all
> > sub-projects.
> >
> > ### Updating CMake More Regularly
> > In the past we have clung to old versions of
> > CMake for extended periods of time. This has
> > resulted in significant checking
> > `CMAKE_VERSION` to enable some features or
> > being completely unable to use others. In
> > particular recent CMake development extending
> > generator expressions could provide
> > substantial benefit to the project. If we
> > stick to current upgrade policies, we may not
> > be able to use the current CMake release for
> > another few years.
> >
> > As an alternative proposal, we should consider
> > CMake upgrades independent of OS package
> > manager versioning. That is not to say we
> > should take every new version of CMake,
> > however we should upgrade for compelling
> > reasons.
> >
> > In tree right now we have code gated on CMake
> > 3.7 which enables CMake to generate Ninja
> > rules for tablegen that process tablegen's dep
> > files. This allows accurately rebuilding
> > tablegen when included files change. I propose
> > that this change's benefit should be
> > sufficient to justify moving to CMake 3.7 for
> > the project.
> >
> > Additionally, building LLDB.framework on
> > Darwin requires CMake 3.8 due to bugs in
> > earlier versions of CMake. This could also be
> > a justification for updating.
> > Lastly, getting updated versions of CMake is
> > very easy. Kitware provides Windows, Mac and
> > Linux builds on cmake.org as well as an Ubuntu
> > apt source. If that is insufficient building
> > CMake from source is simple, and has minimal
> > system requirements. Visual Studio contains
> > reasonably up-to-date CMake bundled. As such
> > we should not allow OS release or support
> > cycles to dictate when we upgrade CMake.
> >
> > ### Reducing the Test Matrix
> > The most important guiding principal for
> > development of the LLVM build system must be
> > to reduce the matrix of configurations. The
> > more possible configurations the build system
> > supports the wider the test matrix. This is
> > not to say the build system should support
> > doing less, but rather to support less unique
> > configurations.
> >
> > Many configuration options in the build system
> > just turn on or off different parts of the
> > project. For example, the
> > `LLVM_BUILD_LLVM_DYLIB` option just disables
> > configuring libLLVM. An alternative approach
> > would be to always configure libLLVM, and
> > leave it up to users of the build system to
> > determine whether or not to build it.
> >
> > We also have options to enable and disable
> > configuring individual tools in the LLVM and
> > Clang projects. I believe we should eliminate
> > those options, which will result in the `all`
> > target always being everything. We have
> > explicit clean dependencies for the `check-*`
> > targets so most developer workflows should be
> > un-impacted. Distribution workflows can use
> > the `LLVM_DISTRIBUTION_COMPONENTS` option to
> > hand tailor which parts of LLVM to build and
> > install with better control without as much
> > complication.
> >
> > Many other options exist to support a wide
> > variety of divergent workflows. For example,
> > the `LLVM_EXTERNAL_${PROJECT}_SOURCE_DIR`
> > options exist to allow users to specify custom
> > paths for projects so that, historically, they
> > didn't need to nest clang inside LLVM. With
> > the move to the mono-repo we should define
> > consistent workflows and eliminate options
> > that support divergent workflows.
> >
> > ### Adopting Conventions
> > Much of LLVM's build system is not idiomatic
> > CMake. Some of those differences make sense
> > because LLVM is not a typical software
> > project. I'm unaware of any build
> > configuration system that was designed
> > specifically to build compilers and handle the
> > complex dependency chains that come with that
> > territory.
> >
> > Some of our divergences come from history. We
> > have a great many features implemented in our
> > CMake because CMake didn't support them at the
> > time. We also have patterns that were
> > appropriate before CMake added new features,
> > and have never cleaned them up.
> >
> > One big thing our build system needs is a set
> > of guiding conventions to direct future
> > development. Some key conventions that I
> > believe are crucial:
> >
> > #### Avoid Order Dependent Behavior
> > CMake generator expressions provide the
> > ability to defer logic until after script
> > processing. This allows the build system to
> > avoid direct dependence on the order in which
> > targets are processed. We should not use the
> > `if(TARGET ...)` or `get_target_property`
> > interfaces unless it is completely impossible
> > to avoid.
> >
> > #### Avoid Options to Enable/Disable
> > Configuration
> > If we reduce the test matrix, having a
> > convention to keep it reduced is of vital
> > importance so that we don't find ourselves
> > needing to clean up again in a few years.
> >
> > #### Avoid Caching, Use `mark_as_advanced` and
> > `INTERNAL` Liberally
> > CMake has no strategy for cache invalidation.
> > As such, cached variables add additional
> > maintenance burden because they can break
> > builds sometimes in hard to diagnose ways.
> > That said they are useful. In particular for
> > things like configuration checks that are slow
> > caching the result makes incremental
> > re-configuration much faster. We should use
> > cached values sparingly and only where they
> > provide benefit.
> >
> > Additionally, every cached CMake variable is a
> > configuration point. Variables not marked
> > `INTERNAL` show up in `ccmake` and
> > `cmake-gui`, and variables not
> > `mark_as_advanced` show up to all users. We
> > should use the `INTERNAL` and
> > `mark_as_advanced` options wherever
> > appropriate to limit our supported
> > configuration interface.
> >
> > #### Making Sense of Runtime Builds
> > Right now, there are three different ways to
> > build compiler-rt as part of LLVM and two
> > different ways to build most of the other
> > runtime libraries (libcxxabi, libcxx,
> > libunwind, etc). This situation is confusing
> > even for long time contributors.
> >
> > We need a clearer story for building runtime
> > libraries to reduce the number of different
> > ways they are built and provide simplified
> > workflows for users.
> >
> > It is my opinion that if you are building a
> > runtime library as part of an LLVM/Clang
> > build, it should be configured and built with
> > the in-tree clang as it would be for
> > distribution. If you don't want to build with
> > the in-tree clang, we should encourage people
> > to build the runtime libraries independently
> > of the compiler.
> >
> > My reasoning for this is that distributions of
> > clang are generally built from the default
> > settings in the build and configuration
> > process, and distributions (or installs by new
> > users) which include the runtime libraries
> > should have runtimes built with the just-built
> > compiler. To align these two situations we
> > need the default build configuration of
> > LLVM+Clang+Runtimes to be using the just-built
> > compiler.
> >
> > Adopting this change would mean runtime
> > library projects would only contain build
> > system support for building "standalone"
> > meaning not in the same configuration as LLVM.
> > We would then support runtime libraries built
> > as individual projects or using the LLVM
> > runtimes directory, which separately
> > configures and builds runtime libraries using
> > the just-built clang.
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >
> >
> >
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >
> >
> >
> >
> >_______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20191029/390608c6/attachment.html>
More information about the llvm-dev
mailing list