<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Oct 29, 2019 at 1:46 PM Martin Storsjö via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">Speaking on how to locate/enable projects (like clang/lldb) - even with <br>
the monorepo, I'm still enabling clang/lldb by symlinking them into <br>
llvm/tools, for one specific reason: CMake and relative paths.<br>
<br>
As long as all the source that is being built is below the toplevel CMake <br>
directory, and the build directory itself is within the toplevel CMake <br>
project directory, CMake uses relative paths in all build files. <br>
(<a href="https://github.com/Kitware/CMake/blob/master/Source/cmLocalGenerator.h#L338-L346" rel="noreferrer" target="_blank">https://github.com/Kitware/CMake/blob/master/Source/cmLocalGenerator.h#L338-L346</a> <br>
contains some mention of this.)<br>
<br>
This means, my build layout is that I build in llvm-project/llvm/build. In <br>
this setup, CMake produces build files with paths e.g. <br>
../tools/clang/. If I use LLVM_ENABLE_PROJECTS for enabling clang/lldb, <br>
all clang/lldb source files are referenced by absolute path (instead of <br>
../../clang).<br>
<br>
Using relative paths everywhere has the nice (and for me, at this point, <br>
essential) side effect of making the build output fully independent of the <br>
path to the source tree, allowing ccache to share build artefacts across <br>
two source trees at different locations. This allows builds to run a <br>
couple order of magnitudes faster, if I've built the same code version in <br>
a different code tree on the same machine.<br>
<br>
I guess this could be overcome by adding another top level CMake file at <br>
the top of the monorepo, allowing that to be the main source root of the <br>
build. </blockquote><div><br></div><div>This seems to be a natural evolution of the monorepo to me to have a top-level CMake instead of pointing at the one nested in the `llvm/` directory.</div><div>I wasn't at the round table, was this mentioned?</div><div><br></div><div>-- </div><div>Mehdi</div><div><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">I'm not familiar enough with CMake to know if it's possible to just <br>
make this a couple-lines CMakeLists.txt that includes llvm/CMakeLists.txt, <br>
or if something else is needed to make this work.<br>
<br>
// Martin<br>
<br>
On Tue, 29 Oct 2019, Shoaib Meenai via llvm-dev wrote:<br>
<br>
> <br>
> LLVM_EXTERNAL_*_SOURCE_DIR can be used for specifying paths to external<br>
> clang, etc., and I agree that with the monorepo, there’s one canonical<br>
> location for those sub-projects to live, and we don’t need to support it for<br>
> those subprojects. However, LLVM_EXTERNAL_*_SOURCE_DIR can also be used in<br>
> conjunction with LLVM_EXTERNAL_PROJECTS to specify the paths to other<br>
> projects you want to include in your build but don’t necessarily want to<br>
> place beside the llvm directory, e.g. Swift. We’ll continue supporting it<br>
> for the latter use case, correct?<br>
> <br>
> <br>
> <br>
> From: llvm-dev <<a href="mailto:llvm-dev-bounces@lists.llvm.org" target="_blank">llvm-dev-bounces@lists.llvm.org</a>> on behalf of Chris Bieneman<br>
> via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>><br>
> Reply-To: Chris Bieneman <<a href="mailto:beanz@apple.com" target="_blank">beanz@apple.com</a>><br>
> Date: Tuesday, October 29, 2019 at 10:10 AM<br>
> To: llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>><br>
> Subject: Re: [llvm-dev] RFC: LLVM Build System Future Direction<br>
> <br>
> <br>
> <br>
> Sorry for the delay in writing this up and sending it out, but I wanted to<br>
> recap the discussion from the roundtable on October 23rd. The roundtable ran<br>
> for almost two hours and we discussed at most of the main points in my RFC.<br>
> Thank you everyone who participated and contributed to the discussion!<br>
> <br>
> <br>
> <br>
> TL;DR: We should move to CMake 3.15 (RFC incoming). We should make `all`<br>
> really `all`. We should strive to reduce complexity and remove options,<br>
> specifically options that aren't relevant to the monorepo. We should work to<br>
> standardize workflows. We need to keep thinking about how to build runtime<br>
> projects.<br>
> <br>
> <br>
> <br>
> In my recap I'd like to fully disclose that I was standing up through most<br>
> of the roundtable directing the conversation so I don't have much in the way<br>
> of notes to go off. I may get details or things wrong, so please chime in to<br>
> correct me.<br>
> <br>
> <br>
> <br>
> We had a brief discussion around raising the minimum required CMake version.<br>
> The general consensus was that since CMake provides binary packages for most<br>
> common OSs, and building CMake from source has lower system requirements<br>
> than LLVM and is very simple, nobody saw any barrier to adopting new<br>
> versions. Initially I suggested moving to CMake 3.11 or 3.12, I believe it<br>
> was James Knight who said the actual cost of updating the bots is the hard<br>
> part in raising the version, so maybe we should just take the newest. That<br>
> reasoning made sense to everyone at the roundtable and there were no<br>
> objections at the roundtable to moving to CMake 3.15. Look for an RFC from<br>
> me shortly that will propose that and lay out a timeline.<br>
> <br>
> <br>
> <br>
> We spent a lot of time talking about a handful of other topics that all<br>
> spring out from my RFC. There was a general agreement that the number of<br>
> options in the build system is unwieldy and the combinatoric effect of the<br>
> options is making the build system difficult to maintain. There was a<br>
> general agreement at least in principal that we should work to reduce the<br>
> complexity of the build system.<br>
> <br>
> <br>
> <br>
> We did discuss specific build system functionality that could be simplified<br>
> or removed, and on some points there was agreement, and on others there is<br>
> need for more discussion. A few examples:<br>
> <br>
> <br>
> <br>
> There was general agreement on the concept from my RFC that the `all` target<br>
> should always really be `all`. That would mean removing the<br>
> `LLVM_TOOL_*_BUILD` options.<br>
> <br>
> <br>
> <br>
> There was also agreement that in the monorepo it no longer makes sense to<br>
> have an option to specify the source locations of sub-projects. That means<br>
> we can remove the `LLVM_EXTERNAL_*_SOURCE_DIR` variables.<br>
> <br>
> <br>
> <br>
> There was disagreement over whether or not standalone builds of non-runtime<br>
> sub-projects should remain. This specifically would relate to clang and<br>
> lldb, and whether or not they can be built against installed or separately<br>
> built copies of LLVM. There were points on both side of this discussion, and<br>
> it will require more discussion to resolve.<br>
> <br>
> <br>
> <br>
> The roundtable also focused a lot of time on the runtime libraries, and how<br>
> they should be built. It is clear from the discussion that the runtime<br>
> libraries need to support being built standalone (separately from LLVM) in<br>
> order to fit into the various distribution strategies. Specifically many of<br>
> the runtime libraries are sometimes shipped as OS components rather than as<br>
> part of the toolchain, so you need to be able to build them separately from<br>
> the toolchain. Those same libraries can also be shipped as part of a<br>
> toolchain, so we need to support that workflow too. The need to support<br>
> these disparate workflows has led to much of the complexity. The proposal<br>
> from my RFC of standardizing runtimes builds as "standalone", and using the<br>
> LLVM runtimes directory (which uses CMake's ExternalProject) to configure<br>
> and build runtimes had pretty wide support in the discussion. I suspect this<br>
> is the direction we should move in, but with some slight changes.<br>
> <br>
> <br>
> <br>
> One of the points that came up was that building runtimes for multiple<br>
> platforms at once can be a configuration nightmare requiring hundreds of<br>
> build settings. It was suggested that it might be easier if you could build<br>
> all of the runtime libraries with a single CMake invocation separately from<br>
> LLVM. There are some dependency complications around the ordering of the<br>
> builtin libraries build, but otherwise this is largely doable.<br>
> <br>
> <br>
> <br>
> There are also problems related to cross-runtime dependencies which have<br>
> come up recently. Some of these problems can be addressed more cleanly with<br>
> modern CMake generator expressions, and other issues may require a larger<br>
> restructuring of code. For years there have been discussions thrown around<br>
> about breaking the builtins libraries out of compiler-rt. Maybe now is the<br>
> time to consider doing that.<br>
> <br>
> <br>
> <br>
> The last topic that was brought up at the end of the roundtable was about<br>
> supporting IDE generators. Saleem Abdulrasool pointed out that Visual Studio<br>
> ships a CMake integration that generates Ninja builds and supports the IDE<br>
> UI interactions as if it were a Visual Studio project without being a<br>
> project generator. Not supporting IDE build systems would clean up a lot of<br>
> complexity in our build system. As was pointed out during the discussion,<br>
> Xcode has no such support and is used by members of the community. There was<br>
> discussion of whether or not an Xcode+Ninja generator could be created which<br>
> would use Xcode's external build system mechanism to create an Xcode project<br>
> for browsing, editing, and debugging, while using Ninja to build. The<br>
> conversation had no real resolution other than "that would be cool" and "it<br>
> would be nice to remove all the `if (XCODE)` blocks".<br>
> <br>
> <br>
> <br>
> Thank you everyone who participated in the roundtable! If there is anything<br>
> I missed please help fill in the blanks.<br>
> <br>
> <br>
> <br>
> Thanks,<br>
> <br>
> -Chris<br>
> <br>
> <br>
><br>
> On Oct 24, 2019, at 10:28 AM, Alex Denisov via llvm-dev<br>
> <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<br>
> <br>
> <br>
> <br>
> That works perfectly. <br>
> <br>
> <br>
> <br>
> But I’m referring to the case when I want to add LLVM as a sub<br>
> project, which would allow me to debug/modify LLVM as if it is my<br>
> code.<br>
> <br>
> As I said, this also works, but some parts are not straightforward.<br>
> <br>
> <br>
> <br>
> On Thu 24. Oct 2019 at 18:15, Louis Dionne <<a href="mailto:ldionne@apple.com" target="_blank">ldionne@apple.com</a>> wrote:<br>
><br>
> <br>
> <br>
> <br>
><br>
> On Oct 24, 2019, at 02:17, Alex Denisov via<br>
> llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<br>
> <br>
> <br>
> <br>
> Hi Chris,<br>
> <br>
> <br>
> <br>
> This is a great initiative and it feels like the right<br>
> direction.<br>
> <br>
> <br>
> <br>
> I'd like to add another point to the list: using LLVM as a<br>
> library, i.e. being able to add it as a CMake subproject.<br>
> <br>
> Currently it works pretty good, but some parts can be<br>
> improved (somehow). E.g.:<br>
> <br>
> <br>
> <br>
> I believe the CMake-correct way of doing that is to produce LLVM<br>
> export files when installing LLVM, which would allow<br>
> find_package to work out of the box. You’d get LLVM targets you<br>
> can link against and all dependencies would be propagated by<br>
> CMake properly.<br>
> <br>
> <br>
> <br>
> <br>
> <br>
> Louis<br>
> <br>
> <br>
><br>
> - getting LLVM's version: the only way I've found<br>
> is to parse the CMakeLists.txt with regexes and hope<br>
> it doesn't break in the future<br>
> <br>
> - getting include dirs info: IIRC currently it works<br>
> transitively when one links against some library, but it<br>
> feels like a weak point to me<br>
> <br>
> - getting other configuration options (for example<br>
> whether LLVM is built with or without exceptions)<br>
> <br>
> - controlling how LLVM is built: the only way I found is<br>
> to force set a variable (i.e. set (LLVM_BUILD_32_BITS ON<br>
> CACHE BOOL "" FORCE)) before adding LLVM as a subproject<br>
> <br>
> <br>
> <br>
> I think my list is not exhaustive since it covers only my<br>
> use cases, so maybe there are other opinions/requests from<br>
> others.<br>
> <br>
> <br>
> <br>
> Cheers,<br>
> <br>
> Alex.<br>
> <br>
> <br>
> <br>
> On Mon, Oct 21, 2019 at 8:26 PM Chris Bieneman via<br>
> llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<br>
><br>
> Over the past few years the LLVM CMake build<br>
> system has gained a lot of new features, which<br>
> have powered new workflows and capabilities.<br>
> This development has largely been individual<br>
> efforts without focus or long-term direction.<br>
> The build system is of incredible importance<br>
> to LLVM as it is a primary interface for<br>
> contributors to build and test their changes.<br>
><br>
> This year, LLVM is making a huge<br>
> infrastructure shift to GitHub. Along with<br>
> that shift many of the previously supported<br>
> build options may not make sense. This is a<br>
> prefect opportunity to revisit the state of<br>
> our build infrastructure with an eye toward<br>
> the future.<br>
><br>
> I would like this RFC, and any discussion it<br>
> sparks, to serve as a starting point for a<br>
> conversation we can continue at the LLVM<br>
> Developer Meeting to set goals and conventions<br>
> for the development of the build system.<br>
><br>
> Tom Stellard has scheduled a roundtable on<br>
> CMake from 10:45-11:55 on Wednesday, Oct 23.<br>
><br>
> ## The Problem<br>
> Lacking clear direction and oversight the<br>
> build system is evolving in rapidly divergent<br>
> ways. Further since we don't have a formalized<br>
> process for unifying workflows and deprecating<br>
> old behaviors LLVM's build system has a<br>
> convoluted feature set.<br>
><br>
> This manifests itself in many unfortunate<br>
> ways. Some examples are:<br>
><br>
> (1) There are three different ways to build<br>
> compiler-rt as part of LLVM<br>
> (2) There are lots of incompatible build<br>
> configurations that need to be accounted for,<br>
> some that aren't (like<br>
> -DLLVM_BUILD_LLVM_DYLIB=On<br>
> -DBUILD_SHARED_LIBS=On, which will explode at<br>
> runtime)<br>
><br>
> As the build system gains complexity<br>
> maintaining the build system is getting more<br>
> expensive to the community.<br>
><br>
> ## Future Directions<br>
> The following are proposals to enable the<br>
> build system to better facilitate LLVM<br>
> development and provide a usable, extensible,<br>
> and stable build system for LLVM and all<br>
> sub-projects.<br>
><br>
> ### Updating CMake More Regularly<br>
> In the past we have clung to old versions of<br>
> CMake for extended periods of time. This has<br>
> resulted in significant checking<br>
> `CMAKE_VERSION` to enable some features or<br>
> being completely unable to use others. In<br>
> particular recent CMake development extending<br>
> generator expressions could provide<br>
> substantial benefit to the project. If we<br>
> stick to current upgrade policies, we may not<br>
> be able to use the current CMake release for<br>
> another few years.<br>
><br>
> As an alternative proposal, we should consider<br>
> CMake upgrades independent of OS package<br>
> manager versioning. That is not to say we<br>
> should take every new version of CMake,<br>
> however we should upgrade for compelling<br>
> reasons.<br>
><br>
> In tree right now we have code gated on CMake<br>
> 3.7 which enables CMake to generate Ninja<br>
> rules for tablegen that process tablegen's dep<br>
> files. This allows accurately rebuilding<br>
> tablegen when included files change. I propose<br>
> that this change's benefit should be<br>
> sufficient to justify moving to CMake 3.7 for<br>
> the project.<br>
><br>
> Additionally, building LLDB.framework on<br>
> Darwin requires CMake 3.8 due to bugs in<br>
> earlier versions of CMake. This could also be<br>
> a justification for updating.<br>
> Lastly, getting updated versions of CMake is<br>
> very easy. Kitware provides Windows, Mac and<br>
> Linux builds on <a href="http://cmake.org" rel="noreferrer" target="_blank">cmake.org</a> as well as an Ubuntu<br>
> apt source. If that is insufficient building<br>
> CMake from source is simple, and has minimal<br>
> system requirements. Visual Studio contains<br>
> reasonably up-to-date CMake bundled. As such<br>
> we should not allow OS release or support<br>
> cycles to dictate when we upgrade CMake.<br>
><br>
> ### Reducing the Test Matrix<br>
> The most important guiding principal for<br>
> development of the LLVM build system must be<br>
> to reduce the matrix of configurations. The<br>
> more possible configurations the build system<br>
> supports the wider the test matrix. This is<br>
> not to say the build system should support<br>
> doing less, but rather to support less unique<br>
> configurations.<br>
><br>
> Many configuration options in the build system<br>
> just turn on or off different parts of the<br>
> project. For example, the<br>
> `LLVM_BUILD_LLVM_DYLIB` option just disables<br>
> configuring libLLVM. An alternative approach<br>
> would be to always configure libLLVM, and<br>
> leave it up to users of the build system to<br>
> determine whether or not to build it.<br>
><br>
> We also have options to enable and disable<br>
> configuring individual tools in the LLVM and<br>
> Clang projects. I believe we should eliminate<br>
> those options, which will result in the `all`<br>
> target always being everything. We have<br>
> explicit clean dependencies for the `check-*`<br>
> targets so most developer workflows should be<br>
> un-impacted. Distribution workflows can use<br>
> the `LLVM_DISTRIBUTION_COMPONENTS` option to<br>
> hand tailor which parts of LLVM to build and<br>
> install with better control without as much<br>
> complication.<br>
><br>
> Many other options exist to support a wide<br>
> variety of divergent workflows. For example,<br>
> the `LLVM_EXTERNAL_${PROJECT}_SOURCE_DIR`<br>
> options exist to allow users to specify custom<br>
> paths for projects so that, historically, they<br>
> didn't need to nest clang inside LLVM. With<br>
> the move to the mono-repo we should define<br>
> consistent workflows and eliminate options<br>
> that support divergent workflows.<br>
><br>
> ### Adopting Conventions<br>
> Much of LLVM's build system is not idiomatic<br>
> CMake. Some of those differences make sense<br>
> because LLVM is not a typical software<br>
> project. I'm unaware of any build<br>
> configuration system that was designed<br>
> specifically to build compilers and handle the<br>
> complex dependency chains that come with that<br>
> territory.<br>
><br>
> Some of our divergences come from history. We<br>
> have a great many features implemented in our<br>
> CMake because CMake didn't support them at the<br>
> time. We also have patterns that were<br>
> appropriate before CMake added new features,<br>
> and have never cleaned them up.<br>
><br>
> One big thing our build system needs is a set<br>
> of guiding conventions to direct future<br>
> development. Some key conventions that I<br>
> believe are crucial:<br>
><br>
> #### Avoid Order Dependent Behavior<br>
> CMake generator expressions provide the<br>
> ability to defer logic until after script<br>
> processing. This allows the build system to<br>
> avoid direct dependence on the order in which<br>
> targets are processed. We should not use the<br>
> `if(TARGET ...)` or `get_target_property`<br>
> interfaces unless it is completely impossible<br>
> to avoid.<br>
><br>
> #### Avoid Options to Enable/Disable<br>
> Configuration<br>
> If we reduce the test matrix, having a<br>
> convention to keep it reduced is of vital<br>
> importance so that we don't find ourselves<br>
> needing to clean up again in a few years.<br>
><br>
> #### Avoid Caching, Use `mark_as_advanced` and<br>
> `INTERNAL` Liberally<br>
> CMake has no strategy for cache invalidation.<br>
> As such, cached variables add additional<br>
> maintenance burden because they can break<br>
> builds sometimes in hard to diagnose ways.<br>
> That said they are useful. In particular for<br>
> things like configuration checks that are slow<br>
> caching the result makes incremental<br>
> re-configuration much faster. We should use<br>
> cached values sparingly and only where they<br>
> provide benefit.<br>
><br>
> Additionally, every cached CMake variable is a<br>
> configuration point. Variables not marked<br>
> `INTERNAL` show up in `ccmake` and<br>
> `cmake-gui`, and variables not<br>
> `mark_as_advanced` show up to all users. We<br>
> should use the `INTERNAL` and<br>
> `mark_as_advanced` options wherever<br>
> appropriate to limit our supported<br>
> configuration interface.<br>
><br>
> #### Making Sense of Runtime Builds<br>
> Right now, there are three different ways to<br>
> build compiler-rt as part of LLVM and two<br>
> different ways to build most of the other<br>
> runtime libraries (libcxxabi, libcxx,<br>
> libunwind, etc). This situation is confusing<br>
> even for long time contributors.<br>
><br>
> We need a clearer story for building runtime<br>
> libraries to reduce the number of different<br>
> ways they are built and provide simplified<br>
> workflows for users.<br>
><br>
> It is my opinion that if you are building a<br>
> runtime library as part of an LLVM/Clang<br>
> build, it should be configured and built with<br>
> the in-tree clang as it would be for<br>
> distribution. If you don't want to build with<br>
> the in-tree clang, we should encourage people<br>
> to build the runtime libraries independently<br>
> of the compiler.<br>
><br>
> My reasoning for this is that distributions of<br>
> clang are generally built from the default<br>
> settings in the build and configuration<br>
> process, and distributions (or installs by new<br>
> users) which include the runtime libraries<br>
> should have runtimes built with the just-built<br>
> compiler. To align these two situations we<br>
> need the default build configuration of<br>
> LLVM+Clang+Runtimes to be using the just-built<br>
> compiler.<br>
><br>
> Adopting this change would mean runtime<br>
> library projects would only contain build<br>
> system support for building "standalone"<br>
> meaning not in the same configuration as LLVM.<br>
> We would then support runtime libraries built<br>
> as individual projects or using the LLVM<br>
> runtimes directory, which separately<br>
> configures and builds runtime libraries using<br>
> the just-built clang.<br>
> _______________________________________________<br>
> LLVM Developers mailing list<br>
> <a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
> <a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
> <br>
> _______________________________________________<br>
> LLVM Developers mailing list<br>
> <a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
> <a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
> <br>
> <br>
> <br>
> _______________________________________________<br>
> LLVM Developers mailing list<br>
> <a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
> <a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
> <br>
> <br>
> <br>
> <br>
>_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote></div></div>