[libcxx-dev] [llvm-dev] Upcoming change with how libc++, libc++abi and libunwind are being built

[Andrzej Warzynski via libcxx-dev]

Hi Louis,

This sounds like a huge simplification, thank you!

With this change, will we still be able to use LLVM_ENABLE_PROJECTS to 
build libc++, libc++abi and libunwind? IIUC, you are disabling this 
option entirely, right? Will CMake warn us about the 
incorrect/deprecated usage? Should the LLVM documentation be updated [1]?

Thank you,
-Andrzej

[1] https://llvm.org/docs/CMake.html

On 11/10/2021 17:17, Louis Dionne via llvm-dev wrote:
> Hi folks,
> 
> This message is both a heads up and a request for comments on an 
> upcoming change we want to make to the way we build 
> libc++/libc++abi/libunwind.
> 
> For a long time now, those three projects have supported various ways to 
> be built:
> 
> 1. The Project build, where one would use <root>/llvm as the root of the 
> CMake invocation and pass LLVM_ENABLE_PROJECTS.
> 2. The Standalone build, where one would perform one CMake invocation 
> per project being built, and fill in various CMake variables to tell 
> each project where to find bits of the other projects.
> 3. The "Runtimes" or "Bootstrapping" build, where one would use 
> <root>/llvm as the root of the CMake invocation and pass 
> LLVM_ENABLE_RUNTIMES. This would result in Clang being built, and then 
> the runtimes being built with that fresh Clang.
> 4. The "Unified Standalone" build, where one would use 
> <root>/libcxx/utils/ci/runtimes as the root of the CMake invocation and 
> pass LLVM_ENABLE_PROJECTS. This was introduced to work around the fact 
> that the Project build doesn't work on several embedded platforms, and I 
> think it is only used at Apple.
> 
> Through the years, this has gotten rather confusing and difficult to 
> support. We currently support all of those on a best-effort basis and we 
> do have build bots checking each of them, however the complexity is 
> often getting in the way of providing proper support and improving the 
> build system of those projects. We've been trying to migrate to a 
> simpler way to build the runtimes that works for all use cases (notably 
> embedded platforms) for a long time, and we recently got to a point 
> where this is possible (thanks mainly to @phosek and @mstorsjo). To 
> summarize what we're aiming for, we'll now have two ways of building 
> libc++/lib++abi/libunwind:
> 
> 1. The "Default" build, where the CMake invocation is rooted at 
> <root>/runtimes and one uses LLVM_ENABLE_RUNTIMES.
> 2. The "Bootstrapping" build, where the CMake invocation is rooted at 
> <root>/llvm and one uses LLVM_ENABLE_RUNTIMES (this is the current 
> "Bootstrapping" build unchanged).
> 
> If you have been using the "Project", "Standalone" or "Unified 
> Standalone" builds, you should be moving to the new "Default" build. 
> This should be really easy to achieve, basically you can just change 
> LLVM_ENABLE_PROJECTS to LLVM_ENABLE_RUNTIMES, perform a single 
> invocation of CMake instead of multiple invocations, and pass all the 
> LIBCXX_FOO, LIBCXXABI_FOO and LIBUNWIND_FOO options to that single CMake 
> invocation. See the release note in the PR linked below for more details.
> 
> We are aiming to deprecate all the old ways to build as soon as 
> possible, and to keep supporting them until after the LLVM 14 release, 
> at which point we would like to remove support for those and get started 
> with making improvements. The review at https://reviews.llvm.org/D111356 
> <https://reviews.llvm.org/D111356> enshrines this deprecation in the 
> documentation. For technical discussion on this proposed change, please 
> comment on that review to keep everything in one place.
> 
> Cheers,
> Louis
> 
> Questions you might be asking yourself:
> 
> Q: Why doesn't the Project build work for embedded platforms?
> A: A lot of CMake code in <root>/llvm expects that we're building for a 
> target with the usual capabilities for a build host, but that isn't the 
> case for several embedded targets. While it doesn't make sense to build 
> LLVM or Clang for those embedded targets (which would never be used as 
> build hosts), it makes sense to build the runtimes for those targets if 
> you are cross-compiling.
> 
> Q: Currently, we use the Standalone build and are not required to have 
> all of the Monorepo available. Does this change our ability to do that?
> A: Not in the mid term. Building any of the runtimes already requires 
> all the runtimes to be co-located. The only thing you could get away 
> with previously was not having the part in <root>/llvm. With the 
> "Default" build, this will be possible as soon as we move a few CMake 
> helpers from <root>/llvm/cmake to <root>/cmake. Then, you'll only have 
> to have <root>/runtimes, <root>/cmake and the projects you are building.
> 
> Q: What's the difference between the "Runtimes build" and the 
> "Bootstrapping build"?
> A: None. We're trying to move away from the term "Runtimes build" 
> because it's waaaay too confusing, and doesn't describe the fact that 
> we're really building the runtimes with a just-built Clang.
> 
> Q: What's up with naming the new default build the "Default build"?
> A: The name sucks, but we had to pick one and the "Runtimes build" was 
> already taken for the build that does a bootstrap. Suggestions are 
> welcome -- basically what this performs is a normal build of the 
> runtimes using your system compiler, like you used to do with 
> LLVM_ENABLE_PROJECTS (but it's better).
> 
> Q: What are the benefits of this migration beyond simplifying the code 
> for maintainers?
> A: Reducing the number of ways we can build the runtimes means that all 
> the workflows for building the runtimes will become simpler for users 
> (especially vendors). Furthermore, the new "Default" build is as simple 
> as the old Project build, but it is faster to configure and works on 
> embedded platforms. In the near future, it should also be possible to 
> perform the equivalent of an old "Project build" without having all of 
> the monorepo available.
> 
> Q: Does this affect other "runtimes" projects like compiler-rt?
> A: Not for the time being. We will probably attempt to move those over 
> in the future as well, however for the time being I can only confirm 
> that this works fine for libc++, libc++abi and libunwind, so that's 
> where my deprecation efforts are focused.
> 
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>