[cfe-dev] [llvm-dev] Upcoming change with how libc++, libc++abi and libunwind are being built

[Michael Kruse via cfe-dev]

What should the buildbots do after this change? Should they all use
the bootstrap build? I am worried that this would reduces the test
coverage since it would only build the runtimes with the top-of-tree
clang. As by [1], at least libc++ supports also compilation with gcc
and AppleClang.

Name suggestion for the "default" build: "single target" build. Since
only the bootstrap build supports multiple targets with
LLVM_RUNTIME_TARGETS.

[1] https://libcxx.llvm.org/

Am Mo., 11. Okt. 2021 um 11:18 Uhr schrieb Louis Dionne via llvm-dev
<llvm-dev at lists.llvm.org>:
>
> Hi folks,
>
> This message is both a heads up and a request for comments on an upcoming change we want to make to the way we build libc++/libc++abi/libunwind.
>
> For a long time now, those three projects have supported various ways to be built:
>
> 1. The Project build, where one would use <root>/llvm as the root of the CMake invocation and pass LLVM_ENABLE_PROJECTS.
> 2. The Standalone build, where one would perform one CMake invocation per project being built, and fill in various CMake variables to tell each project where to find bits of the other projects.
> 3. The "Runtimes" or "Bootstrapping" build, where one would use <root>/llvm as the root of the CMake invocation and pass LLVM_ENABLE_RUNTIMES. This would result in Clang being built, and then the runtimes being built with that fresh Clang.
> 4. The "Unified Standalone" build, where one would use <root>/libcxx/utils/ci/runtimes as the root of the CMake invocation and pass LLVM_ENABLE_PROJECTS. This was introduced to work around the fact that the Project build doesn't work on several embedded platforms, and I think it is only used at Apple.
>
> Through the years, this has gotten rather confusing and difficult to support. We currently support all of those on a best-effort basis and we do have build bots checking each of them, however the complexity is often getting in the way of providing proper support and improving the build system of those projects. We've been trying to migrate to a simpler way to build the runtimes that works for all use cases (notably embedded platforms) for a long time, and we recently got to a point where this is possible (thanks mainly to @phosek and @mstorsjo). To summarize what we're aiming for, we'll now have two ways of building libc++/lib++abi/libunwind:
>
> 1. The "Default" build, where the CMake invocation is rooted at <root>/runtimes and one uses LLVM_ENABLE_RUNTIMES.
> 2. The "Bootstrapping" build, where the CMake invocation is rooted at <root>/llvm and one uses LLVM_ENABLE_RUNTIMES (this is the current "Bootstrapping" build unchanged).
>
> If you have been using the "Project", "Standalone" or "Unified Standalone" builds, you should be moving to the new "Default" build. This should be really easy to achieve, basically you can just change LLVM_ENABLE_PROJECTS to LLVM_ENABLE_RUNTIMES, perform a single invocation of CMake instead of multiple invocations, and pass all the LIBCXX_FOO, LIBCXXABI_FOO and LIBUNWIND_FOO options to that single CMake invocation. See the release note in the PR linked below for more details.
>
> We are aiming to deprecate all the old ways to build as soon as possible, and to keep supporting them until after the LLVM 14 release, at which point we would like to remove support for those and get started with making improvements. The review at https://reviews.llvm.org/D111356 enshrines this deprecation in the documentation. For technical discussion on this proposed change, please comment on that review to keep everything in one place.
>
> Cheers,
> Louis
>
> Questions you might be asking yourself:
>
> Q: Why doesn't the Project build work for embedded platforms?
> A: A lot of CMake code in <root>/llvm expects that we're building for a target with the usual capabilities for a build host, but that isn't the case for several embedded targets. While it doesn't make sense to build LLVM or Clang for those embedded targets (which would never be used as build hosts), it makes sense to build the runtimes for those targets if you are cross-compiling.
>
> Q: Currently, we use the Standalone build and are not required to have all of the Monorepo available. Does this change our ability to do that?
> A: Not in the mid term. Building any of the runtimes already requires all the runtimes to be co-located. The only thing you could get away with previously was not having the part in <root>/llvm. With the "Default" build, this will be possible as soon as we move a few CMake helpers from <root>/llvm/cmake to <root>/cmake. Then, you'll only have to have <root>/runtimes, <root>/cmake and the projects you are building.
>
> Q: What's the difference between the "Runtimes build" and the "Bootstrapping build"?
> A: None. We're trying to move away from the term "Runtimes build" because it's waaaay too confusing, and doesn't describe the fact that we're really building the runtimes with a just-built Clang.
>
> Q: What's up with naming the new default build the "Default build"?
> A: The name sucks, but we had to pick one and the "Runtimes build" was already taken for the build that does a bootstrap. Suggestions are welcome -- basically what this performs is a normal build of the runtimes using your system compiler, like you used to do with LLVM_ENABLE_PROJECTS (but it's better).
>
> Q: What are the benefits of this migration beyond simplifying the code for maintainers?
> A: Reducing the number of ways we can build the runtimes means that all the workflows for building the runtimes will become simpler for users (especially vendors). Furthermore, the new "Default" build is as simple as the old Project build, but it is faster to configure and works on embedded platforms. In the near future, it should also be possible to perform the equivalent of an old "Project build" without having all of the monorepo available.
>
> Q: Does this affect other "runtimes" projects like compiler-rt?
> A: Not for the time being. We will probably attempt to move those over in the future as well, however for the time being I can only confirm that this works fine for libc++, libc++abi and libunwind, so that's where my deprecation efforts are focused.
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev