[libcxx-dev] [RFC] A vision for building the runtimes
Martin Storsjö via libcxx-dev
libcxx-dev at lists.llvm.org
Fri Oct 23 03:09:42 PDT 2020
On Thu, 22 Oct 2020, Louis Dionne via libcxx-dev wrote:
> My goal with this proposal is to achieve:
> 1. Decoupling from the top-level LLVM CMake setup (which doesn't work, see above)
> 2. A simple build that works everywhere, including embedded platforms
> 3. Remove the need to manually tie together the various runtimes (as in the Standalone builds)
> My proposal is basically to have a "Unified Standalone" build for all the runtimes. It would look similar to a Monorepo build in essence (i.e. you'd have a single CMake invocation where you would specify the flags for all runtime projects at once), but it wouldn't be using the top-level LLVM CMake setup . Specifically:
> 1. Add a `runtimes/CMakeLists.txt` file that includes the runtimes subprojects that are requested through -DLLVM_ENABLE_PROJECTS (open to bikeshed), and sets up minimal stuff like the `llvm-lit` wrapper and Python, but none of the harmful stuff that's done by the top-level LLVM CMake.
> 2. Deprecate the old individual Standalone builds for this new "Unified Standalone build".
> 3. Users migrate to the new Unified Standalone build. Users include the current "Runtimes" build, some places in compiler-rt, and various organizations.
> 4. Remove support for the old individual Standalone builds.
Sounds potentially good, but I hope the removal can wait until e.g. after
the next stable release sometime (ideally removed from the master branch
only after the next stable release has happened, not only forked off), as
I try to have a single set of build scripts that work both for the latest
stable release and the current top-of-tree master branch.
I presume the main place where you want to avoid other kinds of builds is
for the combination of libcxxabi+libcxx, and that's indeed a bit of a mess
at the moment.
Would it still be possible to build only compiler-rt/lib/builtins and not
all of compiler-rt?
The prime reason why the current "runtimes" (as you call toolchain) builds
aren't uasble for me, is that while it does the right thing (first build a
compiler, then use that compiler to build runtimes) is that I need to do a
number of other things inbetween building the compiler and various bits of
the runtime, and I need (or just want?) to micromanage the process.
My current procedure for bootstrapping a cross toolchain from scratch
amounts to this:
- Build the compiler and tools
- Install the mingw headers and base runtime, set up compiler frontend
- Build compiler-rt/lib/builtins only
(At this point, I have an essentially complete toolchain for C code.)
- Build libunwind+libcxxabi+libcxx (with tight interdependencies)
- Build all of compiler-rt (sanitizers require having a working set of C++
headers and other things)
So while it'd be nice to bundle as much as possible up in a Unified
Standalone build, I very much like the fact that I can pick the individual
build steps in the order I want, needed to assemble things from scratch
for my setup. If I later can merge more of them into a single cmake
invocation (or at least fewer), that'd be an optional bonus. Building all
of libunwind+libcxxabi+libcxx in one cmake build is something I'd like to
do in any case.
With Unified Standalone builds, I can probably do most of that by still
doing many individual cmake invocations, picking a different set of
runtime projects to build each time. But what about e.g. picking only
>  If you're wondering what that would look like:
> $ mkdir <monorepo-root>/build
> $ cd <monorepo-root>/build
> $ cmake ../runtimes -DLLVM_ENABLE_PROJECTS="libcxx;libcxxabi;libunwind" \
> -C <path-to-your-cache-if-desired> \
> -DLIBCXX_ENABLE_XXX \
> -DLIBCXXABI_ENABLE_XXX \
> <other options>
> $ ninja install-cxx install-cxxabi
That looks good, and I tried it out with the cmake file you shared - with
a small amount of tweaks I can use that in my current setups - I'll send a
patch for that for discussion.
One thing I'm left wanting in this setup, is that for various reasons I've
currently been passing different sets of extra flags (compiler and linker
flags) to each of libunwind/libcxxabi/libcxx. Some of it is included in
the patch I'll post soon, reducing the need for it though. But I've been
building e.g. libunwind with -Wno-dll-attribute-on-redeclaration, but not
the other libs. (Yes if that's a permanent setup, I should probably
upstream setting that flag as well.)
But for general cases, when trying out various build setups, being able to
pass individual CMAKE_CXX_FLAGS or CMAKE_SHARED_LINKER_FLAGS to each of
libunwind/libcxxabi/libcxx would be very useful, especially if doing
something that isn't necessarily supported (yet).
More information about the libcxx-dev