[llvm-dev] [libcxx-dev] [RFC] A vision for building the runtimes

Fri Oct 23 07:23:41 PDT 2020

> On Oct 23, 2020, at 06:09, Martin Storsjö <martin at martin.st> wrote:
> 
> Hi,
> 
> On Thu, 22 Oct 2020, Louis Dionne via libcxx-dev wrote:
> 
>> Proposal
>> --------------
>> 
>> My goal with this proposal is to achieve:
>> 1. Decoupling from the top-level LLVM CMake setup (which doesn't work, see above)
>> 2. A simple build that works everywhere, including embedded platforms
>> 3. Remove the need to manually tie together the various runtimes (as in the Standalone builds)
>> 
>> My proposal is basically to have a "Unified Standalone" build for all the runtimes. It would look similar to a Monorepo build in essence (i.e. you'd have a single CMake invocation where you would specify the flags for all runtime projects at once), but it wouldn't be using the top-level LLVM CMake setup [1]. Specifically:
>> 
>> 1. Add a `runtimes/CMakeLists.txt` file that includes the runtimes subprojects that are requested through -DLLVM_ENABLE_PROJECTS (open to bikeshed), and sets up minimal stuff like the `llvm-lit` wrapper and Python, but none of the harmful stuff that's done by the top-level LLVM CMake.
>> 2. Deprecate the old individual Standalone builds for this new "Unified Standalone build".
>> 3. Users migrate to the new Unified Standalone build. Users include the current "Runtimes" build, some places in compiler-rt, and various organizations.
>> 4. Remove support for the old individual Standalone builds.
> 
> Sounds potentially good, but I hope the removal can wait until e.g. after the next stable release sometime (ideally removed from the master branch only after the next stable release has happened, not only forked off), as I try to have a single set of build scripts that work both for the latest stable release and the current top-of-tree master branch.
> 
> I presume the main place where you want to avoid other kinds of builds is for the combination of libcxxabi+libcxx, and that's indeed a bit of a mess at the moment.
> 
> Would it still be possible to build only compiler-rt/lib/builtins and not all of compiler-rt?

Yes. We wouldn't make any such changes to compiler-rt. For now, I don't even want to try to include compiler-rt in that Unified Standalone build: I want to focus on making libc++, libc++abi and libunwind work in that setup.

> So while it'd be nice to bundle as much as possible up in a Unified Standalone build, I very much like the fact that I can pick the individual build steps in the order I want, needed to assemble things from scratch for my setup. If I later can merge more of them into a single cmake invocation (or at least fewer), that'd be an optional bonus. Building all of libunwind+libcxxabi+libcxx in one cmake build is something I'd like to do in any case.

I hear you. As I said above, my goal is basically to have a single invocation for libunwind, libc++abi and libc++. We can then try to also make compiler-rt work the same, but I think it might require significant refactoring of compiler-rt's build before we can do that I don't know enough to say for sure, but I'm sure I can get assistance from e.g. beanz and phosek if we do this.

> 
> With Unified Standalone builds, I can probably do most of that by still doing many individual cmake invocations, picking a different set of runtime projects to build each time. But what about e.g. picking only compiler-rt/lib/builtins?

Yes, no plan to disable that.

> 
> One thing I'm left wanting in this setup, is that for various reasons I've currently been passing different sets of extra flags (compiler and linker flags) to each of libunwind/libcxxabi/libcxx. Some of it is included in the patch I'll post soon, reducing the need for it though. But I've been building e.g. libunwind with -Wno-dll-attribute-on-redeclaration, but not the other libs. (Yes if that's a permanent setup, I should probably upstream setting that flag as well.)

Why doesn't this flag apply to all the runtimes?

> 
> But for general cases, when trying out various build setups, being able to pass individual CMAKE_CXX_FLAGS or CMAKE_SHARED_LINKER_FLAGS to each of libunwind/libcxxabi/libcxx would be very useful, especially if doing something that isn't necessarily supported (yet).

I believe the correct place for that is in a downstream-only patch being applied to your fork of llvm. That's how we do it at Apple, and it avoids having to maintain all kinds of hooks to customize X and Y upstream. I'm not saying these hooks don't even make sense -- they make sense most of the time. But if we can avoid adding an option to customize all the flags for each runtime, I think that would be ideal.

> On Oct 23, 2020, at 06:44, Martin Storsjö <martin at martin.st> wrote:
> 
> On Thu, 22 Oct 2020, Louis Dionne via libcxx-dev wrote:
> 
>> - Monorepo
>>   This is the "easy" and most common way to build the runtimes. It builds the runtimes as subprojects of LLVM (with LLVM_ENABLE_PROJECTS), with the same compiler that's used to build LLVM.
> 
> Btw, unrelated to the topic at hand, but related to monorepo builds in general, as I've brought up some time before - it'd be great for various reasons to be able to do such a build (building the tools, not the runtimes) by pointing cmake directly at the llvm-project root instead of at the llvm subdir.
> 
> Last time this was brought up, Chris [1] mentioned that it'd be doable, but would require a bit of work to make the produced files (<builddir>/bin/clang etc) end up in the root of the build dir, instead of in a subdir, like <builddir>/llvm/bin/clang. My cmake-fu is a bit too weak to easily pinpoint what needs to be fixed to proceed with this though...

Interesting -- I also brought this up recently: http://lists.llvm.org/pipermail/llvm-dev/2020-June/142384.html

However, the issue I've been hitting with this is that it's a huge chunk of work to bite. Making the runtimes work only is something I can chew, but it's another story entirely to make it work for arbitrary sub-projects.

Louis