[libcxx-dev] [llvm-dev] RFC: A top level monorepo CMake file

Thu Jun 18 11:46:09 PDT 2020

> On Jun 18, 2020, at 14:40, Josh Stone <jistone at redhat.com> wrote:
> 
> On 6/18/20 11:27 AM, Steven Wu via llvm-dev wrote:
>> I like the proposal but I would like to go even further. If we are going
>> to create a top level CMake file, we should just go ahead and eliminate
>> all the standalone build configuration. The standalone build should just
>> be `cmake <monorepo-root> -DLLVM_ENABLE_PROJECTS=standalone-project
>> ...`. That means less build configuration to maintain which is always good.
> 
> This would break the release of standalone source tarballs, no?

I guess it would, however they are already broken at least for libc++, libc++abi and libunwind. These projects don't support being built outside of the monorepo layout. They require many similar things and share a lot of code, and we've started implementing nice simplifications since we agreed to require the monorepo layout.

IMO, this is an unavoidable (nice) consequence of moving to the git monorepo.

Louis

> 
>> That also means that we can refactor some of the cmake modules to put
>> into the top level directories so sub-projects can just reuse them
>> without duping the code worrying they are not available in the
>> standalone build today.
>> 
>> Steven
>> 
>>> On Jun 18, 2020, at 10:57 AM, Louis Dionne via llvm-dev
>>> <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>>> 
>>> Hi folks,
>>> 
>>> Building any LLVM project currently requires invoking CMake inside
>>> <monorepo-root>/llvm, while setting the projects to enable in the
>>> LLVM_ENABLE_PROJECTS variable. This has the downside that CMake
>>> processing for the LLVM subproject happens even when one doesn't
>>> really need or want it. It's also not great from a build hygiene
>>> perspective, as LLVM globally sets some flags and subprojects pick
>>> them up, when they don't really mean to. For example, see this
>>> workaround:
>>> https://github.com/llvm/llvm-project/blob/master/libcxx/CMakeLists.txt#L503-L507,
>>> where we need to account for some flags that might have been set
>>> globally by LLVM.
>>> 
>>> I'm not sure about other projects, however this is quite problematic
>>> for projects part of the C++ runtime (libc++/libc++abi/libunwind).
>>> Indeed, we often try to build those projects targetting not widely
>>> supported platforms, where the overall LLVM build doesn't work. For
>>> example, trying to use the LLVM_ENABLE_PROJECTS approach for building
>>> libc++ for Apple's DriverKit environment doesn't work, since it has a
>>> few unusual things that the LLVM build chokes on. However, building
>>> libc++ standalone works just fine because it has far fewer
>>> requirements. It's also not just an issue of working vs not working:
>>> because of global flag pollution, building libc++ standalone and as
>>> part of the rest of LLVM can result in slightly different flags being
>>> used, which could cause important and hard-to-diagnose issues.
>>> 
>>> Hence, I think we should introduce a way to build LLVM projects (or at
>>> least the runtimes) without going through
>>> <monorepo-root>/llvm/CMakeLists.txt. What I suggest is to have a
>>> top-level <monorepo-root>/CMakeLists.txt whose sole job is to include
>>> subprojects. We could also place basic LLVM-wide things like the check
>>> for the minimum CMake version there. More specifically, I would like
>>> to be able to do:
>>> 
>>> Â  Â  $ cd <monorepo-root>
>>> Â  Â  $ mkdir build
>>> Â  Â  $ (cd build && cmake <monorepo-root>
>>> -DLLVM_ENABLE_PROJECTS="<projects-to-enable>")
>>> 
>>> Pretty much the only difference with today is that you'd use `cmake
>>> <monorepo-root>` instead of `cmake <monorepo-root>/llvm`.
>>> 
>>> Like I said, this is a problem for the runtime projects, but I'm not
>>> sure about other projects. For the runtime projects, another option
>>> would be to only allow standalone builds. However, the runtime
>>> projects are often built in lockstep, and so running three CMake
>>> commands when one would suffice is both annoying and also an easy way
>>> to screw things up. Furthermore, the current standalone builds add
>>> complexity to the projects, because they require the ability to point
>>> to arbitrary headers/libraries from the other projects, when we really
>>> always want to point to the just-built ones.
>>> 
>>> Relationship with Petr Hosek's "Runtimes" build
>>> ---------------------------------------------------------------
>>> What I'm proposing isn't a replacement for itl. The "Runtimes" build
>>> can be seen as a driver that sets up the individual
>>> libc++/libc++abi/libunwind builds with the just-built toolchain, and
>>> for the provided targets. That's really great, however it is built *on
>>> top of* the basic libc++/libc++abi/libunwind builds. So basically,
>>> after my proposal, the "Runtimes" build could simply build all
>>> elements from the runtime with a single CMake invocation, as opposed
>>> to multiple invocations.
>>> 
>>> Thoughts?
>>> Louis
>>> 
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> 
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> 
> 