[libcxx-dev] [llvm-dev] RFC: A top level monorepo CMake file

[Steven Wu via libcxx-dev]

> On Jun 18, 2020, at 11:40 AM, Josh Stone <jistone at redhat.com> wrote:
> 
> On 6/18/20 11:27 AM, Steven Wu via llvm-dev wrote:
>> I like the proposal but I would like to go even further. If we are going
>> to create a top level CMake file, we should just go ahead and eliminate
>> all the standalone build configuration. The standalone build should just
>> be `cmake <monorepo-root> -DLLVM_ENABLE_PROJECTS=standalone-project
>> ...`. That means less build configuration to maintain which is always good.
> 
> This would break the release of standalone source tarballs, no?

That is a problem we can fix. We just need to ship those tarballs with some processing by including the top-level cmake and cmake modules, and removing all other subdirectories.
I don't think we have split standalone repo so it is just about release management. Yes, all the clients will have to build standalone projects differently so it is a workflow break but it should be acceptable.

Steven

> 
>> That also means that we can refactor some of the cmake modules to put
>> into the top level directories so sub-projects can just reuse them
>> without duping the code worrying they are not available in the
>> standalone build today.
>> 
>> Steven
>> 
>>> On Jun 18, 2020, at 10:57 AM, Louis Dionne via llvm-dev
>>> <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>>> 
>>> Hi folks,
>>> 
>>> Building any LLVM project currently requires invoking CMake inside
>>> <monorepo-root>/llvm, while setting the projects to enable in the
>>> LLVM_ENABLE_PROJECTS variable. This has the downside that CMake
>>> processing for the LLVM subproject happens even when one doesn't
>>> really need or want it. It's also not great from a build hygiene
>>> perspective, as LLVM globally sets some flags and subprojects pick
>>> them up, when they don't really mean to. For example, see this
>>> workaround:
>>> https://github.com/llvm/llvm-project/blob/master/libcxx/CMakeLists.txt#L503-L507,
>>> where we need to account for some flags that might have been set
>>> globally by LLVM.
>>> 
>>> I'm not sure about other projects, however this is quite problematic
>>> for projects part of the C++ runtime (libc++/libc++abi/libunwind).
>>> Indeed, we often try to build those projects targetting not widely
>>> supported platforms, where the overall LLVM build doesn't work. For
>>> example, trying to use the LLVM_ENABLE_PROJECTS approach for building
>>> libc++ for Apple's DriverKit environment doesn't work, since it has a
>>> few unusual things that the LLVM build chokes on. However, building
>>> libc++ standalone works just fine because it has far fewer
>>> requirements. It's also not just an issue of working vs not working:
>>> because of global flag pollution, building libc++ standalone and as
>>> part of the rest of LLVM can result in slightly different flags being
>>> used, which could cause important and hard-to-diagnose issues.
>>> 
>>> Hence, I think we should introduce a way to build LLVM projects (or at
>>> least the runtimes) without going through
>>> <monorepo-root>/llvm/CMakeLists.txt. What I suggest is to have a
>>> top-level <monorepo-root>/CMakeLists.txt whose sole job is to include
>>> subprojects. We could also place basic LLVM-wide things like the check
>>> for the minimum CMake version there. More specifically, I would like
>>> to be able to do:
>>> 
>>> Â  Â  $ cd <monorepo-root>
>>> Â  Â  $ mkdir build
>>> Â  Â  $ (cd build && cmake <monorepo-root>
>>> -DLLVM_ENABLE_PROJECTS="<projects-to-enable>")
>>> 
>>> Pretty much the only difference with today is that you'd use `cmake
>>> <monorepo-root>` instead of `cmake <monorepo-root>/llvm`.
>>> 
>>> Like I said, this is a problem for the runtime projects, but I'm not
>>> sure about other projects. For the runtime projects, another option
>>> would be to only allow standalone builds. However, the runtime
>>> projects are often built in lockstep, and so running three CMake
>>> commands when one would suffice is both annoying and also an easy way
>>> to screw things up. Furthermore, the current standalone builds add
>>> complexity to the projects, because they require the ability to point
>>> to arbitrary headers/libraries from the other projects, when we really
>>> always want to point to the just-built ones.
>>> 
>>> Relationship with Petr Hosek's "Runtimes" build
>>> ---------------------------------------------------------------
>>> What I'm proposing isn't a replacement for itl. The "Runtimes" build
>>> can be seen as a driver that sets up the individual
>>> libc++/libc++abi/libunwind builds with the just-built toolchain, and
>>> for the provided targets. That's really great, however it is built *on
>>> top of* the basic libc++/libc++abi/libunwind builds. So basically,
>>> after my proposal, the "Runtimes" build could simply build all
>>> elements from the runtime with a single CMake invocation, as opposed
>>> to multiple invocations.
>>> 
>>> Thoughts?
>>> Louis
>>> 
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> 
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> 
>