[libcxx-dev] [llvm-dev] RFC: A top level monorepo CMake file

[Manoj Gupta via libcxx-dev]

On Thu, Jun 18, 2020 at 2:41 PM Steven Wu via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

>
>
> > On Jun 18, 2020, at 2:25 PM, Tom Stellard <tstellar at redhat.com> wrote:
> >
> > On 06/18/2020 11:27 AM, Steven Wu via llvm-dev wrote:
> >> I like the proposal but I would like to go even further. If we are
> going to create a top level CMake file, we should just go ahead and
> eliminate all the standalone build configuration. The standalone build
> should just be `cmake <monorepo-root>
> -DLLVM_ENABLE_PROJECTS=standalone-project ...`. That means less build
> configuration to maintain which is always good.
> >>
> >
> > Eliminating stand-alone builds would be an inconvenience for
> > us in Fedora, since this is how we build LLVM packages.
> > However, I think we may have different definitions of what a
> > stand-alone build is.
> >
> > Does your proposal mean that
> > `cmake <monorepo-root> -DLLVM_ENABLE_PROJECTS=lld ..` would build lld
> > only without llvm and be a replacement for what I consider to
> > be a stand-alone build, which is `cmake <monorepo-root>/lld ...`


> Let me clear up the terminology here. In current llvm-project, we have a
> full build and standalone build. Let say for libcxx (using libcxx or other
> runtime libraries because they can build without anything from llvm
> subdirectory).
>
> This is a full build that can produce libcxx dylib:
> `cmake <monorepo-root>/llvm -DLLVM_ENABLE_PROJECTS=llvm;libcxx
> -DLLVM_DISTRIBUTION_COMPONENTS=libcxx.dylib ...`
>
> This is a standalone build that doesn't pull in anything from llvm
> directory:
> `cmake <monorepo-root>/libcxx -D...`
>
> Those two cmake configurations are going down different path and can
> result in different behavior if not careful. The goal is just turning those
> into one configuration. You can still do a standalone build of runtime
> library by enabling just that project and it is going to produce the same
> build configuration as the full build.
>

Can you clarify what is going to be built with `cmake <monorepo-root>
-DLLVM_ENABLE_PROJECTS=libcxx ?

We need to cross-compile the libraries using the pre-installed clang in SDK
and have to use the standalone builds i.e. cmake <monorepo-root>/libcxx. We
do not wish to rebuild llvm or clang when cross-compiling the libraries.
If cmake <monorepo-root> -DLLVM_ENABLE_PROJECTS=libcxx is going to build
more than libcxx, it is a *big* no-go for us (Chrome OS). Same for other
libraries like libcxxabi/libunwind.

Also,  we do not currently link libc++ with llvm's libunwind as it misses
many functionalities provided by libgcc_s/libgcc_eh in particular on ARM32
(related to GCC/GLIBC extensions) and ARM was not willing to invest in
that.
So, whilte it'd be nice to build libc++abi/libc++/libunwind in a single
innovation, linking with libunwind should NOT be mandatory.

Thanks,
Manoj

Steven
>
> >
> > -Tom
> >
> >> That also means that we can refactor some of the cmake modules to put
> into the top level directories so sub-projects can just reuse them without
> duping the code worrying they are not available in the standalone build
> today.
> >>
> >> Steven
> >>
> >>> On Jun 18, 2020, at 10:57 AM, Louis Dionne via llvm-dev <
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
> >>>
> >>> Hi folks,
> >>>
> >>> Building any LLVM project currently requires invoking CMake inside
> <monorepo-root>/llvm, while setting the projects to enable in the
> LLVM_ENABLE_PROJECTS variable. This has the downside that CMake processing
> for the LLVM subproject happens even when one doesn't really need or want
> it. It's also not great from a build hygiene perspective, as LLVM globally
> sets some flags and subprojects pick them up, when they don't really mean
> to. For example, see this workaround:
> https://github.com/llvm/llvm-project/blob/master/libcxx/CMakeLists.txt#L503-L507,
> where we need to account for some flags that might have been set globally
> by LLVM.
> >>>
> >>> I'm not sure about other projects, however this is quite problematic
> for projects part of the C++ runtime (libc++/libc++abi/libunwind). Indeed,
> we often try to build those projects targetting not widely supported
> platforms, where the overall LLVM build doesn't work. For example, trying
> to use the LLVM_ENABLE_PROJECTS approach for building libc++ for Apple's
> DriverKit environment doesn't work, since it has a few unusual things that
> the LLVM build chokes on. However, building libc++ standalone works just
> fine because it has far fewer requirements. It's also not just an issue of
> working vs not working: because of global flag pollution, building libc++
> standalone and as part of the rest of LLVM can result in slightly different
> flags being used, which could cause important and hard-to-diagnose issues.
> >>>
> >>> Hence, I think we should introduce a way to build LLVM projects (or at
> least the runtimes) without going through
> <monorepo-root>/llvm/CMakeLists.txt. What I suggest is to have a top-level
> <monorepo-root>/CMakeLists.txt whose sole job is to include subprojects. We
> could also place basic LLVM-wide things like the check for the minimum
> CMake version there. More specifically, I would like to be able to do:
> >>>
> >>>    $ cd <monorepo-root>
> >>>    $ mkdir build
> >>>    $ (cd build && cmake <monorepo-root>
> -DLLVM_ENABLE_PROJECTS="<projects-to-enable>")
> >>>
> >>> Pretty much the only difference with today is that you'd use `cmake
> <monorepo-root>` instead of `cmake <monorepo-root>/llvm`.
> >>>
> >>> Like I said, this is a problem for the runtime projects, but I'm not
> sure about other projects. For the runtime projects, another option would
> be to only allow standalone builds. However, the runtime projects are often
> built in lockstep, and so running three CMake commands when one would
> suffice is both annoying and also an easy way to screw things up.
> Furthermore, the current standalone builds add complexity to the projects,
> because they require the ability to point to arbitrary headers/libraries
> from the other projects, when we really always want to point to the
> just-built ones.
> >>>
> >>> Relationship with Petr Hosek's "Runtimes" build
> >>> ---------------------------------------------------------------
> >>> What I'm proposing isn't a replacement for itl. The "Runtimes" build
> can be seen as a driver that sets up the individual
> libc++/libc++abi/libunwind builds with the just-built toolchain, and for
> the provided targets. That's really great, however it is built *on top of*
> the basic libc++/libc++abi/libunwind builds. So basically, after my
> proposal, the "Runtimes" build could simply build all elements from the
> runtime with a single CMake invocation, as opposed to multiple invocations.
> >>>
> >>> Thoughts?
> >>> Louis
> >>>
> >>> _______________________________________________
> >>> LLVM Developers mailing list
> >>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> >>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >>
> >>
> >>
> >> _______________________________________________
> >> LLVM Developers mailing list
> >> llvm-dev at lists.llvm.org
> >> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >>
> >
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/libcxx-dev/attachments/20200618/52224f39/attachment-0001.html>