[llvm-dev] [RFC] Modernize CMake LLVM "Components"/libLLVM Facility

[Mehdi AMINI via llvm-dev]

On Sun, Jan 3, 2021 at 1:50 PM Stella Laurenzo via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> Hi folks, happy new year!
>
> *Proposal:*
>
>    - See comments at the top of LLVMComponents.cmake
>    <https://github.com/stellaraccident/llvm-project/blob/newcomponents/llvm/cmake/modules/LLVMComponents.cmake>
>    in my fork
>    <https://github.com/stellaraccident/llvm-project/tree/newcomponents>.
>    - Draft phab: https://reviews.llvm.org/D94000
>
>
> *Background:*
> As I've been working on NPCOMP <https://github.com/llvm/mlir-npcomp> trying
> to come up with a release flow for MLIR derived Python projects (see
> py-mlir-release <https://github.com/stellaraccident/mlir-py-release>),
> I've repeatedly run into issues with how the LLVM build system generates
> shared libraries. While the problems have been varied, I pattern match most
> of them to a certain "pragmatic" nature to how components/libLLVM/libMLIR
> have come to be: in my experience, you can fix most individual dynamic
> linkage issues with another work-around, but the need for this tends to be
> rooted in a lack of definition and structure to the libraries themselves,
> causing various kinds of problems and scenarios that don't arise if
> developed to stricter standards. (This isn't a knock on anyone -- I know
> how these things tend to grow. My main observation is that I think we have
> outgrown the ad-hoc nature of shared libraries in the LLVM build now).
>
> I think I'm hitting this because reasonable Python projects and releases
> pre-supposes a robust dynamic linkage story. Also, I use Windows and am
> very aware that LLVM basically does not support dynamic linking on Windows
> -- and cannot without more structure (and in my experience, this structure
> would also benefit the robustness of dynamic linking on the others).
>
> Several of us got together to discuss this in November
> <https://llvm.discourse.group/t/meeting-notes-mlir-build-install-and-shared-libraries/2257>.
> We generally agreed that BUILD_SHARED_LIBS was closer to what we wanted vs
> libLLVM/libMLIR, but the result is really only factored for development
> (i.e. not every add_library should result in a shared object -- the shared
> library surface should mirror public interface boundaries and add_library
> mirrors private boundaries). The primary difference between the two is:
>
>    - BUILD_SHARED_LIBS preserves the invariant that every translation
>    unit will be "homed" in one library at link time (either .so/.dll or .a)
>    and the system will never try to link together shared and static
>    dependencies of the same thing (which is what libLLVM/libMLIR do today). It
>    turns out that this is merely a good idea on most platforms but is the core
>    requirement on native Windows (leaving out mingw, which uses some clever
>    and dirty tricks to try to blend the worlds).
>    - LLVM_BUILD_LLVM_DYLIB treats libLLVM.so as a "bucket" to throw
>    things that might benefit from shared linkage, but end binaries end up also
>    needing to link against the static libraries in case if what you want isn't
>    in libLLVM.so. When this is done just right, it can work (on Unix) but it
>    is very fragile and prone to multiple definition and other linkage issues
>    that can be extremely hard to track down.
>
> *What I did:*
>
>    1. Well, first, I tried looking the other way for a few months and
>    hoping someone else would fix it :)
>    2. When I started trying to generalize some of the shared library
>    handling for MLIR and NPCOMP, I noted that the LLVM_LINK_COMPONENTS (as in
>    named groups of things) are in the right direction of having a structure to
>    the libraries, and I found that I could actually rebase all of what the
>    LLVM_LINK_COMPONENTS was trying to do on the same facility, relegating the
>    existing LLVM_LINK_COMPONENTS to a name normalization layer on top of a
>    more generic "LLVM Components" facility that enforces stricter layering and
>    more control than the old libLLVM.so facility did.
>    3. I rewrote it twice to progressively more modern CMake and was able
>    to eliminate all of the ad-hoc dependency tracking in favor of
>    straight-forward use of INTERFACE libraries and $<TARGET_PROPERTY>
>    generator expressions for selecting static or dynamic component trees based
>    on global flags and the presence (or absence) of per-executable
>    LLVM_LINK_STATIC properties
>       1. Note that since this is rooted only in CMake features and not
>       LLVM macros, out of tree, non-LLVM projects should be able to depend on
>       LLVM components in their own targets.
>    4. I hacked up AddLLVM/LLVM-Build/LLVM-Config to (mostly) use the new
>    facility (leaving out a few things that can be fixed but aren't conceptual
>    issues), applied a bunch of fixes to the tree that were revealed by
>    stricter checks and got all related tests passing for LLVM and MLIR (on X86
>    -- some mechanical changes need to be made to other targets) for both
>    dynamic and static builds.
>
> *What I'd like to do:*
>
>    - Get some consensus that we'd like to improve things in this area and
>    that the approach I'm taking makes sense. I can do a lot of the work, but I
>    don't want to waste my time, and this stuff is fragile if we keep it in an
>    intermediate state for too long (I'm already paying this price downstream).
>    - Land LLVMComponents.cmake
>    <https://github.com/stellaraccident/llvm-project/blob/newcomponents/llvm/cmake/modules/LLVMComponents.cmake>
>    as the basis of the new facility.
>    - Finish implementing the "Redirection" feature that would allow us to
>    emulate an aggregate libLLVM as it is today.
>    - Start pre-staging the various stricter constraints to the build tree
>    that will be needed to swap AddLLVM to use the new facility.
>    - Rewrite component-related AddLLVM/LLVM-Build/LLVM-Config bits in a
>    more principled way to use the new facility (or remove features entirely
>    that are no longer needed) -- what I did in the above patch was just a
>    minimal amount of working around for a POC.
>    - Agree on whether we should try to have the two co-exist for a time
>    or do a more clean break with the old.
>    - Start applying the facility to downstream projects like MLIR and
>    NPCOMP.
>
> *What I would need:*
>
>    - Help, testing and expertise. I am reasonably confident in my
>    understanding of how to make shared libraries work and how to use CMake,
>    but the legacy in LLVM here is deep -- I likely pattern matched some old
>    features as no longer needed when they actually are (I am not clear at all
>    on how much of LLVM-Config is still relevant).
>    - Pointers to who the stakeholders are that I should be coordinating
>    with.
>
> Comments?
>

Looks great! In particular it is interesting to see how more modern CMake
features could replace some of the custom-LLVM CMake macros that are likely
almost a decade old now.

One thing I wonder about trying to see BUILD_SHARED_LIBS as some desirable
for a production environment: I seem to remember that there were
non-trivial performance regression when using many .so instead of a single
libLLVM.so (even a single libLLVM.so was showing a measurable performance
impact for clang IIRC).

-- 
Mehdi

>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210103/424875ca/attachment.html>