[llvm-dev] [RFC] Modernize CMake LLVM "Components"/libLLVM Facility

[Tom Stellard via llvm-dev]

On 1/3/21 1:49 PM, Stella Laurenzo via llvm-dev wrote:
> Hi folks, happy new year!
> 
> *Proposal:*
> 
>   * See comments at the top of LLVMComponents.cmake
>     <https://github.com/stellaraccident/llvm-project/blob/newcomponents/llvm/cmake/modules/LLVMComponents.cmake>
>     in my fork
>     <https://github.com/stellaraccident/llvm-project/tree/newcomponents>.
>   * Draft phab: https://reviews.llvm.org/D94000
> 
> 
> *Background:*
> As I've been working on NPCOMP 
> <https://github.com/llvm/mlir-npcomp> trying to come up with a release 
> flow for MLIR derived Python projects (see py-mlir-release 
> <https://github.com/stellaraccident/mlir-py-release>), I've repeatedly 
> run into issues with how the LLVM build system generates shared 
> libraries. While the problems have been varied, I pattern match most of 
> them to a certain "pragmatic" nature to how components/libLLVM/libMLIR 
> have come to be: in my experience, you can fix most individual dynamic 
> linkage issues with another work-around, but the need for this tends to 
> be rooted in a lack of definition and structure to the libraries 
> themselves, causing various kinds of problems and scenarios that don't 
> arise if developed to stricter standards. (This isn't a knock on anyone 
> -- I know how these things tend to grow. My main observation is that I 
> think we have outgrown the ad-hoc nature of shared libraries in the LLVM 
> build now).
> 
> I think I'm hitting this because reasonable Python projects and releases 
> pre-supposes a robust dynamic linkage story. Also, I use Windows and am 
> very aware that LLVM basically does not support dynamic linking on 
> Windows -- and cannot without more structure (and in my experience, this 
> structure would also benefit the robustness of dynamic linking on the 
> others).
> 
> Several of us got together to discuss this in November 
> <https://llvm.discourse.group/t/meeting-notes-mlir-build-install-and-shared-libraries/2257>. 
> We generally agreed that BUILD_SHARED_LIBS was closer to what we wanted 
> vs libLLVM/libMLIR, but the result is really only factored for 
> development (i.e. not every add_library should result in a shared object 
> -- the shared library surface should mirror public interface boundaries 
> and add_library mirrors private boundaries). The primary difference 
> between the two is:
> 
>   * BUILD_SHARED_LIBS preserves the invariant that every translation
>     unit will be "homed" in one library at link time (either .so/.dll or
>     .a) and the system will never try to link together shared and static
>     dependencies of the same thing (which is what libLLVM/libMLIR do
>     today). It turns out that this is merely a good idea on most
>     platforms but is the core requirement on native Windows (leaving out
>     mingw, which uses some clever and dirty tricks to try to blend the
>     worlds).
>   * LLVM_BUILD_LLVM_DYLIB treats libLLVM.so as a "bucket" to throw
>     things that might benefit from shared linkage, but end binaries end
>     up also needing to link against the static libraries in case if what
>     you want isn't in libLLVM.so. When this is done just right, it can
>     work (on Unix) but it is very fragile and prone to multiple
>     definition and other linkage issues that can be extremely hard to
>     track down.
> 
> *What I did:*
> 
>  1. Well, first, I tried looking the other way for a few months and
>     hoping someone else would fix it :)
>  2. When I started trying to generalize some of the shared library
>     handling for MLIR and NPCOMP, I noted that the LLVM_LINK_COMPONENTS
>     (as in named groups of things) are in the right direction of having
>     a structure to the libraries, and I found that I could actually
>     rebase all of what the LLVM_LINK_COMPONENTS was trying to do on the
>     same facility, relegating the existing LLVM_LINK_COMPONENTS to a
>     name normalization layer on top of a more generic "LLVM Components"
>     facility that enforces stricter layering and more control than the
>     old libLLVM.so facility did.
>  3. I rewrote it twice to progressively more modern CMake and was able
>     to eliminate all of the ad-hoc dependency tracking in favor of
>     straight-forward use of INTERFACE libraries and $<TARGET_PROPERTY>
>     generator expressions for selecting static or dynamic component
>     trees based on global flags and the presence (or absence) of
>     per-executable LLVM_LINK_STATIC properties
>      1. Note that since this is rooted only in CMake features and not
>         LLVM macros, out of tree, non-LLVM projects should be able to
>         depend on LLVM components in their own targets.
>  4. I hacked up AddLLVM/LLVM-Build/LLVM-Config to (mostly) use the new
>     facility (leaving out a few things that can be fixed but aren't
>     conceptual issues), applied a bunch of fixes to the tree that were
>     revealed by stricter checks and got all related tests passing for
>     LLVM and MLIR (on X86 -- some mechanical changes need to be made to
>     other targets) for both dynamic and static builds.
> 
> *What I'd like to do:*
> 
>   * Get some consensus that we'd like to improve things in this area and
>     that the approach I'm taking makes sense. I can do a lot of the
>     work, but I don't want to waste my time, and this stuff is fragile
>     if we keep it in an intermediate state for too long (I'm already
>     paying this price downstream).
>   * Land LLVMComponents.cmake
>     <https://github.com/stellaraccident/llvm-project/blob/newcomponents/llvm/cmake/modules/LLVMComponents.cmake>
>     as the basis of the new facility.

Do you have a proposed list of components yet for LLVM?

>   * Finish implementing the "Redirection" feature that would allow us to
>     emulate an aggregate libLLVM as it is today.
>   * Start pre-staging the various stricter constraints to the build tree
>     that will be needed to swap AddLLVM to use the new facility.
>   * Rewrite component-related AddLLVM/LLVM-Build/LLVM-Config bits in a
>     more principled way to use the new facility (or remove features
>     entirely that are no longer needed) -- what I did in the above patch
>     was just a minimal amount of working around for a POC.
>   * Agree on whether we should try to have the two co-exist for a time
>     or do a more clean break with the old.
>   * Start applying the facility to downstream projects like MLIR and NPCOMP.
> 

It sounds like what you are proposing is BUILD_SHARED_LIBS=ON but with 
fewer total libraries, is this an accurate summary?

I would prefer for any large change like this that we do not add any net 
new configuration options (meaning if we add a new option we should 
remove an old one)to LLVM as we already have too many.  Would this be 
able to replace BUILD_SHARED_LIBS=ON?

- Tom

> *What I would need:*
> 
>   * Help, testing and expertise. I am reasonably confident in my
>     understanding of how to make shared libraries work and how to use
>     CMake, but the legacy in LLVM here is deep -- I likely pattern
>     matched some old features as no longer needed when they actually are
>     (I am not clear at all on how much of LLVM-Config is still relevant).
>   * Pointers to who the stakeholders are that I should be coordinating with.
> 
> Comments?
> 
> Thanks!
> - Stella
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>