[llvm-dev] Building SVN head with CMake - shared libraries?

[Chris Bieneman via llvm-dev]

I didn’t realize we had deprecated the dependency walking code, which actually makes me a little sad, but I’ve been thinking about this, and I think we’re making this all too hard on ourselves.

A lot of the work we’re doing here is actually just duplicating functionality already in llvm-config. Doing a normal in-tree build we can’t use llvm-config because we need the information at configuration time before llvm-config is built. For this use case, where you’ve already built LLVM and are essentially building LLDB as an out-of-tree project, we could use llvm-config because it should already be there.

Doing that would mean we don’t have to duplicate logic between the llvm-config tool and our CMake builds, and not duplicating things is generally good.

-Chris

> On Jan 20, 2016, at 3:13 AM, Dan Liew <dan at su-root.co.uk> wrote:
> 
> On 19 January 2016 at 18:31, Chris Bieneman <beanz at apple.com> wrote:
>> Dan,
>> 
>> Thanks for looking so deeply at this. I think the fundamental problem is that we still have our infrastructure in an odd state as we’re moving off autoconf.
>> 
>> Brad’s explanation of the component mappings is correct, but with an added complication that we also have LLVMBuild which handles the dependency connections between static libraries. That allows a user to specify the specific components that their project depends on, and our build scripts will also add any components that those components depend on (see: `llvm_expand_dependencies`).
> 
> Eh? ``llvm_expand_dependencies()`` shouldn't be used. The only use in
> LLVM's tree is being called indirectly via
> ``llvm_map_components_to_libraries()`` which is marked deprecated (has
> been since LLVM 3.5 - we ought to remove it soon). I think
> ``llvm_map_components_to_libnames()`` to supposed to resolve all the
> dependencies.
> 
> 
>> In our CMake `llvm_config` and `llvm_add_*` serve very different purposes. In general all the functions and macros that start with “${project}_*” are intended as internal to our CMake, and we really don’t encourage others to call them at all. We also have functions and macros that match the pattern "add_${project}_*" which are intended for use only within a given project, those are generally built on top of the “llvm_add_*” functions.
> 
> Our docs say differently. See [1] , we suggest using
> ``add_llvm_loadable_module()``.
> 
>> The intention is that out-of-tree clients (like CastXML) would use llvm_config, which would roughly map to the functionality of the llvm-config command line tool that we’ve used with autoconf. All of these code paths end up using “llvm_map_components_to_libnames", for the actual mapping. Part of the problem here is that we haven’t kept the functionality of that function in parity with the llvm-config tool.
>> 
>> Today the llvm-config tool knows which components get put into libLLVM.so, and knows how and when to map to the shared library. I think that is the missing bit of functionality here. I don’t even think adding that will be hard, we just need to make sure that LLVM_DYLIB_COMPONENTS, and LLVM_LINK_LLVM_DYLIB are both saved of in LLVMConfig.cmake.
> 
> The ``LLVM_INSTALL_COMPONENT_LIBRARIES`` option I proposed (and gave a
> rough patch for) would also need to be exported but I think
> implementing that would be a lot easier after a refactor.
> 
> As Brad mentioned we do not want to tell people to move off
> ``llvm_map_components_to_libnames()`` just because it makes our
> refactoring easier. ``llvm_map_components_to_libnames()`` is the
> documented public interface since LLVM 3.5 and it would silly to
> change it when we can easily avoid it. What I think we need to do is
> 
> - Remove ``llvm_map_components_to_libraries()`` and all the dead code
> that results from that (i.e. ``expand_topologically`` and
> ``llvm_expand_dependencies``).
> - Rename the current implementation of
> ``llvm_map_components_to_libnames()`` to
> ``llvm_map_components_to_module_libs()`` (need a better name for this,
> we can't call it ``..._to_static_libnames`` because of the
> ``BUILD_SHARED_LIBS`` option)
> - Implement ``llvm_map_components_to_libnames()`` to have a little
> more functionality than before. Internally it may call
> ``llvm_map_components_to_module_libs()``.
> 
> The signature would be something like this. Sorry if I screw up
> terminology (i.e. modules, components... I'm not sure what I'm
> supposed to use here.)
> 
> llvm_map_component_to_libnames(<OUT_VAR> [ DYLIB_ONLY | MODULES_ONLY |
> DYLIB_FALLBACK_MODULES ]  <component1> <component2> ...)
> 
> The optional ``DYLIB_ONLY``, ``MODULES_ONLY`` and
> ``DYLIB_FALLBACK_MODULES`` arguments do the following
> 
> - ``DYLIB_ONLY`` - If specified will use the monolithic
> ``libLLVM.so``. If we know that this library was
> not built then we throw a fatal error. If one of the requested
> components was not built in the library then
> throw a fatal error.
> 
> - ``MODULES_ONLY`` - If specified will use the modules (e.g.
> LLVMSupport.a, etc...). When ``LLVM_INSTALL_COMPONENT_LIBRARIES``
> (perhaps rename to LLVM_INSTALL_MODULE_LIBRARIES?)
>  is implemented in the future and set to off (i.e. the module libs
> won't get installed) throw a fatal error. We may need to be careful
> here. The modules libraries are always available from the build tree
> but not from the install tree.
> 
> - ``DYLIB_FALLBACK_MODULES`` - Try to use the monolithic
> ``libLLVM.so``. If the library wasn't built use the modules instead.
> If some of the components aren't in the monolithic ``libLLVM.so`` then
> use the modules for those
> (i.e. the libraries returned is a mixture of the monolithic library
> and the module libraries).
> 
> If neither of the above options are specified then the default in
> ``MODULES_ONLY`` (i.e. the existing behaviour). However when
> ``LLVM_INSTALL_COMPONENT_LIBRARIES`` is implemented and if it is set
> to
> off (i.e. the modules won't be installed) and then the default changes
> to ``DYLIB_ONLY`` when being used from an install tree.
> 
> The point of the above is to give the client explicit control if they
> want it (e.g. they might always want the monolithic libLLVM.so). The
> default works in as many situations as possible.
> 
> Does this make sense? If so I'd happily have a go at this but I
> wouldn't be able to take a look until next week.
> 
> 
> [1] http://llvm.org/docs/CMake.html#developing-llvm-passes-out-of-source