[llvm-dev] [RFC] LLVM Directory Structure Changes (was Re: [PATCH] D20992: [CMake] Add LLVM runtimes directory)

Craig, Ben via llvm-dev llvm-dev at lists.llvm.org
Thu Jun 9 14:09:04 PDT 2016


>>   * To get cmake to work, I have to set HAVE_CXX_ATOMICS_WITHOUT_LIB,
>>     even though I have no intention of building LLVM.  I then get to
>>     set LIBCXX_HAVE_CXX_ATOMICS_WITHOUT_LIB too, because reasons.
>>
> This is bad. I’m curious why you need to set those ever. Have you 
> diagnosed this? For you to need to set that it means the host 
> toolchain isn’t properly passing the CMake checks.
It looks like I don't need to set these anymore, but the comment that I 
left myself at the time was that HAVE_CXX_ATOMICS_WITHOUT_LIB looked at 
the state of my local machine's toolchain, as opposed to the toolchain 
that I was about to use.  My local machine's toolchain is gcc 4.6.3, and 
it doesn't have an <atomic> header.

>>   * Multi-libs require multiple independent build directories, with
>>     all the associated cmake overhead.
>>
> I strongly believe that for building multi-lib or more generally when 
> building for multiple targets you want multiple build directories, and 
> in particular the multiple-cmake invocations. I believe this because 
> you want the checks to be relevant for the target, and the only way to 
> do that is to run the checks once per target.
>
> That said, I also strongly believe that for any user of our projects 
> we should find a way to have a single simple CMake invocation that 
> gets the end result that you want. I don’t believe these are mutually 
> exclusive goals.
>
> One of the development steps of the new runtime directory will be 
> supporting specifying multiple targets to build the runtimes for, and 
> having CMake construct the appropriate number of build directories and 
> manage building them all through a single top-level configuration and 
> build directory. If you’re skeptical about how doable this is I’d 
> encourage you to look at this bot -> 
> http://lab.llvm.org:8011/builders/clang-3stage-ubuntu.
>
> That bot does a full 3-stage clang build from a single CMake invocation:
>
> cmake -C ../llvm.src/tools/clang/cmake/caches/3-stage.cmake -GNinja 
> -DLLVM_TARGETS_TO_BUILD=all 
> -DLLVM_BINUTILS_INCDIR=/opt/binutils/include ../llvm.src
>
I'm fine if I can invoke cmake once and get multiple library variants 
out.  If that means that behind-the-scenes, cmake has multiple build 
sub-directories, then that's fine by me.  On Windows, this already 
happens to some degree, as the Visual Studio project is allowed to 
switch between Release, Debug, RelWithDebInfo, etc without re-running cmake.

>> So why not run out of tree instead you may ask?
>>
>>   * No lit tests or lit utilities (FileCheck, not, etc…)
>>
> I don’t know if the runtimes you’re building are setup for this or 
> not, but you can get out-of-tree tests working if you have an LLVM 
> installation on the system or a build directory that you can point at. 
> Compiler-RT does this. It isn’t ideal but it is workable.
>
> We should have a better solution.
The nightly builds that my organization produces is generally a 
"customer build", and not a "developer build".  Stated otherwise, it 
doesn't include llvm-config, libclang.a, or any of the other things that 
people building against llvm would want.  That means that if I wanted to 
use this particular out-of-tree solution, I would still need to clone 
down LLVM and rebuild and reinstall it on occasion.

>> So some things I would like to see...
>>
>>   * Standalone runtime builds should use the "normal" build
>>     interfaces (bare make, make all, make check, make install. 
>>     s/make/ninja as desired).
>>
> I think this is doable, but I’m hesitant to rope it in with what I’m 
> trying to do here. Nothing I want to do would prevent this or make it 
> any harder than it already is.
Completely fair.  I figured I'd get my grievances and wishlist out 
first, just so that some agreement on a direction can be figured out.  
It does extend beyond the directory re-org patch.

>>   * Break out testing infrastructure to a common repo, so that the
>>     runtimes can have access to the testing "banana" without dragging
>>     along the LLVM "gorilla”.
>>
> I’m hesitant to suggest more and more repos because I think there are 
> some challenges and additional burdens with that. I do understand the 
> benefit of what you’re asking for here, and I think it is worth 
> considering. I think there is an argument for splitting out the LLVM 
> testing infrastructure, as well as an argument for splitting out the 
> LLVM build infrastructure.
>
> In both cases I think those changes are larger than what I’m 
> proposing, but worth considering.
>
> -Chris
>
> Obligatory troll: Maybe we should move to github and change the whole 
> repo structure in the process?
I figured the github thread was crazy enough without me sabotaging it 
with this kind of suggestion :)

>>
>> On 6/9/2016 12:20 PM, Chris Bieneman via llvm-dev wrote:
>>> Moving to llvm-dev (I think this has gone a bit further than a patch 
>>> review discussion)
>>>
>>> In hindsight I probably should have explained more of my thinking on 
>>> this with the patch, or done an RFC on llvm-dev to start with. I’l 
>>> do that now, and answer the questions along the way. I sent a 
>>> separate email discussing Justin’s patch review feedback.
>>>
>>> In the build system today there is no strong distinction between 
>>> ‘projects’ and ‘tools’. There are a few subtle differences, but I’m 
>>> not sure any of them really matter. The differences are:
>>>
>>> (1) The projects directory is always configured, tools can be 
>>> disabled using LLVM_INCLUDE_TOOLS=Off (projects and tools can both 
>>> be individually disabled too)
>>> (2) Projects are configured before tools, so tools can rely on 
>>> targets being created for projects (we don’t really use this, and 
>>> anywhere we are is probably a bug)
>>> (3) Some projects have special handling. For example test-suite 
>>> isn’t actually treated as a project, it has special handling in 
>>> LLVM/CMakeLists.txt:727, and Compiler-RT is handled by clang if you 
>>> set LLVM_BUILD_EXTERNAL_COMPILER_RT=On.
>>>
>>> With this in mind I was thinking about the general usability of our 
>>> build system. The distinction between a project and a tool is not 
>>> very clear. At a high level I see three different use cases that are 
>>> covered by our current projects & tools directories.
>>>
>>> (1) Projects that are configured with LLVM
>>> (2) Runtime projects that should be configured using the just-built 
>>> tools
>>> (3) The LLVM test-suite, which is really just external tests that 
>>> should be configured and run with the just-built tools
>>>
>>> My proposal is that we make the tools subdirectory the *only* place 
>>> for projects that fall into category 1. I don’t think there is any 
>>> technical reason to drop an in-tree project into projects over tools 
>>> today, and I think we migrating people who are doing that away from 
>>> it should be easy.
>>>
>>> Second I want to add a “runtimes” directory to LLVM to cover case 2 
>>> (see D20992). The idea behind this is to use common code in LLVM to 
>>> support building runtimes. This will allow the full LLVM toolchain 
>>> to be visible during configuration. I will abstract this 
>>> functionality into an installed CMake module so that Clang can use 
>>> it for out-of-tree clang builds.
>>>
>>> Lastly we need to give the test-suite a new home. I’m not super 
>>> concerned with where we do that. It could be under tests, it could 
>>> just be at the root of the LLVM directory. I don’t think it matters 
>>> too much because it is a one-off. Thoughts welcome.
>>>
>>> My proposed patch makes the runtimes directory work for Compiler-RT, 
>>> but it doesn’t yet handle libcxxabi, libcxx and libunwind. There is 
>>> some special case handling between libcxxabi and libcxx that will 
>>> need to be handled to make the dependencies work between the two, 
>>> and I still need to work that out.
>>>
>>> If we want to go with this proposal I envision the transition being 
>>> multi-staged:
>>>
>>> (1) Adding the new functionality, getting it up and fully working 
>>> for all runtime projects - this will involve changes to runtime projects
>>> (2) Work with bot maintainers to migrate bots, and fix any issues 
>>> that come up
>>> (3) Add support for a new secondary location for the test-suite
>>> (4) Set a date for removing the projects directory, post patches 
>>> including updated documentation
>>> (5) Remove the projects directory entirely
>>>
>>> Thoughts?
>>> -Chris
>>>
>>>> On Jun 8, 2016, at 6:59 PM, Chandler Carruth <chandlerc at gmail.com 
>>>> <mailto:chandlerc at gmail.com>> wrote:
>>>>
>>>> On Wed, Jun 8, 2016 at 4:39 PM Justin Bogner via llvm-commits 
>>>> <llvm-commits at lists.llvm.org> wrote:
>>>>
>>>>     Chris Bieneman <beanz at apple.com <mailto:beanz at apple.com>> writes:
>>>>     > beanz created this revision.
>>>>     > beanz added reviewers: chandlerc, bogner.
>>>>     > beanz added a subscriber: llvm-commits.
>>>>     >
>>>>     > There are a few LLVM projects that produce runtime libraries.
>>>>     Ideally
>>>>     > runtime libraries should be built differently than other
>>>>     projects,
>>>>     > specifically they should be built using the just-built toolchain.
>>>>     >
>>>>     > There is support for building compiler-rt in this way from
>>>>     the clang
>>>>     > build. Moving this logic into the LLVM build is interesting
>>>>     because it
>>>>     > provides a simpler way to extend the just-built toolchain to
>>>>     include
>>>>     > LLD and the LLVM object file tools.
>>>>     >
>>>>     > Once this functionality is better fleshed out and tested
>>>>     we’ll want to
>>>>     > encapsulate it in a module that can be used for clang standalone
>>>>     > builds, and we’ll want to make it the default way to build
>>>>     compiler-rt.
>>>>     >
>>>>     > With this patch applied there is no immediate change in the
>>>>     build.
>>>>     > Moving compiler-rt out from llvm/projects into llvm/runtimes
>>>>     enables
>>>>     > the functionality.
>>>>
>>>>     This seems reasonable, but I am a little worried about how
>>>>     transitioning
>>>>     to the new system will work. Will everyone have to move their
>>>>     compiler-rt checkout? Will we continue to support compiler-rt
>>>>     in either
>>>>     place? Both of these are workable, but neither is great. Thoughts?
>>>>
>>>>
>>>> I share your concerns, but I also kind of like the direction this 
>>>> is going.
>>>>
>>>> But there is a higher-level meta-point: do we want to keep the 
>>>> 'projects' directory *at all*.
>>>>
>>>> Every single resident of it I can think of except for the 
>>>> test-suite is either dead (dragonegg) or a runtime library.
>>>>
>>>> I think we should either have all the build-integrated projects in 
>>>> a single 'projects' directory (including LLD and Clang), or we 
>>>> should have none of them and use more domain relevant organization 
>>>> (today "tools", you're adding "runtimes", maybe we move the 
>>>> test-suite to go under one of the test directories).
>>>>
>>>> I think we should have a consistent plan here before moving stuff. 
>>>> But once we have it, I think we shouldn't be afraid of 
>>>> re-organizing stuff to make more sense, and just work to get folks 
>>>> to update their checkouts.
>>>>
>>>> -Chandler
>>>>
>>>>
>>>>     > This code has a few improvements over the method provided by
>>>>     > LLVM_BUILD_EXTERNAL_COMPILER_RT. Specifically the sub-ninja
>>>>     command is
>>>>     > always invoked, so changes to compiler-rt source files will
>>>>     get built
>>>>     > properly, so this patch can be used for iterative development
>>>>     with
>>>>     > just-built tools.
>>>>     >
>>>>     >http://reviews.llvm.org/D20992
>>>>     >
>>>>     > Files:
>>>>     >   CMakeLists.txt
>>>>     >  cmake/modules/LLVMExternalProjectUtils.cmake
>>>>     >   runtimes/CMakeLists.txt
>>>>     >
>>>>     > Index: runtimes/CMakeLists.txt
>>>>     >
>>>>     ===================================================================
>>>>     > --- /dev/null
>>>>     > +++ runtimes/CMakeLists.txt
>>>>     > @@ -0,0 +1,16 @@
>>>>     > +include(LLVMExternalProjectUtils)
>>>>     > +
>>>>     > +# Discover the projects that use CMake in the subdirectories.
>>>>     > +# Note that explicit cmake invocation is required every time
>>>>     a new project is
>>>>     > +# added or removed.
>>>>     > +
>>>>     > +add_custom_target(runtimes)
>>>>     > +
>>>>     > +file(GLOB entries *)
>>>>     > +foreach(entry ${entries})
>>>>     > +  if(IS_DIRECTORY ${entry} AND EXISTS ${entry}/CMakeLists.txt)
>>>>     > +    get_filename_component(projName ${entry} NAME)
>>>>     > + llvm_ExternalProject_Add(${projName} ${entry} USE_TOOLCHAIN)
>>>>     > +    add_dependencies(runtimes ${projName})
>>>>     > +  endif()
>>>>     > +endforeach(entry)
>>>>     > Index: cmake/modules/LLVMExternalProjectUtils.cmake
>>>>     >
>>>>     ===================================================================
>>>>     > --- cmake/modules/LLVMExternalProjectUtils.cmake
>>>>     > +++ cmake/modules/LLVMExternalProjectUtils.cmake
>>>>     > @@ -29,7 +29,8 @@
>>>>     >  #     Extra targets in the subproject to generate targets for
>>>>     >  #   )
>>>>     >  function(llvm_ExternalProject_Add name source_dir)
>>>>     > -  cmake_parse_arguments(ARG
>>>>     "USE_TOOLCHAIN;EXCLUDE_FROM_ALL;NO_INSTALL"
>>>>     > +  cmake_parse_arguments(ARG
>>>>     > + "USE_TOOLCHAIN;EXCLUDE_FROM_ALL;NO_INSTALL;ALWAYS_CLEAN"
>>>>     >      "SOURCE_DIR"
>>>>     >
>>>>     "CMAKE_ARGS;TOOLCHAIN_TOOLS;RUNTIME_LIBRARIES;DEPENDS;EXTRA_TARGETS"
>>>>     ${ARGN})
>>>>     >    canonicalize_tool_name(${name} nameCanon)
>>>>     > @@ -52,6 +53,10 @@
>>>>     >      endif()
>>>>     >    endforeach()
>>>>     >
>>>>     > +  if(ARG_ALWAYS_CLEAN)
>>>>     > +    set(always_clean clean)
>>>>     > +  endif()
>>>>     > +
>>>>     >    list(FIND TOOLCHAIN_TOOLS clang FOUND_CLANG)
>>>>     >    if(FOUND_CLANG GREATER -1)
>>>>     >      set(CLANG_IN_TOOLCHAIN On)
>>>>     > @@ -135,6 +140,14 @@
>>>>     >      CMAKE_ARGS ${${nameCanon}_CMAKE_ARGS}
>>>>     >                 ${compiler_args}
>>>>     >  -DCMAKE_INSTALL_PREFIX=${CMAKE_INSTALL_PREFIX}
>>>>     > +  -DLLVM_CONFIG_PATH=${LLVM_RUNTIME_OUTPUT_INTDIR}/llvm-config
>>>>     > +  -DLLVM_LIBRARY_OUTPUT_INTDIR=${LLVM_LIBRARY_OUTPUT_INTDIR}
>>>>     > +  -DLLVM_RUNTIME_OUTPUT_INTDIR=${LLVM_RUNTIME_OUTPUT_INTDIR}
>>>>     > +  -DLLVM_LIBDIR_SUFFIX=${LLVM_LIBDIR_SUFFIX}
>>>>     > +  -DLLVM_ENABLE_WERROR=${LLVM_ENABLE_WERROR}
>>>>     > +  -DPACKAGE_VERSION=${PACKAGE_VERSION}
>>>>     > +  -DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}
>>>>     > +  -DCMAKE_MAKE_PROGRAM=${CMAKE_MAKE_PROGRAM}
>>>>
>>>>     How did you decide which variables need to be passed through
>>>>     like this?
>>>>     The set seems somewhat arbitrary, but I may be missing something
>>>>     obvious.
>>>>
>>>>     >                 ${ARG_CMAKE_ARGS}
>>>>     >  ${PASSTHROUGH_VARIABLES}
>>>>     >      INSTALL_COMMAND ""
>>>>     > @@ -152,7 +165,7 @@
>>>>     >      ExternalProject_Add_Step(${name} force-rebuild
>>>>     >        COMMAND ${run_build}
>>>>     >        COMMENT "Forcing rebuild of ${name}"
>>>>     > -      DEPENDEES configure clean
>>>>     > +      DEPENDEES configure ${always_clean}
>>>>
>>>>     I'm not sure I understand what this does. If I had to guess I'd
>>>>     say that
>>>>     when ALWAYS_CLEAN is passed the rebuild of the external project
>>>>     always
>>>>     invokes clean first, but if it's not passed we'll just invoke the
>>>>     external build and allow it to be incremental if appropriate.
>>>>     Is that
>>>>     right?
>>>>
>>>>     >        DEPENDS ${ALWAYS_REBUILD} ${ARG_DEPENDS} ${TOOLCHAIN_BINS}
>>>>     >        ${cmake_3_4_USES_TERMINAL} )
>>>>     >    endif()
>>>>     > Index: CMakeLists.txt
>>>>     >
>>>>     ===================================================================
>>>>     > --- CMakeLists.txt
>>>>     > +++ CMakeLists.txt
>>>>     > @@ -720,6 +720,8 @@
>>>>     >    add_subdirectory(tools)
>>>>     >  endif()
>>>>     >
>>>>     > +add_subdirectory(runtimes)
>>>>     > +
>>>>     >  if( LLVM_INCLUDE_EXAMPLES )
>>>>     >    add_subdirectory(examples)
>>>>     >  endif()
>>>>     > @@ -730,7 +732,8 @@
>>>>     > llvm_ExternalProject_Add(test-suite
>>>>     ${LLVM_MAIN_SRC_DIR}/projects/test-suite
>>>>     >        USE_TOOLCHAIN
>>>>     >        EXCLUDE_FROM_ALL
>>>>     > -      NO_INSTALL)
>>>>     > +      NO_INSTALL
>>>>     > +      ALWAYS_CLEAN)
>>>>     >    endif()
>>>>     >    add_subdirectory(test)
>>>>     >    add_subdirectory(unittests)
>>>>     >
>>>>
>>>>     _______________________________________________
>>>>     llvm-commits mailing list
>>>>     llvm-commits at lists.llvm.org <mailto:llvm-commits at lists.llvm.org>
>>>>     http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>> -- 
>> Employee of Qualcomm Innovation Center, Inc.
>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>

-- 
Employee of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160609/757f7e3e/attachment.html>


More information about the llvm-dev mailing list