[LLVMdev] compiler-rt CMake build

Greg Fitzgerald garious at gmail.com
Mon Mar 24 19:36:11 PDT 2014 

Hi Alexey,

> What if we don't have stage0-clang?

I'd like to answer this question in detail.  I think if we're on the
same page about which compiler to use to build CompilerRT and where it
comes from, then the rest will, with any luck, fall into place.

Here is a dependency diagram for building and running the sanitizer test suite:

http://yuml.me/0324b101

Note that it mentions 3 compilers: Clang, the C++ compiler used to
build Clang, and a C++ cross-compiler used to build CompilerRT.  If no
cross-compiler exists, we can bootstrap by building a Stage-0 Clang
with a C++ compiler.  The full dependency diagram then becomes:

http://yuml.me/2633fbbd

When a C++ cross-compiler already exists (either by building it or
using clang-3.4), the dependency graph can be reduced by using the
same compiler to build Clang, LLVM and CompilerRT:

http://yuml.me/4b0473ae

http://yuml.me/9ff774bc

When building in this way, CompilerRT can expect the
CMAKE_CXX_COMPILER to be a cross-compiler with the same command-line
interface as Clang (cross-compile using -target and --sysroot flags).
This Stage-0 compiler cannot be used to run the Sanitizer test suite.
Instead, the Clang build should use add_subdirectory() to add the
CompilerRT 'test' directory to its build.  No need for the CMake's
ExternalProject feature.

-Greg


On Sun, Mar 23, 2014 at 11:17 AM, Alexey Samsonov <samsonov at google.com> wrote:
> Hi Greg,
>
> On Fri, Mar 21, 2014 at 11:18 PM, Greg Fitzgerald <garious at gmail.com> wrote:
>>
>> Hi Alexey,
>>
>> CMAKE_PREFIX_PATH is a convenient mechanism for exposing prebuilt
>> install directories to a CMake build.  It contains a colon-separated
>> list of paths.  Each path points to a directory that contains
>> directory names that are meaningful to CMake, including 'bin', 'lib'
>> and 'share'.
>>
>> If, for example, you want the path to 'llvm-config', instead of
>> creating a new CMake variable, you instead write
>> "find_program(LLVM_CONFIG_PATH llvm-config)".  CMake will search for
>> an executable named 'llvm-config' in each 'bin' directory of each path
>> in CMAKE_PREFIX_PATH.
>>
>> In a simplest case, where there is only one dependency, using
>> CMAKE_PREFIX_PATH means we do not have to standardize on a variable
>> name.  For llvm-config, should it be LLVM_CONFIG_PATH or
>> COMPILER_RT_LLVM_CONFIG_PATH?  In a component build, the former is
>> sufficient, but in the monolithic build, it makes more sense to
>> namespace every variable.  We can avoid painting that bike shed using
>> CMAKE_PREFIX_PATH.
>>
>> When one component has multiple dependencies from another, using
>> CMAKE_PREFIX_PATH really starts saving keystrokes.  You can use it to
>> find headers (sanitizer/asan_interface.h), binaries (FileCheck),
>> libraries, or CMake modules.
>>
>> A good alternative to CMAKE_PREFIX_PATH is to create one variable per
>> install directory.  So compiler-rt might accept LLVM_INSTALL_DIR, and
>> clang accept COMPILER_RT_INSTALL_DIR and LLVM_INSTALL_DIR.  There are
>> two advantages to this route:
>>
>> 1) No need for CMake features such as 'find_program'.  Instead, you
>> can be explicit:
>>     ${LLVM_INSTALL_DIR}/bin/llvm-config
>> 2) No need to organize the install directory in any particular way.
>> If a small component wants to combine its 'bin' and 'lib' directories
>> into one top-level directory, that's no problem.
>>
>> The disadvantage of this system is the need to standardize across
>> components on a naming convention.  COMPILER_RT_DIR, COMPILER-RT_DIR,
>> COMPILER_RT_INSTALL_DIR, COMPILER-RT_SHIP_DIR?  Ideally you'd only
>> have to write "deps = llvm compiler-rt" and a package manager could
>> automatically translate that to names that match the CMake options
>> (LLVM_DIR, COMPILER_RT_DIR).  But like before, using CMAKE_PREFIX_PATH
>> avoids the issue.
>
>
> Thanks for the great detailed explanations. I should check if we can
> simplify
> clang - compiler-rt build systems relationship with this.
>
>>
>>
>> Regarding the use of ExternalProject within Clang, I think it could
>> create more problems than it solves.  For one, it can't be used from
>> within the monolithic build (unless you have a duplicate llvm build,
>> or point CompilerRT to a subset of the currently-building LLVM build).
>
>
> Could you elaborate on this? That's exactly what I'm trying to do now -
> when I call "ExternalProject_Add(compiler-rt)" from Clang build tree, I
> use just-built llvm-config to configure it, and set up the paths so that
> "make compiler-rt" in compiler-rt build tree will build the libraries in the
> proper locations, where Clang driver expects to find them. Maybe, I don't
> understand what you mean when you say "monolithic build"
>
>
>>
>>  Secondly, it's not a general enough solution for other LLVM
>> components.  Say, for example, you wanted to use ExternalProject to
>> build LLVM from both Clang and CompilerRT.
>
>
> Why would one want to build LLVM from Clang and CompilerRT? My approach
> is targeted at solving a very specific problem.
>
>>
>>  If the parameters passed
>> to LLVM were identical, CMake could detect that and not build a
>> duplicate copy.  If they parameters are different in any way, you have
>> a problem.  Do you duplicate the build or flag an error?  We can avoid
>> this issue by allowing a super-project or package manager build LLVM
>> and pass its path to both Clang and CompilerRT.
>>
>>
>> Regarding the use of just-built-clang for both building CompilerRT and
>> running its test suite, please consider relaxing that to a
>> stage0-clang for building and a just-built-clang for testing.
>
>
> What if we don't have stage0-clang, and use gcc as a host compiler?
>
>>
>>  Using
>> the just-built-clang to build compiler-rt is rough on package
>> managers.  You're basically stating that the compiler-rt build depends
>> on some unreleased feature of clang to *build* compiler-rt.
>
>
> No, I want "use just-built clang" to be the default workflow. Currently, you
> may use any host compiler to build compiler-rt sources (gcc, Clang, MSVC),
> and we certainly want to keep it that way and not rely on some non-standard
> or Clang-specific features. But we probably want to ship/release Clang with
> a set of compiler-rt libraries that are built with *this* Clang. Again, one
> use case
> when this can be necessary is targeting different platforms. For example, if
> you use linux-x86 and you build LLVM/Clang capable of targeting both x86 and
> arm,
> it certainly makes sense to build necessary compiler-rt libraries for all
> arches that Clang
> can target. But you host compiler will most likely not be able to build arm
> version of
> compiler-rt libs.
>
>>
>>  To a
>> package manager, that means it should first build clang, then use that
>> clang to build compiler-rt, and then build a version of clang that
>> includes compiler-rt.
>
>
> As I mentioned earlier, it's enough to build clang, and use it to build
> compiler-rt
> in a proper location, no futher steps will be necessary.
>
>>
>>  It implies that every time there is a new
>> commit to clang, compiler-rt should be rebuilt and retested.  If the
>> compiler-rt build depended only a just-released-clang, then a commit
>> to clang would imply only that the sanitizer tests should be rebuilt
>> and tested.
>>
>> -Greg
>>
>>
>>
>> On Fri, Mar 21, 2014 at 3:59 AM, Alexey Samsonov <samsonov at google.com>
>> wrote:
>> > On Thu, Mar 20, 2014 at 10:12 PM, Greg Fitzgerald <garious at gmail.com>
>> > wrote:
>> >>
>> >> > ExternalProject_Add(compiler-rt ...)
>> >>
>> >> So that was quite the experiment.  Looking at
>> >> clang/runtime/CMakeLists.txt, I'm not seeing a lot of bang for buck
>> >> here, and it looks like this file is prone to bit rot.
>> >
>> >
>> > Could you please elaborate on this? In fact, I don't plan to give up on
>> > this
>> > experiment.
>> > On the contrary, I wanted to move LLVM_BUILD_EXTERNAL_COMPILER_RT to the
>> > bots, migrate the workflow of our team to use this mode, and eventually
>> > make
>> > it the default.
>> >
>> > The problem is that we want top-level targets from compiler-rt build
>> > tree be
>> > visible in the llvm/clang
>> > top-level build tree. I thought this could be achieved by ugly
>> > propagation
>> > of top-level compiler-rt
>> > targets with add_custom_target() command. (like I did this for
>> > check-compiler-rt command).
>> >
>> >
>> >>
>> >>  I think we
>> >> should consider punting on this one and looking for a more incremental
>> >> strategy.  For instance, how about starting by moving the call
>> >> 'add_llvm_external_project(compiler-rt)' from
>> >> llvm/projects/CMakesLists.txt to clang/runtime?  We can tweak this
>> >> macro (or a clang variant) to optionally do something like:
>> >>
>> >>     include(AddCompilerRt)
>> >>
>> >> That way if CMAKE_PREFIX_PATH includes a path to the compiler-rt
>> >> install directory, it can import all the same build targets that would
>> >> be created by add_subdirectory(compiler-rt).
>> >
>> >
>> > Probably I don't understand how CMAKE_PREFIX_PATH works. How can
>> > top-level
>> > targets from one build tree be visible in another build tree?
>> >
>> >>
>> >>
>> >> If I were to use CMake's ExternalProject feature, it would be from a
>> >> top-level superproject.  I would use it to manage dependencies and to
>> >> populate CMAKE_PREFIX_PATH for each dependency.  CMake, Make or a
>> >> package manager, it wouldn't matter which you used for this part (I've
>> >> used Make for this).  Ultimately, you want to end up with a top-level
>> >> build that performs the following:
>> >>
>> >>     mkdir -p llvm/out && cd llvm/out
>> >>     cmake .. -DCMAKE_INSTALL_PREFIX=ship
>> >>     ninja check-all install
>> >>     cd -
>> >>
>> >>     mkdir -p compiler-rt/out && cd compiler-rt/out
>> >>     cmake .. -DCMAKE_PREFIX_PATH=`pwd`/../../llvm/out/ship
>> >> -DCMAKE_INSTALL_PREFIX=ship
>> >>     ninja check-all install
>> >>     cd -
>> >>
>> >>     mkdir -p clang/out && cd clang/out
>> >>     cmake ..
>> >>
>> >> -DCMAKE_PREFIX_PATH=`pwd`/../../llvm/out/ship:`pwd`/../../compiler-rt/out/ship
>> >> -DCMAKE_INSTALL_PREFIX=ship
>> >>     ninja check-all install
>> >>     cd -
>> >>
>> >> In this scenario, the compiler-rt build creates a cmake module that
>> >> clang includes.  It uses that cmake module to find the compiler-rt
>> >> install targets and copy them into "lib/clang/<version>".  The clang
>> >> build then calls 'add_subdirectory' on the compiler-rt 'test'
>> >> directory, pointing it to the just-built-clang.  So the dependency
>> >> tree for clang is:
>> >>
>> >>     sanitizer-tests -> clang -> compiler-rt -> c-compiler
>> >
>> >
>> >
>> > ... But we want to build compiler-rt libraries with just-built Clang as
>> > well. This is what
>> > Makefile-based build does, and what we can benefit from.
>> >
>> >>
>> >>
>> >> Thoughts?
>> >>
>> >> Thanks,
>> >> Greg
>> >>
>> >>
>> >> On Thu, Feb 27, 2014 at 1:19 AM, Alexey Samsonov <samsonov at google.com>
>> >> wrote:
>> >> >
>> >> > On Wed, Feb 26, 2014 at 9:58 PM, Brad King <brad.king at kitware.com>
>> >> > wrote:
>> >> >>
>> >> >> On 02/26/2014 12:43 PM, Alexey Samsonov wrote:
>> >> >> > Do you think it makes sense to land my ExternalProject_Add patch
>> >> >> > so that others can experiment with it? I can add quit with a
>> >> >> > fatal_error/warning if the build tree rules are generated with
>> >> >> > Ninja.
>> >> >>
>> >> >> Since it is conditional on LLVM_BUILD_EXTERNAL_COMPILER_RT, yes.
>> >> >
>> >> >
>> >> > Submitted as r202367.
>> >> >
>> >> >>
>> >> >>
>> >> >> > parallelism doesn't work when I run "make check-compiler-rt -j8"
>> >> >> > in the original build tree, presumably because we call
>> >> >> > "cd /path/to/compiler-rt/build/tree && make check-all" there.
>> >> >>
>> >> >> Right.  The ExternalProject module has a special case for the
>> >> >> Makefile generators to make with $(MAKE) instead of "make":
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> http://cmake.org/gitweb?p=cmake.git;a=blob;f=Modules/ExternalProject.cmake;hb=v2.8.12.2#l846
>> >> >>
>> >> >> so that flags like -j propagate automatically.  You could do
>> >> >> that too:
>> >> >>
>> >> >>   if(CMAKE_GENERATOR MATCHES "Make")
>> >> >>     set(check_compiler_rt "$(MAKE)" "check-all")
>> >> >>   else()
>> >> >>     set(check_compiler_rt ${CMAKE_COMMAND} --build .
>> >> >>       --target check-all --config $<CONFIGURATION>)
>> >> >>   endif()
>> >> >>
>> >> >>   ExternalProject_Get_Property(compiler-rt BINARY_DIR)
>> >> >>   add_custom_target(check-compiler-rt
>> >> >>     COMMAND ${check_compiler_rt}
>> >> >>     DEPENDS compiler-rt
>> >> >>     WORKING_DIRECTORY ${BINARY_DIR}
>> >> >>     VERBATIM
>> >> >>     )
>> >> >>
>> >> >
>> >> > This worked, thanks! Currently I also print fatal_error message if I
>> >> > detect
>> >> > Ninja as a CMAKE_GENERATOR.
>> >> >
>> >> >
>> >> > --
>> >> > Alexey Samsonov, MSK
>> >> >
>> >> > _______________________________________________
>> >> > LLVM Developers mailing list
>> >> > LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> >> > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>> >> >
>> >
>> >
>> >
>> >
>> > --
>> > Alexey Samsonov, MSK
>
>
>
>
> --
> Alexey Samsonov, MSK

More information about the llvm-dev mailing list