[cfe-dev] CMake builld system for compiler-rt rework

[Evgeniy Stepanov]

On Tue, Oct 30, 2012 at 2:03 PM, Alexey Samsonov <samsonov at google.com>wrote:

> +cfe-dev
>
>
> On Tue, Oct 30, 2012 at 1:50 PM, Alexey Samsonov <samsonov at google.com>wrote:
>
>> Hi there!
>>
>> tl;dr we should build compiler-rt with just-built clang
>>
>> A number of issues with CMake build of compiler-rt libraries and tests
>> are caused by the fact that we build compiler-rt with host compiler
>> instead of
>> just-built Clang (examples: http://llvm.org/bugs/show_bug.cgi?id=13864,
>> http://llvm.org/bugs/show_bug.cgi?id=14109)
>>
>> Using Clang looks like the Right thing to do, and it's probably time to
>> make this happen:
>> 1) configure+make build system uses fresh Clang
>> 2) using just one fresh Clang will allow us to drop support for all kinds
>> of host compilers users
>> are building compiler-rt with:
>>   * we will be able to build compiler-rt with -Werror
>>   * we will be able to build versions of compiler-rt libraries for all
>> target triples supported by Clang (not by host compiler)
>>   * we will be able to run compiler-rt tests, regardless of host
>> compiler. For example, my gcc-4.4 build of compiler-rt
>>     fails on "check-ubsan" tests because of lacking 128-bit int support.
>>
>> OK, how can we build with fresh Clang? (One can't simply tell CMake to
>> build some targets by a compiler produced
>> while building another target).
>>
>> Chandler (and other developers) frequently mention the bootstrap process
>> (external to CMake), that (IIUC) build Clang
>> and then uses it to re-build LLVM[+Clang?]+compiler-rt.
>> Can we:
>> 1) *always* build compiler-rt as a part of bootstrap process (run CMake
>> to build clang, then run it again with Clang as a C compiler to build
>> compiler-rt).
>> 2) document the usage.
>> 3) make a script that would run bootstrap process in (1), so that the
>> users can invoke a single command instead of running a sequence of
>> instructions?
>>
>> So, when CMake is called with Clang compiler, it can run a set of
>> try_compile() tests to determine possible target triples and build
>> necessary compiler-rt
>> libraries for all these triples. But I'm afraid it's *not* that easy:
>> 1) if a compiler-rt library ever needs to use LLVM libraries, it needs to
>> have these LLVM libs built for the same target. That is, in general we may
>> require
>> a bunch of LLVM libraries compiled for different targets, which is not
>> supported by LLVM build system at all.
>> 2) part (1) is already true for compiler-rt unit tests: there is no way
>> we can build 64-bit and 32-bit version of the same unit test in a single
>> build tree, as we'd
>> need two different versions of googletest and LLVMSupport.
>> 3) Notable case is Android (eugenis@ may comment more on this) -
>> building for Android requires many changes and is hardly possible in the
>> same
>> build tree we use for regular (say, x86_64-linux) build tree.
>>
>
The difference is you can not get away with simply changing CFLAGS (-m32 vs
-m64) in this case. You need to add quite a few other flags, and, more
often than not, host compiler can not target Android at all. So, while we
can do some hacks to build both x86_32 and x86_64 runtime libraries in the
same build tree (with the host compiler), we really need a separate tree,
and a different compiler, for Android targets.

Recursive trees (below) seem like the only option.


>
>> We can try to resolve this by using recursive build trees:
>> * for each target create its own "recursive" LLVM build tree (e.g.
>> projects/compiler-rt/Linux-x86_64) and invoke CMake
>> there with some additional compile/link flags.
>> * describe a single target (say, "clang_rt") in a CMakeLists.txt in each
>> target-specific recursive build tree.
>> * Build rule for "clang_rt.linux-x86_64" in main build tree in  will
>> simply invoke building "clang_rt" in necessary
>> target-specific subtree and copy result from there.
>>
>> Overall, I believe this is doable (but pretty ugly) and have a raw
>> working prototype for this. If we decide to go this way,
>> I'm ready to complete this if anyone signs up for a careful review (as my
>> CMake skills are far from good).
>>
>> Problems:
>> 1) this adds significant complexity to CMake build system. Rules for
>> building specific libraries may become easier and shorter, but all the
>> infrastructure
>> (building dependencies between separate build trees is... challenging)
>> would be messy and contain a lot of implicit details. Moreover, each
>> CMakeLists.txt
>> would have to behave in two different modes (whether it's included from
>> target-specific build tree or from main one).
>>
>
Why two modes? Sounds like it just needs to build whatever it is building
with the current CC/CXX/CFLAGS/LDFLAGS/etc (which, of course, depend on
each tree configuration).


> 2) building with debug Clang is slow (default for many developers).
>> Though, compiler-rt libraries are not that large for now, so we've observed
>> this problem
>> with ASan unit tests (we may split it and make use of parallelism).
>>
>
And, for correctness, any change that affects the compiler binary would
trigger full rebuild of the runtime libraries. As we bring more llvm code
into compiler_rt (like we have done with LLVMSupport), this will slow down
development tremendously.

We need to have some kind of "fast" mode that would not rebuild runtime
every time.


>
>> WDYT?
>>
>> --
>> Alexey Samsonov, MSK
>>
>
>
>
> --
> Alexey Samsonov, MSK
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20121030/1568a82a/attachment.html>