<br><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Oct 30, 2012 at 4:09 PM, Alexey Samsonov <span dir="ltr"><<a href="mailto:samsonov@google.com" target="_blank">samsonov@google.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><br><br><div class="gmail_quote"><div><div class="h5">On Tue, Oct 30, 2012 at 3:58 PM, Evgeniy Stepanov <span dir="ltr"><<a href="mailto:eugenis@google.com" target="_blank">eugenis@google.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div>On Tue, Oct 30, 2012 at 2:03 PM, Alexey Samsonov <span dir="ltr"><<a href="mailto:samsonov@google.com" target="_blank">samsonov@google.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">+cfe-dev<div><div><br><br><div class="gmail_quote">On Tue, Oct 30, 2012 at 1:50 PM, Alexey Samsonov <span dir="ltr"><<a href="mailto:samsonov@google.com" target="_blank">samsonov@google.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
Hi there!<div><br></div><div>tl;dr we should build compiler-rt with just-built clang</div><div><br></div><div>A number of issues with CMake build of compiler-rt libraries and tests</div><div>are caused by the fact that we build compiler-rt with host compiler instead of</div>
<div>just-built Clang (examples: <a href="http://llvm.org/bugs/show_bug.cgi?id=13864" target="_blank">http://llvm.org/bugs/show_bug.cgi?id=13864</a>, <a href="http://llvm.org/bugs/show_bug.cgi?id=14109" target="_blank">http://llvm.org/bugs/show_bug.cgi?id=14109</a>)</div>
<div><br></div><div>Using Clang looks like the Right thing to do, and it's probably time to make this happen:</div><div>1) configure+make build system uses fresh Clang</div><div>2) using just one fresh Clang will allow us to drop support for all kinds of host compilers users</div>
<div>are building compiler-rt with:</div><div> * we will be able to build compiler-rt with -Werror</div><div> * we will be able to build versions of compiler-rt libraries for all target triples supported by Clang (not by host compiler)</div>
<div> * we will be able to run compiler-rt tests, regardless of host compiler. For example, my gcc-4.4 build of compiler-rt</div><div> fails on "check-ubsan" tests because of lacking 128-bit int support.</div>
<div><br></div><div>OK, how can we build with fresh Clang? (One can't simply tell CMake to build some targets by a compiler produced</div><div>while building another target).</div><div><br></div><div>Chandler (and other developers) frequently mention the bootstrap process (external to CMake), that (IIUC) build Clang</div>
<div>and then uses it to re-build LLVM[+Clang?]+compiler-rt.</div><div>Can we:</div><div>1) *always* build compiler-rt as a part of bootstrap process (run CMake to build clang, then run it again with Clang as a C compiler to build compiler-rt).</div>
<div>2) document the usage.</div><div>3) make a script that would run bootstrap process in (1), so that the users can invoke a single command instead of running a sequence of instructions?</div><div><br></div><div>So, when CMake is called with Clang compiler, it can run a set of try_compile() tests to determine possible target triples and build necessary compiler-rt</div>
<div>libraries for all these triples. But I'm afraid it's *not* that easy:</div><div>1) if a compiler-rt library ever needs to use LLVM libraries, it needs to have these LLVM libs built for the same target. That is, in general we may require</div>
<div>a bunch of LLVM libraries compiled for different targets, which is not supported by LLVM build system at all.</div><div>2) part (1) is already true for compiler-rt unit tests: there is no way we can build 64-bit and 32-bit version of the same unit test in a single build tree, as we'd</div>
<div>need two different versions of googletest and LLVMSupport.</div><div>3) Notable case is Android (eugenis@ may comment more on this) - building for Android requires many changes and is hardly possible in the same</div>
<div>build tree we use for regular (say, x86_64-linux) build tree.</div></blockquote></div></div></div></blockquote><div><br></div></div></div><div>The difference is you can not get away with simply changing CFLAGS (-m32 vs -m64) in this case. You need to add quite a few other flags, and, more often than not, host compiler can not target Android at all. So, while we can do some hacks to build both x86_32 and x86_64 runtime libraries in the same build tree (with the host compiler), we really need a separate tree, and a different compiler, for Android targets.</div>
<div><br></div><div>Recursive trees (below) seem like the only option.</div><div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div><div>
<div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div><br></div><div>We can try to resolve this by using recursive build trees:</div>
<div>* for each target create its own "recursive" LLVM build tree (e.g. projects/compiler-rt/Linux-x86_64) and invoke CMake</div>
<div>there with some additional compile/link flags.</div><div>* describe a single target (say, "clang_rt") in a CMakeLists.txt in each target-specific recursive build tree.</div><div>* Build rule for "clang_rt.linux-x86_64" in main build tree in will simply invoke building "clang_rt" in necessary</div>
<div>target-specific subtree and copy result from there.</div><div><br></div><div>Overall, I believe this is doable (but pretty ugly) and have a raw working prototype for this. If we decide to go this way,</div><div>I'm ready to complete this if anyone signs up for a careful review (as my CMake skills are far from good).</div>
<div><br></div><div>Problems:</div>
<div>1) this adds significant complexity to CMake build system. Rules for building specific libraries may become easier and shorter, but all the infrastructure</div><div>(building dependencies between separate build trees is... challenging) would be messy and contain a lot of implicit details. Moreover, each CMakeLists.txt</div>
<div>would have to behave in two different modes (whether it's included from target-specific build tree or from main one).</div></blockquote></div></div></div></blockquote><div><br></div></div><div>Why two modes? Sounds like it just needs to build whatever it is building with the current CC/CXX/CFLAGS/LDFLAGS/etc (which, of course, depend on each tree configuration).</div>
</div></div></blockquote><div><br></div></div></div><div>Nah, I mean your CMakeLists.txt would probably look like this:</div><div><br></div><div>if(IN_TARGET_SPECIFIC_BUILD_TREE)</div><div> add_library(clang_rt.asan ${ASAN_SOURCES} ...)</div>
<div> ..</div><div>else()</div><div> AddRuntimeForTarget(Linux-x86_64 clang_rt clang_rt.x86_64)</div><div> AddRuntimeForTarget(Android clang_rt clang_rt.arm-android)</div><div>endif()</div><div><br></div><div>Alternatively, you can move all calls of AddRuntimeForTarget (AddTestForTarget etc...) to some other file.</div>
<div class="im">
<div> </div></div></div></blockquote><div><br></div><div>But we don't need this in every CMakeLists.txt, right? This switch goes under tools/clang/runtime/. Every other CMakeLists.txt simply knows how to build stuff for the current configuration - we just don't execute compiler-rt's rules in the main tree (or do we?).</div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div class="gmail_quote"><div class="im"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class="gmail_extra"><div class="gmail_quote"><div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div><div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div>2) building with debug Clang is slow (default for many developers). Though, compiler-rt libraries are not that large for now, so we've observed this problem</div><div>with ASan unit tests (we may split it and make use of parallelism).</div>
</blockquote></div></div></div></blockquote><div><br></div></div><div>And, for correctness, any change that affects the compiler binary would trigger full rebuild of the runtime libraries.</div></div></div></blockquote><div>
<br></div></div><div>Good point.</div><div class="im"><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class="gmail_extra"><div class="gmail_quote"><div>As we bring more llvm code into compiler_rt (like we have done with LLVMSupport), this will slow down development tremendously.</div>
</div></div></blockquote><div><br></div></div><div>This would slow down the development for Clang guys :) If you touch compiler-rt parts of code, the compiler is not changed, and you</div><div>don't have to rebuild everything.</div>
<div class="im">
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div class="gmail_extra"><div class="gmail_quote">
<div>We need to have some kind of "fast" mode that would not rebuild runtime every time. </div>
</div></div></blockquote><div><br></div></div><div>Yes, smth like "make clang-only" that wouldn't use fresh Clang to rebuild compiler-rt libraries.</div><div class="im"><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class="gmail_extra"><div class="gmail_quote"><div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div><div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div><div><br></div><div>WDYT?</div><div><br></div>--<br><div>Alexey Samsonov, MSK</div>
</div>
</blockquote></div><br><br clear="all"><div><br></div></div></div><span><font color="#888888">-- <br><div>Alexey Samsonov, MSK</div><br>
</font></span></blockquote></div></div><br></div>
</blockquote></div></div><span class=""><font color="#888888"><br><br clear="all"><div><br></div>-- <br><div>Alexey Samsonov, MSK</div><br>
</font></span></blockquote></div><br></div>