[llvm-dev] [RFC] Strategies for Bootstrapping Compiler-RT builtins

[Chris Bieneman via llvm-dev]

> On Nov 2, 2015, at 9:42 AM, Justin Bogner <mail at justinbogner.com> wrote:
> 
> Chris Bieneman via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> writes:
>> In the effort to flesh out the CMake build system a problematic issue
>> has come up, and I’d like some feedback on how to best handle it.
>> 
>> For reference this issue has been reported by a few users, one
>> proposed patches that don’t really address the underlying problem
>> here:
>> http://reviews.llvm.org/D13131
>> 
>> The problem comes when bootstrapping a cross-compiler toolchain. In
>> order to have a cross-compiling toolchain that can build a “hello
>> world” application you need four basic components:
>> 
>> (1) clang
>> (2) ld
>> (3) libclang_rt (builtins)
>> (4) runtime libraries
>> 
>> Today building this toolchain with CMake is impossible because you
>> cannot configure the cross-compiled builtins. The failure is a result
>> of CMake’s try_compile function always testing a full compile then
>> link operation. When bootstrapping a cross-compiler this will always
>> fail because linking even the simplest applications fails when you
>> don’t have libclang_rt prebuilt.
>> 
>> So, how do we fix this? I have a couple ideas, and am open to more.
>> 
>> (1) Roll our own CMake checks
>> 
>> We could roll our own replacement to try_compile and the various check
>> macros that we need. In my opinion this is probably the right
>> solution, but it does have downsides.
>> 
>> The big downside is that when bootstrapping compiler-rt it will need
>> to build differently. In particular, it is probable that a bootstrap
>> build of compiler-rt will not be able to perform the necessary checks
>> to build the runtimes, so when bootstrapping we’ll need to disable
>> building all the runtime libraries. We can probably find clever ways
>> to hide a bunch of the complexity here, but it is not going to be
>> clean.
>> 
>> (2) Provide a way to bootstrap the builtins without CMake
>> 
>> Another alternative would be to provide a way to bootstrap the builtin
>> libraries without CMake. The builtin libraries are actually very
>> simple to compile. It is possible to roll a custom build script for
>> use only bootstrapping the builtins that could run on any platform and
>> just get to a functional compiler. The biggest downside here is that
>> bootstrapping on all supported platforms with all supported compilers
>> is actually a non-trivial matrix, and supporting and maintaining that
>> could be a real pain. This is my least favorite option.
>> 
>> (3) Split the builtins and the runtime libraries
>> 
>> This is the most complicated approach, but I also think it is the best
>> approach. One of the underlying problems here is that the builtin
>> libraries and the runtime libraries have very different requirements
>> for building. The builtins really only require a functional compiler
>> and archiver, and the runtime libraries require a full linker +
>> runtime libraries (libc & libcxx). These additional build-time
>> requirements actually make things very complicated because when
>> bootstrapping a cross toolchain compiler-rt needs to build in two
>> different places in the build order; once before libcxx, and once
>> after.
>> 
>> I believe that the cleanest solution to this problem is going to be to
>> separate the builtins and the sanitizers.
> 
> To be clear, you mean split the builtins (as in lib/builtins) and all of
> the runtime libraries (as in everything else under lib: sanitizers,
> profiling runtimes, etc). Correct?

Yes. lib/builtins is the part that causes all the problems.

-Chris

> 
>> Doing this and rolling our own CMake checks would allow us to have a
>> fully CMake solution for building a cross-targeting toolchain. We
>> might even be able to get support for try_compile checks that don’t
>> link from CMake which would allow us to get rid of the hand-rolled
>> checks in the future (I have already started a thread on the
>> cmake-developers list).
>> 
>> Logistically this solution could take many forms. We could break
>> compiler-rt out into two repositories, which would be a huge
>> undertaking, or we could leave it as a single repository and have the
>> builtins be able to build as a sub-project. I think we can make it
>> work such that compiler-rt can be built either from the top-level
>> directory to build it all, or from the builtins sub directory to
>> support bootstrapping cross-compilers.
>> 
>> Either way, supporting this approach will require significant cleanup
>> and refactoring because we’ll need to separate out the build system
>> functionality into three categories: things that apply to builtins,
>> things that apply to runtimes, things that apply to both. That
>> separation will need to be somewhat clearly maintained so that we can
>> prevent inadvertent stream crossing, because that is almost always
>> bad.
>> 
>> Thoughts? Additional suggestions?
> 
> ISTM that (3) essentially includes the work from (1), but ends up in a
> more maintainable state. (3) feels like the right approach to me, as (1)
> will be error prone and hard to maintain, and (2) is kludgy and complex
> in a way that could scare people away from contributing.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20151102/b1901044/attachment.html>