[llvm-dev] [RFC] LLVM Directory Structure Changes (was Re: [PATCH] D20992: [CMake] Add LLVM runtimes directory)

[Chris Bieneman via llvm-dev]

I think I may be explaining the dependency problem poorly. This isn't so much a limitation in CMake, or a problem with our CMake implementation. This is a limitation in basically every build configuration management system I've ever seen.

The problem comes from that fact that generally builds are structured as configure -> build. What we actually need is "configure some" -> "build some" -> "configure some more" -> "build some more" -> repeat until done.

The problem occurs when you are bootstrapping a toolchain and target. In general a bootstrap build involves building a compiler for your target (which may or may not match your host), then building the various libraries and tools, working up to the full OS.

For our projects the first step of a bootstrap is building a clang that runs on host supporting the target. After building that clang you need to build the runtime libraries. For sake of simplicity let's assume we're only building compiler-rt's current libraries. After building clang the logical step would be CMake + Ninja for compiler-rt then you're done right? Except, that doesn't work. You can't *configure* the sanitizer build until after you've fully configured and built the builtin libraries.

Pardon the bad formatting here, but this next sentence is really important, so I'm going to try and apply some emphasis.

***Generally speaking, you cannot *configure* a project until all the tools and libraries your build system is running tests against are built.***

If you configure a project before the tools and libraries are available your configuration system may make the wrong decisions. To put it in more normal compiler terminology, it is undefined behavior.

We have viable workarounds for this in our build today. We can extend these workarounds to make bootstrapping simpler. There are some rough edges that are caused by limitations in CMake, but there is no fundamental limitation in CMake that makes this not work.

At Apple we workaround this by having some out-of-tree script goop that configures and builds clang, then configures and builds the builtins, then configures and builds the runtimes. Due to complications interfacing with our internal build system, we'll likely always have that script goop, but I'd really like to be able to run clean bootstrap builds against open source without the crazy complications.

I believe that splitting compiler-rt's sanitizer libraries and builtin archives into separate projects will allow the project structure to more clearly represent the way it fits into the project build graph. It is not strictly necessary for anything, so if people generally don't agree I won't fight for it.

-Chris

> On Jun 12, 2016, at 4:51 AM, Renato Golin <renato.golin at linaro.org> wrote:
> 
> On 12 June 2016 at 00:35, Sean Silva via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>>> I think the fundamental distinction needs to be following dependency
>>> graphs because if we don’t get rid of the circular dependency in
>>> bootstrapping there is no reason to make any changes.
>> 
>> +1 for what Chandler said here.
>> 
>> I don't think CMake itself per se cares about actual VCS repo breakdown. It
>> should be possible to bring sanity without any change to VCS structure.
> 
> Funny enough "compiler-rt" is a fitting name to anything that provides
> run-time libraries that users shouldn't be fiddling with, and that
> includes builtins, sanitizers, profiling, the lot.
> 
> Not getting into the name and split bike shed, I agree with Sean that
> this is a CMake issue, not a directory structure one.
> 
> Simply put, RT's CMake is a mess.
> 
> If we can build each component (builtins, sans, profs) independently,
> then the overall build dependency can follow without a split, by
> basically creating internal independent targets with a phony target
> that builds them all being the default.
> 
> So, if in the main CMake you ask for libc++, RT and unwind, the build
> dependency will be from specific internal targets in libc++ to other
> internal targets in RT and so on.
> 
> This could also be a way out to separate the "builtins" target into
> "builtins.<arch>" into different internal targets to allow
> cross-compilation. Though, that'd requiring having a complete
> toolchain for each of the supported target, so it can't be default on
> the targets built in LLVM.
> 
> cheers,
> --renato