[llvm-dev] [RFC] LLVM Directory Structure Changes (was Re: [PATCH] D20992: [CMake] Add LLVM runtimes directory)

[Sean Silva via llvm-dev]

On Tue, Jun 14, 2016 at 9:40 AM, Chris Bieneman <beanz at apple.com> wrote:

> I think I may be explaining the dependency problem poorly. This isn't so
> much a limitation in CMake, or a problem with our CMake implementation.
> This is a limitation in basically every build configuration management
> system I've ever seen.
>
> The problem comes from that fact that generally builds are structured as
> configure -> build. What we actually need is "configure some" -> "build
> some" -> "configure some more" -> "build some more" -> repeat until done.
>
> The problem occurs when you are bootstrapping a toolchain and target. In
> general a bootstrap build involves building a compiler for your target
> (which may or may not match your host), then building the various libraries
> and tools, working up to the full OS.
>
> For our projects the first step of a bootstrap is building a clang that
> runs on host supporting the target. After building that clang you need to
> build the runtime libraries. For sake of simplicity let's assume we're only
> building compiler-rt's current libraries. After building clang the logical
> step would be CMake + Ninja for compiler-rt then you're done right? Except,
> that doesn't work. You can't *configure* the sanitizer build until after
> you've fully configured and built the builtin libraries.
>
> Pardon the bad formatting here, but this next sentence is really
> important, so I'm going to try and apply some emphasis.
>
> ***Generally speaking, you cannot *configure* a project until all the
> tools and libraries your build system is running tests against are built.***
>
> If you configure a project before the tools and libraries are available
> your configuration system may make the wrong decisions. To put it in more
> normal compiler terminology, it is undefined behavior.
>
> We have viable workarounds for this in our build today. We can extend
> these workarounds to make bootstrapping simpler. There are some rough edges
> that are caused by limitations in CMake, but there is no fundamental
> limitation in CMake that makes this not work.
>
> At Apple we workaround this by having some out-of-tree script goop that
> configures and builds clang, then configures and builds the builtins, then
> configures and builds the runtimes. Due to complications interfacing with
> our internal build system, we'll likely always have that script goop, but
> I'd really like to be able to run clean bootstrap builds against open
> source without the crazy complications.
>
> I believe that splitting compiler-rt's sanitizer libraries and builtin
> archives into separate projects will allow the project structure to more
> clearly represent the way it fits into the project build graph. It is not
> strictly necessary for anything, so if people generally don't agree I won't
> fight for it.
>

If it isn't necessary, I would avoid doing it. There is a cost which is O(#
LLVM developers) and incurred against the entire community and its
constituent companies. This should not be done lightly.

-- Sean Silva


>
> -Chris
>
> > On Jun 12, 2016, at 4:51 AM, Renato Golin <renato.golin at linaro.org>
> wrote:
> >
> > On 12 June 2016 at 00:35, Sean Silva via llvm-dev
> > <llvm-dev at lists.llvm.org> wrote:
> >>> I think the fundamental distinction needs to be following dependency
> >>> graphs because if we don’t get rid of the circular dependency in
> >>> bootstrapping there is no reason to make any changes.
> >>
> >> +1 for what Chandler said here.
> >>
> >> I don't think CMake itself per se cares about actual VCS repo
> breakdown. It
> >> should be possible to bring sanity without any change to VCS structure.
> >
> > Funny enough "compiler-rt" is a fitting name to anything that provides
> > run-time libraries that users shouldn't be fiddling with, and that
> > includes builtins, sanitizers, profiling, the lot.
> >
> > Not getting into the name and split bike shed, I agree with Sean that
> > this is a CMake issue, not a directory structure one.
> >
> > Simply put, RT's CMake is a mess.
> >
> > If we can build each component (builtins, sans, profs) independently,
> > then the overall build dependency can follow without a split, by
> > basically creating internal independent targets with a phony target
> > that builds them all being the default.
> >
> > So, if in the main CMake you ask for libc++, RT and unwind, the build
> > dependency will be from specific internal targets in libc++ to other
> > internal targets in RT and so on.
> >
> > This could also be a way out to separate the "builtins" target into
> > "builtins.<arch>" into different internal targets to allow
> > cross-compilation. Though, that'd requiring having a complete
> > toolchain for each of the supported target, so it can't be default on
> > the targets built in LLVM.
> >
> > cheers,
> > --renato
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160614/9c9201d1/attachment.html>