[llvm-dev] [cfe-dev] GN build roundtable summary; adding GN build files to the repo

Thu Nov 1 07:14:50 PDT 2018

Lots of feedback! Sounds like several people are interested in this, which
is cool.

Trying to address a few points here:

 > f these files exist in the repo, I could imagine that reviewers will
> reject patches that might/likely break the gn files.

I will make the language in https://reviews.llvm.org/D53944 stronger.
Reviewers should not ask for this. If it helps, we could put the gn build
files into a directory called "experimental" or similar.

> Do you think there will be some point in time that these files will be
> out of 'experimental' status?

Not anytime soon.

> Some folks who don't like GN or don't see the point.

That's fine, see first sentence in my mail :-)

> my strong preference would be to accompany that with a plan to phase
cmake out.

I very strongly don't think this should happen. cmake does many things
right, for example running on all the platforms under the sun and
supporting very long-tail issues. By saying "cmake stays the main build
system" there's a long list of things we don't need to worry about. If in a
few years GN can build on all the platforms under the sun, everything in
LLVM works in the GN build, and everyone happens to use it, we could
discuss this at that point, but in my opinion this is extremely unlikely to
happen. If this is a requirement, I guarantee this thread will fizzle out.
Also, I'm very much not interested in pushing for moving llvm off cmake
personally.

> lldb xcode and cmake painful

I think this is in part because it's tricky to update the xcode build files
on non-Apple hardware, and because it's agreed upon which of the two is the
canonical one. Here, we very explicitly say that cmake is the official
build system that's covered by bots, and your patch has to work there, else
it gets reverted. And keeping the gn build up to date is solely the
responsibility of people using it.

This also gives a good metric on if anyone uses the gn build: If they're
bitrotted for long stretches, we know it's time to delete them again.

> I'm curious - would you please outline the problems you've encountered?

Sure. I did that a bit in https://reviews.llvm.org/D53944 (the
"Intoduction" section) and in my talk. The issues are:
1. cmake is slow, so it caches things
2. but cache eviction is hard, so it's very hard to change an existing
build dir e.g. from debug to release – for many config values you need to
create a completely new build dir to be sure the new values are actually
used
3. gn has support for building the same target with different toolchains in
a single build dir, making it e.g. possible to model bootstrap builds or
compiler-rt compiles with the built clang much nicer (I haven't implemented
that part yet)

I know I read the following two somewhere but can't find it, so this is an
approximate cite:

> the gn bots should be private, not public

I agree that no bots on the official LLVM waterfall (
http://lab.llvm.org:8011/console) should use GN. They would be
off-waterfall bots that don't email you and that you wouldn't know about if
you didn't go looking for them. They wouldn't even run on LLVM's buildbot
instance.

> so keep them in a separate repo

Right now, my approach adds the BUILD.gn files in the directory they belong
(clang/lib/Support/BUILD.gn, etc). That's not easily possible with a
separate repo. Using the secondary tree, this is easier -- but it's still
harder than having the files just there, and I don't see which problems are
solved by keeping them out of tree.

>  Most of us don't want to spend their time on build system tasks, right?

I do like doing them in gn. I find updating gn build files after syncing
very relaxing :-) (And I have a script that helps me, so it really isn't
much work -- see gn/sync_source_lists_from_cmake.py in my repo) For
example, I'm pretty happy with how the setup for how to select which archs
to build turned out in the LLVM GN build.

> Is there a good example where multiple build systems actually coexist
well?

webkit has a bunch, but I wouldn't say that works super well. I'm not aware
of any project that has tried experimental, unsupported build systems for
enthusiasts in addition to a single main supported build system.

--

To reiterate, this is the setup I use for my development. I feel it makes
me more happy and productive. Others are interested in it too. I think
there's value to be able to experiment in-tree with this since it makes
collaboration on the gn files much easier and has zero cost for people not
using it. I am not interested in long mailing list discussions about build
systems, so I'm very much not interested in proposing to remove cmake.
Also, I think requiring that build system experimentation has to have a
plan for converging on a single build system again eventually is a
basically impossibly high bar.

On Thu, Nov 1, 2018 at 4:44 AM Csaba Raduly <rcsaba at gmail.com> wrote:

> On Thu, Nov 1, 2018 at 1:22 AM Vedant Kumar via cfe-dev
> <cfe-dev at lists.llvm.org> wrote:
> >
> > Hi all,
> >
> >> On Oct 31, 2018, at 11:18 AM, Nico Weber via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
> >>
> >> Hi,
> >>
> >> first things first: If you're happy with cmake, you can stop reading
> now.
> >>
>
> I'm not, I just put up with it :)
>
> > ...
> > I think it's very valuable that we have one shared build system. If
> you'd like to check in GN files, my strong preference would be to accompany
> that with a plan to phase cmake out.
> >
> > I really don't mean to be flippant here -- I'm in awe of the work you've
> already done to set up GN files, and I know transitioning away from cmake
> would be a massive amount of work. Hear me out :)
> >
> > My perspective comes from having worked on lldb for a while. lldb has
> two build systems (xcodebuild & cmake). I suppose opinions differ on
> whether that works well. Speaking for myself, having two build systems has
> been a massive source of frustration. I regularly see commits which break
> one of the two systems because of course they do. No one wants to test
> their commit a second time against a build system they don't really use.
> Replicating complex bits of build system logic also is a chore -- I've CC'd
> Stefan who might be able to say more about that.
> >
> > My concern is that introducing gn files into llvm will cause a bit of a
> fracture. If the policy is that cmake users don't have to worry about
> breaking the gn build, I think gn users would be less inclined to fix the
> cmake build. If most developers decide to switch to gn, that would leave
> cmake adopters with a higher (possibly unmanageable) maintenance burden.
> It's also confusing for new users, as they already have a lot of different
> ways of checking out and building.
> >
> > I know your plan is to have the maintenance burden of gn files placed on
> gn users, and that you haven't experienced an unmanageable number of breaks
> over the past 8 months. In lldb (a lower volume project), I actually do
> think the constant build breaks are hard to manage. And I'm just not sure
> the temptation to only update gn files could be resisted, as "works in my
> build system" tends to shut down conversations. I'm worried the same thing
> would happen in llvm.
> >
> > best,
> > vedant
> >
> >
>
> Precisely. How would the proposed cmake + gn build system be different
> from the (deprecated and then removed) autoconf + cmake ?
>
> Csaba
> --
> You can get very substantial performance improvements
> by not doing the right thing. - Scott Meyers, An Effective C++11/14 Sampler
> So if you're looking for a completely portable, 100% standards-conformat
> way
> to get the wrong information: this is what you want. - Scott Meyers
> (C++TDaWYK)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20181101/28a09124/attachment-0001.html>