[llvm-dev] Contributing Bazel BUILD files similar to gn
Tom Stellard via llvm-dev
llvm-dev at lists.llvm.org
Thu Oct 29 08:22:57 PDT 2020
On 10/28/20 7:18 PM, Geoffrey Martin-Noble via llvm-dev wrote:
> Hi all,
> tl;dr: We'd like to contribute Bazel BUILD files for LLVM and MLIR in a
> side-directory in the monorepo, similar to the gn build.
> Some of us have been working on open-source Bazel BUILD files for the
> LLVM Project. You may have seen us hanging out in the #build-systems
> discord channel. As you may know, Google uses Bazel internally and has
> maintained a Bazel BUILD of LLVM for years. Especially with the
> introduction of MLIR, we've got more and more OSS projects with a Bazel
> BUILD depending on LLVM (e.g. IREE <https://github.com/google/iree>and
> TensorFlow <https://github.com/tensorflow/tensorflow>). We're also not
> the only ones using Bazel: e.g. PlaidML also has a Bazel BUILD of LLVM
> that they've borrowed from TF
> Each of these projects has to jump through some weird hoops to keep
> their version of the Bazel BUILD files in sync with the code, which
> requires some fragile combination of scripts and human intervention.
> Instead, we'd like to move general-purpose Bazel BUILD files into the
> LLVM Project monorepo. We expect to follow the model of the GN build
> where these will be maintained by interested contributors rather than
> expecting the general community to maintain them.
> To facilitate and test this we've been developing a standalone
> repository that just has the Bazel BUILD files. It symlinks together the
> directory trees on top of a submodule as we would need in the monorepo
> to to avoid in-tree BUILD files. The configuration is at
> https://github.com/google/llvm-bazel. We now have those in a good place
> and think they would be useful upstream.
Can you explain some of the benefits to using Bazel instead of CMake?
I'm a little concerned about having two 'unsupported' buildsystems
living in tree, and I'm not sure what would stop us from continuing to
add more. I would feel better if we had a set of guidelines to define
the criteria for adding a new buildsytem and also criteria for when we
can remove them.
Would you be able to amend this proposal to include some general
guidelines for adding/removing new buildsystems, so that we can discuss
> # Details
> ## What
> Bazel BUILD files for the LLVM, MLIR, and Clang (PR out for review
> <https://github.com/google/llvm-bazel/pull/72>) subprojects, potentially
> expanding to others, as needed. Basically everything currently at
> ## Where
> In https://github.com/google/llvm-bazelthe BUILD files live in a single
> directory tree matching the structure of the overall llvm-project
> directory. For users, @llvm-project is a single Bazel repository
> includes both LLVM and MLIR subprojects. To maintain this structure, we
> would probably want to put a `bazel` directory in the monorepo's utils
> directory <https://github.com/llvm/llvm-project/tree/master/utils>,
> which currently only contains a directory for arcanist. This is
> different from gn, which is under the LLVM subproject's utils directory
> <https://github.com/llvm/llvm-project/tree/master/llvm/utils/gn>. We
> could similarly put the Bazel BUILD files under llvm/utils/bazel but
> have them be for the entire llvm project (the subsets that are
> supported). This seems like an odd structure to me, but I know that the
> CMake build for LLVM also builds the other subprojects
> so maybe this would be preferable.
> Alternatively we could split each subproject into a separate Bazel
> repository and put the Bazel build files under each subproject. I think
> this fragments the configuration of the BUILD without much benefit.
> ## Configurations
> We currently have configurations for Linux GCC and Clang, MacOS GCC and
> Clang, and Windows MSVC. Support for other configurations can be added
> as-desired, but supporting all possible LLVM build configurations is not
> the goal.
> ## Support
> Support would be similar to the gn build. Contributors could optionally
> update the Bazel BUILD files as part of their patches, but would be
> under no obligation to do so.
> ## Preserving History
> I don't *think* the history of llvm-bazel is interesting enough to try
> to merge it into the monorepo and I was planning to submit this as a
> single patch, but please let me know if you disagree.
> ## Benefits to the community
> Projects that depend on LLVM and use the Bazel build system can
> avoid duplicating fragile effort. We'll spend more time contributing
> to LLVM instead :-D
> Bazel is stricter than CMake in many ways (e.g. it requires that
> even header dependencies be declared) and can catch layering issues
> very easily. There's even an optional layering_check feature we
> could turn on if its use would benefit the community. (though
> currently the existing problematic layering makes it a burden to
> maintain on our own). Even without that additional check, as I've
> been keeping the Bazel build green, I've found and fixed a number of
> layering issues in the past couple weeks (e.g.
> Here's a patch <https://reviews.llvm.org/D90352>adding the Bazel build
> system. It's basically just `cp -r llvm-bazel/llvm-bazel
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
More information about the llvm-dev