[llvm-dev] Contributing Bazel BUILD files similar to gn

[Geoffrey Martin-Noble via llvm-dev]

Hi all,

tl;dr: We'd like to contribute Bazel BUILD files for LLVM and MLIR in a
side-directory in the monorepo, similar to the gn build.

Some of us have been working on open-source Bazel BUILD files for the LLVM
Project. You may have seen us hanging out in the #build-systems discord
channel. As you may know, Google uses Bazel internally and has maintained a
Bazel BUILD of LLVM for years. Especially with the introduction of MLIR,
we've got more and more OSS projects with a Bazel BUILD depending on LLVM
(e.g. IREE <https://github.com/google/iree> and TensorFlow
<https://github.com/tensorflow/tensorflow>). We're also not the only ones
using Bazel: e.g. PlaidML also has a Bazel BUILD of LLVM that they've borrowed
from TF
<https://github.com/plaidml/plaidml/blob/master/vendor/llvm/llvm.BUILD>.
Each of these projects has to jump through some weird hoops to keep their
version of the Bazel BUILD files in sync with the code, which requires some
fragile combination of scripts and human intervention. Instead, we'd like
to move general-purpose Bazel BUILD files into the LLVM Project monorepo.
We expect to follow the model of the GN build where these will be
maintained by interested contributors rather than expecting the general
community to maintain them.

To facilitate and test this we've been developing a standalone repository
that just has the Bazel BUILD files. It symlinks together the directory
trees on top of a submodule as we would need in the monorepo to to avoid
in-tree BUILD files. The configuration is at
https://github.com/google/llvm-bazel. We now have those in a good place and
think they would be useful upstream.

# Details

## What

Bazel BUILD files for the LLVM, MLIR, and Clang (PR out for review
<https://github.com/google/llvm-bazel/pull/72>) subprojects, potentially
expanding to others, as needed. Basically everything currently at
https://github.com/google/llvm-bazel.

## Where

In https://github.com/google/llvm-bazel the BUILD files live in a single
directory tree matching the structure of the overall llvm-project
directory. For users, @llvm-project is a single Bazel repository
<https://docs.bazel.build/versions/master/build-ref.html#repositories> that
includes both LLVM and MLIR subprojects. To maintain this structure, we
would probably want to put a `bazel` directory in the monorepo's utils
directory <https://github.com/llvm/llvm-project/tree/master/utils>, which
currently only contains a directory for arcanist. This is different from
gn, which is under the LLVM subproject's utils directory
<https://github.com/llvm/llvm-project/tree/master/llvm/utils/gn>. We could
similarly put the Bazel BUILD files under llvm/utils/bazel but have them be
for the entire llvm project (the subsets that are supported). This seems
like an odd structure to me, but I know that the CMake build for LLVM
also builds
the other subprojects
<https://github.com/llvm/llvm-project/blob/529ac33197f6/llvm/tools/CMakeLists.txt#L34-L41>,
so maybe this would be preferable.

Alternatively we could split each subproject into a separate Bazel
repository and put the Bazel build files under each subproject. I think
this fragments the configuration of the BUILD without much benefit.

## Configurations

We currently have configurations for Linux GCC and Clang, MacOS GCC and
Clang, and Windows MSVC. Support for other configurations can be added
as-desired, but supporting all possible LLVM build configurations is not
the goal.

## Support

Support would be similar to the gn build. Contributors could optionally
update the Bazel BUILD files as part of their patches, but would be under
no obligation to do so.

## Preserving History

I don't *think* the history of llvm-bazel is interesting enough to try to
merge it into the monorepo and I was planning to submit this as a single
patch, but please let me know if you disagree.

## Benefits to the community

   -

   Projects that depend on LLVM and use the Bazel build system can avoid
   duplicating fragile effort. We'll spend more time contributing to LLVM
   instead :-D
   -

   Bazel is stricter than CMake in many ways (e.g. it requires that even
   header dependencies be declared) and can catch layering issues very easily.
   There's even an optional layering_check feature we could turn on if its use
   would benefit the community. (though currently the existing problematic
   layering makes it a burden to maintain on our own). Even without that
   additional check, as I've been keeping the Bazel build green, I've found
   and fixed a number of layering issues in the past couple weeks (e.g.
   https://reviews.llvm.org/rGb49787df9a
   <https://reviews.llvm.org/rGb49787df9a535f03761c340dca7ec3ec1155133d>
   and https://reviews.llvm.org/rGc17ae2916c
   <https://reviews.llvm.org/rGc17ae2916ccf45a0c1717bd5f11598cc4fff342a>).


Here's a patch <https://reviews.llvm.org/D90352> adding the Bazel build
system. It's basically just `cp -r llvm-bazel/llvm-bazel
llvm-project/utils/bazel`.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20201028/89728611/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3992 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20201028/89728611/attachment-0001.bin>