[llvm-dev] Contributing Bazel BUILD files similar to gn

Thu Oct 29 18:44:47 PDT 2020

(see below)

On 10/28/20 6:18 PM, Geoffrey Martin-Noble via llvm-dev wrote:
 > Hi all,
 >
 > tl;dr: We'd like to contribute Bazel BUILD files for LLVM and MLIR in a
 > side-directory in the monorepo, similar to the gn build.
 >
 > Some of us have been working on open-source Bazel BUILD files for the 
LLVM
 > Project. You may have seen us hanging out in the #build-systems discord
 > channel. As you may know, Google uses Bazel internally and has 
maintained a
 > Bazel BUILD of LLVM for years. Especially with the introduction of MLIR,
 > we've got more and more OSS projects with a Bazel BUILD depending on LLVM
 > (e.g. IREE <https://github.com/google/iree> and TensorFlow
 > <https://github.com/tensorflow/tensorflow>). We're also not the only ones
 > using Bazel: e.g. PlaidML also has a Bazel BUILD of LLVM that they've 
borrowed
 > from TF
 > <https://github.com/plaidml/plaidml/blob/master/vendor/llvm/llvm.BUILD>.
 > Each of these projects has to jump through some weird hoops to keep their
 > version of the Bazel BUILD files in sync with the code, which 
requires some
 > fragile combination of scripts and human intervention. Instead, we'd like
 > to move general-purpose Bazel BUILD files into the LLVM Project monorepo.
 > We expect to follow the model of the GN build where these will be
 > maintained by interested contributors rather than expecting the general
 > community to maintain them.
 >
 > To facilitate and test this we've been developing a standalone repository
 > that just has the Bazel BUILD files. It symlinks together the directory
 > trees on top of a submodule as we would need in the monorepo to to avoid
 > in-tree BUILD files. The configuration is at
 > https://github.com/google/llvm-bazel. We now have those in a good 
place and
 > think they would be useful upstream.
 >
 > # Details
 >
 > ## What
 >
 > Bazel BUILD files for the LLVM, MLIR, and Clang (PR out for review
 > <https://github.com/google/llvm-bazel/pull/72>) subprojects, potentially
 > expanding to others, as needed. Basically everything currently at
 > https://github.com/google/llvm-bazel.
 >
 > ## Where
 >
 > In https://github.com/google/llvm-bazel the BUILD files live in a single
 > directory tree matching the structure of the overall llvm-project
 > directory. For users, @llvm-project is a single Bazel repository
 > 
<https://docs.bazel.build/versions/master/build-ref.html#repositories> that
 > includes both LLVM and MLIR subprojects. To maintain this structure, we
 > would probably want to put a `bazel` directory in the monorepo's utils
 > directory <https://github.com/llvm/llvm-project/tree/master/utils>, which
 > currently only contains a directory for arcanist. This is different from
 > gn, which is under the LLVM subproject's utils directory
 > <https://github.com/llvm/llvm-project/tree/master/llvm/utils/gn>. We 
could
 > similarly put the Bazel BUILD files under llvm/utils/bazel but have 
them be
 > for the entire llvm project (the subsets that are supported). This seems
 > like an odd structure to me, but I know that the CMake build for LLVM
 > also builds
 > the other subprojects
 > 
<https://github.com/llvm/llvm-project/blob/529ac33197f6/llvm/tools/CMakeLists.txt#L34-L41>,
 > so maybe this would be preferable.
 >
 > Alternatively we could split each subproject into a separate Bazel
 > repository and put the Bazel build files under each subproject. I think
 > this fragments the configuration of the BUILD without much benefit.
 >
 > ## Configurations
 >
 > We currently have configurations for Linux GCC and Clang, MacOS GCC and
 > Clang, and Windows MSVC. Support for other configurations can be added
 > as-desired, but supporting all possible LLVM build configurations is not
 > the goal.
 >
 > ## Support
 >
 > Support would be similar to the gn build. Contributors could optionally
 > update the Bazel BUILD files as part of their patches, but would be under
 > no obligation to do so.
 >
 > ## Preserving History
 >
 > I don't *think* the history of llvm-bazel is interesting enough to try to
 > merge it into the monorepo and I was planning to submit this as a single
 > patch, but please let me know if you disagree.
 >
 > ## Benefits to the community
 >
 >    -
 >
 >    Projects that depend on LLVM and use the Bazel build system can avoid
 >    duplicating fragile effort. We'll spend more time contributing to LLVM
 >    instead :-D
 >    -
 >
 >    Bazel is stricter than CMake in many ways (e.g. it requires that even
 >    header dependencies be declared) and can catch layering issues 
very easily.
 >    There's even an optional layering_check feature we could turn on 
if its use
 >    would benefit the community. (though currently the existing 
problematic
 >    layering makes it a burden to maintain on our own). Even without that
 >    additional check, as I've been keeping the Bazel build green, I've 
found
 >    and fixed a number of layering issues in the past couple weeks (e.g.
 >    https://reviews.llvm.org/rGb49787df9a
 > <https://reviews.llvm.org/rGb49787df9a535f03761c340dca7ec3ec1155133d>
 >    and https://reviews.llvm.org/rGc17ae2916c
 > <https://reviews.llvm.org/rGc17ae2916ccf45a0c1717bd5f11598cc4fff342a>).
 >
 >
 > Here's a patch <https://reviews.llvm.org/D90352> adding the Bazel build
 > system. It's basically just `cp -r llvm-bazel/llvm-bazel
 > llvm-project/utils/bazel`.

Doesn't the last paragraph mean all benefits derived from this can be
described either as:
   (1) users do not need to clone the llvm-bazel git repo but get the
       files in llvm-project, or
   (2) "interested contributors" could send patches to llvm-project
       instead of llvm-bazel to update the bazel build.

TBH, I have no interest in using bazel nor anything against it being
merged per se. I just find it curious that we merge another build system
"at no cost" for the community (I think I picked that up in the thread
but I might have imagined the phrasing). I mean, there is always "a
cost"* so it boils down to determine if the benefit is worth it.

~ Johannes

* i.a., people will assume we (=the LLVM community) maintain(s) a bazel
   build, which can certainly be a benefit but also a cost", e.g., when
   the build is not properly maintained, support is scarce, etc. and
   emails come in complaining about it (not thinking of prior examples
   here.)

 > _______________________________________________
 > LLVM Developers mailing list
 > llvm-dev at lists.llvm.org
 > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev