[llvm-dev] Contributing Bazel BUILD files similar to gn

Thu Oct 29 15:17:02 PDT 2020

I want to jump in with some general support for this addition as a
non-googler + frequent bazel user. I find moving between projects that use
bazel much more palatable than moving between bazel + cmake projects.

I also think there's a huge benefit in having strict dependencies. In my
experience this is especially true for tests. This way you know you've
always correctly rebuilt the necessary inputs to a single test, vs using
`lit` directly and having to know / remember which set of binaries are
required to be rebuilt based on your current changes.
--
Keith Smiley

On Wed, Oct 28, 2020 at 4:18 PM Geoffrey Martin-Noble via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> Hi all,
>
> tl;dr: We'd like to contribute Bazel BUILD files for LLVM and MLIR in a
> side-directory in the monorepo, similar to the gn build.
>
> Some of us have been working on open-source Bazel BUILD files for the LLVM
> Project. You may have seen us hanging out in the #build-systems discord
> channel. As you may know, Google uses Bazel internally and has maintained a
> Bazel BUILD of LLVM for years. Especially with the introduction of MLIR,
> we've got more and more OSS projects with a Bazel BUILD depending on LLVM
> (e.g. IREE <https://github.com/google/iree> and TensorFlow
> <https://github.com/tensorflow/tensorflow>). We're also not the only ones
> using Bazel: e.g. PlaidML also has a Bazel BUILD of LLVM that they've borrowed
> from TF
> <https://github.com/plaidml/plaidml/blob/master/vendor/llvm/llvm.BUILD>.
> Each of these projects has to jump through some weird hoops to keep their
> version of the Bazel BUILD files in sync with the code, which requires some
> fragile combination of scripts and human intervention. Instead, we'd like
> to move general-purpose Bazel BUILD files into the LLVM Project monorepo.
> We expect to follow the model of the GN build where these will be
> maintained by interested contributors rather than expecting the general
> community to maintain them.
>
> To facilitate and test this we've been developing a standalone repository
> that just has the Bazel BUILD files. It symlinks together the directory
> trees on top of a submodule as we would need in the monorepo to to avoid
> in-tree BUILD files. The configuration is at
> https://github.com/google/llvm-bazel. We now have those in a good place
> and think they would be useful upstream.
>
> # Details
>
> ## What
>
> Bazel BUILD files for the LLVM, MLIR, and Clang (PR out for review
> <https://github.com/google/llvm-bazel/pull/72>) subprojects, potentially
> expanding to others, as needed. Basically everything currently at
> https://github.com/google/llvm-bazel.
>
> ## Where
>
> In https://github.com/google/llvm-bazel the BUILD files live in a single
> directory tree matching the structure of the overall llvm-project
> directory. For users, @llvm-project is a single Bazel repository
> <https://docs.bazel.build/versions/master/build-ref.html#repositories>
> that includes both LLVM and MLIR subprojects. To maintain this structure,
> we would probably want to put a `bazel` directory in the monorepo's utils
> directory <https://github.com/llvm/llvm-project/tree/master/utils>, which
> currently only contains a directory for arcanist. This is different from
> gn, which is under the LLVM subproject's utils directory
> <https://github.com/llvm/llvm-project/tree/master/llvm/utils/gn>. We
> could similarly put the Bazel BUILD files under llvm/utils/bazel but have
> them be for the entire llvm project (the subsets that are supported). This
> seems like an odd structure to me, but I know that the CMake build for LLVM
> also builds the other subprojects
> <https://github.com/llvm/llvm-project/blob/529ac33197f6/llvm/tools/CMakeLists.txt#L34-L41>,
> so maybe this would be preferable.
>
> Alternatively we could split each subproject into a separate Bazel
> repository and put the Bazel build files under each subproject. I think
> this fragments the configuration of the BUILD without much benefit.
>
> ## Configurations
>
> We currently have configurations for Linux GCC and Clang, MacOS GCC and
> Clang, and Windows MSVC. Support for other configurations can be added
> as-desired, but supporting all possible LLVM build configurations is not
> the goal.
>
> ## Support
>
> Support would be similar to the gn build. Contributors could optionally
> update the Bazel BUILD files as part of their patches, but would be under
> no obligation to do so.
>
> ## Preserving History
>
> I don't *think* the history of llvm-bazel is interesting enough to try to
> merge it into the monorepo and I was planning to submit this as a single
> patch, but please let me know if you disagree.
>
> ## Benefits to the community
>
>    -
>
>    Projects that depend on LLVM and use the Bazel build system can avoid
>    duplicating fragile effort. We'll spend more time contributing to LLVM
>    instead :-D
>    -
>
>    Bazel is stricter than CMake in many ways (e.g. it requires that even
>    header dependencies be declared) and can catch layering issues very easily.
>    There's even an optional layering_check feature we could turn on if its use
>    would benefit the community. (though currently the existing problematic
>    layering makes it a burden to maintain on our own). Even without that
>    additional check, as I've been keeping the Bazel build green, I've found
>    and fixed a number of layering issues in the past couple weeks (e.g.
>    https://reviews.llvm.org/rGb49787df9a
>    <https://reviews.llvm.org/rGb49787df9a535f03761c340dca7ec3ec1155133d>
>    and https://reviews.llvm.org/rGc17ae2916c
>    <https://reviews.llvm.org/rGc17ae2916ccf45a0c1717bd5f11598cc4fff342a>).
>
>
> Here's a patch <https://reviews.llvm.org/D90352> adding the Bazel build
> system. It's basically just `cp -r llvm-bazel/llvm-bazel
> llvm-project/utils/bazel`.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20201029/4e3a06be/attachment.html>