[llvm-dev] Contributing Bazel BUILD files similar to gn

Sun Nov 1 02:00:16 PST 2020

Hi,

Doesn't an email like this maybe point to another problem that, if solved,
would make situations like this not a problem at all.
One would have 2 git repos.
The main one and an overlay one. Much like overlay filesystems work.
They would be linked in such a way that the developer would not need to
manually add dependencies but the git repos would be linked (linked at the
commit hash tree level) so that if one did a checkout from one, the correct
matching version of the other repo would also be checked out.
Then the main repo would just need a "readme", saying: To build with Bazel
add this overlay repo.

But on the topic of having multiple build systems for LLVM.
If I was to wish to upstream a commit to LLVM, I would:
1) Want to know which build system that commit should work with.
2) Not have to make sure my commit worked with both build systems.

I worked on one project that had two build systems, and most of the time,
only one or the other build would actually work, because committers only
fixed the build system they actually used.

Kind Regards

James

On Wed, 28 Oct 2020 at 23:18, Geoffrey Martin-Noble via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> Hi all,
>
> tl;dr: We'd like to contribute Bazel BUILD files for LLVM and MLIR in a
> side-directory in the monorepo, similar to the gn build.
>
> Some of us have been working on open-source Bazel BUILD files for the LLVM
> Project. You may have seen us hanging out in the #build-systems discord
> channel. As you may know, Google uses Bazel internally and has maintained a
> Bazel BUILD of LLVM for years. Especially with the introduction of MLIR,
> we've got more and more OSS projects with a Bazel BUILD depending on LLVM
> (e.g. IREE <https://github.com/google/iree> and TensorFlow
> <https://github.com/tensorflow/tensorflow>). We're also not the only ones
> using Bazel: e.g. PlaidML also has a Bazel BUILD of LLVM that they've borrowed
> from TF
> <https://github.com/plaidml/plaidml/blob/master/vendor/llvm/llvm.BUILD>.
> Each of these projects has to jump through some weird hoops to keep their
> version of the Bazel BUILD files in sync with the code, which requires some
> fragile combination of scripts and human intervention. Instead, we'd like
> to move general-purpose Bazel BUILD files into the LLVM Project monorepo.
> We expect to follow the model of the GN build where these will be
> maintained by interested contributors rather than expecting the general
> community to maintain them.
>
> To facilitate and test this we've been developing a standalone repository
> that just has the Bazel BUILD files. It symlinks together the directory
> trees on top of a submodule as we would need in the monorepo to to avoid
> in-tree BUILD files. The configuration is at
> https://github.com/google/llvm-bazel. We now have those in a good place
> and think they would be useful upstream.
>
> # Details
>
> ## What
>
> Bazel BUILD files for the LLVM, MLIR, and Clang (PR out for review
> <https://github.com/google/llvm-bazel/pull/72>) subprojects, potentially
> expanding to others, as needed. Basically everything currently at
> https://github.com/google/llvm-bazel.
>
> ## Where
>
> In https://github.com/google/llvm-bazel the BUILD files live in a single
> directory tree matching the structure of the overall llvm-project
> directory. For users, @llvm-project is a single Bazel repository
> <https://docs.bazel.build/versions/master/build-ref.html#repositories>
> that includes both LLVM and MLIR subprojects. To maintain this structure,
> we would probably want to put a `bazel` directory in the monorepo's utils
> directory <https://github.com/llvm/llvm-project/tree/master/utils>, which
> currently only contains a directory for arcanist. This is different from
> gn, which is under the LLVM subproject's utils directory
> <https://github.com/llvm/llvm-project/tree/master/llvm/utils/gn>. We
> could similarly put the Bazel BUILD files under llvm/utils/bazel but have
> them be for the entire llvm project (the subsets that are supported). This
> seems like an odd structure to me, but I know that the CMake build for LLVM
> also builds the other subprojects
> <https://github.com/llvm/llvm-project/blob/529ac33197f6/llvm/tools/CMakeLists.txt#L34-L41>,
> so maybe this would be preferable.
>
> Alternatively we could split each subproject into a separate Bazel
> repository and put the Bazel build files under each subproject. I think
> this fragments the configuration of the BUILD without much benefit.
>
> ## Configurations
>
> We currently have configurations for Linux GCC and Clang, MacOS GCC and
> Clang, and Windows MSVC. Support for other configurations can be added
> as-desired, but supporting all possible LLVM build configurations is not
> the goal.
>
> ## Support
>
> Support would be similar to the gn build. Contributors could optionally
> update the Bazel BUILD files as part of their patches, but would be under
> no obligation to do so.
>
> ## Preserving History
>
> I don't *think* the history of llvm-bazel is interesting enough to try to
> merge it into the monorepo and I was planning to submit this as a single
> patch, but please let me know if you disagree.
>
> ## Benefits to the community
>
>    -
>
>    Projects that depend on LLVM and use the Bazel build system can avoid
>    duplicating fragile effort. We'll spend more time contributing to LLVM
>    instead :-D
>    -
>
>    Bazel is stricter than CMake in many ways (e.g. it requires that even
>    header dependencies be declared) and can catch layering issues very easily.
>    There's even an optional layering_check feature we could turn on if its use
>    would benefit the community. (though currently the existing problematic
>    layering makes it a burden to maintain on our own). Even without that
>    additional check, as I've been keeping the Bazel build green, I've found
>    and fixed a number of layering issues in the past couple weeks (e.g.
>    https://reviews.llvm.org/rGb49787df9a
>    <https://reviews.llvm.org/rGb49787df9a535f03761c340dca7ec3ec1155133d>
>    and https://reviews.llvm.org/rGc17ae2916c
>    <https://reviews.llvm.org/rGc17ae2916ccf45a0c1717bd5f11598cc4fff342a>).
>
>
> Here's a patch <https://reviews.llvm.org/D90352> adding the Bazel build
> system. It's basically just `cp -r llvm-bazel/llvm-bazel
> llvm-project/utils/bazel`.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20201101/bfcad26c/attachment.html>