[llvm-dev] RFC: Contributing Bazel BUILD files in the "peripheral" support tier

[Geoffrey Martin-Noble via llvm-dev]

I previously <https://groups.google.com/g/llvm-dev/c/u07o3QREVUg/> proposed
contributing Bazel build files to the LLVM monorepo, supported *only* by
interested community members and not to interfere with or affect the
existing CMake configuration. As part of that conversation, it became clear
that the LLVM policies for more "peripheral" components were not clearly
documented. We now have a shiny new Support Policy
<http://llvm.org/docs/SupportPolicy.html>. Thank you, Renato for moving
that forward. Please take a look at it, if you haven't already.

I would now like to re-raise the proposal to contribute Bazel build files
to the LLVM monorepo. I am starting a new thread, as the last one became
rather fragmented.

This build configuration would be added at the peripheral support level
<http://llvm.org/docs/SupportPolicy.html#peripheral-tier> to a new
`utils/bazel` directory. I've prepared a patch
<https://reviews.llvm.org/D90352> of what I am proposing to add. It
includes a README indicating the level of support. It is largely a port of
the Bazel build files Google uses internally and has maintained for several
years.

This should have approximately the same impact on the community as the
current GN build in `llvm/utils/gn` does today. That is, it should not
affect anyone who doesn't care.

I've commented on the specific requirements
<http://llvm.org/docs/SupportPolicy.html#id2> listed in the support policy
inline:

Code in this tier must:
>


* Have a clear benefit for residing in the main repository, catering to an
> active sub-community (upstream or downstream).
>
A number of projects build LLVM with Bazel (e.g. IREE
<https://github.com/google/iree>, TensorFlow
<https://github.com/tensorflow/tensorflow>, PlaidML
<https://github.com/plaidml/plaidml/blob/master/vendor/llvm/llvm.BUILD>).
Google also uses Bazel to build in its internal source repository. This
includes a substantial number of developers and active contributors to
LLVM. Adding this to the monorepo would provide a natural coordination
point for these projects and avoid fragmentation (projects currently have
their own copies of the BUILD files) or Google-centric governance (e.g.
signing Google's CLA).

> * Be actively maintained by such sub-community and have its problems
> addressed in a timely manner.
>
We can commit to maintaining and addressing issues with the configuration.
Google has maintained its internal version of this configuration for a few
years.

>

Code in this tier must not:
> * Break or invalidate core tier code or infrastructure. If that happens
> accidentally, reverting functionality and working on the issues offline is
> the only acceptable course of action.
>
There should be no interaction between the Bazel build configuration and
any core code or infrastructure.

> * Negatively affect development of core tier code, with the sub-community
> involved responsible for making changes to address specific concerns.
>
 This should not affect development of core tier components. One reason we
propose adding this to the root utils/ directory instead of under
llvm/utils (where GN is located) is to avoid unnecessarily sending messages
to llvm-commits. Others have raised the concern that the existence of an
alternative build system might lead to lack of maintenance for the CMake
build system. Given that supporting CMake will remain a requirement and
maintenance of a Bazel build system will continue to happen regardless, we
do not expect any significant impact in this way.

> * Negatively affect other peripheral tier code, with the sub-communities
> involved tasked to resolve the issues, still making sure the solution
> doesn’t break or invalidate the core tier.
>
Similarly, this should have no interaction with other components in the
peripheral tier. We will address conflicts if they arise.

> * Impose sub-optimal implementation strategies on core tier components as
> a result of idiosyncrasies in the peripheral component.
>
We do not expect any negative constraints on normal development of core
tiers. Bazel is stricter about layering, which may help quickly identify
layering issues in incoming commits.

> * Have build infrastructure that spams all developers about their
> breakages.
>
Build infrastructure will be configured to only notify opted-in developers.

> * Fall into disrepair. This is a reflection of lack of an active
> sub-community and will result in removal.

Build bots with accompanying status badges will be prominently linked from
the README. Currently a Linux/Clang build bot exists and can be easily
reconfigured after the code move. A windows build bot will be added soon.

>

Code in this tier should:
> * Have infrastructure to test, whenever meaningful, with either no
> warnings or notification contained within the sub-community.
>
* Have support and testing that scales with the complexity and resilience
> of the component, with the bar for simple and gracefully-degrading
> components (such as editor bindings) much lower than for complex components
> that must remain fresh with HEAD (such as experimental back-ends or
> alternative build systems).
>
Build bot coverage already exists and will be expanded as described above.

> * Have a document making clear the status of implementation, level of
> support available, who the sub-community is and, if applicable, roadmap for
> inclusion into the core tier.
>
The patch includes a README that should make the support level clear. I am
happy to add additional language or take additional steps to make that more
clear (e.g. adding `unsupported` to the directory path).

> * Be restricted to a specific directory or have a consistent pattern (ex.
> unique file suffix), making it easy to remove when necessary.
>
All configuration is restricted to a single directory and should be trivial
to remove.

A number of people raised the question of "why not a separate repository".
This is indeed possible: It's what we've done with
https://github.com/google/llvm-bazel, which is currently used by
https://github.com/google/iree. It is significantly more infrastructure,
coordination, and complexity for something that is specifically a
configuration for the LLVM project itself, not its own dependent or
adjacent project.

I believe this contribution will significantly improve the situation for
downstream users that use Bazel while having minimal impact on the
community at large.

Thanks,
Geoffrey
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20201116/22a3b454/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3992 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20201116/22a3b454/attachment.bin>