[llvm-dev] RFC: Contributing Bazel BUILD files in the "peripheral" support tier

Tue Nov 17 09:29:59 PST 2020

Since this is being added within the bounds of a now-existing policy, I will withdraw my objections. Thanks for tabling discussion of adding Bazel until the policy on peripheral tier components was settled, and thanks very much to Renato for taking the initiative to push through a new policy!

Unfortunately, I do not know enough about Bazel to provide any sort of useful code review.

Thanks,
   Christopher Tetreault

From: llvm-dev <llvm-dev-bounces at lists.llvm.org> On Behalf Of Geoffrey Martin-Noble via llvm-dev
Sent: Monday, November 16, 2020 7:02 PM
To: LLVM Dev <llvm-dev at lists.llvm.org>
Subject: [EXT] [llvm-dev] RFC: Contributing Bazel BUILD files in the "peripheral" support tier

I previously<https://groups.google.com/g/llvm-dev/c/u07o3QREVUg/%20> proposed contributing Bazel build files to the LLVM monorepo, supported *only* by interested community members and not to interfere with or affect the existing CMake configuration. As part of that conversation, it became clear that the LLVM policies for more "peripheral" components were not clearly documented. We now have a shiny new Support Policy<http://llvm.org/docs/SupportPolicy.html>. Thank you, Renato for moving that forward. Please take a look at it, if you haven't already.

I would now like to re-raise the proposal to contribute Bazel build files to the LLVM monorepo. I am starting a new thread, as the last one became rather fragmented.

This build configuration would be added at the peripheral support level<http://llvm.org/docs/SupportPolicy.html#peripheral-tier> to a new `utils/bazel` directory. I've prepared a patch<https://reviews.llvm.org/D90352> of what I am proposing to add. It includes a README indicating the level of support. It is largely a port of the Bazel build files Google uses internally and has maintained for several years.

This should have approximately the same impact on the community as the current GN build in `llvm/utils/gn` does today. That is, it should not affect anyone who doesn't care.

I've commented on the specific requirements<http://llvm.org/docs/SupportPolicy.html#id2> listed in the support policy inline:

Code in this tier must:

* Have a clear benefit for residing in the main repository, catering to an active sub-community (upstream or downstream).
A number of projects build LLVM with Bazel (e.g. IREE<https://github.com/google/iree>, TensorFlow<https://github.com/tensorflow/tensorflow>, PlaidML<https://github.com/plaidml/plaidml/blob/master/vendor/llvm/llvm.BUILD>). Google also uses Bazel to build in its internal source repository. This includes a substantial number of developers and active contributors to LLVM. Adding this to the monorepo would provide a natural coordination point for these projects and avoid fragmentation (projects currently have their own copies of the BUILD files) or Google-centric governance (e.g. signing Google's CLA).
* Be actively maintained by such sub-community and have its problems addressed in a timely manner.
We can commit to maintaining and addressing issues with the configuration. Google has maintained its internal version of this configuration for a few years.

Code in this tier must not:
* Break or invalidate core tier code or infrastructure. If that happens accidentally, reverting functionality and working on the issues offline is the only acceptable course of action.
There should be no interaction between the Bazel build configuration and any core code or infrastructure.
* Negatively affect development of core tier code, with the sub-community involved responsible for making changes to address specific concerns.
 This should not affect development of core tier components. One reason we propose adding this to the root utils/ directory instead of under llvm/utils (where GN is located) is to avoid unnecessarily sending messages to llvm-commits. Others have raised the concern that the existence of an alternative build system might lead to lack of maintenance for the CMake build system. Given that supporting CMake will remain a requirement and maintenance of a Bazel build system will continue to happen regardless, we do not expect any significant impact in this way.
* Negatively affect other peripheral tier code, with the sub-communities involved tasked to resolve the issues, still making sure the solution doesn’t break or invalidate the core tier.
Similarly, this should have no interaction with other components in the peripheral tier. We will address conflicts if they arise.
* Impose sub-optimal implementation strategies on core tier components as a result of idiosyncrasies in the peripheral component.
We do not expect any negative constraints on normal development of core tiers. Bazel is stricter about layering, which may help quickly identify layering issues in incoming commits.
* Have build infrastructure that spams all developers about their breakages.
Build infrastructure will be configured to only notify opted-in developers.
* Fall into disrepair. This is a reflection of lack of an active sub-community and will result in removal.
Build bots with accompanying status badges will be prominently linked from the README. Currently a Linux/Clang build bot exists and can be easily reconfigured after the code move. A windows build bot will be added soon.

Code in this tier should:
* Have infrastructure to test, whenever meaningful, with either no warnings or notification contained within the sub-community.
* Have support and testing that scales with the complexity and resilience of the component, with the bar for simple and gracefully-degrading components (such as editor bindings) much lower than for complex components that must remain fresh with HEAD (such as experimental back-ends or alternative build systems).
Build bot coverage already exists and will be expanded as described above.
* Have a document making clear the status of implementation, level of support available, who the sub-community is and, if applicable, roadmap for inclusion into the core tier.
The patch includes a README that should make the support level clear. I am happy to add additional language or take additional steps to make that more clear (e.g. adding `unsupported` to the directory path).
* Be restricted to a specific directory or have a consistent pattern (ex. unique file suffix), making it easy to remove when necessary.
All configuration is restricted to a single directory and should be trivial to remove.

A number of people raised the question of "why not a separate repository". This is indeed possible: It's what we've done with https://github.com/google/llvm-bazel, which is currently used by https://github.com/google/iree. It is significantly more infrastructure, coordination, and complexity for something that is specifically a configuration for the LLVM project itself, not its own dependent or adjacent project.

I believe this contribution will significantly improve the situation for downstream users that use Bazel while having minimal impact on the community at large.

Thanks,
Geoffrey
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20201117/b070626a/attachment-0001.html>